Skip to main content
 首页 » 编程设计

python之word_tokenize TypeError : expected string or buffer

2025年05月04日13三少

<分区>

调用 word_tokenize 时出现以下错误:

File "C:\Python34\lib\site-packages\nltk\tokenize\punkt.py", line 1322, 
    in _slices_from_text for match in 
    self._lang_vars.period_context_re().finditer(text): 
TypeError: expected string or buffer 

我有一个大文本文件 (1500.txt),我想从中删除停用词。 我的代码如下:

from nltk.corpus import stopwords 
from nltk.tokenize import word_tokenize 
 
with open('E:\\Book\\1500.txt', "r", encoding='ISO-8859-1') as File_1500: 
    stop_words = set(stopwords.words("english")) 
    words = word_tokenize(File_1500) 
    filtered_sentence = [w for w in words if not w in stop_words] 
    print(filtered_sentence)