我有一个 csv 文件,其中一列包含 10 行文本。对于每一行,我想删除停用词并取回相同的 csv 文件,只是减去停用词。
这是我的代码:
def remove_stopwords(filename):
new_text_list=[]
cr = csv.reader(open(filename,"rU").readlines()[1:])
cachedStopWords = stopwords.words("english")
for row in cr:
text = ' '.join([word for word in row.split() if word not in cachedStopWords])
print text
new_text_list.append(text)
但是我一直收到这个错误:
AttributeError: 'list' object has no attribute 'split'
所以我的 csv 文件中的行似乎无法使用 .split 拆分,因为它们是一个列表。我该如何解决这个问题?
这是我的 csv 文件的样子
Text
I am very pleased with the your software for contractors. It is tailored quite neatly for the construction industry.
We have two different companies, one is real estate management and one is health and nutrition services. It works great for both.
所以上面的例子是我的 csv 文件的前 3 行。 当我运行这行代码时:
cr = csv.reader(open(filename,"rU").readlines()[1:])
print cr[2]
我明白了:
['We have two different companies, one is real estate management and one is health and nutrition services. It works great for both.']
谢谢,
请您参考如下方法:
您的数据文件不是 CSV - 单词由空格而不是逗号分隔。所以你不需要 CSV 模块。相反,只需从文件中读取每一行并使用 row = line.split() 在空白处拆分行。
def remove_stopwords(filename):
new_text_list = []
cachedStopWords = set(stopwords.words("english"))
with open(filename, "rU") as f:
next(f) # skip one line
for line in f:
row = line.split()
text = ' '.join([word for word in row
if word not in cachedStopWords])
print(text)
new_text_list.append(text)
顺便说一下,checking membership in a set是一个 O(1) 操作,而检查 list 中的成员资格是一个 O(n) 操作。因此,将 cachedStopWords 设为一个集合是有利的:
cachedStopWords = set(stopwords.words("english"))
