我有一个关键字列表和一个数据框:
keywords=['椅子',' table ', '风扇']
Description
The table is 6 inches long
The fan is really good
The table fan is cheap
The chair is broken
The chair is on the table
我想搜索关键字列表并创建一个新列,列表中的关键字出现在 Description 列中。
Description Keyword
The table is 6 inches long table
The fan is really good fan
The table fan is cheap table, fan
The chair is broken chair
The chair is on the table chair, table
我已经搜索了一些解决方案,但似乎都没有用。我自己尝试了以下代码:
for i in word_set:
for x in range(0, len(df)):
if(df['Event Message'][x] in (i)):
df['word'] = i
但显然时间复杂度太高,耗费大量时间。任何帮助将不胜感激。
请您参考如下方法:
使用Series.str.findall与 Series.str.join并通过正则表达式或 - |:
keywords=['chair','table', 'fan']
df['Keyword'] = df['Description'].str.findall('|'.join(keywords)).apply(set).str.join(', ')
print (df)
Description Keyword
0 The table is 6 inches long table
1 The fan is really good fan
2 The table fan is cheap table, fan
3 The chair is broken chair
4 The chair is on the table chair, table
如果需要单词边界以避免提取子字符串:
keywords=['chair','tab', 'fan']
pat = '|'.join(r"\b{}\b".format(x) for x in keywords)
df['Keyword1'] = df['Description'].str.findall(pat).apply(set).str.join(', ')
df['Keyword2'] = df['Description'].str.findall('|'.join(keywords)).apply(set).str.join(', ')
print (df)
Description Keyword1 Keyword2
0 The table is 6 inches long tab
1 The fan is really good fan fan
2 The table fan is cheap fan tab, fan
3 The chair is broken chair chair
4 The chair is on the table chair chair, tab
编辑:为了提高性能,可以使用带有拆分和测试成员资格的自定义函数:
keywords=['chair','table', 'fan']
s = set(keywords)
f = lambda x: ', '.join(set([y for y in x.split() if y in s]))
df['Keyword1'] = df['Description'].apply(f)
列表理解也应该更快:
df['Keyword1'] = [', '.join(set([y for y in x.split() if y in s])) for x in df['Description']]
print (df)
Description Keyword1
0 The table is 6 inches long table
1 The fan is really good fan
2 The table fan is cheap fan, table
3 The chair is broken chair
4 The chair is on the table table, chair
谢谢@Henry Yik 的另一种解决方案set.intersection:
df['Keyword1'] = df['Description'].apply(lambda x: ', '.join(set(x.split()).intersection(s)))
