对于以下数据框:
index sentences category
1 the side effects are terrible ! SSRI
2 They are killing me,,, I want to stop SNRI
3 I need to contact my physicians ? SSRI
4 How to stop it.. I am surprised because of its effect. SSRI
5 SSRI
6 NAN SNRI
我正在尝试标记句子列中的句子。 sentences 列有一些空值。这是我的代码,但它不起作用。
df["sentences"] = df.sentences.replace (r'[^a-zA-Z]', '', regex= True, inplace = True)
df["tokenized_sents"] = df["sentences"].apply(nltk.word_tokenize)
我也试过这个:
df["sentences"] = df.sentences.replace (r'[^a-zA-Z]', 'null', regex= True, inplace = True)
它会产生以下错误:
expected string or bytes-like object
有什么建议吗?
请您参考如下方法:
# I added NaN, None to your date for demonstration, please check below first df.
print(df)
df["tokenized_sents"] = df["sentences"].fillna("").map(nltk.word_tokenize)
print(df)
第一次打印,
index sentences category
0 1 the side effects are terrible ! SSRI
1 2 They are killing me,,, I want to stop SNRI
2 3 I need to contact my physicians ? SSRI
3 4 How to stop it.. I am surprised because of its... SSRI
4 5 NaN SNRI
5 5 None None
第二次打印,
index sentences category \
0 1 the side effects are terrible ! SSRI
1 2 They are killing me,,, I want to stop SNRI
2 3 I need to contact my physicians ? SSRI
3 4 How to stop it.. I am surprised because of its... SSRI
4 5 NaN SNRI
5 5 None None
tokenized_sents
0 [the, side, effects, are, terrible, !]
1 [They, are, killing, me, ,, ,, ,, I, want, to,...
2 [I, need, to, contact, my, physicians, ?]
3 [How, to, stop, it.., I, am, surprised, becaus...
4 []
5 []
顺便说一句,如果您显式地使用了 inplace=True,则不必再次将它分配给您原来的 df。
df.sentences.replace(r'[^a-zA-Z]', '', regex=True, inplace=True)
# instead of, df["sentences"] = df.sentences.replace(r'[^a-zA-Z]', '', regex=True, inplace=True)
