Skip to main content
 首页 » 编程设计

python之在 csv 文件中写入字符串时,如何避免重复字符串

2025年01月19日25bjzhanghao

我用 Python3 和 bs4 成功地编写了一个脚本,从维基百科页面获取没有重复的字符串。为此,

算法:

1) 写入有重复项的csv文件

使用上面的文件,

2) 写入 csv 文件没有重复项。

脚本:

#!/usr/bin/env python 
# -*- coding: utf-8 -*- 
from bs4 import BeautifulSoup 
import requests 
import csv 
 
url = 'https://ta.wikisource.org/w/index.php?title=அட்டவணை:அ. மருதகாசி-பாடல்கள்.pdf&action=history' 
content = requests.get(url).content 
soup = BeautifulSoup(content,'lxml') 
#getting the uncleaned contributors 
userBdi = soup.findAll('bdi') 
#list 2 string 
uncleanedContributors =''.join(str(userBdi)[1:-1]).replace('</','<').replace('<bdi>','').replace(',','\n').replace(' ','').replace('பக்கம்','அட்டவணை_பேச்சு').replace('Bot','').replace('BOT','') 
print() 
print('The output of uncleaned contributors') 
print('--------------------------------------') 
print(uncleanedContributors) 
with open('uncleaned-contributors.csv','a') as csvwrite: 
    csvwriter = csvwrite.write(uncleanedContributors+'\n') 
 
content = open('uncleaned-contributors.csv','r').readlines() 
content4set = set(content) 
cleanedcontent = open('cleaned-contributors.csv','w') 
print() 
print('The output of cleaned contributors') 
print('--------------------------------------') 
for i, line in enumerate(content4set,0): 
    cleanedcontent.write("{}.{}".format(str(i+1),line.replace('பக்கம்','அட்டவணை_பேச்சு'))) 
    line=line.strip() 
    print(i, line) 
cleanedcontent.close() 

如何直接写入CSV文件而不重复?有什么办法吗?

请您参考如下方法:

这是解决您问题的一种方法:

from bs4 import BeautifulSoup  
import requests  
import csv  
 
url = 'https://ta.wikisource.org/w/index.php?title=அட்டவணை:அ. மருதகாசி-பாடல்கள்.pdf&action=history' 
content = requests.get(url).content  
soup = BeautifulSoup(content,'lxml')  
#getting the uncleaned contributors  
userBdi = soup.findAll('bdi')  
#list 2 string  
uncleanedContributors =''.join(str(userBdi)[1:-1]).replace('</','<').replace('<bdi>','').replace(',','\n').replace(' ','').replace('பக்கம்','அட்டவணை_பேச்சு').replace('Bot','').replace('BOT','')  
 
cleanedcontent = open('cleaned-contributors.csv','w')  
print()  
print('The output of cleaned contributors')  
print('--------------------------------------')  
def unique_list(l): 
    ulist = [] 
    [ulist.append(x) for x in l if x not in ulist] 
    return ulist 
a = ' '.join(unique_list(uncleanedContributors.split())) 
 
for  i, j in enumerate(a.split(' ')): 
    cleanedcontent.write("{}.{}".format(str(i+1),j.replace('பக்கம்','அட்டவணை_பேச்சு'))) 
    cleanedcontent.write('\n') 
    print(i+1, j) 
 
cleanedcontent.close() 

执行时,

[1]:  
The output of cleaned contributors 
-------------------------------------- 
1 Balajijagadesh 
2 Info-farmer  
3 Tshrinivasan 

上面的解决方案代码给出了您在问题中所需的确切输出,并且能够直接写入 CSV 文件而没有任何重复。