Skip to main content
 首页 » 编程设计

python之csv 阅读器中的通用换行模式使 csv 编写器在文件中写入错误换行符

2025年05月04日8langtianya

当我使用通用行模式 ("rU") cdv.reader 读取 csv 文件时,它会在 csv.writer 中生成\r\n 作为新行。你知道如何忽略 csv.writer 中的新行吗?我不得不在阅读器中使用 ("rU"),因为我的文件包含换行符。

这是我使用的代码

import csv 
 
dict={} 
with open('training_data.csv','rU') as f: 
    reader = csv.reader(f,skipinitialspace=True) 
for line in reader: 
    try: 
        dict[line[2]].append(line[3]) 
    except: 
        dict[line[2]]=[line[3]] 
 
with open('training_result.csv','w') as f: 
writer = csv.writer(f, delimiter='|',dialect='excel-tab') 
for key in dict: 
    writer.writerow([key,','.join(dict[key])]) 

输入是这样的

username, some of tweet that 
want to be processed 
by machine , label 

因为那是换行符和通用线路模式激活,当我捕获数据并想用 csv writer 写入时,它会是一样的

我想要的输出是这样的

username, some of tweet that want to be processed by machine , label 

我应该删除 csv 文件中的所有换行符吗?但它太大了,csv 大约有 150MB,包含 70 万行。有什么办法吗?

我已经尝试过 skipinitialspace 和 dialect 等阅读器属性,但仍然无法解决问题

请您参考如下方法:

我认为这就是您要寻找的结果。你没有提到你的 Python 版本。这是 Python 3。我使用了您上传到 Google 云端硬盘的示例数据。文件解析为 UTF-8。

主要注意事项:

  • csv 有一个 DictReader 来帮助选择要处理的列。
  • CSV 文件应该以二进制模式打开。在 Python 2 中它只是 'rb' 或 'wb' 但在 Python 3 中它意味着 'r',newline='' 和一个编码到 open 调用。
  • line 将是 {'header':'value'} 对的字典。
  • extrasaction 告诉 DictWriter 忽略字典中未在 fieldnames 中列出的额外字段。

示例数据:

twitter.place.full_name,twitter.user.location,interaction.author.username,interaction.content,interaction.created_at 
"Gunungsari, Lombok Barat",Indonesia,__Thasya__,At Sheraton Senggigi Beach Resort äóî https://t.co/1FdTsMYWje,"Mon, 16 Jun 2014 15:32:54 +0000" 
"Cakranegara, Kota Mataram",NULL,__Waone,Mataram,"Mon, 24 Mar 2014 13:13:46 +0000" 
"Pemenang, Lombok Utara",Jakarta,_5at,"perdana, my first nephew from my lil sibling sister,,, 
 
*moga gäó» ketularan songong kayak pamannya >_< http://t.co/UBEwcxWY5c","Sat, 04 Jan 2014 04:20:45 +0000" 
"Pemenang, Lombok Utara",Jakarta,_5at,"@indiraputeri udah pinter bahasa sasak nih skrng,,, inaq rari","Sat, 04 Jan 2014 06:15:52 +0000" 
"Pemenang, Lombok Utara",Jakarta,_5at,@indiraputeri dalemmm bgt nih ndoro .. !!! mksd nya apaan?,"Sat, 04 Jan 2014 05:55:04 +0000" 
"Keruak, Lombok Timur",Jakarta,_5at,"pagi2, hujan, holiday, nasi goreng hangat, kopi hangat, di rumah, + spesial: kumpul keluarga,,, ^_^  *kurang_apa_lagi","Thu, 02 Jan 2014 00:02:47 +0000" 
"Pujut, Lombok Tengah",Jakarta,_5at,"Doäó»a bepergian keluar rumah: 
 
""Bismillaahitawakkaltu äó»alallooh"" 
 
*pasrah-pasrah-pasrah; 
*bandara_international_lombok","Sun, 05 Jan 2014 03:36:48 +0000" 
"Sakra, Lombok Timur",Jakarta,_5at,"Time for riding with my lil bro: 
Mataram - Senggigi - Gili Terawangan 
*jenguk_ponakan_baru; 
*very_early","Fri, 03 Jan 2014 22:09:26 +0000" 
"Sukamulia, Lombok Timur",,1821922,Salam friend,"Sun, 20 Jul 2014 19:23:53 +0000" 

代码:

import csv 
 
# Python 2 version of opens 
#with open('training_data.csv','rb') as inp: 
#    with open('training_result.csv','wb') as outp: 
 
with open('training_data.csv','r',newline='',encoding='utf8') as inp: 
    with open('training_result.csv','w',newline='',encoding='utf8') as outp: 
        reader = csv.DictReader(inp) 
        writer = csv.DictWriter(outp, 
                                fieldnames=['interaction.author.username','interaction.content'], 
                                extrasaction='ignore') 
        writer.writeheader() 
        for line in reader: 
            line['interaction.content'] = line['interaction.content'].replace('\n',' ') 
            writer.writerow(line) 

结果:

interaction.author.username,interaction.content 
__Thasya__,At Sheraton Senggigi Beach Resort äóî https://t.co/1FdTsMYWje 
__Waone,Mataram 
_5at,"perdana, my first nephew from my lil sibling sister,,,  *moga gäó» ketularan songong kayak pamannya >_< http://t.co/UBEwcxWY5c" 
_5at,"@indiraputeri udah pinter bahasa sasak nih skrng,,, inaq rari" 
_5at,@indiraputeri dalemmm bgt nih ndoro .. !!! mksd nya apaan? 
_5at,"pagi2, hujan, holiday, nasi goreng hangat, kopi hangat, di rumah, + spesial: kumpul keluarga,,, ^_^  *kurang_apa_lagi" 
_5at,"Doäó»a bepergian keluar rumah:  ""Bismillaahitawakkaltu äó»alallooh""  *pasrah-pasrah-pasrah; *bandara_international_lombok" 
_5at,Time for riding with my lil bro: Mataram - Senggigi - Gili Terawangan *jenguk_ponakan_baru; *very_early 
1821922,Salam friend