Skip to main content
 首页 » 编程设计

python之列表理解之谜——Python

2024年11月01日10wayfarer

我创建了两个 CSV 列表。一个是原始 CSV 文件,另一个是该文件的 DeDuped 版本。我已将每一个读入一个列表,并且出于所有意图和目的,它们都是相同的格式。每个列表项都是一个字符串。

我正在尝试使用列表理解来找出哪些项目被重复删除了。原来的长度是16939,DeDupe的list长度是15368,相差1571,但是我的list comprehension长度是368。有什么想法?

deduped = open('account_de_ex.csv', 'r') 
deduped_data = deduped.read() 
deduped.close() 
deduped = deduped_data.split("\r") 
 
#read in file with just the account names from the full account list 
account_names = open('account_names.csv', 'r') 
account_data = account_names.read() 
account_names.close() 
account_names = account_data.split("\r") 
 
# Get all the accounts that were deleted in the dedupe - i.e. get the duplicate accounts 
dupes = [ele for ele in account_names if ele not in deduped] 

编辑:对于评论中的一些注释,这里是对我的列表组件和列表本身的测试。几乎相同的差异,20 左右。不是我需要的 1500!谢谢!

print len(deduped) 
deduped = set(deduped) 
print len(deduped) 
 
print len(account_names) 
account_names = set(account_names) 
print len(account_names) 
 
 
15368 
15368 
16939 
15387 

请您参考如下方法:

尝试运行此代码并查看它报告的内容。这需要 Python 2.7 或更高版本的 collections.Counter 但您可以轻松编写自己的计数器代码,或者从另一个答案复制我的示例代码:Python : List of dict, if exists increment a dict value, if not append a new dict

from collections import Counter 
 
# read in original records 
with open("account_names.csv", "rt") as f: 
    rows = sorted(line.strip() for line in f) 
 
# count how many times each row appears 
counts = Counter(rows) 
 
# get a list of tuples of (count, row) that only includes count > 1 
dups = [(count, row) for row, count in counts.items() if count > 1] 
dup_count = sum(count-1 for count in counts.values() if count > 1) 
 
# sort the list from largest number of dups to least 
dups.sort(reverse=True) 
 
# print a report showing how many dups 
for count, row in dups: 
    print("{}\t{}".format(count, row)) 
 
# get de-duped list 
unique_rows = sorted(counts) 
 
# read in de-duped list 
with open("account_de_ex.csv", "rt") as f: 
    de_duped = sorted(line.strip() for line in f) 
 
print("List lengths: rows {}, uniques {}/de_duped {}, result {}".format( 
        len(rows), len(unique_rows), len(de_duped), len(de_duped) + dup_count)) 
 
# lists should match since we sorted both lists 
if unique_rows == de_duped: 
    print("perfect match!") 
else: 
    # if lists don't match, find out what is going on 
    uniques_set = set(unique_rows) 
    deduped_set = set(de_duped) 
 
    # find intersection of the two sets 
    x = uniques_set.intersection(deduped_set) 
 
    # print differences 
    if x != uniques_set: 
        print("Rows in original that are not in deduped:\n{}".format(sorted(uniques_set - x))) 
    if x != deduped_set: 
        print("Rows in deduped that are not in original:\n{}".format(sorted(deduped_set - x)))