Skip to main content
 首页 » 编程设计

python之获取由 Python 中的树状图形成的簇列表

2025年02月15日10落叶无声

我有一个单词列表,我对其执行 TF-IDF 算法以获得前 100 个单词的列表。之后我应该执行聚类。现在我可以完成这两项任务(我正在分享代码和输入文件的相关部分,输出截图)。

我的查询是我想要在输出树状图中形成的簇列表,我该怎么做? Dendrogram 函数返回一个元组 ax,它具有一些坐标和节点列表。我如何操作它们以获得完整的集群列表。

以下是输入文件的摘录。

"recommended stories dylan scott stat advertisement kate sheridan dylan scott dylan scott", 
"email touting former representative mike fergusons genuine connection", 
"email touting former representative mike ferguson \u2019", 
"facebook donald trump fda hhs privacy policy", 
"president trump appoints dr scott gottlieb", 
"trade groups including novartis ag", 
"bush alumni coalition supporting trump", 
"online presidential transition analysis center", 
"tennessee republican representative marsha blackburn", 
"nonprofit global health care company", 
"paula stannard ,\u201d said ladd wiley", 
"bremberg returned calls seeking comment", 
"0 \u2026. 0 \u2026 1c", 
"2016 w ashington \u2014 let", 
"take place ,\u201d said dr", 
"\u201c selling baby parts .\u201d", 
"health care companies whose boards", 
"transition ,\u201d said lisa tofil", 

下面是我使用的代码

punctuations = '''!()-[]{};:'\<>./?@#$%^&*_~''' 
n_a =fin_a= "" 
for file in os.listdir(): 
    if (file.endswith(".kwp")): 
        with open(file) as f: 
            #print(f.read()) 
            a = f.read() 
            a = re.sub(r"\\[a-z0-9A-Z]+","",a) 
            a = re.sub(r"\"","",a) 
            a = re.sub(r"\,","",a) 
            #a = re.sub("\\","",a) 
            #print(a) 
            for ch in a: 
                if (ch not in punctuations): 
                    n_a = n_a + ch 
            n_a = n_a.lower() 
            #print(n_a) 
            #new_f = open("n") 
 
        fin_a = fin_a + n_a  
 
tfidf_vectorizer = TfidfVectorizer(max_df=1,stop_words='english',use_idf=True) 
tfd_mat = tfidf_vectorizer.fit_transform([n_a]) 
dense = tfd_mat.todense() 
#print(len(dense[0].tolist()[0])) 
ep = dense[0].tolist()[0] 
phrase_scores = [pair for pair in zip(range(0, len(ep)), ep) if pair[1] > 0] 
#print(phrase_scores) 
#print(len(phrase_scores)) 
phrase_scores=sorted(phrase_scores, key=lambda t: t[1] * -1)[:100] 
#rint(tfd_mat) 
fin_term = [] 
terms = tfidf_vectorizer.get_feature_names() 
with open("/home/laitkor/Desktop/New_Paul/kwp_top100.txt","w") as fl: 
    for t in range(0,100): 
        #print(t) 
        key,valu = phrase_scores[t] 
        #print(key) 
        #print(valu) 
        fl.write(terms[key]+'\n') 
        fin_term.append(terms[key]) 
#print(fin_term)     
#print(phrase_scores[1:100]) 
dist = 1 - cosine_similarity(phrase_scores[1:100]) 
#print(dist) 
linkage_matrix = ward(dist) 
#print(linkage_matrix) 
fig, ax = plt.subplots(figsize=(30, 30)) # set size 
ax = dendrogram(linkage_matrix, orientation="right", labels=fin_term); 
#print(ax) 
#print(leaves) 
plt.tick_params(\ 
    axis= 'x',          # changes apply to the x-axis 
    which='both',      # both major and minor ticks are affected 
    bottom='off',      # ticks along the bottom edge are off 
    top='off',         # ticks along the top edge are off 
    labelbottom='off') 
 
plt.tight_layout() #show plot with tight layout 
 
#uncomment below to save figure 
plt.savefig('kw[enter image description here][1]p.png', dpi=200) 

下面的链接包含形成的树状图的输出

https://www.screencast.com/t/2MEc3ohBe

请您参考如下方法:

你需要事先知道你想要多少个集群;然后你可以使用:

from scipy.cluster.hierarchy import fcluster 
fl = fcluster(cl,numclust,criterion='maxclust') 

其中 cl 是链接方法的输出,numclust 是您想要获得的簇数。