Skip to main content
 首页 » 编程设计

python之使用 TSNE 的词嵌入可视化不清楚

2025年01月19日10mate10pro

我已经从 Word Embeddings by M. Baroni et al. 下载了预训练的词嵌入模型 我想可视化句子中单词的嵌入。我有两句话:

sentence1 = "Four people died in an accident." 
 
sentence2 = "4 men are dead from a collision" 

我有从上面的链接加载嵌入文件的功能:

def load_data(FileName = './EN-wform.w.5.cbow.neg10.400.subsmpl.txt'): 
 
    embeddings = {} 
    file = open(FileName,'r') 
    i = 0 
    print "Loading word embeddings first time" 
    for line in file: 
        # print line 
 
        tokens = line.split('\t') 
 
        #since each line's last token content '\n' 
        # we need to remove that 
        tokens[-1] = tokens[-1].strip() 
 
        #each line has 400 tokens 
        for i in xrange(1, len(tokens)): 
            tokens[i] = float(tokens[i]) 
 
        embeddings[tokens[0]] = tokens[1:-1] 
    print "finished" 
    return embeddings 
 
e = load_data() 

从这两个句子中,我计算单词的引理并忽略停用词和标点符号,所以现在我的句子变成:

sentence1 = ['Four', 'people', 'died', 'accident'] 
sentence2 = ['4', 'men', 'dead', 'collision'] 

现在,当我尝试使用 TSNE(t 分布随机邻域嵌入)可视化嵌入时,我首先为每个句子存储标签和标记:

#for sentence store labels and embeddings in list 
# tokens contains vector of 400 dimensions for each label 
labels1 = [] 
tokens1 = [] 
for i in sentence1: 
    if i in e: 
        labels1.append(i) 
        tokens1.append(e[i]) 
    else: 
        print i 
 
labels2 = [] 
tokens2 = [] 
for i in sentence2: 
    if i in e: 
        labels2.append(i) 
        tokens2.append(e[i]) 
    else: 
        print i 

对于 TSNE

tsne_model = TSNE(perplexity=40, n_components=2, init='random', n_iter=2000, random_state=23) 
# fit transform for tokens of both sentences 
new_values = tsne_model.fit_transform(tokens1) 
new_values1 = tsne_model.fit_transform(tokens2) 
 
#Plot values 
x = [] 
y = [] 
x1 = [] 
y1 = [] 
 
for value in new_values: 
    x.append(value[0]) 
    y.append(value[1]) 
 
for value in new_values1: 
    x1.append(value[0]) 
    y1.append(value[1]) 
 
 
plt.figure(figsize=(10, 10))  
 
for i in range(len(x)): 
    plt.scatter(x[i],y[i]) 
    plt.annotate(labels[i], 
                 xy=(x[i], y[i]), 
                 xytext=(5, 2), 
                 textcoords='offset points', 
                 ha='right', 
                 va='bottom') 
 
for i in range(len(x1)): 
    plt.scatter(x1[i],y1[i]) 
    plt.annotate(labels[i], 
                 xy=(x1[i], y1[i]), 
                 xytext=(5, 2), 
                 textcoords='offset points', 
                 ha='right', 
                 va='bottom') 
 
plt.show() 

我的问题是,为什么同义词如“collision”和“accident”、“people”和“people”有不同的坐标?如果单词是相同的/同义词,它们不应该更接近吗?

距离 = euclidean_distances(tokens1) # 返回形状 (8,8)

请您参考如下方法:

来自TSNE-documentation :

t-SNE has a cost function that is not convex, i.e. with different initializations we can get different results.

这意味着在对词嵌入进行降维时,不能保证获得相同的坐标。

为了解决这个问题,通过加入你的句子来执行一次而不是两次 fit_transform:

sentence1 = ['Four', 'people', 'died', 'accident'] 
sentence2 = ['4', 'men', 'dead', 'collision'] 
sentences = list(set(sentence1)| set(sentence2)) 

编辑:您的代码中还有一个错误,您正在从错误的列表中绘制标签。