Skip to main content
 首页 » 编程设计

python之Keras 是否具有复制输入词向量并反向传播到一组的功能

2024年12月31日9over140

我的目标是在 Keras 中创建一个基于文本段落预测分类输出的 RNN-CNN 网络。在我当前的模型中,段落首先嵌入到特征向量中,然后馈送到 2 个 cuDNNGRU 层、4 个 Conv1D 和 MaxPooling 层,然后馈送到密集输出层。

但是,我找到了一个涉及到复制初始向量的多 channel 方法的引用,通过 CNN 层运行一组,然后在池化之前将输出与副本相加。这样做是为了防止反向传播到一组向量中,从而保留原始词向量中的一些语义思想。

我试过搜索这个,但唯一与多 channel 和 CNN 相关的是使用多种大小的 n-gram 内核。 Keras 是否提供可用于实现此目的的任何类型的功能?

请您参考如下方法:

是的,您可以使用函数 API 来实现。

这里是一个小例子,可以随意适应你的需要:

embed_input = Input(shape=(300,)) 
embedded_sequences = Embedding(10000, 10)(embed_input) 
embed=SpatialDropout1D(0.5)(embedded_sequences) 
 
gru=Bidirectional(CuDNNGRU(200, return_sequences = True))(embed) 
 
conv=Conv1D(filters=4, 
            padding = "valid", 
            kernel_size=4, 
            kernel_initializer='he_uniform', 
            activation='relu')(gru) 
 
avg_pool = GlobalAveragePooling1D()(conv) 
max_pool = GlobalMaxPooling1D()(conv) 
gru_pool = GlobalAveragePooling1D()(gru) 
 
 
l_merge = concatenate([avg_pool, max_pool, gru_pool]) 
 
 
 
output = Dense(6, activation='sigmoid')(l_merge) 
model = Model(embed_input, output) 
 
 
model.summary() 
 
 
output: 
 
__________________________________________________________________________________________________ 
Layer (type)                    Output Shape         Param #     Connected to                      
================================================================================================== 
input_10 (InputLayer)           (None, 300)          0                                             
__________________________________________________________________________________________________ 
embedding_9 (Embedding)         (None, 300, 10)      100000      input_10[0][0]                    
__________________________________________________________________________________________________ 
spatial_dropout1d_9 (SpatialDro (None, 300, 10)      0           embedding_9[0][0]                 
__________________________________________________________________________________________________ 
bidirectional_8 (Bidirectional) (None, 300, 400)     254400      spatial_dropout1d_9[0][0]         
__________________________________________________________________________________________________ 
conv1d_6 (Conv1D)               (None, 297, 4)       6404        bidirectional_8[0][0]             
__________________________________________________________________________________________________ 
global_average_pooling1d_6 (Glo (None, 4)            0           conv1d_6[0][0]                    
__________________________________________________________________________________________________ 
global_max_pooling1d_6 (GlobalM (None, 4)            0           conv1d_6[0][0]                    
__________________________________________________________________________________________________ 
global_average_pooling1d_7 (Glo (None, 400)          0           bidirectional_8[0][0]             
__________________________________________________________________________________________________ 
concatenate_5 (Concatenate)     (None, 408)          0           global_average_pooling1d_6[0][0]  
                                                                 global_max_pooling1d_6[0][0]      
                                                                 global_average_pooling1d_7[0][0]  
__________________________________________________________________________________________________ 
dense_5 (Dense)                 (None, 6)            2454        concatenate_5[0][0]               
================================================================================================== 
Total params: 363,258 
Trainable params: 363,258 
Non-trainable params: 0 

和图的结构: