介绍

我随机创建形状为 (1000,10) 的训练数据 X。对于标签 Y，它始终等于 X 特征的第一个元素。

例如。假设 x1 = [0.1,0.2,0.3...,0.9]，则 y = 0.1。使用以下代码创建的数据集:

from numpy.random import RandomState 
rdm=RandomState(1) 
data_size=10000 
xdim=10 
X=rdm.rand(data_size,xdim) 
Y = [x1[0] for x1 in X] 

我试图创建一个只有一个节点的一层神经网络来学习这个映射，我认为期望的权重应该是[1,0,0,0,0, 0,0,0,0,0] 并且 biases 应该是 0 以仅提取 x 目的的第一个元素。

tensorflow

这是我在tensorflow中实现的代码。训练不收敛。

import tensorflow as tf 
x=tf.placeholder(tf.float64,shape=(None,xdim)) 
y=tf.placeholder(tf.float64,shape=(None)) 
 
# for simple reason, using zero to initialize both weights and biases 
Weights = tf.Variable(tf.zeros([xdim, 1],dtype=tf.float64)) 
biases = tf.Variable(tf.zeros([1],dtype=tf.float64)) 
y_predict = tf.matmul(x, Weights)+biases 
loss = tf.losses.mean_squared_error(y_predict,y)   
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss) 
 
batch_size=100 
with tf.Session() as sess: 
    sess.run(tf.global_variables_initializer()) 
    for i in range(10001): 
        start = i * batch_size % data_size 
        end = min(start + batch_size,data_size) 
        sess.run(optimizer,feed_dict={x:X[start:end],y:Y[start:end]}) 
        if i % 1000 == 0: 
            ypred,training_loss= sess.run([y_predict,loss],feed_dict={x:X,y:Y}) 
            print("Epoch %d: loss=%g"%(i,training_loss)) 
    print('Weights:\n',sess.run(Weights)) 
    print('biases:\n',sess.run(biases)) 

输出是:

Epoch 0: loss=0.299163 
Epoch 1000: loss=0.0838915 
Epoch 2000: loss=0.0829176 
Epoch 3000: loss=0.0825273 
Epoch 4000: loss=0.08237 
Epoch 5000: loss=0.0823084 
Epoch 6000: loss=0.0822847 
Epoch 7000: loss=0.0822745 
Epoch 8000: loss=0.0822701 
Epoch 9000: loss=0.082268 
Epoch 10000: loss=0.0822669 
Weights: 
 [[ 0.01159591] 
 [ 0.0003244 ] 
 [ 0.00319655] 
 [ 0.00113588] 
 [-0.00079908] 
 [-0.00086694] 
 [ 0.00020551] 
 [-0.00243378] 
 [-0.00260724] 
 [ 0.00052958]] 
biases: 
 [ 0.48771921] 

凯拉斯

import keras 
from keras.models import Sequential 
from keras.layers import Dense,Input 
import numpy as np 
 
model = Sequential() 
model.add(Dense(units=1,input_dim=xdim,kernel_initializer='zeros',bias_initializer='zeros'))  
model.compile(loss='mse', optimizer=keras.optimizers.SGD(lr=0.01)) 
 
 
batch_size=100 
for i in range(10001): 
    start = i * batch_size % data_size 
    end = min(start + batch_size,data_size) 
    cost = model.train_on_batch(X[start:end], np.array(Y[start:end])) 
    if i % 1000 == 0: 
        print("Epoch %d: loss=%g"%(i,cost)) 
print('Weights:\n',model.get_weights()[0]) 
print('biases:\n',model.get_weights()[1]) 

输出:

Using TensorFlow backend. 
Epoch 0: loss=0.284947 
Epoch 1000: loss=0.00321839 
Epoch 2000: loss=0.000247763 
Epoch 3000: loss=5.40826e-05 
Epoch 4000: loss=1.90453e-05 
Epoch 5000: loss=7.40253e-06 
Epoch 6000: loss=2.93623e-06 
Epoch 7000: loss=1.17069e-06 
Epoch 8000: loss=4.67434e-07 
Epoch 9000: loss=1.86726e-07 
Epoch 10000: loss=7.45764e-08 
Weights: 
 [[  9.99678493e-01] 
 [ -3.00021959e-04] 
 [ -2.89586897e-04] 
 [ -2.90223019e-04] 
 [ -2.83820234e-04] 
 [ -2.82248948e-04] 
 [ -2.96013983e-04] 
 [ -3.13797180e-04] 
 [ -3.20409046e-04] 
 [ -3.11669020e-04]] 
biases: 
 [ 0.00153964] 

问题

Keras 好像可以得到正确的结果。但我使用了相同的过程，包括权重和偏差的初始化、损失函数和具有相同学习率的优化器。我不明白为什么会这样，我的代码中是否有任何问题/错误？

请您参考如下方法:

你应该交换 tf.losses.mean_squared_error 的参数在 TensorFlow 实现中:

loss = tf.losses.mean_squared_error(y, y_predict)  

此外，y和y_predict的shape分别是(batch_size,)和(batch_size, 1)，分别。您应该在指定损失函数之前挤压 y_predict，以避免不需要的隐式广播:

y_predict = tf.matmul(x, Weights)+biases 
y_predict = tf.squeeze(y_predict) 
loss = tf.losses.mean_squared_error(y,y_predict)