我正在按照 Tom Hope 的书来学习 Tensorflow,并且我找到了这个逻辑回归示例:
import tensorflow as tf
import numpy as np
N = 20000
def sigmoid(x):
return 1 / (1 + np.exp(-x))
# === Create data and simulate results =====
x_data = np.random.randn(N,3)
w_real = [0.3,0.5,0.1]
b_real = -0.2
wxb = np.matmul(w_real,x_data.T) + b_real
y_data_pre_noise = sigmoid(wxb)
y_data = np.random.binomial(1,y_data_pre_noise)
NUM_STEPS = 50
g1 = tf.Graph()
wb_ = []
with g1.as_default():
x = tf.placeholder(tf.float32,shape=[None,3])
y_true = tf.placeholder(tf.float32,shape=None)
with tf.name_scope('inference') as scope:
w = tf.Variable([[0,0,0]],dtype=tf.float32,name='weights')
b = tf.Variable(0,dtype=tf.float32,name='bias')
y_pred = tf.matmul(w,tf.transpose(x)) + b
with tf.name_scope('loss') as scope:
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels=y_true,logits=y_pred))
with tf.name_scope('train') as scope:
learning_rate = 0.5
optimizer = tf.train.GradientDescentOptimizer(learning_rate)
train = optimizer.minimize(loss)
# Before starting, initialize the variables. We will 'run' this first.
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
for step in range(NUM_STEPS):
sess.run(train,{x: x_data, y_true: y_data})
if (step % 5 == 0):
print(step, sess.run([w,b]))
wb_.append(sess.run([w,b]))
print(50, sess.run([w,b]))
例子按照书中的描述运行,但是,有一点我不明白,为什么作者没有使用tf.sigmoid()进行推理?
with tf.name_scope('inference') as scope:
w = tf.Variable([[0,0,0]],dtype=tf.float32,name='weights')
b = tf.Variable(0,dtype=tf.float32,name='bias')
***y_pred = tf.sigmoid(tf.matmul(w,tf.transpose(x)) + b)***
有什么明显的事情被我忽略了吗?此外,无论是否进行上述修改,结果看起来都非常相似。
请您参考如下方法:
因为损失函数 tf.nn.sigmoid_cross_entropy_with_logits 正在应用 sigmoid itself .在训练中执行两次会使预测偏向中心:网络将无法输出接近 0 或 1 的值。 This question是这个网络的一个例子。
如果有人对概率推理感兴趣,他们可以添加一个单独的操作,例如:
y_proba = tf.sigmoid(y_pred)
...但仍然在训练中使用原始y_pred(在测试时间中使用y_proba)。
