我正在尝试使用 Tensorflow 构建一个生成式 RNN。我有一个预处理数据集，它是 sequence_length x 2048 x 2 numpy 数组的列表。序列具有不同的长度。我一直在查看示例和文档，但我真的无法理解，例如，key 是什么，或者我应该如何创建 input_sequences 字典等。
那么应该如何格式化一个 numpy 数组列表，每个数组代表一个阶数为 n(在本例中为 2)张量的序列，以便能够使用此 batch_sequences_with_states 方法？
请您参考如下方法:
玩具实现
我试过了，我很乐意与您分享我的发现。这是一个玩具示例。我试图创建一个有效的示例并观察输出如何变化。特别是我使用了 lstm 的案例研究。对于您，您可以定义一个转换网络。随意添加更多输入并像往常一样进行调整并按照文档进行操作。 https://www.tensorflow.org/versions/r0.11/api_docs/python/contrib.training/splitting_sequence_inputs_into_minibatches_with_state_saving#batch_sequences_with_states 我尝试了其他更微妙的示例，但我保留了这个简单的版本来展示该操作如何有用。特别是向字典中添加更多元素(输入序列和上下文序列)并观察变化。
两种方法
基本上我会使用两种方法:
- tf.contrib.training.batch_sequences_with_states
- tf.train.batch()
我将从第一个开始，因为它直接有用，然后我将展示如何使用 train.batch 解决类似问题。
我基本上会生成玩具 numpy 数组和张量并将其用于测试操作
import tensorflow as tf
batch_size = 32
num_unroll = 20
num_enqueue_threads = 20
lstm_size = 8
cell = tf.contrib.rnn.BasicLSTMCell(num_units=lstm_size)
#state size
state_size = cell.state_size[0];
initial_state_values = tf.zeros((state_size,), dtype=tf.float32)
# Initial states
initial_state_values = tf.zeros((state_size,), dtype=tf.float32)
initial_states = {"lstm_state": initial_state_values}
# Key should be string
#I used x as input sequence and y as input context. So that the
# keys should be 2.
key = ["1","2"]
#Toy data for our sample
x = tf.range(0, 12, name="x")
y = tf.range(12,24,name="y")
# convert to float
#I converted to float so as not to raise type mismatch erroe
x=tf.to_float(x)
y=tf.to_float(y)
#the input sequence as dictionary
#This is needed according to the tensorflow doc
sequences = {"x": x }
#Context Input
context = {"batch1": y}
# Train batch with sequence state
batch_new = tf.contrib.training.batch_sequences_with_states(
input_key=key,
input_sequences=sequences,
input_context=context,
initial_states=initial_states,
num_unroll=num_unroll,
batch_size=batch_size,
input_length = None,
pad = True,
num_threads=num_enqueue_threads,
capacity=batch_size * num_enqueue_threads * 2)
# To test what we have got type and observe the output of
# the following
# In short once in ipython notebook
# type batch_new.[press tab] to see all options
batch_new.key
batch_new.sequences
#splitting of input. This generate input per epoch
inputs_by_time = tf.split(inputs, num_unroll)
assert len(inputs_by_time) == num_unroll
# Get lstm or conv net output
lstm_output, _ = tf.contrib.rnn.static_state_saving_rnn(
cell,
inputs_by_time,
state_saver=batch_new,
state_name=("lstm_state","lstm_state"))
照常创建图形和队列
带#和*的部分可以根据需要进一步调整。
# Create the graph, etc.
init_op = tf.global_variables_initializer()
#Create a session for running operations in the Graph.
sess = tf.Session()
# Initialize the variables (like the epoch counter).
sess.run(init_op)
# Start input enqueue threads.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess, coord=coord)
# For the part below uncomment
#*those comments with asterics to do other operations
#*try:
#* while not coord.should_stop():
#*Run training steps or whatever
#*sess.run(train_op) # uncomment to run other ops
#*except tf.errors.OutOfRangeError:
#print('Done training -- epoch limit reached')
#*finally:
# When done, ask the threads to stop.
coord.request_stop()
# Wait for threads to finish.
coord.join(threads)
sess.close()
第二种方法
您还可以以一种非常有趣的方式使用 train.batch:
import tensorflow as tf
#[0, 1, 2, 3, 4 ,...]
x = tf.range(0, 11, name="x")
# A queue that outputs 0,1,2,3,..
# slice end is useful for dequeuing
slice_end = 10
# instantiate variable y
y = tf.slice(x, [0], [slice_end], name="y")
# Reshape y
y = tf.reshape(y,[10,1])
y=tf.to_float(y, name='ToFloat')
重要 请注意动态的使用，并使用填充使许多队列排队。随意使用这两个选项。并比较输出!
batched_data = tf.train.batch(
tensors=[y],
batch_size=10,
dynamic_pad=True,
#enqueue_many=True,
name="y_batch"
)
batch_size = 128 ;
lstm_cell = tf.contrib.rnn.LSTMCell(batch_size,forget_bias=1,state_is_tuple=True)
val, state = tf.nn.dynamic_rnn(lstm_cell, batched_data, dtype=tf.float32)
结论
目的是通过简单的例子表明我们可以深入了解 操作的细节。您可以根据您的情况将其调整为卷积网络。
希望这对您有所帮助!
