python之Tensorflow Dataset API 读取 csv 转换 tfrecords

使用 Tensorflow 1.3 的 Dataset API 读取 tfrecords 文件时出现错误

2017-09-03 21:33:53.751096: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Name: <unknown>, Key: features, Index: 0.  Number of float values != expected.  Values size: 14 but output shape: [] 
2017-09-03 21:33:53.751173: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Name: <unknown>, Key: features, Index: 0.  Number of float values != expected.  Values size: 14 but output shape: [] 
 [[Node: ParseSingleExample/ParseExample/ParseExample = ParseExample[Ndense=2, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT], dense_shapes=[[], []], sparse_types=[]](ParseSingleExample/ExpandDims, ParseSingleExample/ParseExample/ParseExample/names, ParseSingleExample/ParseExample/ParseExample/dense_keys_0, ParseSingleExample/ParseExample/ParseExample/dense_keys_1, ParseSingleExample/ParseExample/Reshape, ParseSingleExample/ParseExample/Reshape_1)]] 
Traceback (most recent call last): 
 File "/home/fan/PycharmProjects/DeepStockMarket/t.py", line 32, in <module> 
 print(sess.run(label)) 
 File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 889, in run 
run_metadata_ptr) 
 File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1118, in _run 
 feed_dict_tensor, options, run_metadata) 
 File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1315, in _do_run 
options, run_metadata) 
 File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call 
 raise type(e)(node_def, op, message) 
 tensorflow.python.framework.errors_impl.InvalidArgumentError: Name: <unknown>, Key: features, Index: 0.  Number of float values != expected.  Values size: 14 but output shape: [] 
 [[Node: ParseSingleExample/ParseExample/ParseExample = ParseExample[Ndense=2, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT], dense_shapes=[[], []], sparse_types=[]](ParseSingleExample/ExpandDims, ParseSingleExample/ParseExample/ParseExample/names, ParseSingleExample/ParseExample/ParseExample/dense_keys_0, ParseSingleExample/ParseExample/ParseExample/dense_keys_1, ParseSingleExample/ParseExample/Reshape, ParseSingleExample/ParseExample/Reshape_1)]] 
 [[Node: IteratorGetNext = IteratorGetNext[output_shapes=[[?], [?]], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](OneShotIterator)]] 

tfrecords 文件是使用以下代码从 CSV 文件转换而来的

csv = pd.read_csv("./source/000001/1991/BASIC_000001_1991.csv").values 
with tf.python_io.TFRecordWriter("csv.tfrecords") as writer: 
    for row in csv: 
        features, label = row[1:-1], row[-1] 
        example = tf.train.Example() 
        example.features.feature["features"].float_list.value.extend(features) 
        example.features.feature["label"].float_list.value.append(label) 

我正在使用下面的代码来阅读它

def _p_fn(proto): 
    f = { 
        "features": tf.FixedLenFeature([], tf.float32, default_value=0.0), 
        "label": tf.FixedLenFeature([], tf.float32, default_value=0.0) 
    } 
    parsed_features = tf.parse_single_example(proto, f) 
    features = parsed_features["features"] 
    label = parsed_features["label"] 
    return features, label 
 
f = ["csv.tfrecords"] 
dataset = tf.contrib.data.TFRecordDataset(f) 
dataset = dataset.map(_p_fn) 
dataset = dataset.batch(5) 
iterator = dataset.make_one_shot_iterator() 
features, label = iterator.get_next() 
 
sess = tf.Session() 
print(sess.run(label)) 

有谁知道哪里出了问题吗？非常感谢

请您参考如下方法:

由于您正在使用 FixedLenFeature 和您的 len(feature)>1 您应该清楚地指定形状

 # from the error msg youe feature len = 14 
 f = { 
    "features": tf.FixedLenFeature([14], tf.float32, default_value=tf.zeros([14])), 
    "label": tf.FixedLenFeature([], tf.float32, default_value=0.0) 
 }