python之使用 numpy 或 pandas 处理长格式的 csv 文件

我正在尝试编写一个简单的脚本，将 csv 输出文件从 Fortran 代码转换为 Pandas DataFrame 对象，以便我可以进行更多分析。 csv 有两列，但由多个附加的数据 block 组成，形状为 [n,2](每个样本名称的格式为 RN_x)。我得到了以下代码，但生成的 DataFrame 对象不允许分析。我还在下面附上了一个示例文件(比原始文件短得多)。顺便说一下，数据文件中的第一列是一个日期，但在输出中是一个数字，对应于 si=imulation 中的一天。任何建议将不胜感激。

import numpy as np 
import pandas as pd 
import csv as csv 
readdata = csv.reader(open('C:/data/Test.csv', 'r')) 
data = [] 
for row in readdata: 
    data.append(row) 
a = np.array(data).reshape(11,-1, order = 'F') 
col = a[0,:4].reshape(4) 
row = pd.Index(a[4:,0:1].reshape(7)) 
b = a[4:,5:] 
df = pd.DataFrame(b, index = row, columns = col) 

示例:

RN_48865, 
1,Observed 
1,0 
259,Computed 
1,0.000014 
91,0.000014 
182,0.000014 
274,0.000014 
366,0.000014 
457,0.000014 
548,0.000014 
RN_7445, 
1,Observed 
1,0 
259,Computed 
1,0.000013 
91,0.000013 
182,0.000013 
274,0.000013 
366,0.000013 
457,0.000013 
548,0.000013 
RN_9288, 
1,Observed 
1,0 
259,Computed 
1,0.000011 
91,0.000011 
182,0.000011 
274,0.000011 
366,0.000011 
457,0.000011 
548,0.000011 
RN_10955, 
1,Observed 
1,0 
259,Computed 
1,0.000014 
91,0.000014 
182,0.000014 
274,0.000014 
366,0.000014 
457,0.000014 
548,0.000014 

示例输出:

Index,RN_48865,RN_7445,RN_9288,RN_10955 
1,0.000014,0.000013,0.000011,0.000014 
91,0.000014,0.000013,0.000011,0.000014 
182,0.000014,0.000013,0.000011,0.000014 
274,0.000014,0.000013,0.000011,0.000014 
366,0.000014,0.000013,0.000011,0.000014 
457,0.000014,0.000013,0.000011,0.000014 
548,0.000014,0.000013,0.000011,0.000014 

请您参考如下方法:

您实际上是在问几个问题。这是我从所需输出中可以理解的内容:

source="""RN_48865, 
    1,Observed 
    1,0 
    259,Computed 
    1,0.000014 
    91,0.000014 
    182,0.000014 
    274,0.000014 
    366,0.000014 
    457,0.000014 
    548,0.000014 
    RN_7445, 
    1,Observed 
    1,0 
    259,Computed 
    1,0.000013 
    91,0.000013 
    182,0.000013 
    274,0.000013 
    366,0.000013 
    457,0.000013 
    548,0.000013 
    RN_9288, 
    1,Observed 
    1,0 
    259,Computed 
    1,0.000011 
    91,0.000011 
    182,0.000011 
    274,0.000011 
    366,0.000011 
    457,0.000011 
    548,0.000011 
    RN_10955, 
    1,Observed 
    1,0 
    259,Computed 
    1,0.000014 
    91,0.000014 
    182,0.000014 
    274,0.000014 
    366,0.000014 
    457,0.000014 
    548,0.000014 
""" 
import pandas as pd 
import numpy as np 
import StringIO 
df = pd.read_csv(StringIO.StringIO(source), header=None) 
rns = np.where(df[0].apply(lambda x: x.lstrip().startswith('RN_')))[0] 
length = rns[1] - rns[0] 
index = df[0].iloc[4:length] 
cols = df[0][::length].apply(lambda x: x.lstrip()).values 
result_df = pd.DataFrame(index=index) 
for col_num, col_start in enumerate(range(0, len(df), length)): 
    result_df[cols[col_num]] = df[1][col_num*length+4 : (col_num+1)*length].values 
print result_df 

输出:

     RN_48865   RN_7445   RN_9288  RN_10955 
1    0.000014  0.000013  0.000011  0.000014 
91   0.000014  0.000013  0.000011  0.000014 
182  0.000014  0.000013  0.000011  0.000014 
274  0.000014  0.000013  0.000011  0.000014 
366  0.000014  0.000013  0.000011  0.000014 
457  0.000014  0.000013  0.000011  0.000014 
548  0.000014  0.000013  0.000011  0.000014 

对于日期使用:

pandas.read_csv('file', 
  parse_date=0,  # 0th column 
  date_parser=lambda x: pandas.Timestamp('1995-1-1')+timedelta(x))