Skip to main content
 首页 » 编程设计

python之Pandas之从现有数据创建二进制数据

2024年12月31日14del

我正在尝试从现有数据帧创建二进制数据,但需要很长时间才能完成。有没有更快的方法来完成这个?

我现在拥有的是一个包含多行的数据框,比如 df 例如:

Index   Actions Tries   Ratio 
0       20      200     0,1 
1       10      400     0,025 
2       15      500     0,03 
3       30      700     0,04 

I now want to transform this data to binary data, say df_binary e.g.

Index_old   Index_new   Actions Tries   Ratio   Success 
0           0           20      200     0,1     1 
0           1           20      200     0,1     1 
0           2           20      200     0,1     1 
0           3           20      200     0,1     1 
...      
0           19          20      200     0,1     1  -> 20 times success(1)    
0           20          20      200     0,1     0 
0           21          20      200     0,1     0 
0           22          20      200     0,1     0 
...                  
0           199         20      200     0,1     0  -> 200-20= 180 times fail(0) 
1           200         10      400     0,025   1 
1           201         10      400     0,025   1 
1           202         10      400     0,025   1 

As can be seen from the above example, the Actions / Tries = Ratio. The amount of times this should be replicated is based on Tries, the amount of times succes = 1, is based on Actions. The amount of times success = 0 is based on Tries - Actions.

import pandas as pd 
#create the new DataFrame 
df_binary = pd.DataFrame() 
#iterate over all rows in the original DataFrame (df) 
for index,row in df.iterrows(): 
    #get the number of tries from the row in the df 
    tries = row['Tries'] 
    #get the number of actions from the row in the df 
    actions = row['Actions'] 
    #calculate the number of times the tries did not result in action 
    noActions = tries - actions 
    #create a temporary df used for appending 
    tempDf = row 
 
    #loop for the range given by tries (row['Tries']) e.g. loop 200 times       
    for try in range(tries):   
        if try < actions: 
            #if the number of actions is lower than tries, set success to 1. E.g. try 1 < 20, set success, try 15 < 20, set success 
            tempDf['Success'] = 1 
            #append new data to df_binary 
            df_binary = df_binary.append(tempDf, ignore_index=True) 
        else: 
            #else set success to failure, e.g. try 25 > 20 set failure, try 180 > 20 set failure. 
            tempDf['Success'] = 0 
            #append new data to df_binary 
            df_binary = df_binary.append(tempDf, ignore_index=True) 

在这个例子中,完成的时间不会那么长。但是我实际的新 df_binary 在完成后应该包含大约 1500 万行并且包含更多的列,这需要很长时间才能完成。

有什么方法可以更快地做到这一点?

谢谢!

请您参考如下方法:

这是实现此目的的一种潜在方法,在列表理解中使用 pandas.concat、Series.repeat 和 DataFrame.assign:

successes = np.concatenate([[1]*a + [0]*(t-a) for a, t in zip(df['Actions'], df['Tries'])]) 
 
df_binary = (pd.concat([df[s].repeat(df['Tries']) for s in df], axis=1) 
             .assign(success=successes).reset_index())