Skip to main content
 首页 » 编程设计

python之如何在 pandas 中调用函数

2024年11月24日8EasonJim

我在 Pandas 中有以下数据框

code     job_descr               job_type      
123      sales executive         nan 
124      data scientist          nan 
145      marketing manager       nan 
132      finance                 nan 
144      data analyst            nan 

我想将job_descr分类为job_type如下

sales : Sales 
marketing : Marketing 
finance : Finance 
data science : Analytics 
analyst : Analytics 

我正在用 pandas 进行跟踪

def job_type_redifine(column_name): 
   if column_name.str.contains('sales'): 
       return 'Sales' 
   elif column_name.str.contains('marketing'): 
       return 'Marketing' 
   elif column_name.str.contains('data science|data scientist|analyst|machine learning'): 
    return 'Analytics' 
   else: 
       return 'Others' 
 
 
final_df['job_type'] = final_df.apply(lambda row:  
                       job_type_redifine(row['job_descr']), axis=1) 

所需的数据框

code     job_descr               job_type      
123      sales executive         Sales 
124      data scientist          Analytics 
145      marketing manager       Marketing 
132      finance                 Finance 
144      data analyst            Analytics 

请您参考如下方法:

第一个解决方案是 numpy.select和 Series.str.contains , advatage 正在处理缺失值,但速度较慢:

m1 = final_df['job_descr'].str.contains('sales') 
m2 = final_df['job_descr'].str.contains('marketing') 
m3 = final_df['job_descr'].str.contains('data science|data scientist|analyst|machine learning') 
 
final_df['job_type'] = np.select([m1, m2, m3],  
                                 ['Sales','Marketing','Analytics'], default='Others') 
 
print (final_df) 
   code          job_descr   job_type 
0   123    sales executive      Sales 
1   124     data scientist  Analytics 
2   145  marketing manager  Marketing 
3   132            finance     Others 
4   144       data analyst  Analytics 

解决方案 Series.apply - 测试匹配值是使用 in,这里是按每个值循环,但它更快,因为 pandas 文本函数很慢。缺点是有很多 or 的最后一个条件有点复杂:

def job_type_redifine(column_name): 
   if 'sales' in column_name: 
       return 'Sales' 
   elif 'marketing' in column_name: 
       return 'Marketing' 
   elif  ('data science' in column_name or 'data scientist' in column_name  
         or 'analyst' in column_name or 'machine learning' in column_name): 
      return 'Analytics' 
   else: 
       return 'Others' 
 
 
final_df['job_type'] =  final_df['job_descr'].apply(job_type_redifine) 
print (final_df) 
   code          job_descr   job_type 
0   123    sales executive      Sales 
1   124     data scientist  Analytics 
2   145  marketing manager  Marketing 
3   132            finance     Others 
4   144       data analyst  Analytics 

性能:

#[5000 rows x 3 columns] 
final_df = pd.concat([final_df] * 1000, ignore_index=True) 
 
In [13]: %%timeit 
    ...: m1 = final_df['job_descr'].str.contains('sales') 
    ...: m2 = final_df['job_descr'].str.contains('marketing') 
    ...: m3 = final_df['job_descr'].str.contains('data science|data scientist|analyst|machine learning') 
    ...:  
    ...: final_df['job_type'] = np.select([m1, m2, m3], ['Sales','Marketing','Analytics'], default='Others') 
    ...:  
12.1 ms ± 611 µs per loop (mean ± std. dev. of 7 runs, 100 loops each) 
 
In [14]: %%timeit  
    ...: final_df['job_type1'] =  final_df['job_descr'].apply(job_type_redifine) 
    ...:  
1.95 ms ± 57.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)