Skip to main content
 首页 » 编程设计

python pandas 滚动函数在分组的 DataFrame 中有两个参数

2025年02月15日14pengyingh

这是对我之前问题的某种扩展 python pandas rolling function with two arguments .

如何按组执行相同的操作?假设下面的“C”列用于分组。

我正在努力:

  1. 按“C”列分组
  2. 在每个组中,按“A”排序
  3. 对于每个组,将带有两个参数的滚动函数(如 kendalltau)应用于参数“A”和“B”。

预期的结果将是一个如下所示的 DataFrame:

我一直在尝试上面链接中描述的“传递索引”解决方法,但这种情况的复杂性超出了我的能力范围:-(。这是一个玩具示例,与我正在使用的相差不远,所以为了简单起见,我使用了随机生成的数据。

rand = np.random.RandomState(1) 
dff = pd.DataFrame({'A' : np.arange(20), 
                    'B' : rand.randint(100, 120, 20), 
                    'C' : rand.randint(0, 2, 20)}) 
 
def my_tau_indx(indx): 
    x = dff.iloc[indx, 0] 
    y = dff.iloc[indx, 1] 
    tau = sp.stats.mstats.kendalltau(x, y)[0] 
    return tau 
 
dff['tau'] = dff.sort_values(['C', 'A']).groupby('C').rolling(window = 5).apply(my_tau_indx, args = ([dff.index.values])) 

我所做的每一次修复都会产生另一个错误...

上述问题已由 Nickil Maveli 解决,它适用于 numpy 1.11.0、pandas 0.18.1、scipy 0.17.1 和 conda 4.1.4。它会生成一些警告,但有效。


在我的另一台机器上有最新和最好的 numpy 1.12.0、pandas 0.19.2、scipy 0.18.1、conda 版本 3.10.0 和 BLAS/LAPACK - 它不起作用,我得到下面的回溯。这似乎与版本相关,因为我升级了第一台机器,它也停止工作了……以科学的名义……;-)

正如 Nickil 所建议的,这是由于 numpy 1.11 和 1.12 之间的不兼容。降级 numpy 有帮助。因为我在 Windows 上安装了 BLAS/LAPACK,所以我从 http://www.lfd.uci.edu/~gohlke/pythonlibs/ 安装了 numpy 1.11.3+mkl。 .

Traceback (most recent call last): 
 
File "<ipython-input-4-bbca2c0e986b>", line 16, in <module> 
t = grp.apply(func) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\groupby.py", line 651, in apply 
return self._python_apply_general(f) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\groupby.py", line 655, in _python_apply_general 
self.axis) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\groupby.py", line 1527, in apply 
res = f(group) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\groupby.py", line 647, in f 
return func(g, *args, **kwargs) 
 
File "<ipython-input-4-bbca2c0e986b>", line 15, in <lambda> 
func = lambda x: pd.Series(pd.rolling_apply(np.arange(len(x)), 5, my_tau_indx), x.index) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\stats\moments.py", line 584, in rolling_apply 
kwargs=kwargs) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\stats\moments.py", line 240, in ensure_compat 
result = getattr(r, name)(*args, **kwds) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\window.py", line 863, in apply 
return super(Rolling, self).apply(func, args=args, kwargs=kwargs) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\window.py", line 621, in apply 
center=False) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\window.py", line 560, in _apply 
result = calc(values) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\window.py", line 555, in calc 
return func(x, window, min_periods=self.min_periods) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\window.py", line 618, in f 
kwargs) 
 
File "pandas\algos.pyx", line 1831, in pandas.algos.roll_generic (pandas\algos.c:51768) 
 
File "<ipython-input-4-bbca2c0e986b>", line 8, in my_tau_indx 
x = dff.iloc[indx, 0] 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\indexing.py", line 1294, in __getitem__ 
return self._getitem_tuple(key) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\indexing.py", line 1560, in _getitem_tuple 
retval = getattr(retval, self.name)._getitem_axis(key, axis=axis) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\indexing.py", line 1614, in _getitem_axis 
return self._get_loc(key, axis=axis) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\indexing.py", line 96, in _get_loc 
return self.obj._ixs(key, axis=axis) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\core\frame.py", line 1908, in _ixs 
label = self.index[i] 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\indexes\range.py", line 510, in __getitem__ 
return super_getitem(key) 
 
File "C:\Apps\Anaconda\v2_1_0_x64\envs\python35\lib\site-packages\pandas\indexes\base.py", line 1275, in __getitem__ 
result = getitem(key) 
 
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices 

最后的检查:

请您参考如下方法:

实现的一种方法是遍历每个组并在每个此类组上使用 pd.rolling_apply。

import scipy.stats as ss 
 
def my_tau_indx(indx): 
    x = dff.iloc[indx, 0] 
    y = dff.iloc[indx, 1] 
    tau = ss.mstats.kendalltau(x, y)[0] 
    return tau 
 
grp = dff.sort_values(['A', 'C']).groupby('C', group_keys=False) 
func = lambda x: pd.Series(pd.rolling_apply(np.arange(len(x)), 5, my_tau_indx), x.index) 
t = grp.apply(func) 
dff.reindex(t.index).assign(tau=t) 


编辑:

def my_tau_indx(indx): 
    x = dff.ix[indx, 0] 
    y = dff.ix[indx, 1] 
    tau = ss.mstats.kendalltau(x, y)[0] 
    return tau 
 
grp = dff.sort_values(['A', 'C']).groupby('C', group_keys=False) 
t = grp.rolling(5).apply(my_tau_indx).get('A') 
 
grp.head(dff.shape[0]).reindex(t.index).assign(tau=t)