Skip to main content
 首页 » 编程设计

python之将 numpy 数组中的连续值及其长度分组

2025年04月02日9EasonJim

在 numpy/scipy(或纯 python,如果您愿意)中,将连续区域分组到 numpy 数组中并计算这些区域的长度的好方法是什么?

像这样:

x = np.array([1,1,1,2,2,3,0,0,0,0,0,1,2,3,1,1,0,0,0]) 
y = contiguousGroup(x) 
print y 
 
>> [[1,3], [2,2], [3,1], [0,5], [1,1], [2,1], [3,1], [1,2], [0,3]] 

我曾尝试仅使用循环来执行此操作,但它需要比我想要的更长的时间(6 秒)来创建一个包含大约 3000 万个样本和 20000 个连续区域的列表。

编辑:

现在进行一些速度比较(仅使用 time.clock() 和几百次迭代,如果以秒为单位则更少)。

首先,我的 python 循环代码在 5 个样本上进行了测试。

Number of elements  33718251 
Number of regions  135137 
Time taken = 8.644007 seconds... 
 
Number of elements  42503100 
Number of regions  6985 
Time taken = 10.533305 seconds... 
 
Number of elements  21841302 
Number of regions  7619335 
Time taken = 7.671015 seconds... 
 
Number of elements  19723928 
Number of regions  10799 
Time taken = 5.014807 seconds... 
 
Number of elements  16619539 
Number of regions  19293 
Time taken = 4.207359 seconds... 

现在有了 Divakar 的矢量化解决方案。

Number of elements  33718251 
Number of regions  135137 
Time taken = 0.063470 seconds... 
 
Number of elements  42503100 
Number of regions  6985 
Time taken = 0.046293 seconds... 
 
Number of elements  21841302 
Number of regions  7619335 
Time taken = 1.654288 seconds... 
 
Number of elements  19723928 
Number of regions  10799 
Time taken = 0.022651 seconds... 
 
Number of elements  16619539 
Number of regions  19293 
Time taken = 0.021189 seconds... 

修改后的方法给出大致相同的时间(最坏情况下可能慢 5%)

现在使用 Kasramvd 的生成器方法。

Number of elements  33718251 
Number of regions  135137 
Time taken = 3.834922 seconds... 
 
Number of elements  42503100 
Number of regions  6985 
Time taken = 4.785480 seconds... 
 
Number of elements  21841302 
Number of regions  7619335 
Time taken = 6.806867 seconds... 
 
Number of elements  19723928 
Number of regions  10799 
Time taken = 2.264413 seconds... 
 
Number of elements  16619539 
Number of regions  19293 
Time taken = 1.778873 seconds... 

现在是他的 numpythonic 版本。

Number of elements  33718251 
Number of regions  135137 
Time taken = 0.286336 seconds... 
 
Number of elements  42503100 
Number of regions  6985 
Time taken = 0.174769 seconds... 
 
Memory error sample 3 (too many regions) 
 
Number of elements  19723928 
Number of regions  10799 
Time taken = 0.087028 seconds... 
 
Number of elements  16619539 
Number of regions  19293 
Time taken = 0.084963 seconds... 

无论如何,我认为这个故事的寓意是 numpy 非常好。

请您参考如下方法:

这是一个向量化的方法-

idx = np.concatenate(([0],np.flatnonzero(x[:-1]!=x[1:])+1,[x.size])) 
out = zip(x[idx[:-1]],np.diff(idx)) 

sample 运行-

In [34]: x 
Out[34]: array([1, 1, 1, 2, 2, 3, 0, 0, 0, 0, 0, 1, 2, 3, 1, 1, 0, 0, 0]) 
 
In [35]: out 
Out[35]: [(1, 3), (2, 2), (3, 1), (0, 5), (1, 1), (2, 1), (3, 1), (1, 2), (0, 3)] 

整个数组的串联可能很昂贵。因此,可以建议对组移动索引进行串联的修改版本,就像这样 -

idx0 = np.flatnonzero(x[:-1]!=x[1:]) 
count = np.concatenate(([idx0[0]+1],np.diff(idx0),[x.size-idx0[-1]-1])) 
out = zip(x[np.append(0,idx0+1)],count) 

或者,在最后一步,如果作为 2D 数组的输出没问题,我们可以避免zipping 并使用 NumPy 的 column_stack,就像这样 -

out = np.column_stack((x[np.append(0,idx0+1)],count))