在 numpy/scipy(或纯 python,如果您愿意)中,将连续区域分组到 numpy 数组中并计算这些区域的长度的好方法是什么?
像这样:
x = np.array([1,1,1,2,2,3,0,0,0,0,0,1,2,3,1,1,0,0,0])
y = contiguousGroup(x)
print y
>> [[1,3], [2,2], [3,1], [0,5], [1,1], [2,1], [3,1], [1,2], [0,3]]
我曾尝试仅使用循环来执行此操作,但它需要比我想要的更长的时间(6 秒)来创建一个包含大约 3000 万个样本和 20000 个连续区域的列表。
编辑:
现在进行一些速度比较(仅使用 time.clock() 和几百次迭代,如果以秒为单位则更少)。
首先,我的 python 循环代码在 5 个样本上进行了测试。
Number of elements 33718251
Number of regions 135137
Time taken = 8.644007 seconds...
Number of elements 42503100
Number of regions 6985
Time taken = 10.533305 seconds...
Number of elements 21841302
Number of regions 7619335
Time taken = 7.671015 seconds...
Number of elements 19723928
Number of regions 10799
Time taken = 5.014807 seconds...
Number of elements 16619539
Number of regions 19293
Time taken = 4.207359 seconds...
现在有了 Divakar 的矢量化解决方案。
Number of elements 33718251
Number of regions 135137
Time taken = 0.063470 seconds...
Number of elements 42503100
Number of regions 6985
Time taken = 0.046293 seconds...
Number of elements 21841302
Number of regions 7619335
Time taken = 1.654288 seconds...
Number of elements 19723928
Number of regions 10799
Time taken = 0.022651 seconds...
Number of elements 16619539
Number of regions 19293
Time taken = 0.021189 seconds...
修改后的方法给出大致相同的时间(最坏情况下可能慢 5%)
现在使用 Kasramvd 的生成器方法。
Number of elements 33718251
Number of regions 135137
Time taken = 3.834922 seconds...
Number of elements 42503100
Number of regions 6985
Time taken = 4.785480 seconds...
Number of elements 21841302
Number of regions 7619335
Time taken = 6.806867 seconds...
Number of elements 19723928
Number of regions 10799
Time taken = 2.264413 seconds...
Number of elements 16619539
Number of regions 19293
Time taken = 1.778873 seconds...
现在是他的 numpythonic 版本。
Number of elements 33718251
Number of regions 135137
Time taken = 0.286336 seconds...
Number of elements 42503100
Number of regions 6985
Time taken = 0.174769 seconds...
Memory error sample 3 (too many regions)
Number of elements 19723928
Number of regions 10799
Time taken = 0.087028 seconds...
Number of elements 16619539
Number of regions 19293
Time taken = 0.084963 seconds...
无论如何,我认为这个故事的寓意是 numpy 非常好。
请您参考如下方法:
这是一个向量化的方法-
idx = np.concatenate(([0],np.flatnonzero(x[:-1]!=x[1:])+1,[x.size]))
out = zip(x[idx[:-1]],np.diff(idx))
sample 运行-
In [34]: x
Out[34]: array([1, 1, 1, 2, 2, 3, 0, 0, 0, 0, 0, 1, 2, 3, 1, 1, 0, 0, 0])
In [35]: out
Out[35]: [(1, 3), (2, 2), (3, 1), (0, 5), (1, 1), (2, 1), (3, 1), (1, 2), (0, 3)]
整个数组的串联可能很昂贵。因此,可以建议对组移动索引进行串联的修改版本,就像这样 -
idx0 = np.flatnonzero(x[:-1]!=x[1:])
count = np.concatenate(([idx0[0]+1],np.diff(idx0),[x.size-idx0[-1]-1]))
out = zip(x[np.append(0,idx0+1)],count)
或者,在最后一步,如果作为 2D 数组的输出没问题,我们可以避免zipping 并使用 NumPy 的 column_stack,就像这样 -
out = np.column_stack((x[np.append(0,idx0+1)],count))
