Skip to main content
 首页 » 编程设计

python之Cython Numpy 数组操作比 Python 慢

2024年12月31日10freeliver54

我想用 Cython 优化这段 Python 代码:

def updated_centers(point, start, center): 
    return np.array([__cluster_mean(point[start[c]:start[c + 1]], center[c]) for c in range(center.shape[0])]) 
 
def __cluster_mean(point, center): 
    return (np.sum(point, axis=0) + center) / (point.shape[0] + 1) 

我的 Cython 代码:

cimport cython 
cimport numpy as np 
import numpy as np 
 
# C-compatible Numpy integer type.                                                                                         
DTYPE = np.intc 
 
@cython.boundscheck(False)  # Deactivate bounds checking                                                                   
@cython.wraparound(False)   # Deactivate negative indexing.                                                                
@cython.cdivision(True)     # Deactivate division by 0 checking.                                                           
def updated_centers(double [:,:] point, int [:] label, double [:,:] center): 
    if (point.shape[0] != label.size) or (point.shape[1] != center.shape[1]) or (center.shape[0] > point.shape[0]): 
    raise ValueError("Incompatible dimensions") 
 
    cdef Py_ssize_t i, c, j 
    cdef Py_ssize_t n = point.shape[0] 
    cdef Py_ssize_t m = point.shape[1] 
    cdef Py_ssize_t nc = center.shape[0] 
 
    # Updated centers. We accumulate point and center contributions into this array.                                       
    # Start by adding the (unscaled) center contributions.                                                                 
    new_center = np.zeros([nc, m]) 
    new_center[:] = center 
 
    # Counter array. Will contain cluster sizes (including center, whose contribution                                      
    # is again added here) at the end of the point loop.                                                                   
    cluster_size = np.ones([nc], dtype=DTYPE) 
 
    # Add point contributions.                                                                                             
    for i in range(n): 
        c = label[i] 
        cluster_size[c] += 1 
        for j in range(m): 
            new_center[c, j] += point[i, j] 
 
    # Scale center+point summation to be a mean.                                                                           
    for c in range(nc): 
        for j in range(m): 
        new_center[c, j] /= cluster_size[c] 
 
    return new_center 

但是,Cython 比 python 慢:

Python: %timeit f.updated_centers(point, start, center) 
331 ms ± 11.9 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 
Cython: %timeit fx.updated_centers(point, label, center) 
433 ms ± 14 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 

HTML 显示几乎所有行都是黄色的:分配数组、+=、/=。我预计 Cython 会快一个数量级。我做错了什么?

请您参考如下方法:

您需要告诉 Cython new_center 和 cluster_size 是数组:

cdef double[:, :] new_center = np.zeros((nc, m)) 
... 
cdef int[:] cluster_size = np.ones((nc,), dtype=DTYPE) 
... 

没有这些类型注释,Cython 无法生成高效的 C 代码,并且在访问这些数组时必须调用 Python 解释器。这就是为什么 cython -a 的 HTML 输出中的行访问这些数组是黄色的。

仅通过这两个小修改,我们立即看到了我们想要的加速:

%timeit python_updated_centers(point, start, center) 
392 ms ± 41.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each) 
 
%timeit cython_updated_centers(point, start, center) 
1.18 ms ± 145 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)