python之如何在循环中使用 python multiprocessing Pool.map

我正在使用 Runge-Kutta 运行模拟。在每个时间步，两个独立变量的两个 FFT 是必要的，可以并行化。我实现了这样的代码:

from multiprocessing import Pool 
import numpy as np 
 
pool = Pool(processes=2)    # I like to calculate only 2 FFTs parallel  
                            # in every time step, therefor 2 processes 
 
def Splitter(args): 
    '''I have to pass 2 arguments''' 
    return makeSomething(*args): 
 
def makeSomething(a,b): 
    '''dummy function instead of the one with the FFT''' 
    return a*b 
 
def RungeK(): 
    # ... 
    # a lot of code which create the vectors A and B and calculates  
    # one Kunge-Kutta step for them  
    # ... 
 
    n = 20                         # Just something for the example 
    A = np.arange(50000) 
    B = np.ones_like(A) 
 
    for i in xrange(n):                  # loop over the time steps 
        A *= np.mean(B)*B - A 
        B *= np.sqrt(A) 
        results = pool.map(Splitter,[(A,3),(B,2)]) 
        A = results[0] 
        B = results[1] 
 
    print np.mean(A)                                 # Some output 
    print np.max(B) 
 
if __name__== '__main__': 
    RungeK() 

不幸的是python在到达循环后生成无限数量的进程。之前好像只有两个进程在运行。我的内存也满了。添加一个

pool.close() 
pool.join() 

循环后面并不能解决我的问题，将它放在循环内部对我来说毫无意义。希望你能帮上忙。

请您参考如下方法:

将池的创建移动到RungeK函数中；

def RungeK(): 
    # ... 
    # a lot of code which create the vectors A and B and calculates 
    # one Kunge-Kutta step for them 
    # ... 
 
    pool = Pool(processes=2) 
    n = 20                         # Just something for the example 
    A = np.arange(50000) 
    B = np.ones_like(A) 
 
    for i in xrange(n):  # loop over the time steps 
        A *= np.mean(B)*B - A 
        B *= np.sqrt(A) 
        results = pool.map(Splitter, [(A, 3), (B, 2)]) 
        A = results[0] 
        B = results[1] 
    pool.close() 
    print np.mean(A)  # Some output 
    print np.max(B) 

或者，将其放在主 block 中。

这可能是多处理工作方式的副作用。例如。在 MS Windows 上，您需要能够无副作用地导入主模块(例如创建新进程)。