Skip to main content
 首页 » 编程设计

python之multiprocessing.Pool,不会永久影响类变量

2025年05月04日12小虾米

我正在使用 multiprocessing.Pool 将一个方法的工作分配到多个处理器上。当我向字典中添加某些内容时,它会在执行该方法后丢失。这是为什么?以及如何规避它?

from multiprocessing import Pool 
 
 
class Agent: 
    def __init__(self): 
        self.test_dict = {} 
 
    def apply(self, num): 
        # something very processor intensive here 
        self.test_dict[num] = num 
        print 'inside ', self.test_dict 
 
def F(x): 
    agent, i = x 
    return agent.apply(i) 
 
class SeriesInstance(object): 
    def __init__(self): 
        self.agent = Agent() 
        self.F = F 
 
    def run(self): 
        p = Pool() 
 
        for i in range(5): 
            out = p.map(F, [(self.agent, i),]) 
 
            print 'outside', self.agent.test_dict 
 
 
        p.close() 
        p.join() 
 
        return out 
 
if __name__ == '__main__': 
    SeriesInstance().run() 

输出是这样的,但是outside应该等于inside

inside  {0: 0} 
outside {} 
inside  {1: 1} 
outside {} 
inside  {2: 2} 
outside {} 
inside  {3: 3} 
outside {} 
inside  {4: 4} 
outside {} 

请您参考如下方法:

请查看Sharing state between processes并阅读服务器进程部分。看来您必须创建一个管理器并使用该管理器来创建在您的 Agent 类中使用的 dict 实例。

from multiprocessing import Process, Manager 
 
def f(d, l): 
    d[1] = '1' 
    d['2'] = 2 
    d[0.25] = None 
    l.reverse() 
 
if __name__ == '__main__': 
    with Manager() as manager: 
        d = manager.dict() 
        l = manager.list(range(10)) 
 
        p = Process(target=f, args=(d, l)) 
        p.start() 
        p.join() 
 
        print(d) 
        print(l)