Skip to main content
 首页 » 编程设计

Python 多处理 IOError : [Errno 232] The pipe is being closed

2024年11月01日10sky-heaven

我正在尝试实现这个 tutorial在 python 中进行多重处理,但是当我尝试执行自己的任务时,出现以下错误:

Traceback (most recent call last): 
>>>   File "C:\Python27\lib\multiprocessing\queues.py", line 262, in _feed 
    send(obj) 
IOError: [Errno 232] The pipe is being closed 

这是我正在尝试做的一个可重现的示例,它给出了相同的错误消息:

from multiprocessing import Lock, Process, Queue, current_process 
import time 
 
class Testclass(object): 
    def __init__(self, x): 
        self.x = x 
 
def toyfunction(testclass): 
    testclass.product = testclass.x * testclass.x 
    return testclass 
 
 
def worker(work_queue, done_queue): 
    try: 
        for testclass in iter(work_queue.get, 'STOP'): 
            print(testclass.counter) 
            newtestclass = toyfunction(testclass) 
            done_queue.put(newtestclass) 
 
    except: 
        print('error') 
 
    return True 
 
def main(): 
 
    counter = 1 
 
    database = [] 
    while counter <= 1000: 
        database.append(Testclass(3)) 
        counter += 1 
        print(counter) 
 
 
 
    workers = 8 
    work_queue = Queue() 
    done_queue = Queue() 
    processes = [] 
 
    start = time.clock() 
    counter = 1 
 
    for testclass in database: 
        testclass.counter = counter 
        work_queue.put(testclass) 
        counter += 1 
        print(counter) 
 
 
    print('items loaded') 
    for w in range(workers): 
        p = Process(target=worker, args=(work_queue, done_queue)) 
        p.start() 
        processes.append(p) 
        work_queue.put('STOP') 
 
    for p in processes: 
        p.join() 
 
    done_queue.put('STOP') 
 
    print(time.clock()-start) 
    print("Done") 
 
if __name__ == '__main__': 
    main()     

请您参考如下方法:

我通过在使用事件正常退出进程后清空队列来解决这个问题:

self.event.set() #the process has a timer that checks for this to be set, then shuts itself down 
while not self._q.empty(): #_q is a multiprocess.Queue object used to communicate inter-process 
    try: 
        self._q.get(timeout=0.001) 
    except: 
        pass 
self._q.close()