我的问题涉及替换 join() 函数以避免在使用 python3 的多处理库时已终止进程的失效或僵尸状态。是否有替代方案可以暂停子进程被终止,直到它们从主进程获得绿灯?这允许它们正确终止而不会进入僵尸状态?
我使用以下代码准备了一个快速示例,它启动了 20 个不同的进程,第一个进程需要 10 秒的加载工作,所有其他进程需要 3 秒的加载工作:
import os
import sys
import time
import multiprocessing as mp
from multiprocessing import Process
def exe(i):
print(i)
if i == 1:
time.sleep(10)
else:
time.sleep(3)
procs = []
for i in range(1,20):
proc = Process(target=exe, args=(i,))
proc.start()
procs.append(proc)
for proc in procs:
print(proc) # <-- I'm blocked to join others till the first process finishes its work load
proc.join()
print("finished")
如果您启动脚本,您将看到所有其他进程进入僵尸状态,直到第一个进程释放 join() 函数。这可能会使系统不稳定或过载!
谢谢
请您参考如下方法:
根据 this thread ,Marko Rauhamaa 写道:
If you don't care to know when child processes exit, you can simply ignore the SIGCHLD signal:
import signal signal.signal(signal.SIGCHLD, signal.SIG_IGN)That will prevent zombies from appearing.
POSIX.1-2001 specifies that if the disposition of SIGCHLD is set to SIG_IGN or the SA_NOCLDWAIT flag is set for SIGCHLD (see sigaction(2)), then children that terminate do not become zombies and a call to wait() or waitpid() will block until all children have terminated, and then fail with errno set to ECHILD. (The original POSIX standard left the behavior of setting SIGCHLD to SIG_IGN unspecified. Note that even though the default disposition of SIGCHLD is "ignore", explicitly setting the disposition to SIG_IGN results in different treatment of zombie process children.)
Linux 2.6 conforms to the POSIX requirements. However, Linux 2.4 (and earlier) does not: if a wait() or waitpid() call is made while SIGCHLD is being ignored, the call behaves just as though SIGCHLD were not being ignored, that is, the call blocks until the next child terminates and then returns the process ID and status of that child.
因此,如果您使用的是 Linux 2.6 或 POSIX 兼容的操作系统,使用上述代码将允许子进程退出而不会变成僵尸。如果您使用的不是 POSIX 兼容的操作系统,那么上面的线程会提供许多选项。下面是一个替代方案,有点类似于 Marko Rauhamaa 的 third suggestion .
如果出于某种原因您需要知道子进程何时退出并希望 处理(至少其中一些)不同,那么你可以设置一个队列来 允许子进程在完成时向主进程发出信号。然后 主进程可以按照接收的顺序调用适当的连接 队列中的项目:
import time
import multiprocessing as mp
def exe(i, q):
try:
print(i)
if i == 1:
time.sleep(10)
elif i == 10:
raise Exception('I quit')
else:
time.sleep(3)
finally:
q.put(mp.current_process().name)
if __name__ == '__main__':
procs = dict()
q = mp.Queue()
for i in range(1,20):
proc = mp.Process(target=exe, args=(i, q))
proc.start()
procs[proc.name] = proc
while procs:
name = q.get()
proc = procs[name]
print(proc)
proc.join()
del procs[name]
print("finished")
产生这样的结果
...
<Process(Process-10, stopped[1])> # <-- process with exception still gets joined
19
<Process(Process-2, started)>
<Process(Process-4, stopped)>
<Process(Process-6, started)>
<Process(Process-5, stopped)>
<Process(Process-3, stopped)>
<Process(Process-9, started)>
<Process(Process-7, stopped)>
<Process(Process-8, started)>
<Process(Process-13, started)>
<Process(Process-12, stopped)>
<Process(Process-11, stopped)>
<Process(Process-16, started)>
<Process(Process-15, stopped)>
<Process(Process-17, stopped)>
<Process(Process-14, stopped)>
<Process(Process-18, started)>
<Process(Process-19, stopped)>
<Process(Process-1, started)> # <-- Process-1 ends last
finished
