请教一下大佬多进程的超时问题

import time
import random
from multiprocessing import Pool
def fun(name):
    print(f'{name} 运行')
    time.sleep(random.randrange(5,15))
    print(f'{name} 运行结束')

if __name__ == '__main__':
    pool = Pool(processes=2)
    names = ['anne','alice','biantai','haha']
    for name in names:
        pool.apply_async(fun,(name,))
    pool.close()
    pool.join()
    print('主线程')

我想要如果进程时间超过 10s 就结束掉，打印超时提示，然后让新进程执行，超时的进程放后面再执行，最后让所有的进程都在 10s 内成功运行完成，有什么优雅的方法吗？

Supplement 1 · Jun 17, 2021

import multiprocessing
from multiprocessing.dummy import Pool as ThreadPool
from functools import partial
import time
import random

def worker(name):
    begin = time.time()
    print(f'{name} 运行')
    time.sleep(random.randrange(5, 15))
    end = time.time()
    print(f'{name} 运行结束,运行了{end - begin}秒')
    return end - begin


def collectMyResult(result):
    pass
    # print("Got result {}".format(result))


def abortable_worker(func, *args, **kwargs):
    timeout = kwargs.get('timeout', None)
    name = kwargs.get("name",None)
    p = ThreadPool(1)
    res = p.apply_async(func, args=args)
    try:
        out = res.get(timeout)  # Wait timeout seconds for func to complete.
        return out
    except multiprocessing.TimeoutError:
        print(f"{name}运行超时，已终止")
        raise


if __name__ == "__main__":
    pool = multiprocessing.Pool(maxtasksperchild=1,processes=2)
    names = ["赵","钱","孙","李"]  # list of arguments
    for name in names:
        abortable_func = partial(abortable_worker, worker, timeout=10,name=name)
        pool.apply_async(abortable_func, args=name, callback=collectMyResult)
    pool.close()
    pool.join()

解决了超时就终止的问题了

name

pool

import

2 replies • 2021-06-11 09:48:29 +08:00

zhengxiaowai

Jun 11, 2021

https://docs.python.org/zh-cn/3/library/multiprocessing.html#using-a-pool-of-workers

有 timeout 参数，会抛出异常，另外 "最后让所有的进程都在 10s 内成功运行完成" 你得有足够的进程数量

fmdxx1991

Jun 11, 2021

@zhengxiaowai 这个用 get 尝试过，发现同时运行 2 个子进程的时候，用 get(timeout)会堵塞主进程，直到超时才会执行下一步，这个就有问题了

``` python
for name in names:
p = pool.apply_async(fun,(name,))
p.get(timeout=10)
```

这样写的话，就是每次运行一个进程，然后等待超时，那就没必要用多进程了

放外面的话, 也有很大的问题....