当前位置:   article > 正文

A process in the process pool was terminated abruptly while the future was running or pending

a process in the process pool was terminated abruptly while the future was r

解决方法:

把代码加到main中执行,有时能加速6倍左右

改之前:

  1. p = ProcessPoolExecutor(max_workers=3)
  2. results = p.map(task, URLS,range(3))
  3. p.shutdown(wait=True)
  4. for ret,url in results:
  5.     print(ret,url)

改之后:

  1. if __name__ == "__main__"
  2.     p = ProcessPoolExecutor(max_workers=3)
  3.     results = p.map(task, URLS,range(3))
  4.     p.shutdown(wait=True)
  5.     for ret,url in results:
  6.         print(ret,url)

二.使用ProcessPoolExecutor
在concurrent.futures 库中有ThreadPoolExecutor(多线程),ProcessPoolExecutor(多进程)
ThreadPoolExecutor,ProcessPoolExecutor的区别:
ThreadPoolExecutor:
ThreadPoolExecutor多线程并行执行任务,可以共享当前进程变量,但缺点也很致命,由于python GIL(Global Interpreter Lock 全局解释器锁)
的原因,及时多线程,但其实仍然最多只能占用CPU一个核,准确只能说是并发了,如果指定的任务和线程数不恰当(比如一个任务很

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/码创造者/article/detail/899809
推荐阅读
相关标签
  

闽ICP备14008679号