python 多进程加速for循环示例，包括如何传参。_python多进程for循环

# SuperFastPython.com
# execute tasks in parallel in a for loop
from time import sleep
from random import random
from multiprocessing import Process
 
 
# execute a task
def task(arg, i, string_):
    print("arg=" + str(arg) + " i=" + str(i) + "string_" + string_)
    # generate a random value between 0 and 1
    value = random()
    # block for a fraction of a second
    sleep(value + arg[0])
    print(sum(arg))
 
    # report a message
    print(f'.done {arg[0]}, generated {value}', flush=True)
 
 
# protect the entry point
if __name__ == '__main__':
    pool_num = 3
    args_list = [[1,2,3,4], [5,6,7,8], [9,10,11,12]]
    string_ = "res"
    processes = [Process(target=task, args=(args_list[i], i, string_)) for i in range(pool_num)]
    # start all processes
    for process in processes:
        process.start()
    # wait for all processes to complete
    for process in processes:
        process.join()
    # report that all tasks are completed
    print('Done', flush=True)

过程就是定义几个Process，每个Process执行不同的task。task的区别在于for循环中不同的索引。另外各个Process如果输出文件，那么应写到不同的文件里避免不必要的麻烦。

args_list的由来：对于for循环而言，可以将整个长度进行切分成如上的args_list，再对各个Process传递不同的index值即可。如下所示：

split_size = int(len(accx_cuts) / pool_num)
 
print("allocate indexes")
args_list = []
for i in range(pool_num):
    data_list = [i * split_size + k for k in range(split_size)]
    print(i, data_list[0:5], data_list[-5:])
    args_list.append(data_list)
 
processes = [Process(target=task, args=(args_list[i], i, accx_cuts, accy_cuts, accz_cuts)) for i in range(pool_num)]