1、concurrent包
此包3.2版本之后引入，只提供了一个模块futures
异步并行任务编程模块，提供了一个高级的异步可执行的便利接口。
提供了两个池执行器
ThreadPoolExecutor 异步调用的线程池 的 Executor
ProcessPoolExeutor 异步调用的进程池的 Executor
2、ThreadPoolExecutor 对象 -- 线程
首先需要定义一个池的执行器对象，Executor类子类对象。
Future类
测试：IO 密集型测试，创建一个线城池，开启三个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18 executor = futures.ThreadPoolExecutor(max_workers=3)
19
20 fs = []
21 for i in range(3):
22 # 提交任务
23 future = executor.submit(worker, i)
24 fs.append(future)
25
26
27 while True:
28 time.sleep(1)
29 logging.info(threading.enumerate())
30
31 flag = True
32 for f in fs:
33 logging.info(f.done())
34 flag = flag and f.done()
35
36 if flag:
37 for f in fs:
38 logging.info('the ans is {}'.format(f.result()))
39 executor.shutdown()
40 logging.info(threading.enumerate())
41 break
42
43 logging.info('--------------------')线程数=池容量
1 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_0 9176 begin to work0
2 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_1 9176 begin to work1
3 2018-10-30 10:29:03,486 9176 ThreadPoolExecutor-0_2 9176 begin to work2
4 2018-10-30 10:29:04,486 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>, <Thread(ThreadPoolExecutor-0_0, started daemon 5720)>, <Thread(ThreadPoolExecutor-0_1, started daemon 5512)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5944)>]
5 2018-10-30 10:29:04,486 9176 MainThread 9176 False
6 2018-10-30 10:29:04,486 9176 MainThread 9176 False
7 2018-10-30 10:29:04,486 9176 MainThread 9176 False
8 2018-10-30 10:29:05,486 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>, <Thread(ThreadPoolExecutor-0_0, started daemon 5720)>, <Thread(ThreadPoolExecutor-0_1, started daemon 5512)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5944)>]
9 2018-10-30 10:29:05,486 9176 MainThread 9176 False
10 2018-10-30 10:29:05,486 9176 MainThread 9176 False
11 2018-10-30 10:29:05,487 9176 MainThread 9176 False
12 2018-10-30 10:29:06,488 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>, <Thread(ThreadPoolExecutor-0_0, started daemon 5720)>, <Thread(ThreadPoolExecutor-0_1, started daemon 5512)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5944)>]
13 2018-10-30 10:29:06,488 9176 MainThread 9176 False
14 2018-10-30 10:29:06,488 9176 MainThread 9176 False
15 2018-10-30 10:29:06,488 9176 MainThread 9176 False
16 2018-10-30 10:29:07,488 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>, <Thread(ThreadPoolExecutor-0_0, started daemon 5720)>, <Thread(ThreadPoolExecutor-0_1, started daemon 5512)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5944)>]
17 2018-10-30 10:29:07,488 9176 MainThread 9176 False
18 2018-10-30 10:29:07,488 9176 MainThread 9176 False
19 2018-10-30 10:29:07,489 9176 MainThread 9176 False
20 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_1 9176 finisdhed1
21 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_0 9176 finisdhed0
22 2018-10-30 10:29:08,487 9176 ThreadPoolExecutor-0_2 9176 finisdhed2
23 2018-10-30 10:29:08,489 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>, <Thread(ThreadPoolExecutor-0_0, started daemon 5720)>, <Thread(ThreadPoolExecutor-0_1, started daemon 5512)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5944)>]
24 2018-10-30 10:29:08,489 9176 MainThread 9176 True
25 2018-10-30 10:29:08,489 9176 MainThread 9176 True
26 2018-10-30 10:29:08,489 9176 MainThread 9176 True
27 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is 0
28 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is 1
29 2018-10-30 10:29:08,489 9176 MainThread 9176 the ans is 2
30 2018-10-30 10:29:08,490 9176 MainThread 9176 [<_MainThread(MainThread, started 8332)>]
31 2018-10-30 10:29:08,490 9176 MainThread 9176 --------------------得出：线程池关闭之后，就没有线程空闲线程存在了
测试：IO 密集型测试，创建一个线城池，开启一个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18 executor = futures.ThreadPoolExecutor(max_workers=3)
19
20 fs = []
21 for i in range(1):
22 # 提交任务
23 future = executor.submit(worker, i)
24 fs.append(future)
25
26
27 while True:
28 time.sleep(1)
29 logging.info(threading.enumerate())
30
31 flag = True
32 for f in fs:
33 logging.info(f.done())
34 flag = flag and f.done()
35
36 if flag:
37 for f in fs:
38 logging.info('the ans is {}'.format(f.result()))
39 # logging.info(threading.enumerate())
40 # logging.info('====================')
41 executor.shutdown()
42 logging.info(threading.enumerate())
43 break
44
45 logging.info('--------------------')线程数 < 池容量
1 D:\python3.7\python.exe E:/code_pycharm/tt2.py
2 2018-10-30 10:31:01,309 8408 ThreadPoolExecutor-0_0 8408 begin to work0
3 2018-10-30 10:31:02,309 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 9200)>]
4 2018-10-30 10:31:02,309 8408 MainThread 8408 False
5 2018-10-30 10:31:03,309 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 9200)>]
6 2018-10-30 10:31:03,309 8408 MainThread 8408 False
7 2018-10-30 10:31:04,309 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 9200)>]
8 2018-10-30 10:31:04,309 8408 MainThread 8408 False
9 2018-10-30 10:31:05,309 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 9200)>]
10 2018-10-30 10:31:05,309 8408 MainThread 8408 False
11 2018-10-30 10:31:06,309 8408 ThreadPoolExecutor-0_0 8408 finisdhed0
12 2018-10-30 10:31:06,309 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 9200)>]
13 2018-10-30 10:31:06,309 8408 MainThread 8408 True
14 2018-10-30 10:31:06,310 8408 MainThread 8408 the ans is 0
15 2018-10-30 10:31:06,310 8408 MainThread 8408 [<_MainThread(MainThread, started 8852)>]
16 2018-10-30 10:31:06,310 8408 MainThread 8408 --------------------
17
18 Process finished with exit code 0得出：线程池处于懒惰模式，提交一个线程函数，就开启一个线程
测试：IO 密集型测试，创建一个线城池，开启六个线程，因为此模块没有提供join方法，所以通过编程来实现主线程等待。
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(threadName)s %(process)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18 executor = futures.ThreadPoolExecutor(max_workers=3)
19
20 fs = []
21 for i in range(6):
22 # 提交任务
23 future = executor.submit(worker, i)
24 fs.append(future)
25
26
27 while True:
28 time.sleep(1)
29 logging.info(threading.enumerate())
30
31 flag = True
32 for f in fs:
33 logging.info(f.done())
34 flag = flag and f.done()
35
36 if flag:
37 for f in fs:
38 logging.info('the ans is {}'.format(f.result()))
39 # logging.info(threading.enumerate())
40 # logging.info('====================')
41 executor.shutdown()
42 logging.info(threading.enumerate())
43 break
44
45 logging.info('--------------------')线程数 >池容量
1 D:\python3.7\python.exe E:/code_pycharm/tt2.py
2 2018-10-30 10:36:29,200 7524 ThreadPoolExecutor-0_0 7524 begin to work0
3 2018-10-30 10:36:29,201 7524 ThreadPoolExecutor-0_1 7524 begin to work1
4 2018-10-30 10:36:29,201 7524 ThreadPoolExecutor-0_2 7524 begin to work2
5 2018-10-30 10:36:30,201 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
6 2018-10-30 10:36:30,201 7524 MainThread 7524 False
7 2018-10-30 10:36:30,201 7524 MainThread 7524 False
8 2018-10-30 10:36:30,202 7524 MainThread 7524 False
9 2018-10-30 10:36:30,202 7524 MainThread 7524 False
10 2018-10-30 10:36:30,202 7524 MainThread 7524 False
11 2018-10-30 10:36:30,202 7524 MainThread 7524 False
12 2018-10-30 10:36:31,204 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
13 2018-10-30 10:36:31,204 7524 MainThread 7524 False
14 2018-10-30 10:36:31,204 7524 MainThread 7524 False
15 2018-10-30 10:36:31,204 7524 MainThread 7524 False
16 2018-10-30 10:36:31,204 7524 MainThread 7524 False
17 2018-10-30 10:36:31,204 7524 MainThread 7524 False
18 2018-10-30 10:36:31,204 7524 MainThread 7524 False
19 2018-10-30 10:36:32,204 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
20 2018-10-30 10:36:32,204 7524 MainThread 7524 False
21 2018-10-30 10:36:32,204 7524 MainThread 7524 False
22 2018-10-30 10:36:32,204 7524 MainThread 7524 False
23 2018-10-30 10:36:32,204 7524 MainThread 7524 False
24 2018-10-30 10:36:32,204 7524 MainThread 7524 False
25 2018-10-30 10:36:32,204 7524 MainThread 7524 False
26 2018-10-30 10:36:33,204 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
27 2018-10-30 10:36:33,204 7524 MainThread 7524 False
28 2018-10-30 10:36:33,204 7524 MainThread 7524 False
29 2018-10-30 10:36:33,204 7524 MainThread 7524 False
30 2018-10-30 10:36:33,205 7524 MainThread 7524 False
31 2018-10-30 10:36:33,205 7524 MainThread 7524 False
32 2018-10-30 10:36:33,206 7524 MainThread 7524 False
33 2018-10-30 10:36:34,200 7524 ThreadPoolExecutor-0_0 7524 finisdhed0
34 2018-10-30 10:36:34,200 7524 ThreadPoolExecutor-0_0 7524 begin to work3
35 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_1 7524 finisdhed1
36 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_1 7524 begin to work4
37 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_2 7524 finisdhed2
38 2018-10-30 10:36:34,201 7524 ThreadPoolExecutor-0_2 7524 begin to work5
39 2018-10-30 10:36:34,206 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
40 2018-10-30 10:36:34,206 7524 MainThread 7524 True
41 2018-10-30 10:36:34,206 7524 MainThread 7524 True
42 2018-10-30 10:36:34,206 7524 MainThread 7524 True
43 2018-10-30 10:36:34,206 7524 MainThread 7524 False
44 2018-10-30 10:36:34,206 7524 MainThread 7524 False
45 2018-10-30 10:36:34,206 7524 MainThread 7524 False
46 2018-10-30 10:36:35,206 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
47 2018-10-30 10:36:35,206 7524 MainThread 7524 True
48 2018-10-30 10:36:35,206 7524 MainThread 7524 True
49 2018-10-30 10:36:35,206 7524 MainThread 7524 True
50 2018-10-30 10:36:35,206 7524 MainThread 7524 False
51 2018-10-30 10:36:35,206 7524 MainThread 7524 False
52 2018-10-30 10:36:35,206 7524 MainThread 7524 False
53 2018-10-30 10:36:36,206 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
54 2018-10-30 10:36:36,206 7524 MainThread 7524 True
55 2018-10-30 10:36:36,206 7524 MainThread 7524 True
56 2018-10-30 10:36:36,206 7524 MainThread 7524 True
57 2018-10-30 10:36:36,206 7524 MainThread 7524 False
58 2018-10-30 10:36:36,206 7524 MainThread 7524 False
59 2018-10-30 10:36:36,206 7524 MainThread 7524 False
60 2018-10-30 10:36:37,206 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
61 2018-10-30 10:36:37,206 7524 MainThread 7524 True
62 2018-10-30 10:36:37,206 7524 MainThread 7524 True
63 2018-10-30 10:36:37,206 7524 MainThread 7524 True
64 2018-10-30 10:36:37,207 7524 MainThread 7524 False
65 2018-10-30 10:36:37,207 7524 MainThread 7524 False
66 2018-10-30 10:36:37,207 7524 MainThread 7524 False
67 2018-10-30 10:36:38,207 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
68 2018-10-30 10:36:38,207 7524 MainThread 7524 True
69 2018-10-30 10:36:38,209 7524 MainThread 7524 True
70 2018-10-30 10:36:38,210 7524 MainThread 7524 True
71 2018-10-30 10:36:38,210 7524 MainThread 7524 False
72 2018-10-30 10:36:38,210 7524 MainThread 7524 False
73 2018-10-30 10:36:38,210 7524 MainThread 7524 False
74 2018-10-30 10:36:39,200 7524 ThreadPoolExecutor-0_0 7524 finisdhed3
75 2018-10-30 10:36:39,201 7524 ThreadPoolExecutor-0_1 7524 finisdhed4
76 2018-10-30 10:36:39,201 7524 ThreadPoolExecutor-0_2 7524 finisdhed5
77 2018-10-30 10:36:39,210 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>, <Thread(ThreadPoolExecutor-0_0, started daemon 8332)>, <Thread(ThreadPoolExecutor-0_1, started daemon 7056)>, <Thread(ThreadPoolExecutor-0_2, started daemon 5080)>]
78 2018-10-30 10:36:39,210 7524 MainThread 7524 True
79 2018-10-30 10:36:39,210 7524 MainThread 7524 True
80 2018-10-30 10:36:39,211 7524 MainThread 7524 True
81 2018-10-30 10:36:39,211 7524 MainThread 7524 True
82 2018-10-30 10:36:39,211 7524 MainThread 7524 True
83 2018-10-30 10:36:39,211 7524 MainThread 7524 True
84 2018-10-30 10:36:39,211 7524 MainThread 7524 the ans is 0
85 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 1
86 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 2
87 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 3
88 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 4
89 2018-10-30 10:36:39,212 7524 MainThread 7524 the ans is 5
90 2018-10-30 10:36:39,213 7524 MainThread 7524 [<_MainThread(MainThread, started 8852)>]
91 2018-10-30 10:36:39,213 7524 MainThread 7524 --------------------
92
93 Process finished with exit code 0得出：三个线程，只要有一个结束，就开启第四个线程
3、ProcessPoolExecutor 对象 -- 进程
进程
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18 # executor = futures.ThreadPoolExecutor(max_workers=3)
19
20 if __name__ == '__main__':
21 executor = futures.ProcessPoolExecutor(max_workers=3)
22 fs = []
23 for i in range(3):
24 # 提交任务
25 future = executor.submit(worker, i)
26 fs.append(future)
27
28
29 while True:
30 time.sleep(1)
31 logging.info(threading.enumerate())
32
33 flag = True
34 for f in fs:
35 logging.info(f.done())
36 flag = flag and f.done()
37
38 if flag:
39 for f in fs:
40 logging.info('the ans is {}'.format(f.result()))
41 # logging.info(threading.enumerate())
42 # logging.info('====================')
43 executor.shutdown()
44 logging.info(threading.enumerate())
45 break
46
47 logging.info('--------------------')进程测试
1 D:\python3.7\python.exe E:/code_pycharm/tt2.py
2 2018-10-30 10:47:48,933 3224 SpawnProcess-1 begin to work0
3 2018-10-30 10:47:48,951 7484 SpawnProcess-2 begin to work1
4 2018-10-30 10:47:48,974 8848 SpawnProcess-3 begin to work2
5 2018-10-30 10:47:49,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
6 2018-10-30 10:47:49,829 6220 MainProcess False
7 2018-10-30 10:47:49,829 6220 MainProcess False
8 2018-10-30 10:47:49,829 6220 MainProcess False
9 2018-10-30 10:47:50,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
10 2018-10-30 10:47:50,829 6220 MainProcess False
11 2018-10-30 10:47:50,829 6220 MainProcess False
12 2018-10-30 10:47:50,829 6220 MainProcess False
13 2018-10-30 10:47:51,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
14 2018-10-30 10:47:51,829 6220 MainProcess False
15 2018-10-30 10:47:51,829 6220 MainProcess False
16 2018-10-30 10:47:51,829 6220 MainProcess False
17 2018-10-30 10:47:52,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
18 2018-10-30 10:47:52,829 6220 MainProcess False
19 2018-10-30 10:47:52,829 6220 MainProcess False
20 2018-10-30 10:47:52,829 6220 MainProcess False
21 2018-10-30 10:47:53,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
22 2018-10-30 10:47:53,829 6220 MainProcess False
23 2018-10-30 10:47:53,829 6220 MainProcess False
24 2018-10-30 10:47:53,829 6220 MainProcess False
25 2018-10-30 10:47:53,934 3224 SpawnProcess-1 finisdhed0
26 2018-10-30 10:47:53,951 7484 SpawnProcess-2 finisdhed1
27 2018-10-30 10:47:53,974 8848 SpawnProcess-3 finisdhed2
28 2018-10-30 10:47:54,829 6220 MainProcess [<_MainThread(MainThread, started 3604)>, <Thread(QueueManagerThread, started daemon 9156)>, <Thread(QueueFeederThread, started daemon 8476)>]
29 2018-10-30 10:47:54,829 6220 MainProcess True
30 2018-10-30 10:47:54,829 6220 MainProcess True
31 2018-10-30 10:47:54,829 6220 MainProcess True
32 2018-10-30 10:47:54,829 6220 MainProcess the ans is 0
33 2018-10-30 10:47:54,829 6220 MainProcess the ans is 1
34 2018-10-30 10:47:54,829 6220 MainProcess the ans is 2
35 2018-10-30 10:47:54,854 6220 MainProcess [<_MainThread(MainThread, started 3604)>]
36 2018-10-30 10:47:54,854 6220 MainProcess --------------------
37
38 Process finished with exit code 0查看 python进程数，占用三个CPU
4、支持上下文管理
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18 # executor = futures.ThreadPoolExecutor(max_workers=3)
19
20 if __name__ == '__main__':
21 executor = futures.ProcessPoolExecutor(max_workers=3)
22
23 with executor:
24 fs = []
25 for i in range(4):
26 # 提交任务
27 future = executor.submit(worker, i)
28 fs.append(future)
29
30
31 while True:
32 time.sleep(1)
33 logging.info(threading.enumerate())
34
35 flag = True
36 for f in fs:
37 logging.info(f.done())
38 flag = flag and f.done()
39
40 if flag:
41 for f in fs:
42 logging.info('the ans is {}'.format(f.result()))
43 # logging.info(threading.enumerate())
44 # logging.info('====================')
45 break
46
47 logging.info('--------------------')with一般放到刚创建池对象后面，最终自动清除池
1 D:\python3.7\python.exe E:/code_pycharm/tt2.py
2 2018-10-30 10:54:18,388 8244 SpawnProcess-1 begin to work0
3 2018-10-30 10:54:18,404 5944 SpawnProcess-2 begin to work1
4 2018-10-30 10:54:18,433 5608 SpawnProcess-3 begin to work2
5 2018-10-30 10:54:19,294 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
6 2018-10-30 10:54:19,294 8476 MainProcess False
7 2018-10-30 10:54:19,294 8476 MainProcess False
8 2018-10-30 10:54:19,294 8476 MainProcess False
9 2018-10-30 10:54:19,294 8476 MainProcess False
10 2018-10-30 10:54:20,294 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
11 2018-10-30 10:54:20,294 8476 MainProcess False
12 2018-10-30 10:54:20,294 8476 MainProcess False
13 2018-10-30 10:54:20,294 8476 MainProcess False
14 2018-10-30 10:54:20,294 8476 MainProcess False
15 2018-10-30 10:54:21,294 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
16 2018-10-30 10:54:21,294 8476 MainProcess False
17 2018-10-30 10:54:21,294 8476 MainProcess False
18 2018-10-30 10:54:21,294 8476 MainProcess False
19 2018-10-30 10:54:21,294 8476 MainProcess False
20 2018-10-30 10:54:22,294 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
21 2018-10-30 10:54:22,294 8476 MainProcess False
22 2018-10-30 10:54:22,294 8476 MainProcess False
23 2018-10-30 10:54:22,294 8476 MainProcess False
24 2018-10-30 10:54:22,294 8476 MainProcess False
25 2018-10-30 10:54:23,294 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
26 2018-10-30 10:54:23,294 8476 MainProcess False
27 2018-10-30 10:54:23,294 8476 MainProcess False
28 2018-10-30 10:54:23,294 8476 MainProcess False
29 2018-10-30 10:54:23,295 8476 MainProcess False
30 2018-10-30 10:54:23,388 8244 SpawnProcess-1 finisdhed0
31 2018-10-30 10:54:23,388 8244 SpawnProcess-1 begin to work3
32 2018-10-30 10:54:23,404 5944 SpawnProcess-2 finisdhed1
33 2018-10-30 10:54:23,433 5608 SpawnProcess-3 finisdhed2
34 2018-10-30 10:54:24,295 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
35 2018-10-30 10:54:24,295 8476 MainProcess True
36 2018-10-30 10:54:24,295 8476 MainProcess True
37 2018-10-30 10:54:24,295 8476 MainProcess True
38 2018-10-30 10:54:24,295 8476 MainProcess False
39 2018-10-30 10:54:25,295 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
40 2018-10-30 10:54:25,295 8476 MainProcess True
41 2018-10-30 10:54:25,295 8476 MainProcess True
42 2018-10-30 10:54:25,295 8476 MainProcess True
43 2018-10-30 10:54:25,295 8476 MainProcess False
44 2018-10-30 10:54:26,295 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
45 2018-10-30 10:54:26,295 8476 MainProcess True
46 2018-10-30 10:54:26,295 8476 MainProcess True
47 2018-10-30 10:54:26,295 8476 MainProcess True
48 2018-10-30 10:54:26,295 8476 MainProcess False
49 2018-10-30 10:54:27,296 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
50 2018-10-30 10:54:27,296 8476 MainProcess True
51 2018-10-30 10:54:27,296 8476 MainProcess True
52 2018-10-30 10:54:27,296 8476 MainProcess True
53 2018-10-30 10:54:27,296 8476 MainProcess False
54 2018-10-30 10:54:28,296 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
55 2018-10-30 10:54:28,296 8476 MainProcess True
56 2018-10-30 10:54:28,296 8476 MainProcess True
57 2018-10-30 10:54:28,296 8476 MainProcess True
58 2018-10-30 10:54:28,296 8476 MainProcess False
59 2018-10-30 10:54:28,391 8244 SpawnProcess-1 finisdhed3
60 2018-10-30 10:54:29,296 8476 MainProcess [<_MainThread(MainThread, started 8516)>, <Thread(QueueManagerThread, started daemon 5892)>, <Thread(QueueFeederThread, started daemon 8632)>]
61 2018-10-30 10:54:29,296 8476 MainProcess True
62 2018-10-30 10:54:29,296 8476 MainProcess True
63 2018-10-30 10:54:29,296 8476 MainProcess True
64 2018-10-30 10:54:29,296 8476 MainProcess True
65 2018-10-30 10:54:29,296 8476 MainProcess the ans is 0
66 2018-10-30 10:54:29,296 8476 MainProcess the ans is 1
67 2018-10-30 10:54:29,296 8476 MainProcess the ans is 2
68 2018-10-30 10:54:29,296 8476 MainProcess the ans is 3
69 2018-10-30 10:54:29,327 8476 MainProcess --------------------
70
71 Process finished with exit code 0结果
1 import threading
2 from concurrent import futures
3
4 import logging
5 import time
6
7 FORMAT = '%(asctime)-15s\t %(process)s %(processName)s %(message)s'
8 logging.basicConfig(level=logging.INFO, format=FORMAT)
9
10 # 这是一个IO 密集型的函数，建议使用多线程
11 def worker(n):
12 logging.info('begin to work{}'.format(n))
13 time.sleep(5)
14 logging.info('finisdhed{}'.format(n))
15 return n
16
17 # 创建线程池，容量为3个
18
19 if __name__ == '__main__':
20 # executor = futures.ProcessPoolExecutor(max_workers=3)
21 executor = futures.ThreadPoolExecutor(max_workers=3)
22
23 with executor:
24 fs = []
25 for i in range(4):
26 # 提交任务
27 future = executor.submit(worker, i)
28 fs.append(future)
29
30
31 while True:
32 time.sleep(1)
33 logging.info(threading.enumerate())
34
35 flag = True
36 for f in fs:
37 logging.info(f.done())
38 flag = flag and f.done()
39
40 if flag:
41 for f in fs:
42 logging.info('the ans is {}'.format(f.result()))
43 # logging.info(threading.enumerate())
44 # logging.info('====================')
45 break
46 logging.info(threading.enumerate())
47
48 logging.info('--------------------')多线程测试
1 D:\python3.7\python.exe E:/code_pycharm/tt2.py
2 2018-10-30 10:58:12,862 5604 MainProcess begin to work0
3 2018-10-30 10:58:12,862 5604 MainProcess begin to work1
4 2018-10-30 10:58:12,862 5604 MainProcess begin to work2
5 2018-10-30 10:58:13,862 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
6 2018-10-30 10:58:13,862 5604 MainProcess False
7 2018-10-30 10:58:13,862 5604 MainProcess False
8 2018-10-30 10:58:13,862 5604 MainProcess False
9 2018-10-30 10:58:13,862 5604 MainProcess False
10 2018-10-30 10:58:14,863 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
11 2018-10-30 10:58:14,863 5604 MainProcess False
12 2018-10-30 10:58:14,863 5604 MainProcess False
13 2018-10-30 10:58:14,863 5604 MainProcess False
14 2018-10-30 10:58:14,863 5604 MainProcess False
15 2018-10-30 10:58:15,863 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
16 2018-10-30 10:58:15,863 5604 MainProcess False
17 2018-10-30 10:58:15,863 5604 MainProcess False
18 2018-10-30 10:58:15,863 5604 MainProcess False
19 2018-10-30 10:58:15,863 5604 MainProcess False
20 2018-10-30 10:58:16,863 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
21 2018-10-30 10:58:16,863 5604 MainProcess False
22 2018-10-30 10:58:16,863 5604 MainProcess False
23 2018-10-30 10:58:16,863 5604 MainProcess False
24 2018-10-30 10:58:16,863 5604 MainProcess False
25 2018-10-30 10:58:17,863 5604 MainProcess finisdhed2
26 2018-10-30 10:58:17,863 5604 MainProcess begin to work3
27 2018-10-30 10:58:17,863 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
28 2018-10-30 10:58:17,863 5604 MainProcess finisdhed1
29 2018-10-30 10:58:17,863 5604 MainProcess finisdhed0
30 2018-10-30 10:58:17,863 5604 MainProcess False
31 2018-10-30 10:58:17,864 5604 MainProcess True
32 2018-10-30 10:58:17,864 5604 MainProcess True
33 2018-10-30 10:58:17,864 5604 MainProcess False
34 2018-10-30 10:58:18,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
35 2018-10-30 10:58:18,864 5604 MainProcess True
36 2018-10-30 10:58:18,864 5604 MainProcess True
37 2018-10-30 10:58:18,864 5604 MainProcess True
38 2018-10-30 10:58:18,864 5604 MainProcess False
39 2018-10-30 10:58:19,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
40 2018-10-30 10:58:19,864 5604 MainProcess True
41 2018-10-30 10:58:19,864 5604 MainProcess True
42 2018-10-30 10:58:19,864 5604 MainProcess True
43 2018-10-30 10:58:19,864 5604 MainProcess False
44 2018-10-30 10:58:20,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
45 2018-10-30 10:58:20,864 5604 MainProcess True
46 2018-10-30 10:58:20,864 5604 MainProcess True
47 2018-10-30 10:58:20,864 5604 MainProcess True
48 2018-10-30 10:58:20,864 5604 MainProcess False
49 2018-10-30 10:58:21,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
50 2018-10-30 10:58:21,864 5604 MainProcess True
51 2018-10-30 10:58:21,864 5604 MainProcess True
52 2018-10-30 10:58:21,864 5604 MainProcess True
53 2018-10-30 10:58:21,864 5604 MainProcess False
54 2018-10-30 10:58:22,863 5604 MainProcess finisdhed3
55 2018-10-30 10:58:22,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>, <Thread(ThreadPoolExecutor-0_0, started daemon 4192)>, <Thread(ThreadPoolExecutor-0_1, started daemon 9128)>, <Thread(ThreadPoolExecutor-0_2, started daemon 9036)>]
56 2018-10-30 10:58:22,864 5604 MainProcess True
57 2018-10-30 10:58:22,864 5604 MainProcess True
58 2018-10-30 10:58:22,864 5604 MainProcess True
59 2018-10-30 10:58:22,864 5604 MainProcess True
60 2018-10-30 10:58:22,864 5604 MainProcess the ans is 0
61 2018-10-30 10:58:22,864 5604 MainProcess the ans is 1
62 2018-10-30 10:58:22,864 5604 MainProcess the ans is 2
63 2018-10-30 10:58:22,864 5604 MainProcess the ans is 3
64 2018-10-30 10:58:22,864 5604 MainProcess [<_MainThread(MainThread, started 3956)>]
65 2018-10-30 10:58:22,864 5604 MainProcess --------------------多线程测试，最终线程都关闭了
总结：
该库统一了线程池，进程池的调用，简化了编程。
缺点是无法设置线程名。
为什么要坚持，想一想当初！
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
