Python 多进程 多线程

实际上Python比java提供了更激进的并发模式 (当然java也可以做到,不过没人愿意启动那么多虚拟机吧)

它使用多进程的方式来处理并发需求

下面是一个简单例子,看起来跟多线程都差不多

from multiprocessing import Process
import os

# 子进程要执行的代码
def run_proc(name):
    print ('Run child process %s (%s)...' % (name, os.getpid()));

if __name__=='__main__':
    print ('Parent process %s.' % os.getpid());
    p = Process(target=run_proc, args=('test',));
    print ('Process will start.');
    p.start();
    p.join();
    print ('Process end.');

然后我们来尝试一下上一个计算pi的例子,看看多进程模式下python的并发性能

from multiprocessing import Process
import os
from decimal import *;
import sys;
import math;
import time;

def bellard(n):
   pi=Decimal(0)
   k=0
   while k < n:
       pi+=(Decimal(-1)**k/(1024**k))*( Decimal(256)/(10*k+1)+Decimal(1)/(10*k+9)-Decimal(64)/(10*k+3)-Decimal(32)/(4*k+1)-Decimal(4)/(10*k+5)-Decimal(4)/(10*k+7)-Decimal(1)/(4*k+3))
       k+=1
   pi=pi*1/(2**6)
   return pi

def say(name):
    start = time.clock();
    for i in range(100):
        p = bellard(1000);
#        print("running..."+str(p));
    end = time.clock();
    print ("read: %f s" % (end - start));

def run_proc(name):
    print ('Run child process %s (%s)...' % (name, os.getpid()));

if __name__=='__main__':
    print ('Parent process %s.' % os.getpid());
    p = Process(target=say, args=('test',));
    print ('Process will start.');
    p.start();
    p.join();
    print ('Process end.');

单进程下结果

Parent process 8184. Process will start. read: 25.787227 s Process end.

双进程稍微修改一下程序

from multiprocessing import Process
import os
from decimal import *;
import sys;
import math;
import time;

def bellard(n):
   pi=Decimal(0)
   k=0
   while k < n:
       pi+=(Decimal(-1)**k/(1024**k))*( Decimal(256)/(10*k+1)+Decimal(1)/(10*k+9)-Decimal(64)/(10*k+3)-Decimal(32)/(4*k+1)-Decimal(4)/(10*k+5)-Decimal(4)/(10*k+7)-Decimal(1)/(4*k+3))
       k+=1
   pi=pi*1/(2**6)
   return pi

def say(name):
    start = time.clock();
    for i in range(100):
        p = bellard(1000);
#        print("running..."+str(p));
    end = time.clock();
    print ("read: %f s" % (end - start));

def run_proc(name):
    print ('Run child process %s (%s)...' % (name, os.getpid()));

if __name__=='__main__':
    print ('Parent process %s.' % os.getpid());
    p = Process(target=say, args=('test',));
    p2 = Process(target=say, args=('test',));
    print ('Process will start.');
    p.start();
    p2.start();
#    p.join();
    print ('Process end.');

Parent process 7020. Process will start. Process end. read: 26.193302 s read: 26.187929 s

可以看到并没有消耗两倍的时间,是真实并发

4 进程消耗如下

Parent process 7796. Process will start. Process end. read: 29.337095 s read: 29.761048 s read: 29.792244 s read: 30.208371 s

最后8进程,在4核cpu上消耗了差不多两倍时间

Parent process 5844. Process will start. Process end. read: 51.267404 s read: 55.194120 s read: 58.598105 s read: 58.615099 s read: 59.172344 s read: 59.067506 s read: 59.797957 s read: 60.902044 s