Python学习之进程和并发
从python2.4 以后，subprocess模块负责衍生出新的进程，和标准输入，标准输出，标准错误输出交互,并监听返回值。Subprocess模块是用来取代一些老的模块，例如os.system, os.spawn, os.popen和popen2
In [34]: subprocess.call('df -h',shell=True)
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/xvda1 485M 55M 405M 12% /boot
/dev/xvdb 99G 21G 74G 22% /data
Out[34]: 0还可以引入shell变量
In [105]: subprocess.call('du -sh $HOME',shell=True)
34M /home/caribbean
Out[105]: 0使用subprocess.call时可以不显示输出。
In [170]: import subprocess
In [171]: ret=subprocess.call("ping -c 5 10.10.41.10",shell=True,stdout=open('/dev/null','w'),stderr=subprocess.STDOUT)如果不关心shell命令的输出，而只是想执行shell命令，可以使用subprocess.call，如果想捕获shell命令的输出结果，可以使用subprocess.Popen. subprocess.call还会中断等待响应，subprocess.Popen不会
使用Subprocess的返回值
0 成功
1 通用错误
2 误使用shell内置变量
126 调用的命令无法执行
127 命令无法找到
128 无效的参数退出
130 脚本被Ctrl-C 终端
In [250]: subprocess.call("ls /foo",shell=True)
ls: cannot access /foo: No such file or directory
Out[250]: 2In [252]: ret=subprocess.call("ls /foo",shell=True)
ls: cannot access /foo: No such file or directory
In [253]: if ret==0:
.....: print "success"
.....: else:
.....: print "failure"
.....:
.....:
failuresubprocess.call调用命令的返回值可以用在条件语句中
In [382]: if platfORM.system() == "linux":
.....: ret=subprocess.call('cp /tmp/a.txt /tmp/b.txt',shell=True)
.....: if ret==0:
.....: print "Success,the copy was made on %s %s" %(platform.system(),platform.release())
.....:
.....:
Success,the copy was made on Linux 2.6.32-431.el6.x86_64使用subprocess.Popen捕获输出结果
In [399]:
In [400]: import subprocess
In [401]: p=subprocess.Popen('df -h',shell=True,stdout=subprocess.PIPE)
In [402]: out=p.stdout.readlines()
In [403]: for line in out:
.....: print line.strip()
.....:
.....:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/xvda1 485M 55M 405M 12% /boot
/dev/xvdb 99G 21G 73G 23% /data
In [404]:Readlines()返回的是一个带换行符的列表
In [413]: print out
--------> print(out)
['Filesystem Size Used Avail Use% Mounted on\n', '/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /\n', 'tmpfs 1.9G 0 1.9G 0% /dev/shm\n', '/dev/xvda1 485M 55M 405M 12% /boot\n', '/dev/xvdb 99G 21G 73G 23% /data\n']
In [414]:In [440]: for line in out:
.....: print line
.....:
.....:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/xvda1 485M 55M 405M 12% /boot
/dev/xvdb 99G 21G 73G 23% /data
In [441]: for line in out:
.....: print line.strip()
.....:
.....:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/xvda1 485M 55M 405M 12% /boot
/dev/xvdb 99G 21G 73G 23% /data
In [442]:subprocess管道工厂
In [513]: def multi(*args):
.....: for cmd in args:
.....: p=subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)
.....: out=p.stdout.read()
.....: print out
.....:
.....:
In [514]: multi("df -h","ls -lh /tmp","tail -f /var/log/system.log")
tail: cannot open `/var/log/system.log' for reading: No such file or directory
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root 6.7G 1.7G 4.6G 28% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/xvda1 485M 55M 405M 12% /boot
/dev/xvdb 99G 22G 73G 23% /data
total 6.5M
-rw-r--r--. 1 root root 18K May 28 17:55 a.txt
-rw-r--r--. 1 caribbean operation 52K Jul 23 12:00 baidu.html
-rw-r--r--. 1 caribbean operation 18K Aug 1 16:11 b.txt
-rw-r--r--. 1 daemon daemon 6.4M Aug 1 14:36 PHP_errors.log
drwxr-xr-x. 11 root root 4.0K Jul 30 14:40 pip-build-root
-rw-r--r--. 1 root root 0 Jul 31 14:39 send_msg_s1.txt
-rw-r--r--. 1 root root 0 Jul 31 14:39 send_msg_s2.txt
-rw-r--r--. 1 root root 0 Jul 31 14:39 send_msg_s3.txt
-rw-------. 1 root root 0 Dec 17 2013 yum.log
-rw-------. 1 root root 877 Jul 25 10:10 yum_save_tx-2014-07-25-10-10iJ0cju.yumtx
-rw-rw-rw-. 1 Mysql mysql 2.4K Jul 11 13:54 yunbaomysql2.txt
-rw-rw-rw-. 1 mysql mysql 2.4K Jul 11 13:45 yunbaomysql.txtIn [23]: p=subprocess.Popen("wc -c",shell=True,stdin=subprocess.PIPE)
In [24]: p.communicate("charactersinWord")
16
Out[24]: (None, None)和echo charactersinword | wc -c 等价
In [28]: with open('temp.txt','w') as file:
....: file.write('charactersinword')
....:
....:
In [29]: file=open('temp.txt')
In [30]: f=file.read()
In [31]: p=subprocess.Popen("wc -c",shell=True,stdin=subprocess.PIPE)
In [32]: p.communicate(f)
16在shell中等价于
$echo charactersinword > temp.txt
$ wc -c < temp.txt
$ cat /etc/passwd|grep 0:0 |cut -d ':' -f 7
等价于
In [75]: p1=subprocess.Popen("cat /etc/passwd",shell=True,stdout=subprocess.PIPE)
In [76]: p2=subprocess.Popen("grep 0:0",shell=True,stdin=p1.stdout,stdout=subprocess.PIPE)
In [77]: p3=subprocess.Popen("cut -d ':' -f 7",shell=True,stdin=p2.stdout,stdout=subprocess.PIPE)
In [78]: print p3.stdout.read()
-------> print(p3.stdout.read())
/bin/bash等价于
In [114]: import pwd
In [115]: pwd.getpwnam('root')
Out[115]: pwd.struct_passwd(pw_name='root', pw_passwd='x', pw_uid=0, pw_gid=0, pw_gecos='root', pw_dir='/root', pw_shell='/bin/bash')
In [116]: pwd.getpwnam('root')[-1]
Out[116]: '/bin/bash'Subprocess模块还可以同时发送输入和接收输出，同时监听错误输出。
在IPython中可以使用ed upper.py这样的方式直接切换到vim编辑代码
#/usr/bin/python
import subprocess
p=subprocess.Popen("tr a-z A-Z",shell=True,stdin=subprocess.PIPE,stdout=subprocess.PIPE)
output,error=p.communicate("translatetoupper")
print outputIn [10]: ed upper.py
Editing... done. Executing edited code...
TRANSLATETOUPPERPython中的线程
#/usr/bin/python
import threading
import time
count=1
class KissThread(threading.Thread):
def run(self):
global count
print "Thread # %s: Pretending to do stuff" %count
count +=1
time.sleep(2)
print "done with stuff"
for t in range(5):
KissThread().start()$ python thread1.py
Thread # 1: Pretending to do stuff
Thread # 2: Pretending to do stuff
Thread # 3: Pretending to do stuff
Thread # 4: Pretending to do stuff
Thread # 5: Pretending to do stuff
done with stuff
done with stuff
done with stuff
done with stuff
done with stuffcommon.py
#/usr/bin/python
import subprocess
import time
IP_LIST=['Google.com',
'yahoo.com',
'yelp.com',
'amazone.com',
'freebase.com',
'baidu.com',
'taobao.com']
cmd_stub='ping -c 5 %s'
def do_ping(addr):
print time.asctime(),"DOING PING FOR", addr
cmd=cmd_stub % (addr,)
return subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)nothread.py
#/usr/bin/python
from common import IP_LIST,do_ping
import time
z=[]
for ip in IP_LIST:
p=do_ping(ip)
z.append((p,ip))
for p,ip in z:
print time.asctime(),"WAITING FOR",ip
p.wait()
print time.asctime(),ip,"RETURNED",p.returncode$ python nothread.py
Mon Aug 11 15:01:58 2014 DOING PING FOR google.com
Mon Aug 11 15:01:58 2014 DOING PING FOR yahoo.com
Mon Aug 11 15:01:58 2014 DOING PING FOR yelp.com
Mon Aug 11 15:01:58 2014 DOING PING FOR amazone.com
Mon Aug 11 15:01:58 2014 DOING PING FOR freebase.com
Mon Aug 11 15:01:58 2014 DOING PING FOR baidu.com
Mon Aug 11 15:01:58 2014 DOING PING FOR taobao.com
Mon Aug 11 15:01:58 2014 WAITING FOR google.com
Mon Aug 11 15:02:03 2014 google.com RETURNED 0
Mon Aug 11 15:02:03 2014 WAITING FOR yahoo.com
Mon Aug 11 15:02:03 2014 yahoo.com RETURNED 0
Mon Aug 11 15:02:03 2014 WAITING FOR yelp.com
Mon Aug 11 15:02:03 2014 yelp.com RETURNED 0
Mon Aug 11 15:02:03 2014 WAITING FOR amazone.com
Mon Aug 11 15:02:12 2014 amazone.com RETURNED 1
Mon Aug 11 15:02:12 2014 WAITING FOR freebase.com
Mon Aug 11 15:02:12 2014 freebase.com RETURNED 0
Mon Aug 11 15:02:12 2014 WAITING FOR baidu.com
Mon Aug 11 15:02:12 2014 baidu.com RETURNED 0
Mon Aug 11 15:02:12 2014 WAITING FOR taobao.com
Mon Aug 11 15:02:12 2014 taobao.com RETURNED 0ping_thread_basic.py
#/usr/bin/python
from threading import Thread
import subprocess
from Queue import Queue
num_threads=3
queue=Queue()
ips=["10.10.41.254","10.10.41.20","10.10.41.42","10.10.41.15"]
def pinger(i,q):
while True:
ip=q.get()
print "Thread %s: Pinging %s" %(i,ip)
ret=subprocess.call("ping -c 1 %s" %ip,shell=True,stdout=open('/dev/null','w'),stderr=subprocess.STDOUT)
if ret==0:
print "%s: is alive" %ip
else:
print "%s: did not respond" %ip
q.task_done()
for i in range(num_threads):
worker=Thread(target=pinger,args=(i,queue))
worker.setDaemon(True)
worker.start()
for ip in ips:
queue.put(ip)
print "Main Thread Waiting"
queue.join()
print "Done"$ python2.6 ping_thread_basic.py
Main Thread Waiting
Thread 0: Pinging 10.10.41.254
Thread 2: Pinging 10.10.41.20
Thread 1: Pinging 10.10.41.42
10.10.41.254: is alive
Thread 0: Pinging 10.10.41.15
10.10.41.42: is alive
10.10.41.20: is alive
10.10.41.15: is alive
Done相关文章