SDE python

1. S源码分析- python脚本

  工作中好多的工具都是python写得， 比如编码得风格规范，编码得设计规范，文档整合等功能；python 也出现在linux系统中，你会发现有py结尾得脚本文件； 因此有必要进行python得学习，能读懂python。
 

1.1 python 3.0

import sys; x = 'foo'; sys.stdout.write(x + '12')
foo125

差异 5是返回值

 

>>> count = 0
>>> while (count < 9):
...    print ('The count is:', count)
...    count = count + 1
...
The count is: 0
The count is: 1

语句快要多打一个空格。

 

fruits = [‘banana’, ‘apple’, ‘mango’]
for index in range(len(fruits)):
… print (‘Current fruit :’, fruits[index])
…
Current fruit : banana
Current fruit : apple
Current fruit : mango

这个打印字符串直接拼接起来的

 

>>> var1 = 'Hello World!'
>>> var2 = "Python Programming"
>>>
>>> print ("var1[0]: ", var1[0])
var1[0]:  H
>>> print ("var2[1:5]: ", var2[1:5])
var2[1:5]:  ytho

字符串访问区间是前开后闭

 

>>> list = ['physics', 'chemistry', 1997, 2000]
>>>
>>> print (list)
['physics', 'chemistry', 1997, 2000]
>>> del list[2]
>>> print ("After deleting value at index 2 : ", list)
After deleting value at index 2 :  ['physics', 'chemistry', 2000]
>>>

这个数组是中括号， 可以写不同的数据类型 ， 修改如c，删除用函数del

 

>>> tup1 = ('physics', 'chemistry', 1997, 2000)
>>> tup2 = (1, 2, 3, 4, 5 )
>>> tup3 = "a", "b", "c", "d"
>>> tup1
('physics', 'chemistry', 1997, 2000)

括号是元组， 中括号是列表， 元组相当于c语言的常量

 

>>> dict = {'Name': 'Zara', 'Age': 7, 'Class': 'First'}
>>>
>>> print ("dict['Name']: ", dict['Name'])
dict['Name']:  Zara
>>> print ("dict['Age']: ", dict['Age'])
dict['Age']:  7

键值对 中间用冒号隔开

 

>>> dict = {'Name': 'Zara', 'Age': 7, 'Name': 'Manni'}
>>>
>>> print ("dict['Name']: ", dict['Name'])
dict['Name']:  Manni
>>> dict
{'Name': 'Manni', 'Age': 7}
>>>

键值对 重复，将会被覆盖

>>> import time;  # This is required to include time module.
>>>
>>> ticks = time.time()
>>> print ("Number of ticks since 12:00am, January 1, 1970:", ticks)
Number of ticks since 12:00am, January 1, 1970: 1646499161.4908187

这是1970 年产生的时间

 

>>> def printme( str ):
...    "This prints a passed string into this function"
...    print (str)
...    return
...
>>> printme(80)
80
>>>

函数有def 开始 然后由冒号结束

 

def changeme( mylist ):
...    "This changes a passed list into this function"
...    print ("Values inside the function before change: ", mylist)
...    mylist[2]=50
...    print ("Values inside the function after change: ", mylist)
...    return
...
>>> # Now you can call changeme function
>>> mylist = [10,20,30]
>>> changeme( mylist )
Values inside the function before change:  [10, 20, 30]
Values inside the function after change:  [10, 20, 50]
>>> print ("Values outside the function: ", mylist)
Values outside the function:  [10, 20, 50]


>>> #!/usr/bin/python3
>>>
>>> # Function definition is here
>>> def changeme( mylist ):
...    "This changes a passed list into this function"
...    mylist = [1,2,3,4] # This would assi new reference in mylist
...    print ("Values inside the function: ", mylist)
...    return
...
>>> # Now you can call changeme function
>>> mylist = [10,20,30]
>>> changeme( mylist )
Values inside the function:  [1, 2, 3, 4]
>>> print ("Values outside the function: ", mylist)
Values outside the function:  [10, 20, 30]

引用修改了成员 ， 值传递 都是重新定义了链表

 

>>> # Function definition is here
>>> def printinfo( name, age = 35 ):
...    "This prints a passed info into this function"
...    print ("Name: ", name)
...    print ("Age ", age)
...    return
...
>>> # Now you can call printinfo function
>>> printinfo( age=50, name="miki" )
Name:  miki
Age  50
>>> printinfo( name="miki" )
Name:  miki
Age  35

比c 语言高级了， 可以像c++ 整默认参数

 

>>> # Function definition is here
>>> def printinfo( arg1, *vartuple ):
...    "This prints a variable passed arguments"
...    print ("Output is: ")
...    print (arg1)
...    for var in vartuple:
...       print (var)
...    return
...
>>> # Now you can call printinfo function
>>> printinfo( 10 )
Output is:
10
>>> printinfo( 70, 60, 50 )
Output is:
70
60
50

可变长参数， 是用*开始得非关键子组成得变量。

 

>>> # Function definition is here
>>> sum = lambda arg1, arg2: arg1 + arg2
>>>
>>>
>>>
>>> # Now you can call sum as a function
>>> print ("Value of total : ", sum( 10, 20 ))
Value of total :  30
>>> print ("Value of total : ", sum( 20, 20 ))
Value of total :  40
>>>

匿名函数 有返回值 ，相当于宏。

 

>>> total = 0 # This is global variable.
>>> # Function definition is here
>>> def sum( arg1, arg2 ):
...    # Add both the parameters and return them."
...    total = arg1 + arg2; # Here total is local variable.
...    print ("Inside the function local total : ", total)
...    return total
...
>>> # Now you can call sum function
>>> sum( 10, 20 )
Inside the function local total :  30
30
>>> print ("Outside the function global total : ", total )
Outside the function global total :  0

还有作用域。

 

vim support.py
 def print_func( par ):
   print ("Hello : ", par)
   return

#!/usr/bin/python3

# Import module support
import support

# Now you can call defined function that module as follows



>>> total = 0 # This is global variable.
>>> # Function definition is here
>>> def sum( arg1, arg2 ):
...    # Add both the parameters and return them."
...    global total
...    total = arg1 + arg2; # Here total is local variable.
...    print ("Inside the function local total : ", total)
...    return total
...
>>> # Now you can call sum function
>>> sum( 10, 20 )
Inside the function local total :  30
30
>>> print ("Outside the function global total : ", total )
Outside the function global total :  30

>>> #!/usr/bin/python3
>>>
>>> # Import module support
>>> import support
>>>
>>> # Now you can call defined function that module as follows
>>> support.print_func("Zara")
Hello :  Zara

引入文件， 然后文件名作为一个对象来处理了

 

>>> # Open a file
>>> fo = open("foo.txt", "wb")
>>> print ("Name of the file: ", fo.name)
Name of the file:  foo.txt
>>> print ("Closed or not : ", fo.closed)
Closed or not :  False
>>> print ("Opening mode : ", fo.mode)
Opening mode :  wb
>>> fo.close()
>>>
>>>
>>> fo.closed
True

文件句柄关闭了， 还可以查状态

 

>>> def KelvinToFahrenheit(Temperature):
...     assert (Temperature >= 0),"Colder than absolute zero!"
...     return ((Temperature-273)*1.8)+32
...
>>> print (KelvinToFahrenheit(273))
32.0
>>> print (int(KelvinToFahrenheit(505.78)))
451
>>> print (KelvinToFahrenheit(-5))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in KelvinToFahrenheit
AssertionError: Colder than absolute zero!
>>> try:
...    fh = open("testfile", "w")
...    fh.write("This is my test file for exception handling!!")
... except IOError:
...    print ("Error: can\'t find file or read data")
... else:
...    print ("Written content in the file successfully")
...    fh.close()
...
45
Written content in the file successfully

异常处理是c++ 里面得。

 

1.2 python 并发

import urllib.request
>>> import time
>>> ts = time.time()
>>> req = urllib.request.urlopen('http://www.yiibai.com')
>>> pageHtml = req.read()
>>> te = time.time()
>>> print("Page Fetching Time : {} Seconds".format (te-ts))
Page Fetching Time : 0.9485414028167725 Seconds

程序运行时间统计， 程序就是在时间上和空间上得优化 。

时间快，让一个人干得活多人干。 形成工厂流水线工作。 让每个工人没有休息。

 

import _thread
import time

def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print ("%s: %s" % ( threadName, time.ctime(time.time()) ))

try:
   _thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   _thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print ("Error: unable to start thread")
while 1:
   pass

pass 是个占位符号， 表示什么也不做。

 

import threading
import time
exitFlag = 0

class myThread (threading.Thread):
   def __init__(self, threadID, name, counter):
      threading.Thread.__init__(self)
      self.threadID = threadID
      self.name = name
      self.counter = counter
   def run(self):
      print ("Starting " + self.name)
      print_time(self.name, self.counter, 5)
      print ("Exiting " + self.name)
def print_time(threadName, delay, counter):
   while counter:
      if exitFlag:
         threadName.exit()
      time.sleep(delay)
      print ("%s: %s" % (threadName, time.ctime(time.time())))
      counter -= 1

thread1 = myThread(1, "Thread-1", 1)
thread2 = myThread(2, "Thread-2", 2)

thread1.start()
thread2.start()
thread1.join()
thread2.join()
print ("Exiting Main Thread")

Starting Thread-1
Starting Thread-2
Thread-1: Sun Mar 6 10:21:00 2022
Thread-2: Sun Mar 6 10:21:01 2022
Thread-1: Sun Mar 6 10:21:01 2022
Thread-1: Sun Mar 6 10:21:02 2022
Thread-2: Sun Mar 6 10:21:03 2022
Thread-1: Sun Mar 6 10:21:03 2022
Thread-1: Sun Mar 6 10:21:04 2022
Exiting Thread-1
Thread-2: Sun Mar 6 10:21:05 2022
Thread-2: Sun Mar 6 10:21:07 2022
Thread-2: Sun Mar 6 10:21:09 2022
Exiting Thread-2
Exiting Main Thread

有构造函数， 显示得调用了自己。 C++第一个参数是自己隐藏着。

class myThread (threading.Thread): 这一句应该是继承 。
 

import threading
import time
import random
def Thread_execution(i):
   print("Execution of Thread {} started\n".format(i))
   sleepTime = random.randint(1,4)
   time.sleep(sleepTime)
   print("Execution of Thread {} finished".format(i))
for i in range(4):
   thread = threading.Thread(target=Thread_execution, args=(i,))
   thread.start()
   print("Active Threads:" , threading.enumerate())

Execution of Thread 0 started

Active Threads: [<_MainThread(MainThread, started 139688883672896)>, <Thread(Thread-1, started 139688872834816)>]
Execution of Thread 1 started

Active Threads: [<_MainThread(MainThread, started 139688883672896)>, <Thread(Thread-1, started 139688872834816)>, <Thread(Thread-2, started 139688864442112)>]
Execution of Thread 2 started

Active Threads: [<_MainThread(MainThread, started 139688883672896)>, <Thread(Thread-1, started 139688872834816)>, <Thread(Thread-2, started 139688864442112)>, <Thread(Thread-3, started 139688856049408)>]
Execution of Thread 3 started

Active Threads: [<_MainThread(MainThread, started 139688883672896)>, <Thread(Thread-1, started 139688872834816)>, <Thread(Thread-2, started 139688864442112)>, <Thread(Thread-3, started 139688856049408)>, <Thread(Thread-4, started 139688643917568)>]
Execution of Thread 0 finished
Execution of Thread 2 finished
Execution of Thread 3 finished
Execution of Thread 1 finished

线程启动需要一定时间， 主线程先打印。

threading.enumerate() 枚举线程
 

import threading

x = 0

def increment_global():

   global x
   x += 1

def taskofThread(lock):

   for _ in range(50000):
      lock.acquire()
      increment_global()
      lock.release()

def main():
   global x
   x = 0

   lock = threading.Lock()
   t1 = threading.Thread(target = taskofThread, args = (lock,))
   t2 = threading.Thread(target = taskofThread, args = (lock,))

   t1.start()
   t2.start()

   t1.join()
   t2.join()

if __name__ == "__main__":
   for i in range(5):
      main()
      print("x = {1} after Iteration {0}".format(i,x))

lock = threading.Lock() 线程锁

 

import threading
import queue
import random
import time
def myqueue(queue):
   while not queue.empty():
      item = queue.get()
      if item is not None:
         print("{} removed {} from the queue".format(threading.current_thread(), item))
         queue.task_done()
         time.sleep(1)
q = queue.PriorityQueue()
for i in range(5):
   q.put(i,1)

for i in range(5):
   q.put(i,1)

threads = []
for i in range(2):
   thread = threading.Thread(target=myqueue, args=(q,))
   thread.start()
   threads.append(thread)
for thread in threads:
   thread.join()

<Thread(Thread-1, started 139845510756096)> removed 0 from the queue
<Thread(Thread-2, started 139845502363392)> removed 0 from the queue
<Thread(Thread-1, started 139845510756096)> removed 1 from the queue
<Thread(Thread-2, started 139845502363392)> removed 1 from the queue
<Thread(Thread-1, started 139845510756096)> removed 2 from the queue
<Thread(Thread-2, started 139845502363392)> removed 2 from the queue
<Thread(Thread-1, started 139845510756096)> removed 3 from the queue
<Thread(Thread-2, started 139845502363392)> removed 3 from the queue
<Thread(Thread-1, started 139845510756096)> removed 4 from the queue
<Thread(Thread-2, started 139845502363392)> removed 4 from the queue

这个例子就很nice ， 从队列中取数据， 然后来运行。

 

import pdb
a = "aaa"
b = "bbb"
c = "ccc"
final = a + b + c
pdb.set_trace()
print (final)

pdb.set_trace() 设置断点

python -m cProfile test.py

Myfunction took 7.0075013637542725 seconds to complete its execution.
767 function calls (740 primitive calls) in 7.009 seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function)
6 0.000 0.000 0.000 0.000

from concurrent.futures import ThreadPoolExecutor
from time import sleep
def task(message):
   sleep(2)
   return message

def main():
   executor = ThreadPoolExecutor(5)
   future = executor.submit(task, ("Completed"))
   print(future.done())
   sleep(2)
   print(future.done())
   print(future.result())
if __name__ == '__main__':
   main()

提交任务跟线程池处理

 

from concurrent.futures import ThreadPoolExecutor
from concurrent.futures import as_completed
values = [2,3,4,5]
def square(n):
   return n * n
def main():
   with ThreadPoolExecutor(max_workers = 3) as executor:
      results = executor.map(square, values)
      for result in results:
         print(result)
if __name__ == '__main__':
   main()

线程池， 还有定义别名。

map函数用于将square()函数应用于values数组中的每个值。
 

import time
import concurrent.futures

value = [8000000, 7000000]

def counting(n):
   start = time.time()
   while n > 0:
      n -= 1
   return time.time() - start

def main():
   start = time.time()
   with concurrent.futures.ProcessPoolExecutor() as executor:
      for number, time_taken in zip(value, executor.map(counting, value)):
         print('Start: {} Time taken: {}'.format(number, time_taken))
   print('Total time taken: {}'.format(time.time() - start))

if __name__ == '__main__':
   main()





import time
import concurrent.futures

value = [8000000, 7000000]

def counting(n):
   start = time.time()
   while n > 0:
      n -= 1
   return time.time() - start

def main():
   start = time.time()
   with concurrent.futures.ThreadPoolExecutor() as executor:
      for number, time_taken in zip(value, executor.map(counting, value)):
         print('Start: {} Time taken: {}'.format(number, time_taken))
      print('Total time taken: {}'.format(time.time() - start))

if __name__ == '__main__':
   main()


python test.py
Start: 8000000 Time taken: 0.3391857147216797
Start: 7000000 Time taken: 0.2977735996246338
Total time taken: 0.35625338554382324
tan@tan:~/test$
tan@tan:~/test$
tan@tan:~/test$
tan@tan:~/test$
tan@tan:~/test$ make
python test.py
Start: 8000000 Time taken: 0.6328039169311523
Start: 7000000 Time taken: 0.5814099311828613
Total time taken: 0.6336588859558105

进程比线程的时间快一半

在空间允许的情况 得多开进程
 

import multiprocessing
import time

def nondaemonProcess():
   print("starting my Process")
   time.sleep(8)
   print("ending my Process")
def daemonProcess():
   while True:
      print("Hello")
      time.sleep(2)
      
if __name__ == '__main__':
   nondaemonProcess = multiprocessing.Process(target = nondaemonProcess)
   daemonProcess = multiprocessing.Process(target = daemonProcess)
   daemonProcess.daemon = True
   nondaemonProcess.daemon = False
   daemonProcess.start()
   nondaemonProcess.start()

python test.py
starting my Process
ending my Process

守护线程是是不会打印在终端的 。

 

import multiprocessing

def print_records(records):
   for record in records:
      print("Name: {0}\nScore: {1}\n".format(record[0], record[1]))

def insert_record(record, records):
   records.append(record)
   print("A New record is added\n")

if __name__ == '__main__':
   with multiprocessing.Manager() as manager:

      records = manager.list([('Computers', 1), ('Histoty', 5), ('Hindi',9)])
      new_record = ('English', 3)

      p1 = multiprocessing.Process(target = insert_record, args = (new_record, records))
      p2 = multiprocessing.Process(target = print_records, args = (records,))
      p1.start()
      p1.join()
      p2.start()
      p2.join()

A New record is added

Name: Computers
Score: 1

Name: Histoty
Score: 5

Name: Hindi
Score: 9

Name: English
Score: 3

实现了一个插入和打印

 

 

2. 总结

这次主要学习在语法上， 和线程进程编程上，学过linux对这方面知识点理解起来会更容易些。

下周准备对文字处理和设计模式进行了解。