带关键字的格式化
>>>
>>> print "Hello %(name)s !" % {'name':'James'}
Hello James !
>>>
>>> print "Hello {name} !".format(name="James")
Hello James !
>>>
使用dict.__missing__() 避免出现KeyError
If a subclass of dict defines a method __missing__() and key is not present,
the d[key] operation calls that method with the key(key as argument).
The d[key] operation then returns or raises whatever is returned or raised by the __missing__(key) call.
>>>
>>> class Counter(dict):
... def __missing__(self, key):
... return 0
...
>>> c = Counter()
>>> print c['num']
0
>>> c['num'] += 1
>>> print c['num']
1
>>>
>>> c
{'num': 1}
>>>
__getattr__ 调用默认方法
>>>
>>> class A(object):
... def __init__(self,num):
... self.num = num
... print 'init...'
... def mydefault(self, *args, **kwargs):
... print 'default func...'
... print args
... print kwargs
... def __getattr__(self,name):
... print 'No %(name)s found, goto default...' % {'name':name}
... return self.mydefault
...
>>> a1 = A(9)
init...
>>> a1.fn1()
No fn1 found, goto default...
default func...
()
{}
>>> a1.fn2(1,2)
No fn2 found, goto default...
default func...
(1, 2)
{}
>>> a1.fn3(name='standby',age=18)
No fn3 found, goto default...
default func...
()
{'age': 18, 'name': 'standby'}
>>>
>>>
obj.xxx = aaa 触发类的 __setattr__
obj.xxx 触发类的 __getattr__
obj['xxx'] = 'vvv' 触发类的 __setitem__
obj['xxx'] 触发类的 __getitem__
with app1.app_context(): 触发 __enter__ __exit__
__new__ 和 __init__ 的执行顺序
>>>
>>> class B(object):
... def fn(self):
... print 'B fn'
... def __init__(self):
... print "B INIT"
...
>>> class A(object):
... def fn(self):
... print 'A fn'
... def __new__(cls,a):
... print "NEW", a
... if a>10:
... return super(A, cls).__new__(cls)
... return B()
... def __init__(self,a):
... print "INIT", a
...
>>>
>>> a1 = A(5)
NEW 5
B INIT
>>> a1.fn()
B fn
>>> a2=A(20)
NEW 20
INIT 20
>>> a2.fn()
A fn
>>>
类继承之 __class__
>>>
>>> class A(object):
... def show(self):
... print 'base show'
...
>>> class B(A):
... def show(self):
... print 'derived show'
...
>>> obj = B()
>>> obj.show()
derived show
>>>
>>> obj.__class__
<class '__main__.B'>
>>>
>>> obj.__class__ = A
>>> obj.__class__
<class '__main__.A'>
>>> obj.show()
base show
>>>
对象方法 __call__
>>>
>>> class A(object):
... def obj_func(self, *args, **kwargs):
... print args
... print kwargs
... def __call__(self, *args, **kwargs):
... print 'Object method ...'
... return self.obj_func(*args, **kwargs)
...
>>> a1=A()
>>> a1(9,name='standby',city='beijing')
Object method ...
(9,)
{'city': 'beijing', 'name': 'standby'}
>>>
补充：
>>>
>>> class test(object):
... def __init__(self, value):
... self.x = value
... def __call__(self, value):
... return self.x * value
...
>>> a = test(4)
>>> print a(5)
20
>>> 补充：
- 什么后面可以加括号？（只有4种表现形式）
- 函数 执行函数
- 类 执行类的__init__方法
- 方法 obj.func
- 对象 前提：类里有 __call__ 方法
obj() 直接执行类的 __call__方法
关于类的继承
>>>
>>> class Parent(object):
... x = 1
...
>>> class Child1(Parent):
... pass
...
>>> class Child2(Parent):
... pass
...
>>> Child1.x = 2
>>> Parent.x = 3
>>> print Parent.x, Child1.x, Child2.x
3 2 3
>>>
类属性和对象属性
类属性
>>>
>>> class Student:
... score = []
...
>>> stu1 = Student()
>>> stu2 = Student()
>>> stu1.score.append(99)
>>> stu1.score.append(96)
>>> stu2.score.append(98)
>>>
>>>
>>> stu2.score
[99, 96, 98]
>>>
>>>
对象属性
>>>
>>> class Student:
... def __init__(self):;
... self.lst = []
...
>>> stu1 = Student()
>>> stu2 = Student()
>>>
>>>
>>> stu1.lst.append(1)
>>> stu1.lst.append(2)
>>> stu2.lst.append(9)
>>>
>>> stu1.lst
[1, 2]
>>>
>>> stu2.lst
[9]
>>>
一行代码实现列表偶数位加3后求和
>>> a = [1,2,3,4,5,6]
>>> [item+3 for item in a if a.index(item)%2==0]
[4, 6, 8]
>>> result = sum([item+3 for item in a if a.index(item)%2==0])
>>> result
18
>>>
字符串连接
>>>
>>> name = 'hi ' 'standby' ' !'
>>> name
'hi standby !'
>>>
Python解释器中的 '_'
_ 即Python解释器上一次返回的值
>>>
>>> range(5)
[0, 1, 2, 3, 4]
>>> _
[0, 1, 2, 3, 4]
>>>
嵌套列表推导式
>>>
>>> [(i, j) for i in range(3) for j in range(i)]
[(1, 0), (2, 0), (2, 1)]
>>>
Python3 中的unpack
>>>
>>> first, second, *rest, last = range(10)
>>> first
0
>>> second
1
>>> last
9
>>> rest
[2, 3, 4, 5, 6, 7, 8]
>>>
关于__setattr__ __getattr__ __getitem__ __setitem__ 参考：http://www.cnblogs.com/standby/p/7045718.html
Python把常用数字缓存在内存里 *****
>>>
>>> a = 1
>>> b = 1
>>> a is b
True
>>>
>>>
>>> a = 256
>>> b = 256
>>> a is b
True
>>>
>>> a = 257
>>> b = 257
>>> a is b
False
>>>
>>> a = 300
>>> b = 300
>>> a is b
False
>>> 注意：在[-5,256]之间的数字用在内存中的id号是相同的；
Python为了提高运行效率而将这些常用数字缓存到内存里了，所以他们的id号是相同的；
另外，对a,b,c,....等的赋值也只是一种引用而已：
>>>
>>> id(9)
10183288
>>> num = 9
>>> id(num)
10183288
>>>
Python对于短字符串会使用同一个空间，但是对于长字符串会重新开辟空间
>>>
>>> a = 'I love PythonSomething!'
>>> b = 'I love PythonSomething!'
>>> c = [1, 2, 3]
>>> d = [1, 2, 3]
>>>
>>> a is b
False
>>> c is d
False
>>>
>>> id(a)
139848068316272
>>> id(b)
139848068316336
>>> id(c)
139848068310152
>>> id(d)
139848068309936
>>>
字符串 * 操作
>>>
>>> def func(a):
... a = a + '2'
... a = a*2
... return a
...
>>>
>>> func("hello")
'hello2hello2'
>>>
Python浮点数比较
>>>
>>> 0.1
0.10000000000000001
>>> 0.2
0.20000000000000001
>>> 0.1 + 0.2
0.30000000000000004
>>>
>>> 0.3
0.29999999999999999
>>>
>>> 0.1 + 0.2 == 0.3
False
>>>
Python里的 '~' 取反
>>>
>>> 5
5
>>> ~5
-6
>>> ~~5
5
>>> ~~~5
-6
>>> ~~~~5
5
>>>
~5 即对5取反，得到的是 -6 ， 为什么？
参考：https://www.cnblogs.com/piperck/p/5829867.html 和 http://blog.csdn.net/u011080472/article/details/51280919
- 原码就是符号位加上真值的绝对值；
- 反码的表示方法是：正数的反码就是其本身；负数的反码是在其原码的基础上, 符号位不变，其余各个位取反；
- 补码的表示方式是：正数的补码就是其本身；负数的补码是在其原码的基础上, 符号位不变, 其余各位取反, 最后+1 (即在反码的基础上+1)；
| 真值 | 原码 | 反码 | 补码 | |
| 5 | +000 0101 | 0000 0101 | 0000 0101 | 0000 0101 |
| -5 | -000 0101 | 1000 0101 | 1111 1010 | 1111 1011 |
对5取反即对 0000 0101 取反， 得到 1111 1010，那这个值的十进制是多少呢？
因为 负数在计算机中是以补码形式表示的， 所以实际上就是求哪个值的补码是 1111 1010，
按照上面的规则反向计算：
1111 1010 减1 得到其反码表示：1111 1001
在保证符号位不变，其余各位取反：1000 0110 就是该值的原码，对应真值就是 -000 0110 ，对应十进制就是 -6 。
那么对 -6 取反，得到的是多少呢？
对-6取反即对 -6 的补码取反，就是对1111 1010取反，得到 0000 0101，很明显是一个正数。
而正数原码==正数反码==正数补码，所以该值的原码就是 0000 0101，真值就是 +000 0101，对应十进制就是 5。
bool()
>>>
>>> bool('True')
True
>>> bool('False')
True
>>> bool('')
False
>>> bool()
False
>>>
>>>
>>> bool(1)
True
>>> bool(0)
False
>>>
等价于
>>>
>>> True==False==False
False
>>>
>>> True==False and False==False
False
>>>
>>>
>>> 1 in [0,1]
True
>>> 1 in [0,1] == True
False
>>>
>>> (1 in [0,1]) == True
True
>>>
>>> 1 in ([0,1] == True)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: argument of type 'bool' is not iterable
>>>
>>>
>>>
>>> (1 in [0,1]) and ([0,1] == True)
False
>>>Note that comparisons, membership tests, and identity tests,
all have the same precedence and have a left-to-right chaining feature as described in the Comparisons section.
参考：https://stackoverflow.com/questions/31354429/why-is-true-is-false-false-false-in-python
while 结合 break
>>>
>>> i = 0
>>> while i < 5:
... print(i)
... i += 1
... if i == 3:
... break
... else:
... print(0)
...
0
1
2
>>>
set给list去重
>>>
>>> nums = set([1,1,2,3,3,3,4])
>>>
>>> nums
set([1, 2, 3, 4])
>>>
>>> type(nums)
<type 'set'>
>>>
>>> len(nums)
4
>>>
>>>
>>> li = list(nums)
>>> li
[1, 2, 3, 4]
>>>
>>> type(li)
<type 'list'>
>>>
函数是第一类对象（First-Class Object）
在 Python 中万物皆为对象，函数也不例外，
函数作为对象可以赋值给一个变量、可以作为元素添加到集合对象中、
可作为参数值传递给其它函数，还可以当做函数的返回值，这些特性就是第一类对象所特有的。
函数可以嵌套，函数中里面嵌套的函数不能在函数外面访问，只能是在函数内部使用：
def get_length(text):
def clean(t):
return t[1:]
res = clean(text)
return len(res)
print(get_length('standby'))
Python里的高阶函数
函数接受一个或多个函数作为输入或者函数输出（返回）的值是函数时，我们称这样的函数为高阶函数。
Python内置函数中，典型的高阶函数是 map 函数，map 接受一个函数和一个迭代对象作为参数，
调用 map 时，依次迭代把迭代对象的元素作为参数调用该函数。
def foo(text):
return len(text)
li = map(foo, ["the","zen","of","python"])
print(li) # <map object at 0x0000000001119FD0>
li = list(li)
print(li) # [3, 3, 2, 6]
lambda应用场景
- 函数式编程
有一个列表： list1 = [3,5,-4,-1,0,-2,-6]，需要按照每个元素的绝对值升序排序，如何做？
# 使用lambda的方式
>>>
>>> list1
[3, 5, -4, -1, 0, -2, -6]
>>>
>>> sorted(list1, key=lambda i : abs(i))
[0, -1, -2, 3, -4, 5, -6]
>>>
# 不使用lambda的方式
>>>
>>> def foo(x):
... return abs(x)
...
>>> sorted(list1, key=foo)
[0, -1, -2, 3, -4, 5, -6]
>>>
如何把一个字典按照value进行排序？
>>>
>>> dic = {'a': 9, 'c': 3, 'b': 1, 'd': 7, 'f': 12}
>>> dic
{'a': 9, 'f': 12, 'c': 3, 'd': 7, 'b': 1}
>>>
>>> from collections import Iterable
>>> isinstance(dic.items(),Iterable)
True
>>>
>>> dic.items()
dict_items([('a', 9), ('f', 12), ('c', 3), ('d', 7), ('b', 1)])
>>>
>>> sorted(dic.items(), key=lambda x:x[1])
[('b', 1), ('c', 3), ('d', 7), ('a', 9), ('f', 12)]
>>>
- 闭包
# 不用lambda的方式
>>>
>>> def my_add(n):
... def wrapper(x):
... return x+n
... return wrapper
...
>>> add_3 = my_add(3)
>>> add_3(7)
10
>>>
# 使用lambda的方式
>>>
>>> def my_add(n):
... return lambda x:x+n
...
>>> add_3 = my_add(3)
>>> add_3(7)
10
>>>
方法和函数的区别
#!/usr/bin/python3
from types import MethodType,FunctionType
class Foo(object):
def __init__(self):
pass
def func(self):
print('func...')
obj = Foo()
print(obj.func) # 自动传递 self
# <bound method Foo.func of <__main__.Foo object at 0x7f86121505f8>>
print(Foo.func)
# <function Foo.func at 0x7f861214e488>
print(isinstance(obj.func,MethodType)) # True
print(isinstance(obj.func,FunctionType)) # False
print(isinstance(Foo.func,MethodType)) # False
print(isinstance(Foo.func,FunctionType)) # True
时间戳转换成年月日时分秒
>>> from datetime import datetime
>>> ts=1531123200
>>> date_str = datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
>>> date_str
'2018-07-09 16:00:00'
>>> In [11]: import time
In [12]: ts = int(time.time())
In [13]: ts
Out[13]: 1559549982
In [14]: time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(ts))
Out[14]: '2019-06-03 16:19:42'
In [15]: 年月日时分秒转换成时间戳
>>> date_str
'2018-07-09 16:00:00'
>>>
>>> date_struct=time.strptime(date_str,'%Y-%m-%d %H:%M:%S')
>>> date_struct
time.struct_time(tm_year=2018, tm_mon=7, tm_mday=9, tm_hour=16, tm_min=0, tm_sec=0, tm_wday=0, tm_yday=190, tm_isdst=-1)
>>>
>>> int(time.mktime(date_struct))
1531123200
>>> In [16]: d
Out[16]: '2019-06-03 16:19:42'
In [17]: time.mktime(time.strptime(d, "%Y-%m-%d %H:%M:%S"))
Out[17]: 1559549982.0
In [18]: 获取当前月初和月末时间戳
>>> import time
>>> import datetime
>>>
>>> start_st = datetime.datetime.now()
>>> start_st
datetime.datetime(2018, 7, 17, 16, 45, 39, 95228)
>>> startts = int(time.mktime((start_st.year, start_st.month-1, 1, 0, 0, 0, 0, 0, -1)))
>>> startts # 当前月初时间戳
1527782400
>>>
>>> stopts = int(time.mktime((start_st.year, start_st.month, 1, 0, 0, 0, 0, 0, -1))) - 1
>>> stopts
1530374399 # 当前月末时间戳
>>>IP地址转换成数字，从而进行比较，适用于地址库
>>> ips = ['8.8.8.8','202.106.0.20']
>>>
>>> map(lambda ip:sum([256**j*int(i) for j,i in enumerate(ip.split('.')[::-1])]), ips)
[134744072, 3395944468]
>>>
使用yield逐行读取多个文件并合并
#!/usr/bin/python2.7
def get_line_by_yield():
with open('total.txt','r') as rf_total, open('extra.txt','r') as rf_extra:
for line in rf_total:
extra = rf_extra.readline()
lst = line.strip().split() + extra.strip().split()
yield lst
with open('new_total.txt','w') as wf:
for lst in get_line_by_yield():
wf.write('%s\n' % ''.join(map(lambda i: str(i).rjust(20), lst)))逐行读取
def read_line(path):
with open(path,'r') as rf:
for line in rf:
yield line
检查进程是否 running
In [16]: import signal
In [17]: from os import kill
In [18]: kill(17335, 0) # 17335 是进程ID，第二个参数传0/signal.SIG_DFL 返回值若是None则表示正在运行
In [19]: kill(17335, 15) # 给进程传递15/signal.SIGTERM，即终止该进程
In [20]: kill(17335, 0) # 再次检查发现该进程已经不再running，则raise一个OSError
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-20-cbb7c9624124> in <module>()
----> 1 kill(17335, 0)
OSError: [Errno 3] No such process
In [21]: 
1 In [12]: import signal 2 3 In [13]: signal.SIGKILL 4 Out[13]: 9 5 6 In [14]: signal.SIGTERM 7 Out[14]: 15 8 9 In [15]: signal.__dict__.items() 10 Out[15]: 11 [('SIGHUP', 1), 12 ('SIG_DFL', 0), 13 ('SIGSYS', 31), 14 ('SIGQUIT', 3), 15 ('SIGUSR1', 10), 16 ('SIGFPE', 8), 17 ('SIGPWR', 30), 18 ('SIGTSTP', 20), 19 ('ITIMER_REAL', 0L), 20 ('SIGCHLD', 17), 21 ('SIGCONT', 18), 22 ('SIGIOT', 6), 23 ('SIGBUS', 7), 24 ('SIGXCPU', 24), 25 ('SIGPROF', 27), 26 ('SIGCLD', 17), 27 ('SIGUSR2', 12), 28 ('default_int_handler', <function signal.default_int_handler>), 29 ('pause', <function signal.pause>), 30 ('SIGKILL', 9), 31 ('NSIG', 65), 32 ('SIGTRAP', 5), 33 ('SIGINT', 2), 34 ('SIGIO', 29), 35 ('__package__', None), 36 ('getsignal', <function signal.getsignal>), 37 ('SIGILL', 4), 38 ('SIGPOLL', 29), 39 ('SIGABRT', 6), 40 ('SIGALRM', 14), 41 ('__doc__', 42 'This module provides mechanisms to use signal handlers in Python.\n\nFunctions:\n\nalarm() -- cause SIGALRM after a specified time [Unix only]\nsetitimer() -- cause a signal (described below) after a specified\n float time and the timer may restart then [Unix only]\ngetitimer() -- get current value of timer [Unix only]\nsignal() -- set the action for a given signal\ngetsignal() -- get the signal action for a given signal\npause() -- wait until a signal arrives [Unix only]\ndefault_int_handler() -- default SIGINT handler\n\nsignal constants:\nSIG_DFL -- used to refer to the system default handler\nSIG_IGN -- used to ignore the signal\nNSIG -- number of defined signals\nSIGINT, SIGTERM, etc. -- signal numbers\n\nitimer constants:\nITIMER_REAL -- decrements in real time, and delivers SIGALRM upon\n expiration\nITIMER_VIRTUAL -- decrements only when the process is executing,\n and delivers SIGVTALRM upon expiration\nITIMER_PROF -- decrements both when the process is executing and\n when the system is executing on behalf of the process.\n Coupled with ITIMER_VIRTUAL, this timer is usually\n used to profile the time spent by the application\n in user and kernel space. SIGPROF is delivered upon\n expiration.\n\n\n*** IMPORTANT NOTICE ***\nA signal handler function is called with two arguments:\nthe first is the signal number, the second is the interrupted stack frame.'), 43 ('SIG_IGN', 1), 44 ('getitimer', <function signal.getitimer>), 45 ('SIGURG', 23), 46 ('SIGPIPE', 13), 47 ('SIGWINCH', 28), 48 ('__name__', 'signal'), 49 ('SIGTERM', 15), 50 ('SIGVTALRM', 26), 51 ('ITIMER_PROF', 2L), 52 ('SIGRTMIN', 34), 53 ('SIGRTMAX', 64), 54 ('ITIMER_VIRTUAL', 1L), 55 ('set_wakeup_fd', <function signal.set_wakeup_fd>), 56 ('setitimer', <function signal.setitimer>), 57 ('signal', <function signal.signal>), 58 ('SIGSEGV', 11), 59 ('siginterrupt', <function signal.siginterrupt>), 60 ('SIGXFSZ', 25), 61 ('SIGTTIN', 21), 62 ('SIGSTOP', 19), 63 ('ItimerError', signal.ItimerError), 64 ('SIGTTOU', 22), 65 ('alarm', <function signal.alarm>)] 66 67 In [16]: dict((k, v) for v, k in reversed(sorted(signal.__dict__.items())) 68 ...: if v.startswith('SIG') and not v.startswith('SIG_')) 69 Out[16]: 70 {1: 'SIGHUP', 71 2: 'SIGINT', 72 3: 'SIGQUIT', 73 4: 'SIGILL', 74 5: 'SIGTRAP', 75 6: 'SIGABRT', 76 7: 'SIGBUS', 77 8: 'SIGFPE', 78 9: 'SIGKILL', 79 10: 'SIGUSR1', 80 11: 'SIGSEGV', 81 12: 'SIGUSR2', 82 13: 'SIGPIPE', 83 14: 'SIGALRM', 84 15: 'SIGTERM', 85 17: 'SIGCHLD', 86 18: 'SIGCONT', 87 19: 'SIGSTOP', 88 20: 'SIGTSTP', 89 21: 'SIGTTIN', 90 22: 'SIGTTOU', 91 23: 'SIGURG', 92 24: 'SIGXCPU', 93 25: 'SIGXFSZ', 94 26: 'SIGVTALRM', 95 27: 'SIGPROF', 96 28: 'SIGWINCH', 97 29: 'SIGIO', 98 30: 'SIGPWR', 99 31: 'SIGSYS', 100 34: 'SIGRTMIN', 101 64: 'SIGRTMAX'} 102 103 In [17]:
1 def check_if_process_is_alive(self): 2 try: 3 kill(self.current_pid, 0) 4 kill(self.parent_pid, 0) 5 except: 6 # do something... 7 exit(0)
多指标排序问题
In [26]: lst = [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
In [27]: import operator
In [28]: sorted(lst, key=operator.itemgetter(1))
Out[28]: [('john', 'A', 15), ('jane', 'B', 12), ('dave', 'B', 10)]
In [29]: sorted(lst, key=operator.itemgetter(1,2)) # 先根据第二个域排序，然后再根据第三个域排序
Out[29]: [('john', 'A', 15), ('dave', 'B', 10), ('jane', 'B', 12)]
In [30]:
两个纯数字列表元素个数相等，按序相加求和，得到一个新的列表
length = len(lst1)
lst = reduce(lambda x,y:[x[i]+y[i] for i in range(length)], [lst1,lst2], [0]*length)或者直接使用numpy.array
补充reduce+lambda合并多个列表
In [15]: lst = [[1,2,3],['a','c'],['hello','world'],[2,2,2,111]]
In [16]: reduce(lambda x,y: x+y, lst)
Out[16]: [1, 2, 3, 'a', 'c', 'hello', 'world', 2, 2, 2, 111]
In [17]: 扩展示例1：
lst= [[{u'timestamp': 1545214320, u'value': 222842128},
{u'timestamp': 1545214380, u'value': 224080288},
{u'timestamp': 1545214440, u'value': 253812496},
{u'timestamp': 1545214500, u'value': 295170240},
{u'timestamp': 1545214560, u'value': 221196224},
{u'timestamp': 1545214620, u'value': 252992096}],
[{u'timestamp': 1545214320, u'value': 228121600},
{u'timestamp': 1545214380, u'value': 225682656},
{u'timestamp': 1545214440, u'value': 256428064},
{u'timestamp': 1545214500, u'value': 292691424},
{u'timestamp': 1545214560, u'value': 241462336},
{u'timestamp': 1545214620, u'value': 250864528}],
[{u'timestamp': 1545214320, u'value': 232334304},
{u'timestamp': 1545214380, u'value': 230452032},
{u'timestamp': 1545214440, u'value': 246094880},
{u'timestamp': 1545214500, u'value': 260281088},
{u'timestamp': 1545214560, u'value': 233277120},
{u'timestamp': 1545214620, u'value': 258726192}]]
# 要求：把上述列表合并
# 方法一：使用Python内置函数
In [83]: reduce(lambda x,y:[ { 'timestamp':x[i]['timestamp'], 'value':x[i]['value']+y[i]['value'] } for i in range(6) ], a)
Out[83]:
[{'timestamp': 1545214320, 'value': 683298032},
{'timestamp': 1545214380, 'value': 680214976},
{'timestamp': 1545214440, 'value': 756335440},
{'timestamp': 1545214500, 'value': 848142752},
{'timestamp': 1545214560, 'value': 695935680},
{'timestamp': 1545214620, 'value': 762582816}]
In [84]:
# 方法二：笨办法
In [87]: b = a.pop(0)
In [88]:
In [88]: for i in a:
...: for idx in range(len(i)):
...: b[idx]['value'] += i[idx]['value']
...:
In [89]: b
Out[89]:
[{u'timestamp': 1545214320, u'value': 683298032},
{u'timestamp': 1545214380, u'value': 680214976},
{u'timestamp': 1545214440, u'value': 756335440},
{u'timestamp': 1545214500, u'value': 848142752},
{u'timestamp': 1545214560, u'value': 695935680},
{u'timestamp': 1545214620, u'value': 762582816}]
In [90]: 扩展示例2：
In [48]: a
Out[48]:
[{'A078102C949EC2AB': [1, 2, 3, 4]},
{'457D37015E77700E': [2, 2, 2, 2]},
{'5095060C4552175D': [3, 3, 3, 3]}]
In [49]: reduce(lambda x,y: dict(x.items()+y.items()), a)
Out[49]:
{'457D37015E77700E': [2, 2, 2, 2],
'5095060C4552175D': [3, 3, 3, 3],
'A078102C949EC2AB': [1, 2, 3, 4]}
In [50]:
awk指定字段求和
awk -F '=' '{count+=$4} END{print count}' file.log
找出在列表1中但不在列表2中的元素
list(set(lst1).difference(set(lst2)))
解析url，把字段转换成字典
# 方法一
In [5]: url = 'index?name=standby&age=18&city=beijing'
In [6]: parameter = url.split('?')[1]
In [7]: parameter
Out[7]: 'name=standby&age=18&city=beijing'
In [8]: dict(map(lambda x:x.split('='),parameter.split('&')))
Out[8]: {'age': '18', 'city': 'beijing', 'name': 'standby'}
In [9]:
# 方法二
In [9]: import urlparse
In [10]: query = urlparse.urlparse(url).query
In [11]: query
Out[11]: 'name=standby&age=18&city=beijing'
In [12]: dict([(k, v[0]) for k, v in urlparse.parse_qs(query).items()])
Out[12]: {'age': '18', 'city': 'beijing', 'name': 'standby'}
In [13]:
fromkeys使用的陷阱
In [1]: a = dict.fromkeys(['k1','k2','k3'],{})
In [2]: a
Out[2]: {'k1': {}, 'k2': {}, 'k3': {}}
In [3]: a['k1']['2018-10-10'] = 'hi'
In [4]: a
Out[4]:
{'k1': {'2018-10-10': 'hi'},
'k2': {'2018-10-10': 'hi'},
'k3': {'2018-10-10': 'hi'}}
In [5]:
In [5]: a = dict.fromkeys(['k1','k2','k3'],[])
In [6]: a['k1'].append(999)
In [7]: a
Out[7]: {'k1': [999], 'k2': [999], 'k3': [999]}
In [8]:
In [8]: a = dict.fromkeys(['k1','k2','k3'],0)
In [9]: a['k1'] += 9
In [10]: a
Out[10]: {'k1': 9, 'k2': 0, 'k3': 0}
In [11]:
dateutil库解析时间对象
In [76]: import datetime
In [77]: datetime.datetime.strptime('2019-04-10','%Y-%m-%d')
Out[77]: datetime.datetime(2019, 4, 10, 0, 0)
In [78]: import dateutil
In [79]: dateutil.parser.parse('2019-04-10')
Out[79]: datetime.datetime(2019, 4, 10, 0, 0)
In [80]: dateutil.parser.parse('2019/04/10')
Out[80]: datetime.datetime(2019, 4, 10, 0, 0)
In [81]: dateutil.parser.parse('04/10/2019')
Out[81]: datetime.datetime(2019, 4, 10, 0, 0)
In [82]: dateutil.parser.parse('2019-Apr-10')
Out[82]: datetime.datetime(2019, 4, 10, 0, 0)
In [83]:
合并多个字典
# Python2.7
# 这种方式对资源的一种浪费
# 注意这种方式在Python3中会报错：TypeError: unsupported operand type(s) for +: 'dict_items' and 'dict_items'
In [7]: lst
Out[7]:
[{'k1': [1, 1, 1, 1, 1, 1]},
{'k3': [3, 3, 3, 4, 4, 4]},
{'k5': [5, 5, 5, 6, 6, 6]}]
In [8]: reduce(lambda x,y: dict(x.items()+y.items()), lst)
Out[8]: {'k1': [1, 1, 1, 1, 1, 1], 'k3': [3, 3, 3, 4, 4, 4], 'k5': [5, 5, 5, 6, 6, 6]}
In [9]:
# Python3.6
In [67]: lst
Out[67]:
[{'k1': [1, 1, 1, 1, 1, 1]},
{'k3': [3, 3, 3, 4, 4, 4]},
{'k5': [5, 5, 5, 6, 6, 6]}]
In [68]: reduce(lambda x,y: {**x,**y}, lst)
Out[68]: {'k1': [1, 1, 1, 1, 1, 1], 'k3': [3, 3, 3, 4, 4, 4], 'k5': [5, 5, 5, 6, 6, 6]}
In [69]:
# 另外补充两种兼容Py2和Py3的方法：
# 1. 使用字典的构造函数
reduce(lambda x,y: dict(x, **y), lst)
# 2. 笨办法
{k: v for d in lst for k, v in d.items()}
zip的反操作/unzip
In [2]: lst
Out[2]:
[[1560239100, 16],
[1560239400, 11],
[1560239700, 14],
[1560240000, 18],
[1560240300, 18],
[1560240600, 12],
[1560240900, 19],
[1560241200, 13],
[1560241500, 16],
[1560241800, 16]]
In [3]: tss,vals = [ list(tpe) for tpe in zip(*[ i for i in lst ]) ]
In [4]: tss
Out[4]:
[1560239100,
1560239400,
1560239700,
1560240000,
1560240300,
1560240600,
1560240900,
1560241200,
1560241500,
1560241800]
In [5]: vals
Out[5]: [16, 11, 14, 18, 18, 12, 19, 13, 16, 16]
In [6]:
获取前一天时间并格式化
In [17]: from datetime import timedelta, datetime
In [18]: yesterday = datetime.today() + timedelta(-1)
In [19]: yesterday.strftime('%Y%m%d')
Out[19]: '20190702'
In [20]:
时序数据列表转换位字典结构
In [87]: lst
Out[87]:
[(1562653200, 16408834),
(1562653500, 16180209),
(1562653800, 16178061),
(1562654100, 16147492),
(1562654400, 16103304),
(1562654700, 16182462),
(1562655000, 16334665),
(1562655300, 15440130),
(1562655600, 15433254),
(1562655900, 16607189)]
In [88]: { ts:value for ts,value in lst }
Out[88]:
{1562653200: 16408834,
1562653500: 16180209,
1562653800: 16178061,
1562654100: 16147492,
1562654400: 16103304,
1562654700: 16182462,
1562655000: 16334665,
1562655300: 15440130,
1562655600: 15433254,
1562655900: 16607189}
In [89]: