Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Interesting Things With Python

Xuanyi
September 05, 2013

Interesting Things With Python

This was originally written for my talk in SyPy September 2013. It was originally written as HTML slides. I've converted it to PDF for SpeakerDeck as I have had the time now

Xuanyi

September 05, 2013
Tweet

More Decks by Xuanyi

Other Decks in Programming

Transcript

  1. Zip tricks •  [x amount of] things that I think

    is interesting (YMMV) •  Hacks in Python •  Based on my working experience •  Do NOT put these code into production
  2. The Basics These are things I have found helpful: 1. 

    In place value swapping 2.  Zip tricks 3.  Splat 4.  Enumerate 5.  SimpleHTTPServer
  3. In-place value swapping >>> a = 1! >>> b =

    2! >>> a, b = b, a! >>> print(a, b)! 2, 1
  4. While we're on that topic... >>> a, b, c, d

    = 1, 2, 3, 4! >>> a! 1! >>> b! 2
  5. Zip tricks Zip is really more useful than expected. Take

    this 2 x 4 matrix! 1 2 3 4! 5 6 7 8! Represented in Python: m = [[1,2,3,4], [5,6,7,8]]! Make it a 4 x 2 matrix
  6. Zip tricks >>> zip(*m)! [(1, 5), (2, 6), (3, 7),

    (4, 8)]! It essentially is this: 1 5! 2 6! 3 7! 4 8
  7. Splat Operator •  That's the name in Perl. •  There

    isn't a name in Python. •  Hard to find information about for beginners. •  I thought it was pointers when I started. •  It unpacks arguments (equivalent to Go's ...something and something...)
  8. Splat Operator def fn(arg1=None, arg2=None, arg3=None, arg4=None):! print(arg1, arg2, arg3,

    arg4)! ...! >>> l = [1,2,3,4]! >>> fn(l)! ([1, 2, 3, 4], None, None, None)! >>> fn(*l)! (1, 2, 3, 4)
  9. Splat Operator Now you can take any amount of arguments!

    ! def fn(*args):! print args! ! >>> l = [1,2,3,4]! >>> fn(l)! ([1, 2, 3, 4])! >>> fn(*l)! (1, 2, 3, 4)
  10. Enumerate enumerate() is a generator that returns the index and

    the value of a iterable. Instead of: for i in range(0, len(l)):! v = l[i]! # do something with the index and value! Do this: for i, v in enumerate(l):! # do something with index and value
  11. Enumerate One more trick with enumerate: Start at any number!

    for i, v in enumerate(l, start=2):! # do something with index and value! Do this, and you will eliminate many off-by- one errors.
  12. Bonus: Pydoc I love Godoc. Then I discovered Godoc was

    inspired by Pydoc pydoc -p 6060! Now go to http://localhost:6060 for documentation!
  13. Bit Twiddling What does this code do? def fn(x): !

    return (3435973837*x)>>35! And this? def fn2(y): ! return ((1717986919*y)>>34)-(y>>31)
  14. Bit Twiddling Answer: They both divide x by 10 Can

    you figure out what the magic numbers 3435973837 and 1717986919 are?
  15. Arrays There is a difference between lists and arrays. Python

    has a built-in array library. import array
  16. When are arrays useful? •  When you have a list,

    and you know the list will only ever be integers, characters or bytes. •  When your list does not change much. •  When you don't need to do math (if you want to do math, use Numpy's Array).
  17. Guido's Example Compare these two functions: import array! def f7(l):!

    return array.array('B', list).tostring()! ! def f1(l):! string = ""! for item in l:! string = string + chr(item)! return string! The one with arrays is 15x faster than the other function.
  18. Caveat Don't be seduced, however. Here's the performance of sorting

    an array vs sorting a list of 1 million integers (naive sorts) ! timeit.timeit("sorted(l)", setup="from __main__ import l", number=100)! 54.20818901062012! ! ! timeit.timeit("sorted(a)", setup="from __main__ import a", number=100)! 93.52262997627258
  19. Heap The data structure, not the memory term. Very useful

    as a priority queue. Here's Guido's code to sort 1 million integers with an array and a heap queue: ! ! import sys, array, tempfile, heapq! assert array.array('i').itemsize == 4! ! def intsfromfile(f):! while True:! a = array.array('i')! a.fromstring(f.read(4000))! if not a:! break! for x in a:! yield x! ! iters = []! while True:! a = array.array('i')! a.fromstring(sys.stdin.buffer.read(40000))! if not a:! break! f = tempfile.TemporaryFile()! array.array('i', sorted(a)).tofile(f)! f.seek(0)! iters.append(intsfromfile(f))! ! a = array.array('i')! for x in heapq.merge(*iters):! a.append(x)! if len(a) >= 1000:! a.tofile(sys.stdout.buffer)! del a[:]! if a:! a.tofile(sys.stdout.buffer)
  20. Sorting Sorting is a matter of context. timeit.timeit("sorted(l)", setup="from __main__

    import l", number=100)! 54.20818901062012! ! timeit.timeit("sorted(a)", setup="from __main__ import a", number=100)! 93.52262997627258! ! timeit.timeit("customSortBA(l)", setup="from __main__ import l, customSort", number=100)! 40.209283113479614! ! timeit.timeit("customSortPP(a)", setup="from __main__ import a, customSort", number=100)! 41.524425983428955! HOW?
  21. Sorting Python uses something called TimSort as it's default sort.

    ! import random! import array! ! l = [x for x in range(0, 1000000)] # generate 1 million integers! ! # shuffle a few times! random.shuffle(l)! random.shuffle(l)! random.shuffle(l)! ! # create an array! a = array.array('i', l)
  22. Sorting By  now  it  should  be  something  obvious  about  the

      integers  to  be  sorted:   –  There  are  no  repeats   –  There  is  a  range  (specifically  from  0-­‐1000000)   –  Sound  familiar?   Sounds  like  database  indices?   A  real  life  example   [insert  example  about  url  classificaHon]  
  23. Sorting The Sorting Algorithm from bitarray import bitarray! ! def

    customSortBA(iterable):! ba = bitarray('0' * len(iterable))! for i in iterable:! ba[i] = True! return [j for j,k in enumerate(ba) if k]! Want to do it in pure Python? def customSortPP(iterable):! a = [None for i in range(0, len(iterable))]! for i in iterable:! a[i] = True! return [j for j,k in enumerate(ba) if k]!
  24. Sorting •  Lots  of  opHmizaHons  can  be  done.   • 

    Limited  in  applicaHon  though.   •  O(3N).  (Technically  O(2M  +  N),  but  in  this  case  M  =  N)   •  But  Timsort  has  a  best  case  of  O(N)  too!   •  Not  a  general  purpose  sort  
  25. Try this! def fn1():! print("Hello World")! ! def fn2():! print("EXTERMINATE!

    EXTERMINATE!")! ! fn1.__code__ = fn2.__code__! ! exec(fn1.__code__)  
  26. Q is fun! import q! ! @q! def fn():! q("In

    fn()")! print("Hello, I'm the Doctor. Run.")! ! fn()! Then chewxy@chewxy-TARDIS-40:~$ tail -f /tmp/q! 19.5s <module>: <function fn>! 93.1s fn: 'In fn()'  
  27. When in doubt, inspect! import inspect! import pprint! ! import

    random! ! pprint.pprint(inspect.getmembers(random))