My journey into Python

The slides are created when I am learning Python. I have created this for better understanding of concepts in Python

Transcript

1. Journey into Python World Guided by Bowrna Prabhakaran
2. Prerequisites 1. Basic python(CPython) Note: Views here are derived from my learning in past 5 months by using Python. I have mentioned here about elegant ways in which python allows to code.
3. Plan for today 1. Iterators 2. Generators 3. Coroutines 4. Context Managers 5. Decorators 6. Knowledge sharing about GIL 7. Gotchas
4. Iterators Reading an item one by one is called iteration. You can iterate over list, dict, string, file. All these are iterable! Means you can loop around them and read your data. All iterables support iter() for x in obj: # some statement -------------- _iter = iter(obj) while 1: try: x = _iter.__next__() except StopIteration: # No more items break Behind Cover: How it works?
5. Example on Iterating over File >>> for line in open("real.txt"): print(line, end='')
6. User defined Iterators: • To implement user-defined one, you have to implement __next__() and __iter__() Example: class countdown(object): def __init__(self,start): self.start = start def __iter__(self): return countdown_iter(self.start) class countdown_iter(object): def __init__(self, count): self.count = count def __next__(self): if self.count <= 0: raise StopIteration r = self.count self.count -= 1 return r • Example use: >>> c = countdown(5) >>> for i in c: ... print(i, end=' ') >>> 5 4 3 2 1
7. Generators It is a function that produces series of value. Instead of returning a single value, each time it generates value using the yield statement. When you call your generator function, it creates a generator object
8. • Generator doesn’t need to implement __next__ () or __iter__() • You can now easily write your own iterator • Unlike iterator, generator can be iterated only once. • Yield keyword should be used in function • Generator function starts to yield value as your loop starts to run. When the for loop starts to run, generator function will run till first yield statement and return the value and as it continues it runs each time till yield. • Performs better when you have large data set. • When Generator returns, iteration stops (using conditions)
9. Generator Expressions Creating generators without `yield` statement
10. Using generator to parse a web log: • Here apache web-log, each line look like: 125.25.238.64 - - [24/Feb/2008:01:04:49 -0600] "GET /ply/bookplug.gif HTTP/1.1" 200 12382 • At any point in code, it didn’t load any temp data to memory. • Here is a sample of code, where the total bytes transferred in requests across a log is summed up.. • This can be extended to use case to filter out log based on error codes, grep for pattern and so on..
11. This is what i had to do load the 5 GB log file to pull out the traces..
12. Coroutines While generators acts like producers, Coroutine acts like consumers. Yield expression not only generates value but can receive values and act on it Can be used for listening events, publish/subscription models. def receiver(): while True: item = yield print('Got:', item) rec = receiver() next(rec) # Move to Yield rec.send(‘Hi’) >>> Got: Hi rec.send(‘Hello’) >>> Got: Hello
13. This example shows how you can use Coroutines to grep the string. At the end, invoke g.close() to terminate the coroutine Coroutines are consumers of data, while Generators produce series of data.
14. Decorators They are functions that modify the functionality of other functions. They are wrappers that allow to execute code before and after the function they decorate without modifying the actual function. In python, functions are Objects. This can help you understand how decorators work. As functions are Objects, it can be assigned. As functions are Objects, we can create function inside function. As functions are Objects, we can pass function inside function As functions are Objects, we can return function inside function.
15. Python functions are Objects!!
16. Defining function inside another function( As functions are objects)
17. Returning function in a function call
18. Passing function as argument into another function
19. Actual Decorators
20. Chain of Decorators
21. Pythonic way of arranging the Decorators
22. You could even rearrange the decorators:
23. Decorators Uses: • Logging • Caching - Django cache framework uses Decorators • Counters • Rate Limiting the requests • Payload modifiers and so on
24. Context Managers It is used to allocate and release resources. The most commonly used Context Manager is ‘With’ statement. It helps to avoid lot of boiler plate codes. You need not worry about open and closing the file. Context Managers handle them.
25. Places where Context managers suits well ... 1. Start = time.time() End = time.time() 2. file.open() file.close() 3. db. start_transaction() db.commit()
26. Implementing Context Manager as Class:
27. How about using Generators in Context Managers?
28. Knowledge sharing about GIL (Global Interpreter Lock): Things that we are going to discuss: About single thread, multiple threads and multi processing. How GIL works and about reference counting overview?
29. Single Thread
30. Multi threaded
31. Reference Counting:
32. Multi Processing:
33. Gotchas!! That nightmare bug that i made in production code
34. THANK YOU!!