Beyond the pip

Making sense of the dependency jungle

Transcript

1. Beyond Beyond the pip the pip
2. Disclaimer: this is not about Disclaimer: this is not about pip (in particular). pip (in particular).
4. [Introduction slide]
6. It's the year 2019 It's the year 2019 are we supposed to write our are we supposed to write our own code now?? own code now??
7. NOPE. NOPE. BUT DOUBT. EVERYTHING. BUT DOUBT. EVERYTHING.
9. The good The good
10. The good The good Multitude of libraries
11. The good The good Multitude of libraries Ease of use
12. The good The good Multitude of libraries Ease of use Open source software
13. The bad The bad
14. The bad The bad Poorly written & documented
15. The bad The bad Poorly written & documented Not optimized
16. The bad The bad Poorly written & documented Not optimized Might go missing
17. The ugly The ugly
18. The ugly The ugly Security issues
19. The ugly The ugly Security issues Malicious activity (typosquatting happening)
20. The ugly The ugly Security issues Malicious activity (typosquatting happening) Dependency hell
21. Review time! Review time!
22. Is the library... Is the library...
23. Is the library... Is the library... mature? mature?
24. Is the library... Is the library... mature? mature? used in commercial products? used in commercial products?
25. Is the library... Is the library... mature? mature? used in commercial products? used in commercial products? backed up by other organizations? backed up by other organizations?
26. Is the library... Is the library...
27. Is the library... Is the library... used all through your project? used all through your project?
28. Is the library... Is the library... used all through your project? used all through your project? heavily relying on other libraries? heavily relying on other libraries?
29. Improve the library Improve the library Contribute to open source! Contribute to open source!
30. Use a pattern... Use a pattern... wrap it up! (the code, that is) wrap it up! (the code, that is) from external_dependency import something class SomeExternalDependencyClient: def __init__(self, credentials, name): self.client = something.Client(credentials, name) def get_items(self, ids): self.client.get(ids) def send_items(self, item_list): self.client.batch_insert(item_list)
31. The standard library is The standard library is awesome! awesome! Good resources Good resources Module of the Week by Doug Hellmann PyTricks by Dan Bader
32. You have You have ... ... magic methods magic methods class Hasher(object): def __init__(self, algorithm): self.algorithm = algorithm def __call__(self, file): hash = self.algorithm() with open(file, 'rb') as f: for chunk in iter(lambda: f.read(4096), ''): hash.update(chunk) return hash.hexdigest() md5 = Hasher(hashlib.md5) sha1 = Hasher(hashlib.sha1) md5(somefile)
33. Awesome Awesome decorators decorators from functools import wraps def retry(count=5, exc_type=Exception): def decorator(func): @wraps(func) def result(*args, **kwargs): last_exc = None for _ in range(count): try: return func(*args, **kwargs) except exc_type as e: last_exc = e raise last_exc return result return decorator @retry def might_fail(): # some code here pass
34. class Cache: def __init__(self): self.memo = {} def store(self, fn): def wrapper(*args): if args not in self.memo: self.memo[args] = fn(*args) return self.memo[args] return wrapper def clear(self): self.memo.clear() cache = Cache() @cache.store def somefct(): return expensive_call() cache.clear()
35. Powerful Powerful containers containers >>> from collections import Counter >>> colors = ['blue', 'red', 'blue', 'yellow', 'blue', 'red'] >>> counter = Counter(colors) Counter({'blue': 3, 'red': 2, 'yellow': 1}) >>> counter.most_common()[0][0] 'blue' >>> Point = collections.namedtuple('Point', 'x y') >>> p = Point(1, y=2) Point(x=1, y=2) >>> p.x 1 >>> getattr(p, 'y') 2 >>> p._fields ('x', 'y')
36. and so many MORE! and so many MORE! utilities utilities powerful powerful speci c operations speci c operations dev tools: dev tools: , , , , date & time handling date & time handling regex regex os os pydoc pydoc unittest unittest pdb pdb
37. I still think libraries are cool, I still think libraries are cool, okay? okay?
38. This is it. Thank you!