Solving Python cold start issues in cloud infrastructure

Transcript

1. Copyright © 2019, Oracle. All rights reserved. | 1 Solving Python cold start issues in cloud infrastructure Denis Makogon Principal member of technical stuff Oracle Corp. June 14 2019 London
2. Copyright © 2019, Oracle. All rights reserved. | 2
3. Copyright © 2019, Oracle. All rights reserved. | What the “cold start” is? 3
4. Copyright © 2019, Oracle. All rights reserved. | When? Why? Why should I care? 4
5. Copyright © 2019, Oracle. All rights reserved. | When? - when you start an interpreter 5
6. Copyright © 2019, Oracle. All rights reserved. | Why? - because maintainers don’t care a lot 6
7. Copyright © 2019, Oracle. All rights reserved. | Why should I care? 7
8. Copyright © 2019, Oracle. All rights reserved. | Python 3.7 and a new importing system 8 import importlib
9. Copyright © 2019, Oracle. All rights reserved. | Python 3.7 and imports profiling 9 PYTHONPROFILEIMPORTTIME=1
10. Copyright © 2019, Oracle. All rights reserved. | Sad story first 10 Implement an async HTTP 1.1 web server over UDS that must start within 3 seconds
11. Copyright © 2019, Oracle. All rights reserved. | 11
12. Copyright © 2019, Oracle. All rights reserved. | …oh, wait and … 12 …that must start within 3 seconds on the following constraints: docker run --rm -ti \ --cpu-quota=10000 \ --cpu-period=10000 \ --cpu-shares=128 \ --memory=128m \ —kernel-memory=128m \ --memory-swap=128m
13. Copyright © 2019, Oracle. All rights reserved. | Sad story of async HTTP 13 import time: 95296 | 4301835 | aiohttp import time: 958 | 3702439 | sanic
14. Copyright © 2019, Oracle. All rights reserved. | Sad story of homegrown frameworks 14 import time: 1668 | 2194676 | asyncio import time: 90390 | 2596378 |async_http.app
15. Copyright © 2019, Oracle. All rights reserved. | What causes Python code to be that slow? 15
16. Copyright © 2019, Oracle. All rights reserved. | 1. Stop blaming interpreter, there’s only one slow thing - your code. 16
17. Copyright © 2019, Oracle. All rights reserved. | 2. Coding in __init__.py 17
18. Copyright © 2019, Oracle. All rights reserved. | 3. Stop turning sys.path into a dump of all code that you might not use, ever. 18 Keep in mind, import lookup speed depends on the number of entities in the sys.path
19. Copyright © 2019, Oracle. All rights reserved. | 19 from sanic import Sanic from sanic.response import json app = Sanic() @app.route('/') async def test(request): return json({'hello': 'world'}) if __name__ == '__main__': app.run(host='0.0.0.0', port=8000)
20. Copyright © 2019, Oracle. All rights reserved. | import time: 954 | 954 | aiofiles.base import time: 852 | 852 | aiofiles.threadpool.utils import time: 2020 | 3824 | aiofiles.threadpool.binary import time: 967 | 967 | aiofiles.threadpool.text import time: 1072 | 5863 | aiofiles.threadpool import time: 1377 | 7240 | aiofiles import time: 981 | 981 | sanic.testing import time: 975 | 975 | websockets.exceptions import time: 812 | 812 | websockets.extensions import time: 731 | 731 | websockets.speedups import time: 1895 | 2625 | websockets.framing import time: 2756 | 6193 | websockets.extensions.permessage_deflate import time: 1987 | 1987 | websockets.headers import time: 87046 | 89033 | websockets.handshake import time: 95210 | 95210 | http.client import time: 3160 | 3160 | websockets.version import time: 5609 | 103978 | websockets.http import time: 1005 | 1005 | websockets.compatibility import time: 528 | 528 | websockets.py36 import time: 1360 | 1888 | websockets.py36.protocol import time: 93972 | 96863 | websockets.protocol import time: 938 | 938 | websockets.uri import time: 1502 | 299478 | websockets.client import time: 1154 | 1154 | websockets.server import time: 521 | 301152 | websockets import time: 828 | 301980 | sanic.websocket 20
21. Copyright © 2019, Oracle. All rights reserved. | aiofiles websockets testing fixtures 21 from sanic import Sanic from sanic.response import json app = Sanic() @app.route('/') async def test(request): return json({'hello': 'world'}) if __name__ == '__main__': app.run(host='0.0.0.0', port=8000)
22. Copyright © 2019, Oracle. All rights reserved. | How to make your code work faster? 22
23. Copyright © 2019, Oracle. All rights reserved. | 23
24. Copyright © 2019, Oracle. All rights reserved. | 24
25. Copyright © 2019, Oracle. All rights reserved. | 25
26. Copyright © 2019, Oracle. All rights reserved. | Gate your imports profiling on each commit 26 We all got used to work with codecov, right? Then do the same for each package/module/ code-sample
27. Copyright © 2019, Oracle. All rights reserved. | sys.path must remain clean out of unnecessary code 27 If you are a library maintainer, please stay away from any implicit unnecessary imports unless a user will explicitly ask for them
28. Copyright © 2019, Oracle. All rights reserved. | Always measure your code performance on low constraints 28 10% of a single core with 64 to 128Mb RAM should be enough to see if your application will survive
29. Copyright © 2019, Oracle. All rights reserved. | Use “importlib” 29 If it’s hard to stay away from redundant imports - delay them!
30. Copyright © 2019, Oracle. All rights reserved. | 30
31. Copyright © 2019, Oracle. All rights reserved. | 31 import a <—— module execution return a <—— returns existing one
32. Copyright © 2019, Oracle. All rights reserved. | >>> def blah(): ... import collections ... return collections ... >>> blah() 32 >>> import functools >>> >>> @functools.lru_cache() ... def blah(): ... import collections ... return collections ... >>> blah()
33. Copyright © 2019, Oracle. All rights reserved. | 33 At scale, LRU-cached import works faster than typical sys.path lookup
34. Copyright © 2019, Oracle. All rights reserved. | 34
35. Copyright © 2019, Oracle. All rights reserved. | 35 At most of the times, LRU-based imports are faster up to 39%
36. Copyright © 2019, Oracle. All rights reserved. | 36 6514 LRU-based imports were faster than the rest of imports
37. Copyright © 2019, Oracle. All rights reserved. | 37
38. Copyright © 2019, Oracle. All rights reserved. | 38
39. Copyright © 2019, Oracle. All rights reserved. | 39 Questions?