PyCon India 2018 - Executing scripts in a few milliseconds with MicroPython

Command execution time can become important in a number of applications. Commands executed in command-line completion need to execute in less then 100ms or users will perceive a delay. In Shell scripting one might want to execute commands repeatedly in a for loop and fast execution times makes this more feasible.

Python is a very powerful language but has a much slower startup time compared to other interpreted languages like Perl, Lua and Bash. It can take up to 10 times longer to startup then some of these other languages.

MicroPython was written as a lean implementation of Python 3 with a small subset of the standard library mainly intended to run on microcontrollers. But it happily runs on Unix systems with excellent startup performance, making it an ideal candidate for implementing certain time sensitive commands.

Transcript

1. Executing scripts in milliseconds with MicroPython Marwan AlSabbagh
2. In 2006 I fell in love
3. With Python
4. We did everything together... Command line scripting Web applications Graphical applications Network programing Coroutines
5. Until one fateful day. Something horrible happened.
6. CONKY
8. • Over 300 built-in widgets • Supports custom scripts • I wanted 5 of them • Running each second • 60*60*24*5 = 432,000 Conky Setup
9. Python 15% Bash 0.5% vs CPU Usage
10. • Text editor lint check on save Flake8 takes >300ms • Bash completion pip completion can take >1s Beyond Conky
11. • Elapsed time • Clock cycles Measure Script Execution
12. add.py print(1 + 1) add.sh echo $((1 + 1)) add.pl print 1 + 1 . "\n"; add.lua print(1 + 1) add.awk BEGIN {print 1+1; exit} Test scripts
13. $ time python3.7 add.py 2 real 0m0.010s user 0m0.004s sys 0m0.007s Bash time command
14. $ perf stat -r 10 /usr/bin/python3.7 /opt/py/add.py Performance counter stats (10 runs): 9.146528 task-clock (msec) # 0.981 CPUs utilized 0 context-switches # 0.000 K/sec 0 cpu-migrations # 0.000 K/sec 804 page-faults # 0.088 M/sec 31,926,587 cycles # 3.491 GHz 39,586,183 instructions # 1.24 insn per cycle 8,926,391 branches # 975.932 M/sec 325,273 branch-misses # 3.64% of all branches 0.009324090 seconds time elapsed ( +- 0.25% ) Linux Performance Tool
15. January 15, 2017 $9 USD ESP8266 80 MHz CPU MicroPython
16. MicroPython • Lean implementation of Python 3 • Includes selection of core Python libraries • Supports Unix platforms • usocket (e.g. HTTP client/server) • uos (has uos.system) • ujson, ure, utime, uzlib
17. Hardware • Model: system76 - Galago Pro • OS: Ubuntu Linux 16.04 • CPU: 3.5 GHz i7-7500U • RAM: 16 GB • DISK: 250 GB NVMe SSD • Shipped: July-2017 Software • Python 3.7.0 • MicroPython v1.9.4 (2018-10-04) Hardware/Software Used
18. Execution Time
19. CPU Cycles
20. Part 2 pip bash completion
21. $ time python -c 'import pip' real 0m0.743s user 0m0.696s sys 0m0.048s pip built-in completion • Usability needs < 100ms • Does not complete package names
22. export PIP_NO_INDEX=1 export PIP_FIND_LINKS="http://localhost/wheels/" export PIP_TRUSTED_HOST='localhost' python3 -m virtualenv -p python3 /opt/py/venv/main Demo Details • Nginx hosting over 400 wheels • Auto complete sub-commands and packages • live list of packages on every complete (over http) • Completion includes package name and versions
23. _pip_completion() { compopt -o filenames COMPREPLY=($(compgen -W "$(pipcomp.py $COMP_CWORD)" -- \ "${COMP_WORDS[COMP_CWORD]}")) } complete -F _pip_completion pip bashrc
24. #!/usr/bin/env micropython import urequests, uos, sys def get_packages(url): html = urequests.get(url).text wheels = [i for i in html.split('"') if i.endswith('.whl')] return ['{0}=={1}'.format(*i.split('-')) for i in wheels] def main(): if sys.argv[1] == '1': print('install uninstall freeze list show search help') if sys.argv[1] == '2': url = uos.getenv('PIP_FIND_LINKS') packages = get_packages(url) print('\n'.join(packages)) if __name__ == "__main__": main() pipcomp.py
25. $ perf stat -r 10 /bin/micropython /opt/py/pipcomp.py 1 0.001258725 seconds time elapsed pipcomp Performance Sub-command completion: 1.3 ms $ perf stat -r 10 /bin/micropython /opt/py/pipcomp.py 2 0.009136741 seconds time elapsed Package name completion: 9.1 ms
26. Live Demo!
27. The End https://marwano.com