Executing scripts in a few milliseconds with MicroPython

Command execution time can become important in a number of applications. Commands executed in command-line completion need to execute in less then 100ms or users will perceive a delay. In Shell scripting one might want to execute commands repeatedly in a for loop and fast execution times makes this more feasible.

Python is a very powerful language but has a much slower startup time compared to other interpreted languages like Perl, Lua and Bash. It can take up to 10 times longer to startup then some of these other languages.

MicroPython was written as a lean implementation of Python 3 with a small subset of the standard library mainly intended to run on microcontrollers. But it happily runs on Unix systems with excellent startup performance, making it an ideal candidate for implementing certain time sensitive commands.

Transcript

1. Executing scripts in milliseconds with MicroPython Marwan AlSabbagh
2. In 2006 I fell in love
3. With Python
4. We did everything together... Command line scripting Web applications Graphical applications Network programing Coroutines
5. Until one fateful day. Something horrible happened.
6. CONKY
8. • Over 300 built-in widgets • Supports custom scripts • I wanted 5 of them • Running each second • 60*60*24*5 = 432,000 Conky Setup
9. Python 15% Bash 0.5% vs CPU Usage
10. • Text editor lint check on save Flake8 takes >300ms • Bash completion pip completion can take >1s Beyond Conky
11. • Elapsed time • Clock cycles Measure Script Execution
12. add.py print(1 + 1) add.sh echo $((1 + 1)) add.pl print 1 + 1 . "\n"; add.lua print(1 + 1) add.awk BEGIN {print 1+1; exit} Test scripts
13. $ time python3.5 add.py 2 real 0m0.048s user 0m0.044s sys 0m0.000s Bash time command
14. $ perf stat -r 10 /usr/bin/python3.5 /opt/py/add.py Performance counter stats (10 runs): 17.590596 task-clock (msec)# 0.987 CPUs utilized 0 context-switches # 0.017 K/sec 0 cpu-migrations # 0.000 K/sec 916 page-faults # 0.052 M/sec 50,158,374 cycles # 2.851 GHz 11,392,485 branches # 647.646 M/sec 519,939 branch-misses # 4.56% of all branches 0.017830390 seconds time elapsed ( +- 0.97% ) Linux Performance Tool
15. January 15, 2017 ESP8266 80 MHz CPU MicroPython
16. MicroPython • Lean implementation of Python 3 • Includes selection of core Python libraries • Supports Unix platforms • usocket (e.g. HTTP client/server) • uos (has uos.system) • ujson, ure, utime, uzlib
17. Execution Time
18. CPU Cycles
19. Part 2 pip bash completion
20. $ time python -c 'import pip' real 0m1.065s user 0m1.016s sys 0m0.044s pip built-in completion • Usability needs < 100ms • Does not complete package names
21. export PIP_NO_INDEX=1 export PIP_FIND_LINKS="http://localhost/wheels/" export PIP_TRUSTED_HOST='localhost' python3 -m virtualenv -p python3 /opt/venv/main Demo Details • Nginx hosting over 400 wheels • Auto complete sub-commands and packages • live list of packages on every complete (over http) • Completion includes package name and versions
22. _pip_completion() { compopt -o filenames COMPREPLY=($(compgen -W "$(pipcomp.py $COMP_CWORD)" -- \ "${COMP_WORDS[COMP_CWORD]}")) } complete -F _pip_completion pip bashrc
23. #!/usr/bin/env micropython import urequests, uos, sys def get_packages(url): html = urequests.get(url).text wheels = [i for i in html.split('"') if i.endswith('.whl')] return ['{0}=={1}'.format(*i.split('-')) for i in wheels] def main(): if sys.argv[1] == '1': print('install uninstall freeze list show search help') if sys.argv[1] == '2': url = uos.getenv('PIP_FIND_LINKS') packages = get_packages(url) print('\n'.join(packages)) if __name__ == "__main__": main() pipcomp.py
24. $ perf stat -r 10 /bin/micropython /opt/py/pipcomp.py 1 0.003858786 seconds time elapsed ( +- 5.86% ) pipcomp Performance Sub-command completion: 3.9 ms $ perf stat -r 10 /bin/micropython /opt/py/pipcomp.py 2 0.011441601 seconds time elapsed ( +- 8.77% ) Package name completion: 11.4 ms
25. Live Demo!
26. The End