Sleepy: How to suspend and resume your cpython process from inside

In this talk we'll propose and explain how it is possible suspend & resume a python process execution from within python using no external-help. Pure-python-ffi-free code. This kind of tool is useful with the proliferation of cloud services since they offer restricted execution environment in terms of time/memory and process relocation/priority is required. Besides we’ll get unexpected debugger improvements.

Transcript

1. Sleepy - スリーピー
2. About me: david weil aka: Tenuki / dave @tenukiz • I’m from Argentina • Work (1999-2013): • Some projects: – IMFish Few games for PyWeek.. • http://about.me/tenuki Sleepy – about + intro About Sleepy: • In 2009 we made Sleepy v1 with Fernando Russ [ @frussterix ] • Is our approach to answer this question: How could you run a given code in Google App Engine, without changing it?
3. About me: david weil aka: Tenuki / dave @tenukiz • I’m from Argentina???? • Work (1999-2013): • Some projects: – IMFish Few games for PyWeek.. • http://about.me/tenuki Sleepy – about + intro About Sleepy: • In 2009 we made Sleepy v1 with Fernando Russ [ @frussterix ] • Is our approach to answer this question: How could you run a given code in Google App Engine, without changing it?
4. Argentina??
5. Argentina??
6. Why would you need to suspend python? • To execute a program with a time quota • To resume a process in later .. • … or in a different machine. Enabling: – Load distribution – High availability • To inspect process state / debugging purposes • Implement “continuations” • Resume testing on hardware with no OS (check: Pausable Unittest on EFI Stackless Python by Masamitsu Murase (now at: Room 202)) • Implement application checkpointing!
7. Other approaches • At OS level (unix at least): – in shell Ctrl+Z effectively suspends the process – CryoPID “allows you to capture the state of a running process in Linux and save it to a file” • At Python VM's level (making a C-module or changing the vm) it would be possible to track every object in memory and serialize them to restore when needed.. but we are looking for a pure-python solution.
8. Is it possible in other languages? • Smalltalk → image • Lisp → worlds • C → Again check CryoPID.. however usually this requires source code changes.. • Java/JavaScript/.Net: I don’t know, but would like to, please let me know if you do! • Python? No.. Yes… Let’s see more in detail..
9. Is it possible in Python? • CPython: no..? • Jython: no idea.. • IronPython: no idea.. • Stackless: Yes! * – “One of the main features of Stackless is its ability to pickle and unpickle tasklets” • PyPy: Yes! continuations up to stackless level were added in the lasts years (check: (**) and its continulet) * http://www.stackless.com/wiki/Pickling ** http://doc.pypy.org/en/latest/stackless.html
10. Let’s focus on CPython – Sleepy idea It should be enough if we were able to save and restore every object instance instance in memory (including dependencies). We mean: – User objects: Classes and Instances which makes the application. – Modules the application uses. – Objects created or used by the Virtual-Maching • In particular: program execution state Easy, right?
11. Sleepy – serialization problems We’ll see that in general it is always the same and it happens always because of three reasons. Problem is: There are a few (important) non-serializable objects • They are (mostly) C-defined objects which: – Can’t be saved / Can’t be restored – In some cases the instances just can’t be created from python code – Or can be created but not configured – Or whose state isn’t exposed, forbidding they to be saved / restored One technique: obtain instance, configure it to make it be what you want!
12. Sleepy – serialization problems We’ll see that in general it is always the same and it happens always because of three reasons. Problem is: There are a few (important) non-serializable objects • They are (mostly) C-defined objects which: – Can’t be saved / Can’t be restored – In some cases the instances just can’t be created from python code – Or can be created but not configured – Or whose state isn’t exposed, forbidding they to be saved / restored One technique: obtain instance, configure it to make it be what you want! key concept
13. Sleepy – saving/restoring Python modules • It is not possible to serialize a module in-memory state but we can: load them & restore their state! • We must take care about module’s variables: – In some cases those variables needs to be reset to module’s value of definition, e.g: os.path.sep – However the opposite behavior is required in somecases. For example, we may be interested in keep random generator state: random._inst - random.getstate()
14. Sleepy – program execution state We defined execution state as the program call-stack and all its objects (objects reachable from the call stack). What is the call-stack? Is a list of contexts, each one related to a function call, which was already executed and is still active. Graphically: →main() →train() →BP() →runNN() →main() →train() →BP() →runNN()
15. Sleepy – program execution state We defined execution state as the program call-stack and all its objects (objects reachable from the call stack). What is the call-stack? Is a list of contexts, each one related to a function call, which was already executed and is still active. Graphically: →main() →train() →BP() →runNN() →main() →train() →BP() →runNN() Each one is a Frame
16. Sleepy – program execution state There are at least two ways to retrieve the actual call- stack: • sys._current_frames() → { threadid : [frame_X, .., frame_0 ], ..} where frame_x is a <frame_object> • (*) traceback.extract_stack() → [ frame-info-tuple-X, .., frame-info-tuple-0 ] where frame-info-tuple is: (filename, line_number, module, code_line) (*) current thread only!
17. Sleepy – the frame object The frame object have a function associated which is being executed and holds its “context”, e.g.: the “variables” accessible from that particular function execution instance. The problem is: it wasn’t mean to be “created” from python! Traceback (most recent call last): File "callstack.py", line 41, in <module> new_frame = FrameType() TypeError: cannot create 'frame' instances
18. Sleepy – the frame object However there’s a simple way we can create frame objects ! Ideas?
19. Sleepy – the frame object However there’s a simple way we can make python create frame objects for us! Ideas?
20. Sleepy – the frame object However there’s a simple way we can make python create frame objects for us! Ideas? Just call the associated function ..and the frame will be automatically created! [ This will only be useful if we can configure & control it ]
21. Sleepy – the frame object in detail f_back f_builtins f_code f_exc_traceback f_exc_type f_exc_value f_globals f_lasti f_lineno f_locals f_restricted f_trace <frame object at 0xca1d60> { } # frame’s builtin namespace <code object ..,file ".. .py", line 7> None None None { } # frame’s globals 676 # last bytecode executed 409 # source line nr !!! { } # frame’s locals !!! False None
22. Sleepy – the frame object - control The remaining question is: how can we control the frame execution?
23. Sleepy – the frame object - control How can we control the frame execution? Using the python debugger module: bdb.py. Execute function call Jump to next call Setup frame
24. Sleepy – recreate a call-stack Running the program (with the debugger) will call its entry-point function this way we’ll have the first frame created. It will contain a call to the function of the second frame, and so on.. In our example: main → train → BP → NN • In detail, we will: 1.Make python vm call a function creating its <frame f> 2.Setup frame’s variables f_locals. 3.Jump to next function(*) call position in bytecode: lastno = saved_lineno 4.Repeat as long as functions in call-stack (*) in the call-stack chain
25. Sleepy – <frame f> recreation (Pdb) r > /home/aweil/test_for.py(2)fail_pc() -> for x in xrange(100): (Pdb) l 1 B def fail_pc(): 2 → print 0 3 for x in xrange(100): 4 print 1 5 print 2 6 print 3 7 8 fail_pc() (Pdb) jump 4 *** Jump failed: can't jump into the middle of a block Python’s VM can’t jump across code blocks! Python’s VM can’t jump across code blocks! Current line X
26. Sleepy – bytecode & code blocks 1 def fail_pc(): 2 print 0 3 for x in xrange(100): 4 print 1 5 print 2 6 print 3 3 5 SETUP_LOOP 35 (to 43) 8 LOAD_GLOBAL 0 (xrange) 11 LOAD_CONST 2 (100) 14 CALL_FUNCTION 1 17 GET_ITER >> 18 FOR_ITER FOR_ITER 21 (to 42) 21 STORE_FAST 0 (x) 4 24 LOAD_CONST 3 (1) 27 PRINT_ITEM 28 PRINT_NEWLINE 5 29 LOAD_CONST 4 (2) 32 PRINT_ITEM 33 PRINT_NEWLINE 6 34 LOAD_CONST 5 (3) 37 PRINT_ITEM 38 PRINT_NEWLINE 39 JUMP_ABSOLUTE 18 >> 42 POP_BLOCK POP_BLOCK
27. Sleepy - frame.f_lineno = X int frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno) (*) Setter for f_lineno - you can set f_lineno from within a trace function in order to jump to a given line of code, subject to some restrictions. Most lines are OK to jump to because they don't make any assumptions about the state of the stack (obvious because you could remove the line and the code would still work without any stack errors), but there are some constructs that limit jumping: • Lines with an 'except' statement on them can't be jumped to, because they expect an exception to be on the top of the stack. • Lines that live in a 'finally' block can't be jumped from or to, since the END_FINALLY expects to clean up the stack after the 'try' block. • 'try'/'for'/'while' blocks can't be jumped into because the blockstack needs to be set up before their code runs, and for 'for' loops the iterator needs to be on the stack. * Link to frameobject.c#L78 setlineno() function
28. Sleepy - while • Special cases: break, continue, else. while condition: statement_1 statement_n From a few alternatives, we chosen to rewrite code to generate compatible bytecode but with no code blocks!
29. Sleepy - while :begin if not condition: goto end statement_1 .. statement_n goto begin :end • Special cases: break, continue, else. while condition: statement_1 statement_n From a few alternatives, we chosen to rewrite code to generate compatible bytecode but with no code blocks!
30. Sleepy – for for x in xrange(100): print x return x • Special cases: break, continue, else
31. Sleepy – for for x in xrange(100): print x return x _it = iter(xrange(100)) :next try: X = _it.next() except StopIteration: goto @end print x goto next :end return x • Special cases: break, continue, else
32. Sleepy – try/except deconstruction try: statement_1 ... statement_n except Except_1: block_ex_1 except Except_2, e: block_ex_2 except: bloque_ex else: block_else
33. Sleepy – try/except deconstruction try: statement_1 ... statement_n except Except_1: block_ex_1 except Except_2, e: block_ex_2 except: bloque_ex else: block_else try: statement_1 except Except_1: Temp = Except_1 except Except_2, ex: Temp = Except_2 except: Temp = True if Temp is Except_1: block_ex_1 if Temp is Except_1: block_ex_2 if Temp == True: block_ex if Temp is None: block_else
34. try: statement_n except Except_1: Temp = Except_1 except Except_2, ex: Temp = Except_2 except: Temp = True Sleepy – try/except deconstruction try: statement_1 ... statement_n except Except_1: block_ex_1 except Except_2, e: block_ex_2 except: bloque_ex else: block_else try: statement_2 except Except_1: Temp = Except_1 except Except_2, ex: Temp = Except_2 except: Temp = True if Temp is Except_1: block_ex_1 if Temp is Except_1: block_ex_2 if Temp == True: block_ex if Temp is None: block_else try: statement_1 except Except_1: Temp = Except_1 except Except_2, ex: Temp = Except_2 except: Temp = True
35. Sleepy – bytecode re-writing • Module compiler.pycodegen* includes a python compiler implemented in python: CodeGenerator! • To avoid blocks is it only necessary to overwrite: – visitBreak, visitContinue, visitWhile – visitGoto, visitTryExcept, visitFor – visitTryFinally • We also redefined visitModule to automatically load our helper module: sleepy. * Warning! This module is NOT available in Python 3 ! :-(
36. When rewriting a for loop we also need to consider the possible different kind of iterators, because they are probably implemented in C and in general, they are not persistent: >>> pickle.dumps(iter([ ])) → TypeError: can't pickle list iterator objects To overcome this, we made a function which: – Receive the itereator and acording to it – Returns a new one pure-python compatible In this way, we made the code “resumable”. Sleepy – iterators & co.
37. • We propose a few different ways to check when we want a process to suspended: – Continuous checking (which we implemented) : • Suspend condition check is call by tracing function – It is SLOW !!! – Can’t assert will be stop just when required but is the best effort – Assisted checking: • User should insert calls to suspend condition checking – Mixed 1: tracing not every line but every call – Mixed 2: replace tracing with pdb breakpoints Sleepy – control
38. Sleepy – demo Demo time!
39. • Compiler extension: – Generates block-code free bytecode! – Replaces iterator-creation with ad-hoc function call • Utilities to serialize objects & code-state • Control code execution and trigger process-dump • Utility to custom compile python code on-demand • CLI tool to start and suspend or restore execution Sleepy – components
40. Sleepy – Pros & Cons • Generated bytecode is far from optimal, eg: – A for loop for example: standard for: “SETUP_LOOP + .. + FOR_ITER” sleepy:”try: x=next(iter); except StopIteration” – No optimization is made to bytecode generated, like: 123 JUMP_ABSOLUTE 129 126 JUMP_FORWARD 0 (to 129) 129 JUMP_ABSOLUTE 21 – Execution speed seems 0.5X from std cpython bytecode • compiler.pycodegen don’t support 100% of python2.7 !? • Allows a more flexible debug letting user arbitrarily jump to any line in current function!
41. Sleepy – some notes • Although very delicate and not quite easy AST modification is possible (and turned out to be and extremely powerful) – However I haven’t found tools to perform testing nor simplify the process – Pycodegen allows to trick part of that but there’s lot more to do.. • From this on, we try to be the less cpython dependent.. – Also because there are many python flavors to chose from! • It is absolutely normal from now on that you don’t get along with C-based objects! • We had never used debugger in an application before!
42. Sleepy – Thanks!! Questions?
43. Sleepy - links • Reposity: https://bitbucket.org/tenuki/sleepy/ • Design of the CPython Compiler: http://www.python.org/dev/peps/pep-0339/ • An implementation of import importlib: http://docs.python.org/dev/library/importlib.html • Python Module of the Week: “dis”: http://www.doughellmann.com/PyMOTW/dis/ • New Import Hooks: http://www.python.org/dev/peps/pep-0302/ • The compiler module: http://docs.python.org/2/library/compiler.html • http://docs.python.org/library/imputil.html • http://docs.python.org/library/imp.html • http://www.stackless.com/wiki/Pickling • http://doc.pypy.org/en/latest/stackless.html • Artwork from: http://www.bluebison.net • Python Argentina: http://www.python.org.ar