Sleepy: How to suspend and resume your cpython process from inside

In this talk we'll propose and explain how it is possible suspend & resume a python process execution from within python using no external-help. Pure-python-ffi-free code. This kind of tool is useful with the proliferation of cloud services since they offer restricted execution environment in terms of time/memory and process relocation/priority is required. Besides we’ll get unexpected debugger improvements.

Transcript

1. Sleepy - スリーピー
   

   View Slide

2. About me: david weil
   aka: Tenuki / dave
   @tenukiz
   ●
   I’m from Argentina
   ●
   Work (1999-2013):
   ●
   Some projects:
   – IMFish
   Few games for PyWeek..
   ●
   http://about.me/tenuki
   Sleepy – about + intro
   About Sleepy:
   ●
   In 2009 we made Sleepy
   v1 with Fernando Russ
   [ @frussterix ]
   ●
   Is our approach to answer
   this question:
   How could you run a given
   code in Google App Engine,
   without changing it?
   

   View Slide

3. About me: david weil
   aka: Tenuki / dave
   @tenukiz
   ●
   I’m from Argentina????
   ●
   Work (1999-2013):
   ●
   Some projects:
   – IMFish
   Few games for PyWeek..
   ●
   http://about.me/tenuki
   Sleepy – about + intro
   About Sleepy:
   ●
   In 2009 we made Sleepy
   v1 with Fernando Russ
   [ @frussterix ]
   ●
   Is our approach to answer
   this question:
   How could you run a given
   code in Google App Engine,
   without changing it?
   

   View Slide

4. Argentina??
   

   View Slide

5. Argentina??
   

   View Slide

6. Why would you need to suspend python?
   ●
   To execute a program with a time quota
   ●
   To resume a process in later ..
   ●
   … or in a different machine. Enabling:
   – Load distribution
   – High availability
   ●
   To inspect process state / debugging purposes
   ●
   Implement “continuations”
   ●
   Resume testing on hardware with no OS (check:
   Pausable Unittest on EFI Stackless Python by Masamitsu
   Murase (now at: Room 202))
   ●
   Implement application checkpointing!
   

   View Slide

7. Other approaches
   ●
   At OS level (unix at least):
   – in shell Ctrl+Z effectively suspends the process
   – CryoPID “allows you to capture the state of a running
   process in Linux and save it to a file”
   ●
   At Python VM's level (making a C-module or changing
   the vm) it would be possible to track every object in
   memory and serialize them to restore when needed.. but
   we are looking for a pure-python solution.
   

   View Slide

8. Is it possible in other languages?
   ●
   Smalltalk → image
   ●
   Lisp → worlds
   ●
   C → Again check CryoPID.. however usually this requires
   source code changes..
   ●
   Java/JavaScript/.Net: I don’t know, but would like to,
   please let me know if you do!
   ●
   Python? No.. Yes… Let’s see more in detail..
   

   View Slide

9. Is it possible in Python?
   ●
   CPython: no..?
   ●
   Jython: no idea..
   ●
   IronPython: no idea..
   ●
   Stackless: Yes! *
   – “One of the main features of Stackless is
   its ability to pickle and unpickle tasklets”
   ●
   PyPy: Yes! continuations up to stackless level
   were added in the lasts years (check: (**) and its
   continulet)
   * http://www.stackless.com/wiki/Pickling
   ** http://doc.pypy.org/en/latest/stackless.html
   

   View Slide

10. Let’s focus on CPython – Sleepy idea
    It should be enough if we were able to save and restore
    every object instance instance in memory (including
    dependencies). We mean:
    – User objects: Classes and Instances which makes the
    application.
    – Modules the application uses.
    – Objects created or used by the Virtual-Maching
    ●
    In particular: program execution state
    Easy, right?
    

    View Slide

11. Sleepy – serialization problems
    We’ll see that in general it is always the same and it
    happens always because of three reasons. Problem is:
    There are a few (important) non-serializable objects
    ●
    They are (mostly) C-defined objects which:
    – Can’t be saved / Can’t be restored
    – In some cases the instances just can’t be created from
    python code
    – Or can be created but not configured
    – Or whose state isn’t exposed, forbidding they to be
    saved / restored
    One technique: obtain instance, configure it to make it be
    what you want!
    

    View Slide

12. Sleepy – serialization problems
    We’ll see that in general it is always the same and it
    happens always because of three reasons. Problem is:
    There are a few (important) non-serializable objects
    ●
    They are (mostly) C-defined objects which:
    – Can’t be saved / Can’t be restored
    – In some cases the instances just can’t be created from
    python code
    – Or can be created but not configured
    – Or whose state isn’t exposed, forbidding they to be
    saved / restored
    One technique: obtain instance, configure it to make it be
    what you want!
    key concept
    

    View Slide

13. Sleepy – saving/restoring Python modules
    ●
    It is not possible to serialize a module in-memory state
    but we can: load them & restore their state!
    ●
    We must take care about module’s variables:
    – In some cases those variables needs to be reset to
    module’s value of definition, e.g: os.path.sep
    – However the opposite behavior is required in
    somecases. For example, we may be interested in
    keep random generator state: random._inst -
    random.getstate()
    

    View Slide

14. Sleepy – program execution state
    We defined execution state as the program call-stack and
    all its objects (objects reachable from the call stack).
    What is the call-stack?
    Is a list of contexts, each one related to a function call,
    which was already executed and is still active.
    Graphically: →main()
    →train()
    →BP()
    →runNN()
    →main()
    →train()
    →BP()
    →runNN()
    

    View Slide

15. Sleepy – program execution state
    We defined execution state as the program call-stack and
    all its objects (objects reachable from the call stack).
    What is the call-stack?
    Is a list of contexts, each one related to a function call,
    which was already executed and is still active.
    Graphically: →main()
    →train()
    →BP()
    →runNN()
    →main()
    →train()
    →BP()
    →runNN()
    Each one
    is
    a Frame
    

    View Slide

16. Sleepy – program execution state
    There are at least two ways to retrieve the actual call-
    stack:
    ●
    sys._current_frames() →
    { threadid : [frame_X, .., frame_0 ], ..}
    where frame_x is a
    ●
    (*) traceback.extract_stack() →
    [ frame-info-tuple-X, .., frame-info-tuple-0 ]
    where frame-info-tuple is:
    (filename, line_number, module, code_line)
    (*) current thread only!
    

    View Slide

17. Sleepy – the frame object
    The frame object have a function associated which is being
    executed and holds its “context”, e.g.: the “variables”
    accessible from that particular function execution instance.
    The problem is: it wasn’t mean to be “created” from python!
    Traceback (most recent call last):
    File "callstack.py", line 41, in
    new_frame = FrameType()
    TypeError: cannot create 'frame' instances
    

    View Slide

18. Sleepy – the frame object
    However there’s a simple way we can
    create frame objects !
    Ideas?
    

    View Slide

19. Sleepy – the frame object
    However there’s a simple way we can make
    python create frame objects for us!
    Ideas?
    

    View Slide

20. Sleepy – the frame object
    However there’s a simple way we can make
    python create frame objects for us!
    Ideas?
    Just call the associated function
    ..and the frame will be automatically created!
    [ This will only be useful if we can configure & control it ]
    

    View Slide

21. Sleepy – the frame object in detail
    f_back
    f_builtins
    f_code
    f_exc_traceback
    f_exc_type
    f_exc_value
    f_globals
    f_lasti
    f_lineno
    f_locals
    f_restricted
    f_trace
    
    { } # frame’s builtin namespace
    
    None
    None
    None
    { } # frame’s globals
    676 # last bytecode executed
    409 # source line nr !!!
    { } # frame’s locals !!!
    False
    None
    

    View Slide

22. Sleepy – the frame object - control
    The remaining question is: how can we control the frame
    execution?
    

    View Slide

23. Sleepy – the frame object - control
    How can we control the
    frame execution?
    Using the python
    debugger module: bdb.py.
    Execute
    function
    call
    Jump to
    next call Setup
    frame
    

    View Slide

24. Sleepy – recreate a call-stack
    Running the program (with the debugger) will call its
    entry-point function this way we’ll have the first frame
    created. It will contain a call to the function of the second
    frame, and so on.. In our example:
    main → train → BP → NN
    ●
    In detail, we will:
    1.Make python vm call a function creating its
    2.Setup frame’s variables f_locals.
    3.Jump to next function(*) call position in bytecode:
    lastno = saved_lineno
    4.Repeat as long as functions in call-stack
    (*) in the call-stack chain
    

    View Slide

25. Sleepy – recreation
    (Pdb) r
    > /home/aweil/test_for.py(2)fail_pc()
    -> for x in xrange(100):
    (Pdb) l
    1 B def fail_pc():
    2 → print 0
    3 for x in xrange(100):
    4 print 1
    5 print 2
    6 print 3
    7
    8 fail_pc()
    (Pdb) jump 4
    *** Jump failed: can't jump into the middle of a block
    Python’s VM
    can’t jump
    across
    code blocks!
    Python’s VM
    can’t jump
    across
    code blocks!
    Current line
    X
    

    View Slide

26. Sleepy – bytecode & code blocks
    1 def fail_pc():
    2 print 0
    3 for x in xrange(100):
    4 print 1
    5 print 2
    6 print 3
    3 5 SETUP_LOOP 35 (to 43)
    8 LOAD_GLOBAL 0 (xrange)
    11 LOAD_CONST 2 (100)
    14 CALL_FUNCTION 1
    17 GET_ITER
    >> 18 FOR_ITER
    FOR_ITER 21 (to 42)
    21 STORE_FAST 0 (x)
    4 24 LOAD_CONST 3 (1)
    27 PRINT_ITEM
    28 PRINT_NEWLINE
    5 29 LOAD_CONST 4 (2)
    32 PRINT_ITEM
    33 PRINT_NEWLINE
    6 34 LOAD_CONST 5 (3)
    37 PRINT_ITEM
    38 PRINT_NEWLINE
    39 JUMP_ABSOLUTE 18
    >> 42 POP_BLOCK
    POP_BLOCK
    

    View Slide

27. Sleepy - frame.f_lineno = X
    int frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno) (*)
    Setter for f_lineno - you can set f_lineno from within a trace function in order
    to jump to a given line of code, subject to some restrictions. Most lines are
    OK to jump to because they don't make any assumptions about the state of
    the stack (obvious because you could remove the line and the code would still
    work without any stack errors), but there are some constructs that limit
    jumping:
    ●
    Lines with an 'except' statement on them can't be jumped to, because they
    expect an exception to be on the top of the stack.
    ●
    Lines that live in a 'finally' block can't be jumped from or to, since the
    END_FINALLY expects to clean up the stack after the 'try' block.
    ●
    'try'/'for'/'while' blocks can't be jumped into because the blockstack needs
    to be set up before their code runs, and for 'for' loops the iterator needs to
    be on the stack.
    * Link to frameobject.c#L78 setlineno() function
    

    View Slide

28. Sleepy - while
    ●
    Special cases: break, continue, else.
    while condition:
    statement_1
    statement_n
    From a few alternatives, we chosen to
    rewrite code to generate compatible bytecode
    but with no code blocks!
    

    View Slide

29. Sleepy - while
    :begin
    if not condition:
    goto end
    statement_1
    ..
    statement_n
    goto begin
    :end
    ●
    Special cases: break, continue, else.
    while condition:
    statement_1
    statement_n
    From a few alternatives, we chosen to
    rewrite code to generate compatible bytecode
    but with no code blocks!
    

    View Slide

30. Sleepy – for
    for x in xrange(100):
    print x
    return x
    ●
    Special cases: break, continue, else
    

    View Slide

31. Sleepy – for
    for x in xrange(100):
    print x
    return x
    _it = iter(xrange(100))
    :next
    try:
    X = _it.next()
    except StopIteration:
    goto @end
    print x
    goto next
    :end
    return x
    ●
    Special cases: break, continue, else
    

    View Slide

32. Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else
    

    View Slide

33. Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else
    try:
    statement_1
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    if Temp is Except_1:
    block_ex_1
    if Temp is Except_1:
    block_ex_2
    if Temp == True:
    block_ex
    if Temp is None:
    block_else
    

    View Slide

34. try:
    statement_n
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else
    try:
    statement_2
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    if Temp is Except_1:
    block_ex_1
    if Temp is Except_1:
    block_ex_2
    if Temp == True:
    block_ex
    if Temp is None:
    block_else
    try:
    statement_1
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    

    View Slide

35. Sleepy – bytecode re-writing
    ●
    Module compiler.pycodegen* includes a python
    compiler implemented in python: CodeGenerator!
    ●
    To avoid blocks is it only necessary to overwrite:
    – visitBreak, visitContinue, visitWhile
    – visitGoto, visitTryExcept, visitFor
    – visitTryFinally
    ●
    We also redefined visitModule to automatically load
    our helper module: sleepy.
    * Warning! This module is NOT available in Python 3 ! :-(
    

    View Slide

36. When rewriting a for loop we also need to consider the
    possible different kind of iterators, because they are
    probably implemented in C and in general, they are not
    persistent:
    >>> pickle.dumps(iter([ ])) → TypeError: can't pickle list iterator objects
    To overcome this, we made a function which:
    – Receive the itereator and acording to it
    – Returns a new one pure-python compatible
    In this way, we made the code “resumable”.
    Sleepy – iterators & co.
    

    View Slide

37. ●
    We propose a few different ways to check when we want a
    process to suspended:
    – Continuous checking (which we implemented) :
    ●
    Suspend condition check is call by tracing function
    – It is SLOW !!!
    – Can’t assert will be stop just when required but is
    the best effort
    – Assisted checking:
    ●
    User should insert calls to suspend condition
    checking
    – Mixed 1: tracing not every line but every call
    – Mixed 2: replace tracing with pdb breakpoints
    Sleepy – control
    

    View Slide

38. Sleepy – demo
    Demo time!
    

    View Slide

39. ●
    Compiler extension:
    – Generates block-code free bytecode!
    – Replaces iterator-creation with ad-hoc function call
    ●
    Utilities to serialize objects & code-state
    ●
    Control code execution and trigger process-dump
    ●
    Utility to custom compile python code on-demand
    ●
    CLI tool to start and suspend or restore execution
    Sleepy – components
    

    View Slide

40. Sleepy – Pros & Cons
    ●
    Generated bytecode is far from optimal, eg:
    – A for loop for example:
    standard for: “SETUP_LOOP + .. + FOR_ITER”
    sleepy:”try: x=next(iter); except StopIteration”
    – No optimization is made to bytecode generated, like:
    123 JUMP_ABSOLUTE 129
    126 JUMP_FORWARD 0 (to 129)
    129 JUMP_ABSOLUTE 21
    – Execution speed seems 0.5X from std cpython bytecode
    ●
    compiler.pycodegen don’t support 100% of python2.7 !?
    ●
    Allows a more flexible debug letting user arbitrarily jump to
    any line in current function!
    

    View Slide

41. Sleepy – some notes
    ●
    Although very delicate and not quite easy AST modification
    is possible (and turned out to be and extremely powerful)
    – However I haven’t found tools to perform testing nor
    simplify the process
    – Pycodegen allows to trick part of that but there’s lot
    more to do..
    ●
    From this on, we try to be the less cpython dependent..
    – Also because there are many python flavors to chose
    from!
    ●
    It is absolutely normal from now on that you don’t get
    along with C-based objects!
    ●
    We had never used debugger in an application before!
    

    View Slide

42. Sleepy – Thanks!!
    Questions?
    

    View Slide

43. Sleepy - links
    ●
    Reposity: https://bitbucket.org/tenuki/sleepy/
    ●
    Design of the CPython Compiler: http://www.python.org/dev/peps/pep-0339/
    ●
    An implementation of import importlib: http://docs.python.org/dev/library/importlib.html
    ●
    Python Module of the Week: “dis”: http://www.doughellmann.com/PyMOTW/dis/
    ●
    New Import Hooks: http://www.python.org/dev/peps/pep-0302/
    ●
    The compiler module: http://docs.python.org/2/library/compiler.html
    ●
    http://docs.python.org/library/imputil.html
    ●
    http://docs.python.org/library/imp.html
    ●
    http://www.stackless.com/wiki/Pickling
    ●
    http://doc.pypy.org/en/latest/stackless.html
    ●
    Artwork from: http://www.bluebison.net
    ●
    Python Argentina: http://www.python.org.ar
    

    View Slide