Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Sleepy: How to suspend and resume your cpython process from inside

david weil
September 23, 2016

Sleepy: How to suspend and resume your cpython process from inside

In this talk we'll propose and explain how it is possible suspend & resume a python process execution from within python using no external-help. Pure-python-ffi-free code. This kind of tool is useful with the proliferation of cloud services since they offer restricted execution environment in terms of time/memory and process relocation/priority is required. Besides we’ll get unexpected debugger improvements.

david weil

September 23, 2016
Tweet

More Decks by david weil

Other Decks in Programming

Transcript

  1. Sleepy - スリーピー

    View full-size slide

  2. About me: david weil
    aka: Tenuki / dave
    @tenukiz

    I’m from Argentina

    Work (1999-2013):

    Some projects:
    – IMFish
    Few games for PyWeek..

    http://about.me/tenuki
    Sleepy – about + intro
    About Sleepy:

    In 2009 we made Sleepy
    v1 with Fernando Russ
    [ @frussterix ]

    Is our approach to answer
    this question:
    How could you run a given
    code in Google App Engine,
    without changing it?

    View full-size slide

  3. About me: david weil
    aka: Tenuki / dave
    @tenukiz

    I’m from Argentina????

    Work (1999-2013):

    Some projects:
    – IMFish
    Few games for PyWeek..

    http://about.me/tenuki
    Sleepy – about + intro
    About Sleepy:

    In 2009 we made Sleepy
    v1 with Fernando Russ
    [ @frussterix ]

    Is our approach to answer
    this question:
    How could you run a given
    code in Google App Engine,
    without changing it?

    View full-size slide

  4. Why would you need to suspend python?

    To execute a program with a time quota

    To resume a process in later ..

    … or in a different machine. Enabling:
    – Load distribution
    – High availability

    To inspect process state / debugging purposes

    Implement “continuations”

    Resume testing on hardware with no OS (check:
    Pausable Unittest on EFI Stackless Python by Masamitsu
    Murase (now at: Room 202))

    Implement application checkpointing!

    View full-size slide

  5. Other approaches

    At OS level (unix at least):
    – in shell Ctrl+Z effectively suspends the process
    – CryoPID “allows you to capture the state of a running
    process in Linux and save it to a file”

    At Python VM's level (making a C-module or changing
    the vm) it would be possible to track every object in
    memory and serialize them to restore when needed.. but
    we are looking for a pure-python solution.

    View full-size slide

  6. Is it possible in other languages?

    Smalltalk → image

    Lisp → worlds

    C → Again check CryoPID.. however usually this requires
    source code changes..

    Java/JavaScript/.Net: I don’t know, but would like to,
    please let me know if you do!

    Python? No.. Yes… Let’s see more in detail..

    View full-size slide

  7. Is it possible in Python?

    CPython: no..?

    Jython: no idea..

    IronPython: no idea..

    Stackless: Yes! *
    – “One of the main features of Stackless is
    its ability to pickle and unpickle tasklets”

    PyPy: Yes! continuations up to stackless level
    were added in the lasts years (check: (**) and its
    continulet)
    * http://www.stackless.com/wiki/Pickling
    ** http://doc.pypy.org/en/latest/stackless.html

    View full-size slide

  8. Let’s focus on CPython – Sleepy idea
    It should be enough if we were able to save and restore
    every object instance instance in memory (including
    dependencies). We mean:
    – User objects: Classes and Instances which makes the
    application.
    – Modules the application uses.
    – Objects created or used by the Virtual-Maching

    In particular: program execution state
    Easy, right?

    View full-size slide

  9. Sleepy – serialization problems
    We’ll see that in general it is always the same and it
    happens always because of three reasons. Problem is:
    There are a few (important) non-serializable objects

    They are (mostly) C-defined objects which:
    – Can’t be saved / Can’t be restored
    – In some cases the instances just can’t be created from
    python code
    – Or can be created but not configured
    – Or whose state isn’t exposed, forbidding they to be
    saved / restored
    One technique: obtain instance, configure it to make it be
    what you want!

    View full-size slide

  10. Sleepy – serialization problems
    We’ll see that in general it is always the same and it
    happens always because of three reasons. Problem is:
    There are a few (important) non-serializable objects

    They are (mostly) C-defined objects which:
    – Can’t be saved / Can’t be restored
    – In some cases the instances just can’t be created from
    python code
    – Or can be created but not configured
    – Or whose state isn’t exposed, forbidding they to be
    saved / restored
    One technique: obtain instance, configure it to make it be
    what you want!
    key concept

    View full-size slide

  11. Sleepy – saving/restoring Python modules

    It is not possible to serialize a module in-memory state
    but we can: load them & restore their state!

    We must take care about module’s variables:
    – In some cases those variables needs to be reset to
    module’s value of definition, e.g: os.path.sep
    – However the opposite behavior is required in
    somecases. For example, we may be interested in
    keep random generator state: random._inst -
    random.getstate()

    View full-size slide

  12. Sleepy – program execution state
    We defined execution state as the program call-stack and
    all its objects (objects reachable from the call stack).
    What is the call-stack?
    Is a list of contexts, each one related to a function call,
    which was already executed and is still active.
    Graphically: →main()
    →train()
    →BP()
    →runNN()
    →main()
    →train()
    →BP()
    →runNN()

    View full-size slide

  13. Sleepy – program execution state
    We defined execution state as the program call-stack and
    all its objects (objects reachable from the call stack).
    What is the call-stack?
    Is a list of contexts, each one related to a function call,
    which was already executed and is still active.
    Graphically: →main()
    →train()
    →BP()
    →runNN()
    →main()
    →train()
    →BP()
    →runNN()
    Each one
    is
    a Frame

    View full-size slide

  14. Sleepy – program execution state
    There are at least two ways to retrieve the actual call-
    stack:

    sys._current_frames() →
    { threadid : [frame_X, .., frame_0 ], ..}
    where frame_x is a

    (*) traceback.extract_stack() →
    [ frame-info-tuple-X, .., frame-info-tuple-0 ]
    where frame-info-tuple is:
    (filename, line_number, module, code_line)
    (*) current thread only!

    View full-size slide

  15. Sleepy – the frame object
    The frame object have a function associated which is being
    executed and holds its “context”, e.g.: the “variables”
    accessible from that particular function execution instance.
    The problem is: it wasn’t mean to be “created” from python!
    Traceback (most recent call last):
    File "callstack.py", line 41, in
    new_frame = FrameType()
    TypeError: cannot create 'frame' instances

    View full-size slide

  16. Sleepy – the frame object
    However there’s a simple way we can
    create frame objects !
    Ideas?

    View full-size slide

  17. Sleepy – the frame object
    However there’s a simple way we can make
    python create frame objects for us!
    Ideas?

    View full-size slide

  18. Sleepy – the frame object
    However there’s a simple way we can make
    python create frame objects for us!
    Ideas?
    Just call the associated function
    ..and the frame will be automatically created!
    [ This will only be useful if we can configure & control it ]

    View full-size slide

  19. Sleepy – the frame object in detail
    f_back
    f_builtins
    f_code
    f_exc_traceback
    f_exc_type
    f_exc_value
    f_globals
    f_lasti
    f_lineno
    f_locals
    f_restricted
    f_trace

    { } # frame’s builtin namespace

    None
    None
    None
    { } # frame’s globals
    676 # last bytecode executed
    409 # source line nr !!!
    { } # frame’s locals !!!
    False
    None

    View full-size slide

  20. Sleepy – the frame object - control
    The remaining question is: how can we control the frame
    execution?

    View full-size slide

  21. Sleepy – the frame object - control
    How can we control the
    frame execution?
    Using the python
    debugger module: bdb.py.
    Execute
    function
    call
    Jump to
    next call Setup
    frame

    View full-size slide

  22. Sleepy – recreate a call-stack
    Running the program (with the debugger) will call its
    entry-point function this way we’ll have the first frame
    created. It will contain a call to the function of the second
    frame, and so on.. In our example:
    main → train → BP → NN

    In detail, we will:
    1.Make python vm call a function creating its
    2.Setup frame’s variables f_locals.
    3.Jump to next function(*) call position in bytecode:
    lastno = saved_lineno
    4.Repeat as long as functions in call-stack
    (*) in the call-stack chain

    View full-size slide

  23. Sleepy – recreation
    (Pdb) r
    > /home/aweil/test_for.py(2)fail_pc()
    -> for x in xrange(100):
    (Pdb) l
    1 B def fail_pc():
    2 → print 0
    3 for x in xrange(100):
    4 print 1
    5 print 2
    6 print 3
    7
    8 fail_pc()
    (Pdb) jump 4
    *** Jump failed: can't jump into the middle of a block
    Python’s VM
    can’t jump
    across
    code blocks!
    Python’s VM
    can’t jump
    across
    code blocks!
    Current line
    X

    View full-size slide

  24. Sleepy – bytecode & code blocks
    1 def fail_pc():
    2 print 0
    3 for x in xrange(100):
    4 print 1
    5 print 2
    6 print 3
    3 5 SETUP_LOOP 35 (to 43)
    8 LOAD_GLOBAL 0 (xrange)
    11 LOAD_CONST 2 (100)
    14 CALL_FUNCTION 1
    17 GET_ITER
    >> 18 FOR_ITER
    FOR_ITER 21 (to 42)
    21 STORE_FAST 0 (x)
    4 24 LOAD_CONST 3 (1)
    27 PRINT_ITEM
    28 PRINT_NEWLINE
    5 29 LOAD_CONST 4 (2)
    32 PRINT_ITEM
    33 PRINT_NEWLINE
    6 34 LOAD_CONST 5 (3)
    37 PRINT_ITEM
    38 PRINT_NEWLINE
    39 JUMP_ABSOLUTE 18
    >> 42 POP_BLOCK
    POP_BLOCK

    View full-size slide

  25. Sleepy - frame.f_lineno = X
    int frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno) (*)
    Setter for f_lineno - you can set f_lineno from within a trace function in order
    to jump to a given line of code, subject to some restrictions. Most lines are
    OK to jump to because they don't make any assumptions about the state of
    the stack (obvious because you could remove the line and the code would still
    work without any stack errors), but there are some constructs that limit
    jumping:

    Lines with an 'except' statement on them can't be jumped to, because they
    expect an exception to be on the top of the stack.

    Lines that live in a 'finally' block can't be jumped from or to, since the
    END_FINALLY expects to clean up the stack after the 'try' block.

    'try'/'for'/'while' blocks can't be jumped into because the blockstack needs
    to be set up before their code runs, and for 'for' loops the iterator needs to
    be on the stack.
    * Link to frameobject.c#L78 setlineno() function

    View full-size slide

  26. Sleepy - while

    Special cases: break, continue, else.
    while condition:
    statement_1
    statement_n
    From a few alternatives, we chosen to
    rewrite code to generate compatible bytecode
    but with no code blocks!

    View full-size slide

  27. Sleepy - while
    :begin
    if not condition:
    goto end
    statement_1
    ..
    statement_n
    goto begin
    :end

    Special cases: break, continue, else.
    while condition:
    statement_1
    statement_n
    From a few alternatives, we chosen to
    rewrite code to generate compatible bytecode
    but with no code blocks!

    View full-size slide

  28. Sleepy – for
    for x in xrange(100):
    print x
    return x

    Special cases: break, continue, else

    View full-size slide

  29. Sleepy – for
    for x in xrange(100):
    print x
    return x
    _it = iter(xrange(100))
    :next
    try:
    X = _it.next()
    except StopIteration:
    goto @end
    print x
    goto next
    :end
    return x

    Special cases: break, continue, else

    View full-size slide

  30. Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else

    View full-size slide

  31. Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else
    try:
    statement_1
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    if Temp is Except_1:
    block_ex_1
    if Temp is Except_1:
    block_ex_2
    if Temp == True:
    block_ex
    if Temp is None:
    block_else

    View full-size slide

  32. try:
    statement_n
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    Sleepy – try/except deconstruction
    try:
    statement_1
    ...
    statement_n
    except Except_1:
    block_ex_1
    except Except_2, e:
    block_ex_2
    except:
    bloque_ex
    else:
    block_else
    try:
    statement_2
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True
    if Temp is Except_1:
    block_ex_1
    if Temp is Except_1:
    block_ex_2
    if Temp == True:
    block_ex
    if Temp is None:
    block_else
    try:
    statement_1
    except Except_1:
    Temp = Except_1
    except Except_2, ex:
    Temp = Except_2
    except:
    Temp = True

    View full-size slide

  33. Sleepy – bytecode re-writing

    Module compiler.pycodegen* includes a python
    compiler implemented in python: CodeGenerator!

    To avoid blocks is it only necessary to overwrite:
    – visitBreak, visitContinue, visitWhile
    – visitGoto, visitTryExcept, visitFor
    – visitTryFinally

    We also redefined visitModule to automatically load
    our helper module: sleepy.
    * Warning! This module is NOT available in Python 3 ! :-(

    View full-size slide

  34. When rewriting a for loop we also need to consider the
    possible different kind of iterators, because they are
    probably implemented in C and in general, they are not
    persistent:
    >>> pickle.dumps(iter([ ])) → TypeError: can't pickle list iterator objects
    To overcome this, we made a function which:
    – Receive the itereator and acording to it
    – Returns a new one pure-python compatible
    In this way, we made the code “resumable”.
    Sleepy – iterators & co.

    View full-size slide


  35. We propose a few different ways to check when we want a
    process to suspended:
    – Continuous checking (which we implemented) :

    Suspend condition check is call by tracing function
    – It is SLOW !!!
    – Can’t assert will be stop just when required but is
    the best effort
    – Assisted checking:

    User should insert calls to suspend condition
    checking
    – Mixed 1: tracing not every line but every call
    – Mixed 2: replace tracing with pdb breakpoints
    Sleepy – control

    View full-size slide

  36. Sleepy – demo
    Demo time!

    View full-size slide


  37. Compiler extension:
    – Generates block-code free bytecode!
    – Replaces iterator-creation with ad-hoc function call

    Utilities to serialize objects & code-state

    Control code execution and trigger process-dump

    Utility to custom compile python code on-demand

    CLI tool to start and suspend or restore execution
    Sleepy – components

    View full-size slide

  38. Sleepy – Pros & Cons

    Generated bytecode is far from optimal, eg:
    – A for loop for example:
    standard for: “SETUP_LOOP + .. + FOR_ITER”
    sleepy:”try: x=next(iter); except StopIteration”
    – No optimization is made to bytecode generated, like:
    123 JUMP_ABSOLUTE 129
    126 JUMP_FORWARD 0 (to 129)
    129 JUMP_ABSOLUTE 21
    – Execution speed seems 0.5X from std cpython bytecode

    compiler.pycodegen don’t support 100% of python2.7 !?

    Allows a more flexible debug letting user arbitrarily jump to
    any line in current function!

    View full-size slide

  39. Sleepy – some notes

    Although very delicate and not quite easy AST modification
    is possible (and turned out to be and extremely powerful)
    – However I haven’t found tools to perform testing nor
    simplify the process
    – Pycodegen allows to trick part of that but there’s lot
    more to do..

    From this on, we try to be the less cpython dependent..
    – Also because there are many python flavors to chose
    from!

    It is absolutely normal from now on that you don’t get
    along with C-based objects!

    We had never used debugger in an application before!

    View full-size slide

  40. Sleepy – Thanks!!
    Questions?

    View full-size slide

  41. Sleepy - links

    Reposity: https://bitbucket.org/tenuki/sleepy/

    Design of the CPython Compiler: http://www.python.org/dev/peps/pep-0339/

    An implementation of import importlib: http://docs.python.org/dev/library/importlib.html

    Python Module of the Week: “dis”: http://www.doughellmann.com/PyMOTW/dis/

    New Import Hooks: http://www.python.org/dev/peps/pep-0302/

    The compiler module: http://docs.python.org/2/library/compiler.html

    http://docs.python.org/library/imputil.html

    http://docs.python.org/library/imp.html

    http://www.stackless.com/wiki/Pickling

    http://doc.pypy.org/en/latest/stackless.html

    Artwork from: http://www.bluebison.net

    Python Argentina: http://www.python.org.ar

    View full-size slide