Elizaveta Shashkova - Debugging in Python 3.6: Better, Faster, Stronger

Python 3.6 was released in December of 2016 and it has a lot of new cool features. Some of them are quite easy for using: a developer can read, for example, about f-strings and they can start using them in their programs as soon as possible. But sometimes features are not so evident, and a new frame evaluation API is one of them.
The new frame evaluation API was introduced to CPython in PEP 523 and it allows to specify a per-interpreter function pointer to handle the evaluation of frames. It might not be evident how to use this new feature in everyday life, but it’s quite easy to understand how to build a fast debugger based on it.
In this talk we are going to explain how standard way of debugging in Python works and how a new frame evaluation API may be useful for creating the fast debugger. Also we will consider why such fast debugging was not possible in the previous versions of Python. If someone hasn’t made a final decision to move to Python 3.6 this talk will provide some new reasons to do it.

https://us.pycon.org/2017/schedule/presentation/526/

Transcript

1. Debugging in Python 3.6:
   Better, Faster, Stronger
   Elizaveta Shashkova
   JetBrains

   

   PyCon US 2017
   

   View Slide

2. Bio
   • Software developer of PyCharm IDE at JetBrains
   • Debugger
   • Saint Petersburg, Russia
   2
   

   View Slide

3. Debugging
   • Adding print statements
   • Logging
   3
   

   View Slide

4. Debugging
   • Adding print statements
   • Logging
   • Special tools: debuggers
   4
   

   View Slide

5. Debugger’s Performance
   5
   Run
   Debug
   Debuggers are 30 times slower
   

   View Slide

6. Contents
   • Tracing debugger
   • Python 3.6
   • Frame evaluation debugger
   • Results
   6
   

   View Slide

7. Contents
   • Tracing debugger
   • Python 3.6
   • Frame evaluation debugger
   • Results
   7
   

   View Slide

8. 8
   def tracefunc(frame, event, arg):
   print(frame.f_lineno, event)
   return tracefunc
   sys.settrace(tracefunc)
   1
   2
   3
   4
   5
   6
   sys.settrace(tracefunc) - set the
   system tracing function
   Tracing Function
   

   View Slide

9. def foo():
   friends = ["Bob", "Tom"]
   for f in friends:
   print("Hi %s!” % f)
   return len(friends)
   sys.settrace(tracefunc)
   foo()
   1
   2
   3
   4
   5
   6
   7
   8
   9
   Tracing Function
   9
   

   View Slide

10. 1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    Tracing Function
    10
    

    View Slide

11. 1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    2 line
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    Tracing Function
    11
    

    View Slide

12. 1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    2 line
    3 line
    4 line
    Hi Bob!
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    Tracing Function
    12
    

    View Slide

13. 1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    2 line
    3 line
    4 line
    Hi Bob!
    3 line
    4 line
    Hi Tom!
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    Tracing Function
    13
    

    View Slide

14. 1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    2 line
    3 line
    4 line
    Hi Bob!
    3 line
    4 line
    Hi Tom!
    5 line
    5 return
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    Tracing Function
    14
    

    View Slide

15. Build Python Debugger
    • Breakpoints
    • Stepping
    15
    

    View Slide

16. Tracing Debugger
    • Suspend program if breakpoint’s
    line equals frame.f_lineno
    • Handle events for stepping
    16
    

    View Slide

17. Performance
    def foo():
    friends = ["Bob", "Tom"]
    for f in friends:
    print("Hi %s!” % f)
    return len(friends)
    sys.settrace(tracefunc)
    foo()
    1
    2
    3
    4
    5
    6
    7
    8
    9
    1 call
    2 line
    3 line
    4 line
    Hi Bob!
    3 line
    4 line
    Hi Tom!
    5 line
    5 return
    17
    

    View Slide

18. Example 1
    def calculate():
    sum = 0
    for i in range(10 ** 7):
    sum += i
    return sum
    1
    2
    3
    4
    5
    6
    7
    8
    9
    18
    

    View Slide

19. Example 1
    def calculate():
    sum = 0
    for i in range(10 ** 7):
    sum += i
    return sum
    def tracefunc(frame, event, arg):
    return tracefunc
    1
    2
    3
    4
    5
    6
    7
    8
    9
    19
    

    View Slide

20. Performance
    20
    Run
    Tracing
    Breakpoints
    0,80 sec
    6,85 sec
    19,81 sec
    

    View Slide

21. Performance
    21
    Run
    Tracing
    Breakpoints
    0,80 sec
    6,85 sec
    19,81 sec
    

    View Slide

22. Performance
    22
    Run
    Tracing
    Breakpoints
    0,80 sec
    6,85 sec
    19,81 sec
    

    View Slide

23. Performance
    23
    Run
    Tracing
    Breakpoints
    0,80 sec
    6,85 sec
    19,81 sec
    ~ 25 times slower!
    

    View Slide

24. Problem
    • Tracing call on every line
    24
    

    View Slide

25. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    25
    

    View Slide

26. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    26
    

    View Slide

27. Python 3.6
    27
    

    View Slide

28. Python 3.6
    • New frame evaluation API
    • PEP 523
    28
    

    View Slide

29. PEP 523
    • Handle evaluation of frames
    • Add a new field to code objects
    29
    

    View Slide

30. Frame Evaluation
    def frame_eval(frame, exc):
    func_name = frame.f_code.co_name
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    30
    

    View Slide

31. def frame_eval(frame, exc):
    func_name = frame.f_code.co_name a
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Frame Evaluation
    31
    

    View Slide

32. def frame_eval(frame, exc):
    func_name = frame.f_code.co_name
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Frame Evaluation
    32
    

    View Slide

33. def frame_eval(frame, exc):
    func_name = frame.f_code.co_name
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Frame Evaluation
    33
    

    View Slide

34. def frame_eval(frame, exc):
    func_name = frame.f_code.co_name
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Frame Evaluation
    34
    

    View Slide

35. def frame_eval(frame, exc):
    func_name = frame.f_code.co_name
    line_number = frame.f_lineno
    print(line_number, func_name)
    return _PyEval_EvalFrameDefault(frame, exc)
    def set_frame_eval():
    state = PyThreadState_Get()
    state.interp.eval_frame = frame_eval
    1
    2
    3
    4
    5
    6
    7
    8
    9
    Frame Evaluation
    35
    

    View Slide

36. Example
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    def first():
    second()
    def second():
    third()
    def third():
    pass
    set_frame_eval()
    first()
    36
    

    View Slide

37. Example
    def first():
    second()
    def second():
    third()
    def third():
    pass
    set_frame_eval()
    first()
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    1 first
    4 second
    7 third
    37
    

    View Slide

38. Custom Frame Evaluation
    • It works!
    • Executed while entering a frame
    • Access to frame and code object
    38
    

    View Slide

39. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    39
    

    View Slide

40. Problem
    • Tracing call on every line
    40
    

    View Slide

41. Problem
    • Tracing call on every line
    • Remove the tracing function!
    41
    

    View Slide

42. Replace tracing
    function with frame
    evaluation function
    42
    

    View Slide

43. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    43
    

    View Slide

44. Build Python Debugger
    • Breakpoints
    • Stepping
    44
    

    View Slide

45. Breakpoints
    • Access to the whole code object
    45
    

    View Slide

46. Breakpoints
    • Access to the whole code object
    • Insert breakpoint’s code into
    frame’s code
    46
    

    View Slide

47. Breakpoints
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    1
    2
    3
    4
    5
    6
    7
    8
    47
    

    View Slide

48. Breakpoints
    def maximum(a, b):
    if a > b:
    return a # breakpoint
    else:
    return b
    1
    2
    3
    4
    5
    6
    7
    8
    48
    

    View Slide

49. Breakpoints
    def maximum(a, b):
    if a > b:
    return a # breakpoint
    else:
    return b
    1
    2
    3
    4
    5
    6
    7
    8
    breakpoint()
    49
    

    View Slide

50. Breakpoints
    def maximum(a, b):
    if a > b:
    breakpoint()
    return a # breakpoint
    else:
    return b
    1
    2
    3
    4
    5
    6
    7
    8
    50
    

    View Slide

51. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    51
    

    View Slide

52. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    52
    

    View Slide

53. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    53
    

    View Slide

54. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    54
    

    View Slide

55. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    55
    

    View Slide

56. Python Bytecode
    def maximum(a, b):
    if a > b:
    return a
    else:
    return b
    import dis
    dis.dis(maximum)
    1
    2
    3
    4
    5
    6
    7
    8
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    56
    

    View Slide

57. Python Bytecode
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    57
    

    View Slide

58. 2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    Python Bytecode
    58
    

    View Slide

59. 2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    Python Bytecode
    59
    

    View Slide

60. 2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    Python Bytecode
    60
    

    View Slide

61. Bytecode Modification
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    breakpoint()
    61
    

    View Slide

62. Bytecode Modification
    • Insert breakpoint’s code
    • Update arguments and offsets
    62
    

    View Slide

63. Bytecode Modification
    • Insert breakpoint’s code
    • Update arguments and offsets
    • 200 lines in Python
    63
    

    View Slide

64. Bytecode Modification
    2 0 LOAD_FAST 0 (a)
    2 LOAD_FAST 1 (b)
    4 COMPARE_OP 4 (>)
    6 POP_JUMP_IF_FALSE 12
    3 8 LOAD_FAST 0 (a)
    10 RETURN_VALUE
    5 >> 12 LOAD_FAST 1 (b)
    14 RETURN_VALUE
    16 LOAD_CONST 0 (None)
    18 RETURN_VALUE
    breakpoint()
    ?!
    64
    

    View Slide

65. Breakpoint Bytecode
    def _stop_at_break():
    # a lot of code here
    def breakpoint():
    _stop_at_break()
    1
    2
    3
    4
    5
    6
    7
    8
    0 LOAD_GLOBAL 0
    2 CALL_FUNCTION 0
    4 POP_TOP
    6 LOAD_CONST 0
    8 RETURN_VALUE
    65
    

    View Slide

66. Build Python Debugger
    • Breakpoints
    • Stepping
    66
    

    View Slide

67. Stepping
    • Inserting temporary breakpoint
    on every line
    • Use old tracing function
    67
    

    View Slide

68. Frame evaluation
    debugger is ready!
    68
    

    View Slide

69. Example 1
    def calculate():
    sum = 0
    for i in range(10 ** 7):
    sum += i
    return sum
    1
    2
    3
    4
    5
    69
    

    View Slide

70. Example 1
    70
    Run
    Tracing
    Frame 

    evaluation
    0,80 sec
    19,81 sec
    0,81 sec
    

    View Slide

71. Example 1
    71
    Run
    Tracing
    Frame 

    evaluation
    0,80 sec
    19,81 sec
    0,81 sec
    

    View Slide

72. Example 2
    def foo():
    pass
    def calculate():
    sum = 0
    for i in range(10 ** 7):
    foo()
    sum += i
    return sum
    1
    2
    3
    4
    5
    6
    7
    8
    9
    72
    

    View Slide

73. Example 2
    73
    Run
    Tracing
    Frame 

    evaluation
    1,73 sec
    43,58 sec
    37,41 sec
    

    View Slide

74. PEP 523
    • Handle evaluation of frames
    • Add a new field to code objects
    74
    

    View Slide

75. PEP 523
    • Expand PyCodeObject struct
    • co_extra - “scratch space” for
    the code object
    • Mark frames without breakpoints
    75
    

    View Slide

76. Mark Frames
    def frame_eval(frame, exc):
    flag = _PyCode_GetExtra(frame.f_code, index)
    if flag == NO_BREAKS_IN_FRAME:
    return _PyEval_EvalFrameDefault(frame, exc)

    

    # check for breakpoints
    ...
    1
    2
    3
    4
    5
    6
    7
    8
    9
    76
    

    View Slide

77. Example 2
    77
    Run
    Tracing
    Frame 

    evaluation
    1,73 sec
    43,58 sec
    1,91 sec
    

    View Slide

78. PEP 523
    • Handle evaluation of frames
    • Add a new field to code objects
    78
    

    View Slide

79. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    79
    

    View Slide

80. Contents
    • Tracing debugger
    • Python 3.6
    • Frame evaluation debugger
    • Results
    80
    

    View Slide

81. Real Life Example
    81
    

    View Slide

82. Real Life Example
    • Included into PyCharm 2017.1
    • Works in production
    82
    

    View Slide

83. PyCharm
    83
    Tracing

    w/o Cython
    Tracing

    with Cython
    Frame 

    evaluation
    11,59 sec
    5,66 sec
    0,28 sec
    

    View Slide

84. Frame evaluation
    rocks!
    84
    

    View Slide

85. Disadvantages
    • More complicated
    • Only with CPython
    • Only with Python 3.6
    85
    

    View Slide

86. Frame Evaluation
    • Let’s move to Python 3.6!
    86
    

    View Slide

87. Frame Evaluation
    • Let’s move to Python 3.6!
    • Let’s find another use cases!
    87
    

    View Slide

88. Use cases
    def maximum(a, b):
    if a > b:
    return a # breakpoint
    else:
    return b
    1
    2
    3
    4
    5
    6
    7
    8
    breakpoint()
    88
    

    View Slide

89. PEP 523
    • Pyjion project
    • JIT for Python
    89
    

    View Slide

90. Frame Evaluation
    • Let’s move to Python 3.6!
    • Let’s find another use cases!
    90
    

    View Slide

91. Links
    • Prototype: https://github.com/
    Elizaveta239/frame-eval
    • PyCharm Community Edition
    source code
    91
    

    View Slide

92. Questions?
    • Prototype: https://github.com/
    Elizaveta239/frame-eval
    • PyCharm Community Edition
    source code
    92
    @lisa_shashkova
    

    View Slide