Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Statistical Thread Profiler

Amjith
February 12, 2013

Statistical Thread Profiler

How to write your own statistical thread profiler.

Amjith

February 12, 2013

Transcript

77 slides
  1. @amjithr WRITE YOUR OWN WRITE YOUR OWN STATISTICAL THREAD PROFILER

    STATISTICAL THREAD PROFILER PDX PYTHON ▪ Tuesday, February 12, 13
  2. @amjithr $ python -m cProfile lcm.py 7780242 function calls in

    3.822 seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 3.822 3.822 lcm.py:12(<module>) 1 2.383 2.383 3.822 3.822 lcm.py:12(lcm) 3890120 0.724 0.000 0.724 0.000 {max} 1 0.000 0.000 0.000 0.000 {method 'disable' of 3890119 0.716 0.000 0.716 0.000 {min} Tuesday, February 12, 13
  3. @amjithr $ time python fib.py python fib.py 5.43s user 0.01s

    system 99% cpu 5.451 total $ time python -m cProfile fib.py python -m cProfile fib.py 9.63s user 0.04s system 99% cpu 9.670 total Tuesday, February 12, 13
  4. @amjithr import signal # Use the virtual timer sig =

    signal.ITIMER_VIRTUAL # register a handler signal.signal(sig, handler) # 100ms repeating timer signal.setitimer(sig, 0.1, 0.1) SIGNAL Tuesday, February 12, 13
  5. @amjithr PROs CONs Accurate timer Unix only - Ignored in

    mod_wsgi - Can’t interrupt C-extensions SIGNAL Tuesday, February 12, 13
  6. @amjithr import threading # Create a background thread t =

    threading.Thread(target=handler) t.setDaemon(True) # Start the thread t.start() # Sleep for 100ms in handler time.sleep(0.1) THREAD Tuesday, February 12, 13
  7. @amjithr PROs CONs Cross Platform Inaccurate for CPU tasks mod_wsgi

    compatible Can’t interrupt C-extensions THREAD Tuesday, February 12, 13
  8. @amjithr >>> import sys # Get stack frames of every

    thread >>> frames = sys._current_frames() # Dictionary of { thread_id: frame_obj } >>> frames {4420: <frame object at 0x7fd96c04a650>, 1407: <frame object at 0x7fd96c301730>} INQUIRE Tuesday, February 12, 13
  9. @amjithr function3 function2 function1 Fn Name File Name Line No

    Arguments Globals Locals ... FRAMES Tuesday, February 12, 13
  10. @amjithr function3 function2 function1 Fn Name File Name Line No

    Arguments Globals Locals ... FRAMES Tuesday, February 12, 13
  11. @amjithr >>> import traceback # Extract useful info from frame

    >>> traceback.extract_stack(frame) [('code.py', 88, 'runsource', 'return'), ('code.py', 110, 'runcode', 'print')] EXTRACT Tuesday, February 12, 13
  12. @amjithr Default Dictionary Uniquely identify a function fdata = (f_name,

    file, line) Build a dictionary calls = defaultdict(int) Increment the value when found in frames calls[fn_a] += 1 ATTEMPT Tuesday, February 12, 13
  13. @amjithr Default Dictionary Uniquely identify a function fdata = (f_name,

    file, line) Build a dictionary calls = defaultdict(int) Increment the value when found in frames calls[fn_a] += 1 ATTEMPT Tuesday, February 12, 13
  14. @amjithr Default Dictionary Uniquely identify a function fdata = (f_name,

    file, line) Build a dictionary calls = defaultdict(int) Increment the value when found in frames calls[fn_a] += 1 ATTEMPT Tuesday, February 12, 13
  15. @amjithr fn_a fn_b fn_d T2 fn_a=1 fn_b=1 fn_a=2 fn_b=2 fn_c=1

    fn_d=1 Example Tuesday, February 12, 13
  16. @amjithr SUMMARY Features Limitations Low Overhead No green threads CPython

    & PyPy No Jython support Tuesday, February 12, 13

More Decks by Amjith

Other Decks in Programming