Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GDB: A Gentle Intro

Jason R Clark
November 17, 2015

GDB: A Gentle Intro

We love Ruby’s elegance, simplicity, and flexibility. But our favorite language perches atop a world of native code. When that other world intrudes on your peaceful Ruby, GDB, the venerable GNU debugger, is the tool to turn to.

We’ll examine setting up Ruby to work with GDB. We’ll learn the fundamental commands, and soon you’ll be debugging with ease. We’ll even peer deep into Ruby object internals and face down crashes, deadlocks, and bugs.

Whether you’re writing a native gem, hacking the Ruby VM, or just want a glimpse of the layers below, this talk is for you!

Jason R Clark

November 17, 2015
Tweet

More Decks by Jason R Clark

Other Decks in Technology

Transcript

  1. https://flic.kr/p/8RePxD
    GDB
    A Gentle Intro
    @jasonrclark
    http://git.io/v8GbT
    https://flic.kr/p/o15Y5n

    View Slide

  2. 2
    $ ps -o pid,pcpu,pmem,cmd
    PID %CPU %MEM CMD
    1 0.0 0.1 bash
    10 1.8 0.4 ruby service.rb
    >>

    View Slide

  3. 3
    $ tail production.log
    [14:30:59 (8985)] INFO : Starting!
    $
    >>

    View Slide

  4. 4
    Threading
    https://flic.kr/p/4WFtU8

    View Slide

  5. Deadlock?
    https://flic.kr/p/nkwswv

    View Slide

  6. GDB
    6

    View Slide

  7. Gnu Debugger
    7

    View Slide

  8. MRI
    8

    View Slide

  9. MRI CRuby
    9

    View Slide

  10. 10
    $ gdb -p 10
    ...
    Attaching to process 10
    Reading symbols from /usr/local/bin/
    ruby...done.
    [Thread debugging using libthread_db
    enabled]
    Loaded symbols for /usr/local/lib/ruby/
    2.2.0/x86_64-linux/thread.so
    (gdb)
    >>

    View Slide

  11. 11
    $ gdb -p 10
    ...
    Attaching to process 10
    Reading symbols from /usr/local/bin/
    ruby...done.
    [Thread debugging using libthread_db
    enabled]
    Loaded symbols for /usr/local/lib/ruby/
    2.2.0/x86_64-linux/thread.so
    (gdb)
    >>

    View Slide

  12. (gdb) backtrace
    #0 [email protected]@GLIBC_2.3.2 () at
    ../nptl/sysdeps/unix/sysv/...:185
    #1 0x00007f131f343b03 in
    native_cond_wait(mutex=0x7f131fcdf5,
    cond=0x7f131fcdf558) at
    thread_pthread.c:338
    #2 lock_func (timeout_ms=0, mutex=0x7f,
    th=0x7f131f8c46) at thread.c:4268
    #3 rb_mutex_lock (self=139720114712880)
    at thread.c:4342
    #4 0x00007f131f31c9ee in
    vm_call_cfunc_with_frame (ci=0x0,
    reg_cfp=0x7f131f1a7f70,
    th=0x7f131f8c46e0)
    at vm_insnhelper.c:1380
    >>

    View Slide

  13. 13
    RuntimeError: raise ‘Heck’
    config.ru:13:in `call'
    rack-1.6.4/...tempfile_reaper.rb:15:in `call'
    rack-1.6.4/...lint.rb:49:in `_call'
    rack-1.6.4/...lint.rb:37:in `call'
    rack-1.6.4/...showexceptions.rb:24:in `call'
    rack-1.6.4/...commonlogger.rb:33:in `call'
    rack-1.6.4/...chunked.rb:54:in `call'
    rack-1.6.4/...content_length.rb:15:in `call'
    thin-1.6.3/...connection.rb:86:in `block in
    thin-1.6.3/...connection.rb:84:in `catch'
    thin-1.6.3/...connection.rb:84:in `pre_process'
    thin-1.6.3/...connection.rb:53:in `process'
    thin-1.6.3/...connection.rb:39:in `receive_data
    eventmachine-1.0.7/lib/eventmachine.rb:187:in
    eventmachine-1.0.7/lib/eventmachine.rb:187:in
    thin-1.6.3/...backends/base.rb:73:in `start'
    thin-1.6.3/...server.rb:162:in `start'

    View Slide

  14. (gdb) backtrace
    #0 [email protected]@GLIBC_2.3.2 () at
    ../nptl/sysdeps/unix/sysv/...:185
    #1 0x00007f131f343b03 in
    native_cond_wait(mutex=0x7f131fcdf5,
    cond=0x7f131fcdf558) at
    thread_pthread.c:338
    #2 lock_func (timeout_ms=0, mutex=0x7f,
    th=0x7f131f8c46) at thread.c:4268
    #3 rb_mutex_lock (self=139720114712880)
    at thread.c:4342
    #4 0x00007f131f31c9ee in
    vm_call_cfunc_with_frame (ci=0x0,
    reg_cfp=0x7f131f1a7f70,
    th=0x7f131f8c46e0)
    at vm_insnhelper.c:1380

    View Slide

  15. (gdb) backtrace
    #0 [email protected]@GLIBC_2.3.2 () at
    ../nptl/sysdeps/unix/sysv/...:185
    #1 0x00007f131f343b03 in
    native_cond_wait(mutex=0x7f131fcdf5,
    cond=0x7f131fcdf558) at
    thread_pthread.c:338
    #2 lock_func (timeout_ms=0, mutex=0x7f,
    th=0x7f131f8c46) at thread.c:4268
    #3 rb_mutex_lock (self=139720114712880)
    at thread.c:4342
    #4 0x00007f131f31c9ee in
    vm_call_cfunc_with_frame (ci=0x0,
    reg_cfp=0x7f131f1a7f70,
    th=0x7f131f8c46e0)
    at vm_insnhelper.c:1380

    View Slide

  16. (gdb) backtrace
    #0 [email protected]@GLIBC_2.3.2 () at
    ../nptl/sysdeps/unix/sysv/...:185
    #1 0x00007f131f343b03 in
    native_cond_wait(mutex=0x7f131fcdf5,
    cond=0x7f131fcdf558) at
    thread_pthread.c:338
    #2 lock_func (timeout_ms=0, mutex=0x7f,
    th=0x7f131f8c46) at thread.c:4268
    #3 rb_mutex_lock (self=139720114712880)
    at thread.c:4342
    #4 0x00007f131f31c9ee in
    vm_call_cfunc_with_frame (ci=0x0,
    reg_cfp=0x7f131f1a7f70,
    th=0x7f131f8c46e0)
    at vm_insnhelper.c:1380

    View Slide

  17. (gdb) backtrace
    #0 [email protected]@GLIBC_2.3.2 () at
    ../nptl/sysdeps/unix/sysv/...:185
    #1 0x00007f131f343b03 in
    native_cond_wait(mutex=0x7f131fcdf5,
    cond=0x7f131fcdf558) at
    thread_pthread.c:338
    #2 lock_func (timeout_ms=0, mutex=0x7f,
    th=0x7f131f8c46) at thread.c:4268
    #3 rb_mutex_lock (self=139720114712880)
    at thread.c:4342
    #4 0x00007f131f31c9ee in
    vm_call_cfunc_with_frame (ci=0x0,
    reg_cfp=0x7f131f1a7f70,
    th=0x7f131f8c46e0)
    at vm_insnhelper.c:1380

    View Slide

  18. 18
    (gdb) info threads
    Id Target Id Frame
    4 Thread 0x7f131f1b4700 (LWP 11)
    "ruby-timer-thr" in poll ()
    at ../sysdeps/unix/syscall-temp...
    3 Thread 0x7f131db8a700 (LWP 12)
    “service.rb:6"
    [email protected]@GLIBC_2.3.2 ()
    2 Thread 0x7f131d988700 (LWP 13)
    “service.rb:12"
    vm_exec_core (th=0x7f131d989058...
    * 1 Thread 0x7f131f1aa700 (LWP 10)
    "ruby" [email protected]@GLIBC...
    >>

    View Slide

  19. 19
    (gdb) info threads
    Id Target Id Frame
    4 Thread 0x7f131f1b4700 (LWP 11)
    "ruby-timer-thr" in poll ()
    at ../sysdeps/unix/syscall-temp...
    3 Thread 0x7f131db8a700 (LWP 12)
    “service.rb:6"
    [email protected]@GLIBC_2.3.2 ()
    2 Thread 0x7f131d988700 (LWP 13)
    “service.rb:12"
    vm_exec_core (th=0x7f131d989058...
    * 1 Thread 0x7f131f1aa700 (LWP 10)
    "ruby" [email protected]@GLIBC...

    View Slide

  20. 20
    (gdb) info threads
    Id Target Id Frame
    4 Thread 0x7f131f1b4700 (LWP 11)
    "ruby-timer-thr" in poll ()
    at ../sysdeps/unix/syscall-temp...
    3 Thread 0x7f131db8a700 (LWP 12)
    “service.rb:6"
    [email protected]@GLIBC_2.3.2 ()
    2 Thread 0x7f131d988700 (LWP 13)
    “service.rb:12"
    vm_exec_core (th=0x7f131d989058...
    * 1 Thread 0x7f131f1aa700 (LWP 10)
    "ruby" [email protected]@GLIBC...

    View Slide

  21. 21
    (gdb) thread apply all backtrace
    Thread 3 (Thread 0x7f131db8a700 (LWP 12)):
    #0 [email protected]@GLIBC_2.3.2 () at ..
    #1 0x00007f131f343b03 in native_cond_wait
    #2 lock_func (timeout_ms=0, mutex=0x7f131f
    #3 rb_mutex_lock (self=139720114712840) at
    #4 0x00007f131f31c9ee in vm_call_cfunc_wit
    #5 vm_call_cfunc ([email protected]=0x7f131fcdda
    #6 0x00007f131f32c82e in vm_call_method (t
    #7 0x00007f131f32108b in vm_exec_core (th=
    #8 0x00007f131f3255d8 in vm_exec (th=0x7f1
    #9 0x00007f131f32be0c in invoke_block_from
    #10 0x00007f131f32bfa0 in vm_invoke_proc (t
    >>

    View Slide

  22. 22
    (gdb) thread apply all backtrace
    Thread 3 (Thread 0x7f131db8a700 (LWP 12)):
    #0 [email protected]@GLIBC_2.3.2 () at ..
    #1 0x00007f131f343b03 in native_cond_wait
    #2 lock_func (timeout_ms=0, mutex=0x7f131f
    #3 rb_mutex_lock (self=139720114712840) at
    #4 0x00007f131f31c9ee in vm_call_cfunc_wit
    #5 vm_call_cfunc ([email protected]=0x7f131fcdda
    #6 0x00007f131f32c82e in vm_call_method (t
    #7 0x00007f131f32108b in vm_exec_core (th=
    #8 0x00007f131f3255d8 in vm_exec (th=0x7f1
    #9 0x00007f131f32be0c in invoke_block_from
    >>

    View Slide

  23. 23
    ---Type to continue, or q
    Thread 1 (Thread 0x7f131f1aa700 (LWP 10)):
    #0 [email protected]@GLIBC_2.3.2 () at ..
    #1 0x00007f131f343b03 in native_cond_wait
    #2 lock_func (timeout_ms=0, mutex=0x7f131f
    #3 rb_mutex_lock (self=139720114712880) at
    #4 0x00007f131f31c9ee in vm_call_cfunc_wit
    #5 vm_call_cfunc ([email protected]=0x7f131f8c46
    #6 0x00007f131f32c82e in vm_call_method (t
    #7 0x00007f131f32108b in vm_exec_core (th=
    #8 0x00007f131f3255d8 in vm_exec ([email protected]
    #9 0x00007f131f33319f in rb_iseq_eval_main
    >>

    View Slide

  24. 24
    (gdb) exit
    Undefined command: "exit". Try “help".
    (gdb)
    >>

    View Slide

  25. 25
    (gdb) exit
    Undefined command: "exit". Try “help”.
    (gdb) quit
    $
    >>

    View Slide

  26. https://flic.kr/p/o15Y5n
    26
    Attach
    gdb -p PID

    View Slide

  27. https://flic.kr/p/o15Y5n
    27
    Inspect
    backtrace
    info threads
    thread apply all backtrace

    View Slide

  28. https://flic.kr/p/o15Y5n
    28
    $$$$
    exit
    quit

    View Slide

  29. A Unique
    Problem
    29
    https://flic.kr/p/dPiDp3

    View Slide

  30. 30
    [1] pry(main)> [1,1,2,3].uniq!
    => [1, 2, 3]

    View Slide

  31. 31
    [1] pry(main)> [1,1,2,3].uniq!
    => [1, 2, 3]
    [2] pry(main)> [1,2,3].uniq!
    => nil

    View Slide

  32. 32
    $ cat snowflake.rb
    p [1,1,2,3].uniq!
    p [1,2,3].uniq!

    View Slide

  33. 33
    $ cat snowflake.rb
    p [1,1,2,3].uniq!
    p [1,2,3].uniq!
    $ ruby snowflake.rb
    [1, 2, 3]
    nil

    View Slide

  34. 34
    $ gdb --args ruby snowflake.rb
    GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
    Copyright (C) 2014 Free Software
    Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or
    later gpl.html>
    ...
    For help, type "help".
    Type "apropos word" to search for
    commands related to "word".
    (gdb)
    >>

    View Slide

  35. 35
    (gdb) run
    Starting program: /usr/local/bin/ruby...
    [Thread debugging using libthread_db
    enabled]
    Using host libthread_db library "/lib/
    x86_64-linux-gnu/libthread_db.so.1".
    [New Thread 0x7ffff7ff5700 (LWP 13)]
    [1,2,3]
    nil
    [Thread 0x7ffff7ff5700 (LWP 13) exited]
    [Inferior 1 (process 9) exited normally]
    (gdb)
    >>

    View Slide

  36. 36
    (gdb) run
    Starting program: /usr/local/bin/ruby...
    [Thread debugging using libthread_db
    enabled]
    Using host libthread_db library "/lib/
    x86_64-linux-gnu/libthread_db.so.1".
    [New Thread 0x7ffff7ff5700 (LWP 13)]
    [1,2,3]
    nil
    [Thread 0x7ffff7ff5700 (LWP 13) exited]
    [Inferior 1 (process 9) exited normally]
    (gdb)
    >>

    View Slide

  37. 37

    View Slide

  38. 38

    View Slide

  39. 39

    View Slide

  40. Eeeeeeek!
    https://flic.kr/p/nkwswv

    View Slide

  41. 41
    int the_answer (long value)
    {
    return 42;
    }

    View Slide

  42. 42
    int the_answer (long value)
    {
    return 42;
    }

    View Slide

  43. 43
    int the_answer (long value)
    {
    return 42;
    }

    View Slide

  44. 44
    int the_answer (long value)
    {
    return 42;
    }

    View Slide

  45. 45
    int the_answer (long value)
    {
    return 42;
    }

    View Slide

  46. 46
    static VALUE
    rb_ary_uniq_bang(VALUE ary)
    {
    VALUE hash;
    long hash_size;
    rb_ary_modify_check(ary);
    if (RARRAY_LEN(ary) <= 1)
    return Qnil;
    if (rb_block_given_p())
    hash = ary_make_hash_by(ary);
    else
    hash = ary_make_hash(ary);
    hash_size = RHASH_SIZE(hash);
    if (RARRAY_LEN(ary) == hash_size) {

    View Slide

  47. 47
    static VALUE
    rb_ary_uniq_bang(VALUE ary)
    {
    VALUE hash;
    long hash_size;
    rb_ary_modify_check(ary);
    if (RARRAY_LEN(ary) <= 1)
    return Qnil;
    if (rb_block_given_p())
    hash = ary_make_hash_by(ary);
    else
    hash = ary_make_hash(ary);
    hash_size = RHASH_SIZE(hash);
    if (RARRAY_LEN(ary) == hash_size) {

    View Slide

  48. 48
    static VALUE
    rb_ary_uniq_bang(VALUE ary)
    {
    VALUE hash;
    long hash_size;
    rb_ary_modify_check(ary);
    if (RARRAY_LEN(ary) <= 1)
    return Qnil;
    if (rb_block_given_p())
    hash = ary_make_hash_by(ary);
    else
    hash = ary_make_hash(ary);
    hash_size = RHASH_SIZE(hash);
    if (RARRAY_LEN(ary) == hash_size) {

    View Slide

  49. 49
    $ gdb --args ruby snowflake.rb
    ...
    Reading symbols from ruby...done.
    (gdb) break rb_ary_uniq_bang
    Breakpoint 1 at 0x19a370: file array.c,
    line 4162.
    (gdb)
    >>

    View Slide

  50. 50
    (gdb) run
    Starting program: /usr/local/bin/ruby
    snowflake.rb
    Breakpoint 1, rb_ary_uniq_bang
    (ary=93825000955160) at array.c:4162
    4162 {
    (gdb)
    >>

    View Slide

  51. 51
    (gdb) run
    Starting program: /usr/local/bin/ruby
    snowflake.rb
    Breakpoint 1, rb_ary_uniq_bang
    (ary=93825000955160) at array.c:4162
    4162 {
    (gdb)
    >>

    View Slide

  52. 52
    (gdb) run
    Starting program: /usr/local/bin/ruby
    snowflake.rb
    Breakpoint 1, rb_ary_uniq_bang
    (ary=93825000955160) at array.c:4162
    4162 {
    (gdb)
    >>

    View Slide

  53. https://flic.kr/p/o15Y5n
    53
    Starting
    gdb --args ruby snowflake.rb

    View Slide

  54. https://flic.kr/p/o15Y5n
    54
    C functions!
    int the_answer (long value)

    View Slide

  55. https://flic.kr/p/o15Y5n
    55
    ruby-doc.org
    rb_ary_uniq_bang

    View Slide

  56. https://flic.kr/p/o15Y5n
    56
    Breakpoints
    break rb_ary_uniq_bang

    View Slide

  57. Looking
    Around
    https://flic.kr/p/uq8MA

    View Slide

  58. 58
    (gdb) list
    4157 *
    4158 */
    4159
    4160 static VALUE
    4161 rb_ary_uniq_bang(VALUE ary)
    4162 {
    4163 VALUE hash;
    4164 long hash_size;
    4165
    4166 rb_ary_modify_check(ary);
    >>

    View Slide

  59. 59
    (gdb) display ary
    1: ary = 93825000955160
    >>

    View Slide

  60. Eeeeeeek!
    https://flic.kr/p/nkwswv

    View Slide

  61. 61
    struct Data
    {
    int value;
    }

    View Slide

  62. 62
    struct Data
    {
    int value;
    }
    value
    0xC001C0DE

    View Slide

  63. 63
    struct ValuableData
    {
    Data data;
    int priority;
    }
    value
    priority
    0xC001C0DE

    View Slide

  64. 64
    struct RBasic {
    VALUE flags;
    const VALUE klass;
    }
    flags
    klass
    0xBAADC0DE

    View Slide

  65. struct RArray {
    struct RBasic basic;
    union {
    struct {
    long len;
    union {
    long capa;
    VALUE shared;
    } aux;
    const VALUE *ptr;
    } heap;
    const VALUE ary[RARRAY_EMBED_LEN_MAX];
    } as;
    };
    65
    flags
    klass
    len
    capa
    ...

    View Slide

  66. 66
    RBasic
    RClass
    RObject
    RBignum
    RRational
    RFloat
    RComplex
    RHash
    RSymbol
    RString
    RArray
    RRegexp

    View Slide

  67. 67
    (gdb) display ary
    1: ary = 93825000955160
    flags
    klass
    0x555555da5d18
    >>

    View Slide

  68. 68
    (gdb) display *(struct RArray*)ary
    1: ary = {basic = {flags = 106503,
    klass = 9382499777180}, as = {heap =
    {len = 3, aux = {capa = 5, shared = 5},
    ptr = 0x7}, ary = {3, 5, 7}}}
    >>

    View Slide

  69. 69
    (gdb) display *(struct RArray*)ary
    1: ary = {basic = {flags = 106503,
    klass = 9382499777180}, as = {heap =
    {len = 3, aux = {capa = 5, shared = 5},
    ptr = 0x7}, ary = {3, 5, 7}}}
    >>

    View Slide

  70. 70
    /* for debug print within C code */
    void rb_p(VALUE obj)

    View Slide

  71. 71
    (gdb) call rb_p(ary)
    [">= 0"]
    >>

    View Slide

  72. https://flic.kr/p/o15Y5n
    72
    Source
    list

    View Slide

  73. https://flic.kr/p/o15Y5n
    73
    Look
    display ary

    View Slide

  74. https://flic.kr/p/o15Y5n
    74
    struct
    struct C {
    BOOL scary /* FALSE */
    }

    View Slide

  75. https://flic.kr/p/o15Y5n
    75
    Debug Print
    call rb_p(ary)

    View Slide

  76. 76
    https://flic.kr/p/tLkpZG

    View Slide

  77. 77
    (gdb) call rb_p(ary)
    [">= 0"]
    >>

    View Slide

  78. 78
    (gdb) call rb_backtrace()
    from :1:in `'
    from :1:in `require'
    from rubygems.rb:115:in `'
    from rubygems.rb:1210:in `'
    from rubygems.rb:1210:in `require'
    from rubygems/specification.rb:37:in `’
    from rubygems/specification.rb:146:in `’
    from rubygems/requirement.rb:74:in `default'
    from rubygems/requirement.rb:74:in `new'
    from rubygems/requirement.rb:125:in `initialize'
    from rubygems/requirement.rb:125:in `uniq!'
    >>

    View Slide

  79. 79
    (gdb) call rb_backtrace()
    from :1:in `'
    from :1:in `require'
    from rubygems.rb:115:in `'
    from rubygems.rb:1210:in `'
    from rubygems.rb:1210:in `require'
    from rubygems/specification.rb:37:in `’
    from rubygems/specification.rb:146:in `’
    from rubygems/requirement.rb:74:in `default'
    from rubygems/requirement.rb:74:in `new'
    from rubygems/requirement.rb:125:in `initialize'
    from rubygems/requirement.rb:125:in `uniq!'
    >>

    View Slide

  80. 80
    $ ruby --disable-gems snowflake.rb

    View Slide

  81. Move It!
    https://flic.kr/p/mAst2V

    View Slide

  82. 82
    (gdb) list
    4157 *
    4158 */
    4159
    4160 static VALUE
    4161 rb_ary_uniq_bang(VALUE ary)
    4162 {
    4163 VALUE hash;
    4164 long hash_size;
    4165
    4166 rb_ary_modify_check(ary);
    >>

    View Slide

  83. 83
    (gdb) next
    4166 rb_ary_modify_check(ary);
    >>

    View Slide

  84. 84
    (gdb) step
    rb_ary_modify_check (ary=9382499768664)
    at array.c:312
    312 rb_check_frozen(ary);
    >>

    View Slide

  85. 85
    (gdb) step
    rb_ary_modify_check (ary=9382499768664)
    at array.c:312
    312 rb_check_frozen(ary);
    (gdb) list
    309 static inline void
    310 rb_ary_modify_check(VALUE ary)
    311 {
    312 rb_check_frozen(ary);
    313 }
    >>

    View Slide

  86. 86
    (gdb) finish
    Run till exit from #0
    rb_ary_modify_check (ary=9382500108300)
    at array.c:312
    4167 if (RARRAY_LEN(ary) <= 1)
    (gdb)
    >>

    View Slide

  87. 87
    (gdb) list
    4164 long hash_size;
    4165
    4166 rb_ary_modify_check(ary);
    4167 if (RARRAY_LEN(ary) <= 1)
    4168 return Qnil;
    4169 if (rb_block_given_p())
    4170 hash = ary_make_hash_by(ary);
    4171 else
    4172 hash = ary_make_hash(ary);
    4173
    >>

    View Slide

  88. 88
    (gdb) list
    4164 long hash_size;
    4165
    4166 rb_ary_modify_check(ary);
    4167 if (RARRAY_LEN(ary) <= 1)
    4168 return Qnil;
    4169 if (rb_block_given_p())
    4170 hash = ary_make_hash_by(ary);
    4171 else
    4172 hash = ary_make_hash(ary);
    4173
    >>

    View Slide

  89. 89
    (gdb) list
    4170 hash = ary_make_hash_by(ary);
    4171 else
    4172 hash = ary_make_hash(ary);
    4173
    4174 hash_size = RHASH_SIZE(hash);
    4175 if (RARRAY_LEN(ary) == hash_size) {
    4176 return Qnil;
    4177 }
    4178 rb_ary_modify_check(ary);
    4179 ARY_SET_LEN(ary, 0);
    >>

    View Slide

  90. 90
    (gdb) continue
    Continuing.
    >>

    View Slide

  91. 91

    View Slide

  92. https://flic.kr/p/o15Y5n
    92
    Movin’
    next
    step
    finish
    list
    continue

    View Slide

  93. Silly GDB
    Tricks
    https://flic.kr/p/dHofYV

    View Slide

  94. 94
    (gdb) call (void)close(1)
    (gdb) call (void)close(2)
    (gdb) call (int)open(“gdb.log”, 2, 0)
    (gdb) call (int)open(“gdb.log”, 2, 0)

    View Slide

  95. 95
    (gdb) call(rb_p(rb_eval_string_protect(
    "Thread.list.each do |t|
    puts t;
    puts t.backtrace.join(‘\n');
    end",(int*)0)))
    #
    eval:1:in `backtrace'
    eval:1:in `block in '
    eval:1:in `each'
    eval:1:in `'
    drop.rb:2:in `drop'
    drop.rb:2:in `'
    >>

    View Slide

  96. 96
    (gdb) call(rb_p(rb_eval_string_protect(
    "Thread.list.each do |t|
    puts t;
    puts t.backtrace.join(‘\n');
    end",(int*)0)))
    #
    eval:1:in `backtrace'
    eval:1:in `block in '
    eval:1:in `each'
    eval:1:in `'
    drop.rb:2:in `drop'
    drop.rb:2:in `'

    View Slide

  97. 97
    $ cat gdb_script
    attach 107
    thread all apply backtrace
    call (void)rb_backtrace()
    quit
    >>

    View Slide

  98. 98
    $ gdb --batch -x gdb_script
    ...
    Thread 2 (Thread 0x7f09a71d070 (LWP 108)):
    #0 0x00007f09a638e50d in poll () at
    ../sysdeps/unix/syscall-template.S:81
    #1 0x00007f09a735d in timer_thread_sleep
    at thread_pthread.c:1442
    #2 thread_timer (p=0x7f09a9466018) at
    thread_pthread.c:1549
    #3 0x00007f09a6d9e0a4 in start_thread
    at pthread_create.c:309
    Inferior 1 [process 10] will be detached.
    $
    >>

    View Slide

  99. 99
    $ gem install newrelic_rpm
    $ nrdebug 1234
    Are you sure you want to attach to PID
    1234 ('ruby -e loop do end')}?
    Extracting debug information from this
    process may cause it to hang, crash,
    or otherwise malfunction...
    To continue, type 'continue':
    Generated 'nrdebug-1234-1447787527.log'

    View Slide

  100. 100
    $ cat ~/.gdbinit
    define exit
    quit
    end
    >>

    View Slide

  101. Recap
    101
    https://flic.kr/p/rx2WTU

    View Slide

  102. Deadlock?
    https://flic.kr/p/nkwswv

    View Slide

  103. A Unique
    Problem
    103
    https://flic.kr/p/dPiDp3

    View Slide

  104. Looking
    Around
    https://flic.kr/p/uq8MA

    View Slide

  105. Move It!
    https://flic.kr/p/mAst2V

    View Slide

  106. Silly GDB
    Tricks
    https://flic.kr/p/dHofYV

    View Slide

  107. https://flic.kr/p/8RePxD
    ?
    @jasonrclark
    http://git.io/v8GbT
    https://flic.kr/p/ebUyxU

    View Slide