Building a manhole

Transcript

1. Building a manhole in your app Tuesday, 15 October, 13

2. Apps with Boogz • Situation: Your software is acting up.

   Your software is *not* a web app, but rather a daemon process (like a worker of some sort). • How do you debug? Tuesday, 15 October, 13

3. Tools • Standard “print” statements - there is nothing wrong

   with this. • pdb - very useful and powerful, requires you drop a set_trace(). • django-extension/werkzeug - handy for web applications. Tuesday, 15 October, 13

4. Wouldn’t it be nice to see the bug in action?

   Tuesday, 15 October, 13

5. Build a manhole/ backdoor... Tuesday, 15 October, 13

6. ... with signals • Using standard Unix signals, one can

   communicate with a running process: $ cat test.py #!/usr/bin/python from time import sleep from datetime import datetime import signal def run_debug(signal, frame): import pdb; pdb.set_trace() if __name__ == '__main__': signal.signal(signal.SIGUSR1, run_debug) while True: print datetime.now() sleep(1) Tuesday, 15 October, 13

7. In action: window1 $ python test.py 2013-10-15 16:33:59.697757 2013-10-15 16:34:00.698924

   2013-10-15 16:34:01.700167 2013-10-15 16:34:02.700923 2013-10-15 16:34:03.701884 2013-10-15 16:34:04.703049 2013-10-15 16:34:05.704226 2013-10-15 16:34:06.705380 2013-10-15 16:34:07.706541 2013-10-15 16:34:08.707743 2013-10-15 16:34:09.708956 2013-10-15 16:34:10.710099 2013-10-15 16:34:11.711319 2013-10-15 16:34:12.712514 2013-10-15 16:34:13.713713 --Return-- > /root/test.py(8)run_debug()->None -> import pdb; pdb.set_trace() window2 $ ps ax | grep test.py 28475 pts/1 S+ 0:00 python test.py window2 $ kill -USR1 28458 Tuesday, 15 October, 13

8. What about daemons? • This technique is great when you’re

   running a script inside a terminal, but what about background processes? • They are not attached to a TTY. Tuesday, 15 October, 13

9. reptyr • reptyr - Reparent a running program to a

   new terminal. • Allows you to rewire STDIN/STDOUT/ STDERR to a new terminal. Tuesday, 15 October, 13

10. reptyr in action: window1 $ python test.py 2013-10-15 16:50:22.989920 2013-10-15

    16:50:23.991065 2013-10-15 16:50:24.992256 ^Z [1]+ Stopped python test.py window1 $ bg # puts the job in the background [1]+ python test.py & window1 $ disown # disassociates from the running TTY Tuesday, 15 October, 13

11. reptyr in action...: window2 $ ps ax | grep test.py

    28902 pts/1 S 0:00 python test.py window2 $ reptyr -s 28902 [+] Allocated scratch page: 7f56d35e0000 [+] Opened the new tty in the child: 3 [+] Target is not a session leader, attempting to setsid. [+] Forked a child: 29040 [+] Change pgid for pid 28902 [+] Did setsid() [+] Set the controlling tty 2013-10-15 16:53:51.362533 2013-10-15 16:53:52.363716 2013-10-15 16:53:53.364880 --Return-- > /root/test.py(8)run_debug()->None -> import pdb; pdb.set_trace() (Pdb) window3 $ kill -USR1 28902 Tuesday, 15 October, 13

12. Caveats • reptyr will steal STDIN/STDOUT/ STDERR. • Once stolen,

    it cannot be given back • You will have to restart the process once you’re done debugging to restore order. Tuesday, 15 October, 13

13. Questions? Hany Fahim (@iHandroid) Tuesday, 15 October, 13