TASBot - the perfectionist

A slide deck in collaboration with dwangoAC
Presented at DEF CON 24
Video recording at https://www.youtube.com/watch?v=2kj5ZmoTckI

Transcript

1. dwangoAC TASBot the perfectionist The amazing life & achievements of... Twitch.tv/dwangoAC twitter @MrTASBot
2. Allan 'dwangoAC' Cecil http://acbit.net Presented and written by...
3. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group http://nblug.org http://acbit.net Presented and written by...
4. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://acbit.net Presented and written by...
5. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://tasvideos.org/DwangoAC.html http://tasbot.net http://acbit.net Presented and written by...
7. Speedrunning Human limits
8. Playing games fast http://speeddemosarchive.com/
9. Playing games fast http://speeddemosarchive.com/ • Inspiration: in-game completion timers
10. • SpeedDemosArchive.com and others track fastest completion times • Strict rules + peer review: no cheats, no macros • Typically highly entertaining • Many categories, ranging from "any%" to "low% no major glitches" Playing games fast http://speeddemosarchive.com/ • Inspiration: in-game completion timers
11. Games Done Quick
12. Games Done Quick Speedrunning marathons for charity streamed live on Twitch Classic GDQ (2010), Awesome GDQ (2011-), Summer GDQ (2011-)
13. Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014
14. Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014 Metroid 15:43 World Record https://www.youtube.com/watch?v=67kQ3l-1qMs
15. https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016
16. Punch-Out blindfolded by Sinister1 - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016
17. Even 1-handed, blindfolded... Beyond standard limits! Punch-Out blindfolded by Sinister1 - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016
18. TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.”
19. TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits
20. TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits
21. Harder Faster Better Stronger
22. Harder Faster Better Stronger • Early PC game TAS’s: Savestates, slow motion, and recording tools
23. Harder Faster Better Stronger • Early PC game TAS’s: Savestates, slow motion, and recording tools • ~1999: Doom Done Quick in 19:41
24. https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm
25. https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm
26. • Tools meant hardware limits became the only limits Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
27. ◦ Competitors should admit to doping ◦ Videos made with TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
28. • NESVideos created by Bisqwit in 2004 ◦ Competitors should admit to doping ◦ Videos made with TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
29. • NESVideos created by Bisqwit in 2004 ◦ Now at TASVideos.org with runs for many platforms ◦ Competitors should admit to doping ◦ Videos made with TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
30. the birth of TASBot
31. the birth of TASBot Console verified Pushing hardware limits
32. the birth of TASBot Console verified Pushing hardware limits
33. Console emulators http://tasvideos.org/Lsnes.html lsnes BizHawk http://tasvideos.org/BizHawk.html
34. Rerecording frameworks Hourglass NetHack specific tools http://tasvideos.org/EmulatorResources/Hourglass.html http://tasvideos.org/GameResources/DOS/Nethack.html
35. Emulation accuracy evolution
36. • Clean room reverse engineering ◦ or stolen manuals • Early emulators: highly inaccurate Emulation accuracy evolution
37. • bsnes: extreme accuracy, poor usability • Clean room reverse engineering ◦ or stolen manuals • Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy
38. • bsnes: extreme accuracy, poor usability • Clean room reverse engineering ◦ or stolen manuals • Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan
39. • bsnes: extreme accuracy, poor usability • Clean room reverse engineering ◦ or stolen manuals • Early emulators: highly inaccurate ⇒ match actual hardware, frame for frame Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan
41. Memory searching, Lua scripting, disassembly https://www.lua.org/
42. • More than just frame advance and savestates Memory searching, Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
43. • More than just frame advance and savestates • Find a specific value: save, reset memory search, run ◦ Search based on conditions, repeat Memory searching, Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
44. • More than just frame advance and savestates • Find a specific value: save, reset memory search, run ◦ Search based on conditions, repeat Memory searching, Lua scripting, disassembly • Disassembly of RAM or ROM for complete understanding https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
45. Abusing handwriting recognition https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016
46. Abusing handwriting recognition Editing memory live directly in the game SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=12m28s https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016
48. TAS ⇔ Infosec equivalents • Savestate = VM snapshot • Frame advance = VM CPU step / tick • Glitch = Vulnerability • Arbitrary Code Execution = Exploit • Console verification = Evil maid attack ⇒ TAS = fun, technical, educational
49. AGDQ 2016 https://youtu.be/pj7RE2DcRgc?t=50m23s SMB3 Total Control Glitchfest by Lord Tom
50. Super Mario World Super Mario Bros. TASBot
51. Super Mario World Super Mario Bros. TASBot plays
52. Super Mario World Super Mario Bros. TASBot plays
53. Early console verification devices
54. Early console verification devices • 2009 ◦ a PIC to press NES buttons [true]
55. • 2011 ◦ NESBot [micro500]: first replay of SMB1 ▪ Used at SGDQ 2011 on SMB2 and W&W 3 Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY • 2009 ◦ a PIC to press NES buttons [true]
56. • 2011 ◦ NESBot [micro500]: first replay of SMB1 ▪ Used at SGDQ 2011 on SMB2 and W&W 3 ◦ Droid64 [SoulCal] • 2012 ◦ N64 [micro500] Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY • 2009 ◦ a PIC to press NES buttons [true]
57. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦ NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates
58. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦ NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot"
59. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦ NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot" • 2015 ◦ Multireplay device [true]: self-contained ⇒ faster datarates
60. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦ NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot" • 2015 ◦ Multireplay device [true]: self-contained ⇒ faster datarates ◦ Game Boy Player Player [endrift] (GBA on GameCube)
61. TASBot the perfectionist
62. Super Mario World Super Mario Bros. TASBot
63. Super Mario World Super Mario Bros. TASBot plays
64. Super Mario World Super Mario Bros. TASBot plays
65. Super Mario World Super Mario Bros. TASBot plays in
66. Super Mario World Super Mario Bros. TASBot plays in SMB in SMW by p4plus2 and Masterjun
67. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...
68. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input...
69. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input... A homemade port of the NES classic is sent as payload...
70. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input... A homemade port of the NES classic is sent as payload... A 8-bit game, on a 16-bit system!
71. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool You can write specific sequences in the Object Attribute Memory by using specific objects at specific coordinates,
72. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences...
73. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want.
74. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want. So, just via input...
75. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want. So, just via input... ...you can directly trigger the credits sequence!
76. TASLink ~184 Kbps was too limiting http://taslink.org
77. 32Mhz FPGA Papilio Pro's Spartan 6 LX max poll rate of the serial port (2Mb/s) http://papilio.gadgetfactory.net/index.php?n=Papilio.PapilioPro
78. SMB1+2+3+Lost Levels played simultaneously during SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=58m29s
79. Anatomy of an Arbitrary Code Execution
80. 1. Input exploit Anatomy of an Arbitrary Code Execution Pokemon Red
81. 1. Input exploit 2. Take over the Super GameBoy Anatomy of an Arbitrary Code Execution Pokemon Red
82. 1. Input exploit 2. Take over the Super GameBoy 3. Gain full access to the Super Nintendo Anatomy of an Arbitrary Code Execution Pokemon Red
83. 1. Input exploit 2. Take over the Super GameBoy 3. Gain full access to the Super Nintendo 4. Anything is possible Anatomy of an Arbitrary Code Execution Pokemon Red
86. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
87. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
88. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
89. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
90. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
91. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
92. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2
93. Call to action Join the chat for Q&A at http://twitch.tv/dwangoAC
94. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot...
95. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames!
96. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames! 6000 buttons per second!
97. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames! Some glitches are expected! 6000 buttons per second!
98. DPCM memory ↕ game controller Flood weak controller code to abuse raster interrupt and take over execution conflict http://www.qmtpro.com/~nes/chipimages/#rp2a03 http://arstechnica.com/gaming/2016/07/how-to-beat-super-mario-bros-3-in-less-than-a-second/
99. TAS'ers lethal weapon • More flexible than IDA • Graph view, low level IL and annotation support • Python scripting • NES support: ability to add new mappers
100. ♫♪ Am I…
101. cheating? ♫♪ Am I…
102. cheating? ♫♪ Am I… ♬ No
103. cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬ No, I'm just looking for...
104. cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬ No, I'm just looking for... ♩ And I'm not the only one… ;)
105. Medecins sans Frontières Doctors without borders ♩♬ But more importantly….
106. Medecins sans Frontières Doctors without borders Prevent Cancer Foundation Games Done Quick Raised for charity! over $200k USD ♩♬ But more importantly…. http://tasvideos.org/forum/viewtopic.php?p=437688#437688
107. micro500 Ilari Thanks to:
108. micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty
109. micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty TheAxeMan ange_ greenfly ais523 and many, many others
110. In collaboration with Ange Albertini ? @MrTASBot Twitch.tv/dwangoAC