Pro Yearly is on sale from $80 to $50! »

TASBot - the perfectionist

TASBot - the perfectionist

A slide deck in collaboration with dwangoAC
Presented at DEF CON 24
Video recording at https://www.youtube.com/watch?v=2kj5ZmoTckI

261a01e1b07b7387b0d675322199fb58?s=128

Ange Albertini

August 04, 2016
Tweet

Transcript

  1. dwangoAC TASBot the perfectionist The amazing life & achievements of...

    Twitch.tv/dwangoAC twitter @MrTASBot
  2. Allan 'dwangoAC' Cecil http://acbit.net Presented and written by...

  3. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’

    Group http://nblug.org http://acbit.net Presented and written by...
  4. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’

    Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://acbit.net Presented and written by...
  5. Allan 'dwangoAC' Cecil President of the North Bay Linux Users’

    Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://tasvideos.org/DwangoAC.html http://tasbot.net http://acbit.net Presented and written by...
  6. None
  7. Speedrunning Human limits

  8. Playing games fast http://speeddemosarchive.com/

  9. Playing games fast http://speeddemosarchive.com/ • Inspiration: in-game completion timers

  10. • SpeedDemosArchive.com and others track fastest completion times • Strict

    rules + peer review: no cheats, no macros • Typically highly entertaining • Many categories, ranging from "any%" to "low% no major glitches" Playing games fast http://speeddemosarchive.com/ • Inspiration: in-game completion timers
  11. Games Done Quick

  12. Games Done Quick Speedrunning marathons for charity streamed live on

    Twitch Classic GDQ (2010), Awesome GDQ (2011-), Summer GDQ (2011-)
  13. Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014

  14. Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014 Metroid 15:43 World Record https://www.youtube.com/watch?v=67kQ3l-1qMs

  15. https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016

  16. Punch-Out blindfolded by Sinister1 - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora

    by Halfcoordinated - SGDQ 2016
  17. Even 1-handed, blindfolded... Beyond standard limits! Punch-Out blindfolded by Sinister1

    - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016
  18. TAS verb / noun ~ TASer noun “I’m a TASer

    working on Tetris.” / “I’m TASing Tetris.”
  19. TAS verb / noun ~ TASer noun “I’m a TASer

    working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits
  20. TAS verb / noun ~ TASer noun “I’m a TASer

    working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits
  21. Harder Faster Better Stronger

  22. Harder Faster Better Stronger • Early PC game TAS’s: Savestates,

    slow motion, and recording tools
  23. Harder Faster Better Stronger • Early PC game TAS’s: Savestates,

    slow motion, and recording tools • ~1999: Doom Done Quick in 19:41
  24. https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm

  25. https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm

  26. • Tools meant hardware limits became the only limits Inhuman

    skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
  27. ◦ Competitors should admit to doping ◦ Videos made with

    TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
  28. • NESVideos created by Bisqwit in 2004 ◦ Competitors should

    admit to doping ◦ Videos made with TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
  29. • NESVideos created by Bisqwit in 2004 ◦ Now at

    TASVideos.org with runs for many platforms ◦ Competitors should admit to doping ◦ Videos made with TAS tools should be labeled • Tools meant hardware limits became the only limits • TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/
  30. the birth of TASBot

  31. the birth of TASBot Console verified Pushing hardware limits

  32. the birth of TASBot Console verified Pushing hardware limits

  33. Console emulators http://tasvideos.org/Lsnes.html lsnes BizHawk http://tasvideos.org/BizHawk.html

  34. Rerecording frameworks Hourglass NetHack specific tools http://tasvideos.org/EmulatorResources/Hourglass.html http://tasvideos.org/GameResources/DOS/Nethack.html

  35. Emulation accuracy evolution

  36. • Clean room reverse engineering ◦ or stolen manuals •

    Early emulators: highly inaccurate Emulation accuracy evolution
  37. • bsnes: extreme accuracy, poor usability • Clean room reverse

    engineering ◦ or stolen manuals • Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy
  38. • bsnes: extreme accuracy, poor usability • Clean room reverse

    engineering ◦ or stolen manuals • Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan
  39. • bsnes: extreme accuracy, poor usability • Clean room reverse

    engineering ◦ or stolen manuals • Early emulators: highly inaccurate ⇒ match actual hardware, frame for frame Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan
  40. None
  41. Memory searching, Lua scripting, disassembly https://www.lua.org/

  42. • More than just frame advance and savestates Memory searching,

    Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
  43. • More than just frame advance and savestates • Find

    a specific value: save, reset memory search, run ◦ Search based on conditions, repeat Memory searching, Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
  44. • More than just frame advance and savestates • Find

    a specific value: save, reset memory search, run ◦ Search based on conditions, repeat Memory searching, Lua scripting, disassembly • Disassembly of RAM or ROM for complete understanding https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/
  45. Abusing handwriting recognition https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016

  46. Abusing handwriting recognition Editing memory live directly in the game

    SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=12m28s https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016
  47. None
  48. TAS ⇔ Infosec equivalents • Savestate = VM snapshot •

    Frame advance = VM CPU step / tick • Glitch = Vulnerability • Arbitrary Code Execution = Exploit • Console verification = Evil maid attack ⇒ TAS = fun, technical, educational
  49. AGDQ 2016 https://youtu.be/pj7RE2DcRgc?t=50m23s SMB3 Total Control Glitchfest by Lord Tom

  50. Super Mario World Super Mario Bros. TASBot

  51. Super Mario World Super Mario Bros. TASBot plays

  52. Super Mario World Super Mario Bros. TASBot plays

  53. Early console verification devices

  54. Early console verification devices • 2009 ◦ a PIC to

    press NES buttons [true]
  55. • 2011 ◦ NESBot [micro500]: first replay of SMB1 ▪

    Used at SGDQ 2011 on SMB2 and W&W 3 Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY • 2009 ◦ a PIC to press NES buttons [true]
  56. • 2011 ◦ NESBot [micro500]: first replay of SMB1 ▪

    Used at SGDQ 2011 on SMB2 and W&W 3 ◦ Droid64 [SoulCal] • 2012 ◦ N64 [micro500] Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY • 2009 ◦ a PIC to press NES buttons [true]
  57. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦

    NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates
  58. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦

    NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot"
  59. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦

    NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot" • 2015 ◦ Multireplay device [true]: self-contained ⇒ faster datarates
  60. • 2013 ◦ SNES and Genesis Arduino bot [GhostSonic] ◦

    NES/SNES replay device [true] ▪ Streaming capable and inexpensive but limited datarates • 2014 ◦ Nintendo R.O.B + board + legos: "TASBot" • 2015 ◦ Multireplay device [true]: self-contained ⇒ faster datarates ◦ Game Boy Player Player [endrift] (GBA on GameCube)
  61. TASBot the perfectionist

  62. Super Mario World Super Mario Bros. TASBot

  63. Super Mario World Super Mario Bros. TASBot plays

  64. Super Mario World Super Mario Bros. TASBot plays

  65. Super Mario World Super Mario Bros. TASBot plays in

  66. Super Mario World Super Mario Bros. TASBot plays in SMB

    in SMW by p4plus2 and Masterjun
  67. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...

  68. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...

    Exploits it via input...
  69. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...

    Exploits it via input... A homemade port of the NES classic is sent as payload...
  70. http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...

    Exploits it via input... A homemade port of the NES classic is sent as payload... A 8-bit game, on a 16-bit system!
  71. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool You can write specific sequences in the Object

    Attribute Memory by using specific objects at specific coordinates,
  72. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary

    sequences...
  73. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary

    sequences... ...we can take over execution the way we want.
  74. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary

    sequences... ...we can take over execution the way we want. So, just via input...
  75. https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary

    sequences... ...we can take over execution the way we want. So, just via input... ...you can directly trigger the credits sequence!
  76. TASLink ~184 Kbps was too limiting http://taslink.org

  77. 32Mhz FPGA Papilio Pro's Spartan 6 LX max poll rate

    of the serial port (2Mb/s) http://papilio.gadgetfactory.net/index.php?n=Papilio.PapilioPro
  78. SMB1+2+3+Lost Levels played simultaneously during SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=58m29s

  79. Anatomy of an Arbitrary Code Execution

  80. 1. Input exploit Anatomy of an Arbitrary Code Execution Pokemon

    Red
  81. 1. Input exploit 2. Take over the Super GameBoy Anatomy

    of an Arbitrary Code Execution Pokemon Red
  82. 1. Input exploit 2. Take over the Super GameBoy 3.

    Gain full access to the Super Nintendo Anatomy of an Arbitrary Code Execution Pokemon Red
  83. 1. Input exploit 2. Take over the Super GameBoy 3.

    Gain full access to the Super Nintendo 4. Anything is possible Anatomy of an Arbitrary Code Execution Pokemon Red
  84. None
  85. None
  86. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  87. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  88. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  89. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  90. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  91. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  92. https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

  93. Call to action Join the chat for Q&A at http://twitch.tv/dwangoAC

  94. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot...

  95. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16

    frames!
  96. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16

    frames! 6000 buttons per second!
  97. https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16

    frames! Some glitches are expected! 6000 buttons per second!
  98. DPCM memory ↕ game controller Flood weak controller code to

    abuse raster interrupt and take over execution conflict http://www.qmtpro.com/~nes/chipimages/#rp2a03 http://arstechnica.com/gaming/2016/07/how-to-beat-super-mario-bros-3-in-less-than-a-second/
  99. TAS'ers lethal weapon • More flexible than IDA • Graph

    view, low level IL and annotation support • Python scripting • NES support: ability to add new mappers
  100. ♫♪ Am I…

  101. cheating? ♫♪ Am I…

  102. cheating? ♫♪ Am I… ♬ No

  103. cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬

    No, I'm just looking for...
  104. cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬

    No, I'm just looking for... ♩ And I'm not the only one… ;)
  105. Medecins sans Frontières Doctors without borders ♩♬ But more importantly….

  106. Medecins sans Frontières Doctors without borders Prevent Cancer Foundation Games

    Done Quick Raised for charity! over $200k USD ♩♬ But more importantly…. http://tasvideos.org/forum/viewtopic.php?p=437688#437688
  107. micro500 Ilari Thanks to:

  108. micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty

  109. micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty

    TheAxeMan ange_ greenfly ais523 and many, many others
  110. In collaboration with Ange Albertini ? @MrTASBot Twitch.tv/dwangoAC