Tweet
Tweet

Slide 1

Slide 1 text

dwangoAC TASBot the perfectionist The amazing life & achievements of... Twitch.tv/dwangoAC twitter @MrTASBot

Slide 2

Slide 2 text

Allan 'dwangoAC' Cecil http://acbit.net Presented and written by...

Slide 3

Slide 3 text

Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group http://nblug.org http://acbit.net Presented and written by...

Slide 4

Slide 4 text

Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://acbit.net Presented and written by...

Slide 5

Slide 5 text

Allan 'dwangoAC' Cecil President of the North Bay Linux Users’ Group Senior Engineer at Cyan Ciena http://nblug.org http://www.ciena.com/ http://tasvideos.org/DwangoAC.html http://tasbot.net http://acbit.net Presented and written by...

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Speedrunning Human limits

Slide 8

Slide 8 text

Playing games fast http://speeddemosarchive.com/

Slide 9

Slide 9 text

Playing games fast http://speeddemosarchive.com/ ● Inspiration: in-game completion timers

Slide 10

Slide 10 text

● SpeedDemosArchive.com and others track fastest completion times ● Strict rules + peer review: no cheats, no macros ● Typically highly entertaining ● Many categories, ranging from "any%" to "low% no major glitches" Playing games fast http://speeddemosarchive.com/ ● Inspiration: in-game completion timers

Slide 11

Slide 11 text

Games Done Quick

Slide 12

Slide 12 text

Games Done Quick Speedrunning marathons for charity streamed live on Twitch Classic GDQ (2010), Awesome GDQ (2011-), Summer GDQ (2011-)

Slide 13

Slide 13 text

Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014

Slide 14

Slide 14 text

Abusing games https://youtu.be/kIIzE_H7D2g?t=5m27s AGDQ 2014 Metroid 15:43 World Record https://www.youtube.com/watch?v=67kQ3l-1qMs

Slide 15

Slide 15 text

https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016

Slide 16

Slide 16 text

Punch-Out blindfolded by Sinister1 - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016

Slide 17

Slide 17 text

Even 1-handed, blindfolded... Beyond standard limits! Punch-Out blindfolded by Sinister1 - AGDQ 2014 https://www.youtube.com/watch?v=CvzIb53Lcno https://www.youtube.com/watch?v=JXtUwIW7cL8 Momodora by Halfcoordinated - SGDQ 2016

Slide 18

Slide 18 text

TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.”

Slide 19

Slide 19 text

TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits

Slide 20

Slide 20 text

TAS verb / noun ~ TASer noun “I’m a TASer working on Tetris.” / “I’m TASing Tetris.” Tool-Assisted Superplays Speedruns From human limits To hardware limits

Slide 21

Slide 21 text

Harder Faster Better Stronger

Slide 22

Slide 22 text

Harder Faster Better Stronger ● Early PC game TAS’s: Savestates, slow motion, and recording tools

Slide 23

Slide 23 text

Harder Faster Better Stronger ● Early PC game TAS’s: Savestates, slow motion, and recording tools ● ~1999: Doom Done Quick in 19:41

Slide 24

Slide 24 text

https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm

Slide 25

Slide 25 text

https://www.youtube.com/watch?v=BEcrJLM4GgU http://web.archive.org/web/20031203222907/http://soramimi.egoism.jp/emu.htm

Slide 26

Slide 26 text

● Tools meant hardware limits became the only limits Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/

Slide 27

Slide 27 text

○ Competitors should admit to doping ○ Videos made with TAS tools should be labeled ● Tools meant hardware limits became the only limits ● TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/

Slide 28

Slide 28 text

● NESVideos created by Bisqwit in 2004 ○ Competitors should admit to doping ○ Videos made with TAS tools should be labeled ● Tools meant hardware limits became the only limits ● TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/

Slide 29

Slide 29 text

● NESVideos created by Bisqwit in 2004 ○ Now at TASVideos.org with runs for many platforms ○ Competitors should admit to doping ○ Videos made with TAS tools should be labeled ● Tools meant hardware limits became the only limits ● TASing looked like the Doped Olympics Inhuman skill on display http://tasvideos.org/WelcomeToTASVideos.html https://web.archive.org/web/20060511210906/http://bisqwit.iki.fi/nesvideos/

Slide 30

Slide 30 text

the birth of TASBot

Slide 31

Slide 31 text

the birth of TASBot Console verified Pushing hardware limits

Slide 32

Slide 32 text

the birth of TASBot Console verified Pushing hardware limits

Slide 33

Slide 33 text

Console emulators http://tasvideos.org/Lsnes.html lsnes BizHawk http://tasvideos.org/BizHawk.html

Slide 34

Slide 34 text

Rerecording frameworks Hourglass NetHack specific tools http://tasvideos.org/EmulatorResources/Hourglass.html http://tasvideos.org/GameResources/DOS/Nethack.html

Slide 35

Slide 35 text

Emulation accuracy evolution

Slide 36

Slide 36 text

● Clean room reverse engineering ○ or stolen manuals ● Early emulators: highly inaccurate Emulation accuracy evolution

Slide 37

Slide 37 text

● bsnes: extreme accuracy, poor usability ● Clean room reverse engineering ○ or stolen manuals ● Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy

Slide 38

Slide 38 text

● bsnes: extreme accuracy, poor usability ● Clean room reverse engineering ○ or stolen manuals ● Early emulators: highly inaccurate Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan

Slide 39

Slide 39 text

● bsnes: extreme accuracy, poor usability ● Clean room reverse engineering ○ or stolen manuals ● Early emulators: highly inaccurate ⇒ match actual hardware, frame for frame Emulation accuracy evolution http://arstechnica.com/gaming/2011/08/accuracy-takes-power-one-mans-3ghz-quest-to-build-a-perfect-snes-emulator/ https://web.archive.org/web/20120915125144/http://byuu.org/bsnes/accuracy http://byuu.org/emulation/higan/ higan

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

Memory searching, Lua scripting, disassembly https://www.lua.org/

Slide 42

Slide 42 text

● More than just frame advance and savestates Memory searching, Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/

Slide 43

Slide 43 text

● More than just frame advance and savestates ● Find a specific value: save, reset memory search, run ○ Search based on conditions, repeat Memory searching, Lua scripting, disassembly https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/

Slide 44

Slide 44 text

● More than just frame advance and savestates ● Find a specific value: save, reset memory search, run ○ Search based on conditions, repeat Memory searching, Lua scripting, disassembly ● Disassembly of RAM or ROM for complete understanding https://www.youtube.com/watch?v=RtaS4KEl4Qc https://www.lua.org/

Slide 45

Slide 45 text

Abusing handwriting recognition https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016

Slide 46

Slide 46 text

Abusing handwriting recognition Editing memory live directly in the game SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=12m28s https://youtu.be/mSFHKAvTGNk?t=29m53s AGDQ 2016

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

TAS ⇔ Infosec equivalents ● Savestate = VM snapshot ● Frame advance = VM CPU step / tick ● Glitch = Vulnerability ● Arbitrary Code Execution = Exploit ● Console verification = Evil maid attack ⇒ TAS = fun, technical, educational

Slide 49

Slide 49 text

AGDQ 2016 https://youtu.be/pj7RE2DcRgc?t=50m23s SMB3 Total Control Glitchfest by Lord Tom

Slide 50

Slide 50 text

Super Mario World Super Mario Bros. TASBot

Slide 51

Slide 51 text

Super Mario World Super Mario Bros. TASBot plays

Slide 52

Slide 52 text

Super Mario World Super Mario Bros. TASBot plays

Slide 53

Slide 53 text

Early console verification devices

Slide 54

Slide 54 text

Early console verification devices ● 2009 ○ a PIC to press NES buttons [true]

Slide 55

Slide 55 text

● 2011 ○ NESBot [micro500]: first replay of SMB1 ■ Used at SGDQ 2011 on SMB2 and W&W 3 Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY ● 2009 ○ a PIC to press NES buttons [true]

Slide 56

Slide 56 text

● 2011 ○ NESBot [micro500]: first replay of SMB1 ■ Used at SGDQ 2011 on SMB2 and W&W 3 ○ Droid64 [SoulCal] ● 2012 ○ N64 [micro500] Early console verification devices https://www.youtube.com/watch?v=KQXVgMKJEDY ● 2009 ○ a PIC to press NES buttons [true]

Slide 57

Slide 57 text

● 2013 ○ SNES and Genesis Arduino bot [GhostSonic] ○ NES/SNES replay device [true] ■ Streaming capable and inexpensive but limited datarates

Slide 58

Slide 58 text

● 2013 ○ SNES and Genesis Arduino bot [GhostSonic] ○ NES/SNES replay device [true] ■ Streaming capable and inexpensive but limited datarates ● 2014 ○ Nintendo R.O.B + board + legos: "TASBot"

Slide 59

Slide 59 text

● 2013 ○ SNES and Genesis Arduino bot [GhostSonic] ○ NES/SNES replay device [true] ■ Streaming capable and inexpensive but limited datarates ● 2014 ○ Nintendo R.O.B + board + legos: "TASBot" ● 2015 ○ Multireplay device [true]: self-contained ⇒ faster datarates

Slide 60

Slide 60 text

● 2013 ○ SNES and Genesis Arduino bot [GhostSonic] ○ NES/SNES replay device [true] ■ Streaming capable and inexpensive but limited datarates ● 2014 ○ Nintendo R.O.B + board + legos: "TASBot" ● 2015 ○ Multireplay device [true]: self-contained ⇒ faster datarates ○ Game Boy Player Player [endrift] (GBA on GameCube)

Slide 61

Slide 61 text

TASBot the perfectionist

Slide 62

Slide 62 text

Super Mario World Super Mario Bros. TASBot

Slide 63

Slide 63 text

Super Mario World Super Mario Bros. TASBot plays

Slide 64

Slide 64 text

Super Mario World Super Mario Bros. TASBot plays

Slide 65

Slide 65 text

Super Mario World Super Mario Bros. TASBot plays in

Slide 66

Slide 66 text

Super Mario World Super Mario Bros. TASBot plays in SMB in SMW by p4plus2 and Masterjun

Slide 67

Slide 67 text

http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic...

Slide 68

Slide 68 text

http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input...

Slide 69

Slide 69 text

http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input... A homemade port of the NES classic is sent as payload...

Slide 70

Slide 70 text

http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ https://www.youtube.com/watch?v=YHyaTCuZRzM credits: p4plus2, Masterjun TASBot plays the SNES classic... Exploits it via input... A homemade port of the NES classic is sent as payload... A 8-bit game, on a 16-bit system!

Slide 71

Slide 71 text

https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool You can write specific sequences in the Object Attribute Memory by using specific objects at specific coordinates,

Slide 72

Slide 72 text

https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences...

Slide 73

Slide 73 text

https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want.

Slide 74

Slide 74 text

https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want. So, just via input...

Slide 75

Slide 75 text

https://www.youtube.com/watch?v=vAHXK2wut_I&index=1&list=PLZctv-xoGbfUolvrW5YTi9J1KnY0l0Xch dotsarecool Since CPU instructions are made of specific binary sequences... ...we can take over execution the way we want. So, just via input... ...you can directly trigger the credits sequence!

Slide 76

Slide 76 text

TASLink ~184 Kbps was too limiting http://taslink.org

Slide 77

Slide 77 text

32Mhz FPGA Papilio Pro's Spartan 6 LX max poll rate of the serial port (2Mb/s) http://papilio.gadgetfactory.net/index.php?n=Papilio.PapilioPro

Slide 78

Slide 78 text

SMB1+2+3+Lost Levels played simultaneously during SGDQ 2016 https://youtu.be/EHfw-BEuRO8?t=58m29s

Slide 79

Slide 79 text

Anatomy of an Arbitrary Code Execution

Slide 80

Slide 80 text

1. Input exploit Anatomy of an Arbitrary Code Execution Pokemon Red

Slide 81

Slide 81 text

1. Input exploit 2. Take over the Super GameBoy Anatomy of an Arbitrary Code Execution Pokemon Red

Slide 82

Slide 82 text

1. Input exploit 2. Take over the Super GameBoy 3. Gain full access to the Super Nintendo Anatomy of an Arbitrary Code Execution Pokemon Red

Slide 83

Slide 83 text

1. Input exploit 2. Take over the Super GameBoy 3. Gain full access to the Super Nintendo 4. Anything is possible Anatomy of an Arbitrary Code Execution Pokemon Red

Slide 84

Slide 84 text

No content

Slide 85

Slide 85 text

No content

Slide 86

Slide 86 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 87

Slide 87 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 88

Slide 88 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 89

Slide 89 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 90

Slide 90 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 91

Slide 91 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 92

Slide 92 text

https://archive.org/stream/pocorgtfo10#page/n5/mode/2up http://arstechnica.com/gaming/2015/01/pokemon-plays-twitch-how-a-robot-got-irc-running-on-an-unmodified-snes/ credits: micro500, Ilari, p4plus2

Slide 93

Slide 93 text

Call to action Join the chat for Q&A at http://twitch.tv/dwangoAC

Slide 94

Slide 94 text

https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot...

Slide 95

Slide 95 text

https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames!

Slide 96

Slide 96 text

https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames! 6000 buttons per second!

Slide 97

Slide 97 text

https://youtu.be/EHfw-BEuRO8?t=1h13m50s credits: total_ ais523 From boot... ...to ending, in 16 frames! Some glitches are expected! 6000 buttons per second!

Slide 98

Slide 98 text

DPCM memory ↕ game controller Flood weak controller code to abuse raster interrupt and take over execution conflict http://www.qmtpro.com/~nes/chipimages/#rp2a03 http://arstechnica.com/gaming/2016/07/how-to-beat-super-mario-bros-3-in-less-than-a-second/

Slide 99

Slide 99 text

TAS'ers lethal weapon ● More flexible than IDA ● Graph view, low level IL and annotation support ● Python scripting ● NES support: ability to add new mappers

Slide 100

Slide 100 text

♫♪ Am I…

Slide 101

Slide 101 text

cheating? ♫♪ Am I…

Slide 102

Slide 102 text

cheating? ♫♪ Am I… ♬ No

Slide 103

Slide 103 text

cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬ No, I'm just looking for...

Slide 104

Slide 104 text

cheating? technical challenge & visual entertainment! ♫♪ Am I… ♬ No, I'm just looking for... ♩ And I'm not the only one… ;)

Slide 105

Slide 105 text

Medecins sans Frontières Doctors without borders ♩♬ But more importantly….

Slide 106

Slide 106 text

Medecins sans Frontières Doctors without borders Prevent Cancer Foundation Games Done Quick Raised for charity! over $200k USD ♩♬ But more importantly…. http://tasvideos.org/forum/viewtopic.php?p=437688#437688

Slide 107

Slide 107 text

micro500 Ilari Thanks to:

Slide 108

Slide 108 text

micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty

Slide 109

Slide 109 text

micro500 Ilari Thanks to: p4plus2 Masterjun true total_ psifertex rusty TheAxeMan ange_ greenfly ais523 and many, many others

Slide 110

Slide 110 text

In collaboration with Ange Albertini ? @MrTASBot Twitch.tv/dwangoAC