Capture the Flag: An Owner's Manual

Slide 1

Slide 1 text

Capture the Flag An Owner’s Manual Vito Genovese USENIX Enigma, January 27, 2016

Slide 2

Slide 2 text

What is CTF?

Slide 3

Slide 3 text

Qualiﬁers May 20 through May 22 FREE FUN OMG wow

Slide 4

Slide 4 text

Finals August 5 through August 7

Slide 5

Slide 5 text

Best of the Best Quals >1400 teams Finals 15-20 teams Winner

Slide 6

Slide 6 text

Best of the Best

Slide 7

Slide 7 text

Engineer a Non-Frustrating Game

Slide 8

Slide 8 text

Operate a Reliable Game

Slide 9

Slide 9 text

Have the Empathy to Make the Game Fun

Slide 10

Slide 10 text

Engineering

Slide 11

Slide 11 text

Engineering Process 1. Deﬁne problem 2. Research 3. Decide requirements 4. Brainstorm solutions 5. Pick the best solution 6. Build it 7. See if it's good enough 8. Redo what’s not

Slide 12

Slide 12 text

What kind of game? Jeopardy vs. Attack-defense

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Jeopardy is Easy Scoreboard Standalone challenges

Slide 15

Slide 15 text

Jeopardy is Easy No complex networking No complex admin work (for players)

Slide 16

Slide 16 text

Attack-Defense is Hard Complex network Sensitive to connectivity Teams host services? We host services? Slow services Unavailable services Superman defenses Metagaming

Slide 17

Slide 17 text

Theming Banking Stuxnet Board Game Marijuana culture Money Laundering Botnet SCADA Wizardterrorism Generic hacker

Slide 18

Slide 18 text

Theming web crypto forensics  reverse engineering programming shellcode

Slide 19

Slide 19 text

Jeopardy Scoring SELECT t.id AS team_id, t.name AS team_name, SUM(c.points) AS score, MAX(s.created_at) AS last_solve FROM teams AS t INNER JOIN solutions AS s ON s.team_id = t.id INNER JOIN challenges AS c ON s.challenge_id = c.id WHERE team_id != 1 GROUP BY t.id ORDER BY score DESC, MAX(s.created_at) ASC, MAX(s.id) ASC

Slide 20

Slide 20 text

Attack-Defense Scoring aww jeez

Slide 21

Slide 21 text

Attack-Defense Game Flow PPP atmail scorebot Shellphish

Slide 22

Slide 22 text

Attack-Defense Game Flow PPP atmail scorebot Shellphish deposit

Slide 23

Slide 23 text

Shellphish Attack-Defense Game Flow PPP atmail scorebot steal

Slide 24

Slide 24 text

Shellphish Attack-Defense Game Flow PPP atmail scorebot redeem

Slide 25

Slide 25 text

Shellphish Attack-Defense Game Flow PPP atmail scorebot availability okay availability check

Slide 26

Slide 26 text

Shellphish Attack-Defense Game Flow PPP atmail scorebot failed availability ☠ ☠ can’t steal

Slide 27

Slide 27 text

Attack-Defense Metagaming Any sufﬁciently complex game is metagameable

Slide 28

Slide 28 text

Downtime vs. Being Hacked

Slide 29

Slide 29 text

Reﬂection

Slide 30

Slide 30 text

First Blood

Slide 31

Slide 31 text

Attack-Defense Scoring Zero Sum Finite number of ﬂags Flags per-service

Slide 32

Slide 32 text

Attack-Defense Scoring Can lose N-1 ﬂags to steals per round Stolen ﬂags split among stealers Remainders redistributed fairly

Slide 33

Slide 33 text

Attack-Defense Scoring Downtime means lost steal opportunity Teams lose 2(N-1) ﬂags to downtime

Slide 34

Slide 34 text

Attack-Defense Scoring Remainder and downtime ﬂags are the ﬂags of the people

Slide 35

Slide 35 text

Science of Challenges • Think of cool bugs • Write bugs, tool to check vulnerability • Wrap ‘em in analysis surface • Write smoke test and health checks

Slide 36

Slide 36 text

Art of Challenges The machine is your canvas and the only limit is ~your imagination~

Slide 37

Slide 37 text

Art of Challenges Historic interest Uniqueness Inherent humor

Slide 38

Slide 38 text

Challenges and Team Size Smaller teams don’t solve challenges slower Bigger teams can solve more challenges at once

Slide 39

Slide 39 text

Challenges and Team Size Fewer and Harder Smaller and Smarter

Slide 40

Slide 40 text

Challenges and Team Size

Slide 41

Slide 41 text

Challenges and Operations Engineering great, fun, reliable challenges is the best ops improvement you can make.

Slide 42

Slide 42 text

Operations

Slide 43

Slide 43 text

CTF Operations The dream is for the organizing team to just party and be jerks to teams during the game

Slide 44

Slide 44 text

CTF Operations “Is this down or broken?” “Is this actually exploitable?”

Slide 45

Slide 45 text

CTF Operations It only has to work for a weekend

Slide 46

Slide 46 text

CTF Operations Start on time by being ready early

Slide 47

Slide 47 text

Jeopardy Operations Boston Key Party Servers $27 Quals 2013 Servers $284 Quals 2013 Booze $340

Slide 48

Slide 48 text

Attack-Defense Operations

Slide 49

Slide 49 text

Attack-Defense Operations

Slide 50

Slide 50 text

Attack-Defense Operations We bring hardware to Vegas

Slide 51

Slide 51 text

Bring Hardware Weird architectures

Slide 52

Slide 52 text

Bring Hardware Teams don't want to bring hardware

Slide 53

Slide 53 text

Bring Hardware Don’t trust the uplink

Slide 54

Slide 54 text

Exceptions • Stratum Auhuur who trusted the uplink at cccamp • Also shout out to Shellﬁsh for bringing a server rack to compete at DEF CON

Slide 55

Slide 55 text

Attack-Defense Operations

Slide 56

Slide 56 text

Attack-Defense Dynamics

Slide 57

Slide 57 text

Attack-Defense Dynamics Player time is a limited resource 1 shower 2 meals 3 hours of sleep

Slide 58

Slide 58 text

Attack-Defense Dynamics 1. Player 1 solves Service A 2. Player 1 starts Service B 3. Service A’ is released 4. Player 1 has a choice

Slide 59

Slide 59 text

Defecators & Ventilators Sometimes challenges break

Slide 60

Slide 60 text

Defecators & Ventilators 10 hours / 1 Tester = 10 Hours 10 hours / 20 Teams = 30 Minutes 10 hours / 1000 Teams = 36 Seconds

Slide 61

Slide 61 text

Defecators & Ventilators Perverse incentives

Slide 62

Slide 62 text

Empathy

Slide 63

Slide 63 text

Challenges and Empathy The game is for the players Players want good, fun, working challenges

Slide 64

Slide 64 text

Empathy • We do it for the users/players/audience • picture of CLU goes here

Slide 65

Slide 65 text

Empathy Run the game you want to play

Slide 66

Slide 66 text

Empathy Don’t lie to players Deceive the players iff it makes the game more fun

Slide 67

Slide 67 text

Frustration Trivia & Memes are hit or miss Think of non-US and non-English teams

Slide 68

Slide 68 text

Guessing and Large Solution Spaces Writing a solver for a 28 solution space is fun Writing and paying for a 216 space isn't

Slide 69

Slide 69 text

Preserve Player Agency No hints once a challenge has been solved Think carefully about force-unlocking Jeopardy challenges

Slide 70

Slide 70 text

Preserve Player Enjoyment Force-unlock easy challenges for teams to learn from Force-unlock hard challenges early enough they'll be solvable

Slide 71

Slide 71 text

Hacking Computers is Fun!

Slide 72

Slide 72 text

Engineer a Non-Frustrating Game

Slide 73

Slide 73 text

Operate a Reliable Game

Slide 74

Slide 74 text

Have the Empathy to Make the Game Fun

Slide 75

Slide 75 text

Qualiﬁers May 20 through May 22 https://legitbs.net/ FREE FUN OMG wow

Slide 76

Slide 76 text

Thanks Vito Genovese vito@legitbs.net @vito_lbs GPG B07D616143CAA77B https://legitbs.net @legitbs_ctf