Slide 1

Slide 1 text

What I’ve Learned Writing CTF Challenges Vito Genovese BSides Tampa February 11, 2017

Slide 2

Slide 2 text

What is CTF?

Slide 3

Slide 3 text

What is CTF?

Slide 4

Slide 4 text

Jeopardy Style

Slide 5

Slide 5 text

Attack-Defense

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

CGC Jargon • Cyber Reasoning System • Proof of Vulnerability • Replacement Binary • Challenge Binary

Slide 22

Slide 22 text

BSTies http://bsties.notmalware.ru

Slide 23

Slide 23 text

CTF Challenges Teams attack it

Slide 24

Slide 24 text

CTF Challenges Protects something of in-game value

Slide 25

Slide 25 text

CTF Challenges Ideally: known solvable

Slide 26

Slide 26 text

CTF Challenges Also ideally: appropriately tricky

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

Solving 'em • Correct side of an if statement • Cracking a code • Using SQL injection to trick a database • Stealing the garbage file from the Gibson

Slide 30

Slide 30 text

Actually Building One

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Original Goal

Slide 33

Slide 33 text

Original Goal

Slide 34

Slide 34 text

• Given same STDIN • Same STDOUT
 • Same clock cycles • Same syscalls

Slide 35

Slide 35 text

https://xkcd.com/1319/

Slide 36

Slide 36 text

Why Automate?

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Thousand Cuts

Slide 39

Slide 39 text

Thousand Cuts

Slide 40

Slide 40 text

Thousand Cuts

Slide 41

Slide 41 text

Actually Building It

Slide 42

Slide 42 text

Start the First One 1. Set up CGC service-template 2. Test not-crashing 3. Make binary that doesn’t crash 4. Test crashing 5. Make binary that crashes

Slide 43

Slide 43 text

First One's Done • Ruby script to spit out 334 binaries • Vary buffer sizes • Vary stack cookie • Python script running game flow

Slide 44

Slide 44 text

First Set's Done Figure out how to make second and third sets

Slide 45

Slide 45 text

Second Set Python script to add stuff to source code

Slide 46

Slide 46 text

Third Set Python script to rearrange source code

Slide 47

Slide 47 text

All Sets Done Test the difficulty

Slide 48

Slide 48 text

Integration Testing With Gyno • Finds a bug in like 30s • Fixing the bug took 30m • (fixing the bug a week prior would've taken 5m)

Slide 49

Slide 49 text

More Integration Testing kind of weird

Slide 50

Slide 50 text

easy-prasky • single binary • Baby’s First category • Separate CRS work from first stab at CGCEF • Hacked to be first in 334 cuts

Slide 51

Slide 51 text

Deployment • Determine ops requirements • CGC kernel • Python installed • Open port

Slide 52

Slide 52 text

Deployment Puppetize it!

Slide 53

Slide 53 text

Deployment

Slide 54

Slide 54 text

Running it just kinda worked (once teams Got It)

Slide 55

Slide 55 text

CTF Challenge Process 1.Idea 2.Build 3.Test

Slide 56

Slide 56 text

CTF Challenge Framework • Configuration • Build
 • Testing • Deployment

Slide 57

Slide 57 text

CTF Challenge Framework CGC service-template

Slide 58

Slide 58 text

CTF Challenge Deployment put it on a server lol

Slide 59

Slide 59 text

It's Just Software

Slide 60

Slide 60 text

Software Goals Useful Usable Reliable

Slide 61

Slide 61 text

Useful for Competition Separate teams that solve it from teams that don't Prefer strong correlation with finals ranking

Slide 62

Slide 62 text

Useful for Education Teach new players something Get players over a hang-up

Slide 63

Slide 63 text

Usable Intended difficulty only Minimize corner cases Unintended vulnerabilities?

Slide 64

Slide 64 text

Reliable Remain vulnerable

Slide 65

Slide 65 text

Reliability is Hard Players like to attack Players like to brute Players like to complain

Slide 66

Slide 66 text

Reliability is Hard

Slide 67

Slide 67 text

Reliability is Demanding

Slide 68

Slide 68 text

Reliability is Expensive If you want it to be

Slide 69

Slide 69 text

Reliability is Cheap Build software that can be run reliably

Slide 70

Slide 70 text

Twelve Factor App • VIII. Concurrency • Scale out via the process model • IX. Disposability • Maximize robustness with fast startup and graceful shutdown

Slide 71

Slide 71 text

Twelve Factor Challenges • Limit global state • Process per connection • xinetd ftw • Make adding resources cheap

Slide 72

Slide 72 text

Adding Resources Automate!

Slide 73

Slide 73 text

Operations Automation Puppet or Chef for server provisioning

Slide 74

Slide 74 text

Operations Automation Docker is slow to spin up RunC is fast

Slide 75

Slide 75 text

Shared State • Process per connection not always feasible • Sandwich challenges in 2014 • JRuby & Celluloid slow to start • State still per connection

Slide 76

Slide 76 text

Shared State • Global state and web challenges • Persisted XSS • Tricking a global database

Slide 77

Slide 77 text

Waiting For Your Touch • 2015 web challenge • /r/thebutton knockoff • JRuby, Rails, websockets, postgres

Slide 78

Slide 78 text

Waiting For Your Touch

Slide 79

Slide 79 text

Waiting For The Challenge Slow when not slow 500s Thankfully during the day

Slide 80

Slide 80 text

Waiting For Your Touch 1. Took it down 2. Opened replacement challenge 3. Debugged for an hour

Slide 81

Slide 81 text

Debugging For Your Touch 4. Leaked and exhausted postgres connections 5. Fixed leak 6. Allocated more connections

Slide 82

Slide 82 text

Waiting For My Fix 7. Took an hour 8. Gyno forced a fifteen minute cool-down between "is it fixed" and "it is fixed" 9. Reopened

Slide 83

Slide 83 text

Most Important Part People are the most important

Slide 84

Slide 84 text

Respect People • Respect for yourself • Respect for your team • Respect for players

Slide 85

Slide 85 text

Respect for Yourself

Slide 86

Slide 86 text

Respect for Your Team

Slide 87

Slide 87 text

Respect for Your Team

Slide 88

Slide 88 text

Respect for Your team Make services easy to keep running Document!

Slide 89

Slide 89 text

Respect for Your Team Provide easy to run smoke tests "Is this exploitable?" should be a one-liner that spits out the flag

Slide 90

Slide 90 text

Respect for Players

Slide 91

Slide 91 text

Respect for Players

Slide 92

Slide 92 text

Respect for Players

Slide 93

Slide 93 text

Respect for Players

Slide 94

Slide 94 text

Building Challenges is Software Development Useful Usable Reliable

Slide 95

Slide 95 text

Reliability Imposes Constraints Limit state Limit dependencies Automate

Slide 96

Slide 96 text

Respect Is Fundamental Self Team Players

Slide 97

Slide 97 text

Thanks! Vito Genovese [email protected] @vito_lbs
 GPG B07D616143CAA77B https://legitbs.net @legitbs_ctf