Tweet
Tweet

Slide 1

Slide 1 text

 @binarymist 1

Slide 2

Slide 2 text

2 . 1

Slide 3

Slide 3 text

InfoSecNZ Slack 2 . 2

Slide 4

Slide 4 text

TRADITIONALLY TRADITIONALLY How have we found bugs in so ware? 3 . 1

Slide 5

Slide 5 text

TRADITIONALLY TRADITIONALLY How have we found bugs in so ware? Um... 3 . 1

Slide 6

Slide 6 text

TRADITIONALLY TRADITIONALLY How have we found bugs in so ware? Um... We haven't really 3 . 1

Slide 7

Slide 7 text

The catch all Red Teaming Exercise 3 . 2

Slide 8

Slide 8 text

The catch all Red Teaming Exercise 3 . 2

Slide 9

Slide 9 text

The catch all Red Teaming Exercise 3 . 3

Slide 10

Slide 10 text

The catch all Red Teaming Exercise ≈$20k per week 3 . 3

Slide 11

Slide 11 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks 3 . 3

Slide 12

Slide 12 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months 3 . 3

Slide 13

Slide 13 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project 3 . 3

Slide 14

Slide 14 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project Found: 5 crit, 10 high, 10 med, 10 low severity bugs 3 . 3

Slide 15

Slide 15 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project Found: 5 crit, 10 high, 10 med, 10 low severity bugs Many bugs le unfound waiting to be exploited 3 . 3

Slide 16

Slide 16 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project Found: 5 crit, 10 high, 10 med, 10 low severity bugs Many bugs le unfound waiting to be exploited Business decides to only fix the 5 criticals 3 . 3

Slide 17

Slide 17 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project Found: 5 crit, 10 high, 10 med, 10 low severity bugs Many bugs le unfound waiting to be exploited Business decides to only fix the 5 criticals Each bug avg cost of 15+ x fixed when introduced 3 . 3

Slide 18

Slide 18 text

The catch all Red Teaming Exercise ≈$20k per week ≈Engagement: two weeks ≈So ware project before release: six months ≈$40k per six months - per project Found: 5 crit, 10 high, 10 med, 10 low severity bugs Many bugs le unfound waiting to be exploited Business decides to only fix the 5 criticals Each bug avg cost of 15+ x fixed when introduced 5 bugs x 15 x $320 = $24000 3 . 3

Slide 19

Slide 19 text

Bottom line: Red Teaming 6 months (2 week engagement): $40’000 Only 5 Red Team bugs fixed: cost: $24000 3 . 4

Slide 20

Slide 20 text

Bottom line: Red Teaming Too expensive Too late Too many bugs le unfixed 3 . 5

Slide 21

Slide 21 text

We can do better 3 . 6

Slide 22

Slide 22 text

We can do better And we have to 3 . 6

Slide 23

Slide 23 text

Things are changing But some are not 4 . 1

Slide 24

Slide 24 text

WHAT'S CHANGED? WHAT'S CHANGED? 4 . 2

Slide 25

Slide 25 text

WHAT'S CHANGED? WHAT'S CHANGED? We no longer release every 6 months or year Now it's weekly, daily, hourly, etc More than ever we need to deliver faster 4 . 2

Slide 26

Slide 26 text

The Internet has grown up And so have our attackers 4 . 3

Slide 27

Slide 27 text

More than ever we need to li our game 4 . 4

Slide 28

Slide 28 text

THE MORE THINGS CHANGE THE MORE THINGS CHANGE THE MORE THEY STAY THE SAME THE MORE THEY STAY THE SAME 4 . 5

Slide 29

Slide 29 text

THE MORE THINGS CHANGE THE MORE THINGS CHANGE THE MORE THEY STAY THE SAME THE MORE THEY STAY THE SAME What's the No. 1 area we as Developers/Engineers need the most help with? 4 . 5

Slide 30

Slide 30 text

THE MORE THINGS CHANGE THE MORE THINGS CHANGE THE MORE THEY STAY THE SAME THE MORE THEY STAY THE SAME What's the No. 1 area we as Developers/Engineers need the most help with? APPSEC APPSEC 4 . 5

Slide 31

Slide 31 text

4 . 6

Slide 32

Slide 32 text

4 . 7

Slide 33

Slide 33 text

4 . 8

Slide 34

Slide 34 text

4 . 9

Slide 35

Slide 35 text

 Establish a Security Champion 4 . 9

Slide 36

Slide 36 text

 Establish a Security Champion  Hand-cra ed Penetration Testing 4 . 9

Slide 37

Slide 37 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming 4 . 9

Slide 38

Slide 38 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review 4 . 9

Slide 39

Slide 39 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review  Techniques for Asserting Discipline 4 . 9

Slide 40

Slide 40 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review  Techniques for Asserting Discipline  Techniques for dealing with Consumption of Free & Open Source 4 . 9

Slide 41

Slide 41 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review  Techniques for Asserting Discipline  Techniques for dealing with Consumption of Free & Open Source  Security Focussed TDD 4 . 9

Slide 42

Slide 42 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review  Techniques for Asserting Discipline  Techniques for dealing with Consumption of Free & Open Source  Security Focussed TDD  Evil Test Conditions 4 . 9

Slide 43

Slide 43 text

 Establish a Security Champion  Hand-cra ed Penetration Testing  Pair Programming  Code Review  Techniques for Asserting Discipline  Techniques for dealing with Consumption of Free & Open Source  Security Focussed TDD  Evil Test Conditions  Security Regression Testing 4 . 9

Slide 44

Slide 44 text

SECURITY REGRESSION TESTING SECURITY REGRESSION TESTING 5 . 1

Slide 45

Slide 45 text

WHAT IS WHAT IS 5 . 2

Slide 46

Slide 46 text

WHAT IS WHAT IS SECURITY REGRESSION SECURITY REGRESSION TESTING? TESTING? 5 . 2

Slide 47

Slide 47 text

WHY? WHY? 5 . 3

Slide 48

Slide 48 text

5 . 4

Slide 49

Slide 49 text

5 . 5

Slide 50

Slide 50 text

Bottom line: Red Teaming 6 months (2 week engagement): $40’000 Only 5 Red Team bugs fixed: cost: $24000 5 . 6

Slide 51

Slide 51 text

Purple Teaming 5 . 7

Slide 52

Slide 52 text

Purple Teaming ≈$160 per hour per Engineer 5 . 7

Slide 53

Slide 53 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced 5 . 7

Slide 54

Slide 54 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced Almost 0 cost. Call each bug fix ≈2 hours (≈$320) 5 . 7

Slide 55

Slide 55 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced Almost 0 cost. Call each bug fix ≈2 hours (≈$320) If we fixed every (35) bug found in red teaming exercise it would cost 35 * ≈$320 = ≈$11200 5 . 7

Slide 56

Slide 56 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced Almost 0 cost. Call each bug fix ≈2 hours (≈$320) If we fixed every (35) bug found in red teaming exercise it would cost 35 * ≈$320 = ≈$11200 As opposed to fixing 5 bugs & costing $24000 5 . 7

Slide 57

Slide 57 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced Almost 0 cost. Call each bug fix ≈2 hours (≈$320) If we fixed every (35) bug found in red teaming exercise it would cost 35 * ≈$320 = ≈$11200 As opposed to fixing 5 bugs & costing $24000 >2 x cost to fix only 14% bugs found in Red Teaming 5 . 7

Slide 58

Slide 58 text

Purple Teaming ≈$160 per hour per Engineer Almost every security bug found+fixed as introduced Almost 0 cost. Call each bug fix ≈2 hours (≈$320) If we fixed every (35) bug found in red teaming exercise it would cost 35 * ≈$320 = ≈$11200 As opposed to fixing 5 bugs & costing $24000 >2 x cost to fix only 14% bugs found in Red Teaming As opposed to fixing all 35 for < ½ $ of 5 crit Red Teaming fixes 5 . 7

Slide 59

Slide 59 text

Purple Teaming Security regression testing will always find many more defects Not constrained to time Red Team ≈2 weeks to hack Automated security regression testing: Every day (CI) to hack Every night (nightly build) to hack 5 . 8

Slide 60

Slide 60 text

The Evolution of... 6 . 1

Slide 61

Slide 61 text

6 . 2

Slide 62

Slide 62 text

 6 . 3

Slide 63

Slide 63 text

 Developers write imperative tests for everything 6 . 3

Slide 64

Slide 64 text

 Developers write imperative tests for everything All components required manual setup and config 6 . 3

Slide 65

Slide 65 text

 Developers write imperative tests for everything All components required manual setup and config Components need to be kept up to date 6 . 3

Slide 66

Slide 66 text

 Developers write imperative tests for everything All components required manual setup and config Components need to be kept up to date Minimum of three months work 6 . 3

Slide 67

Slide 67 text

Developers write a little config No additional setup No updating components No writing tests 6 . 4

Slide 68

Slide 68 text

Consumable by your CI/nightly builds Backed by a SaaS Plugable Testers 6 . 5

Slide 69

Slide 69 text

PURPLETEAM ARCHITECTURE PURPLETEAM ARCHITECTURE 7 . 1

Slide 70

Slide 70 text

7 . 2

Slide 71

Slide 71 text

The manual steps, everything else is automatic: 7 . 3

Slide 72

Slide 72 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 7 . 3

Slide 73

Slide 73 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 2. Host Lambda functions 7 . 3

Slide 74

Slide 74 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 2. Host Lambda functions 3. Run your SUT 7 . 3

Slide 75

Slide 75 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 2. Host Lambda functions 3. Run your SUT 4. Run the main -> npm run dc-up docker-compose 7 . 3

Slide 76

Slide 76 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 2. Host Lambda functions 3. Run your SUT 4. Run the main -> npm run dc-up docker-compose 5. Run CLI -> purpleteam test 7 . 3

Slide 77

Slide 77 text

The manual steps, everything else is automatic: 1. Run docker-compose-ui 2. Host Lambda functions 3. Run your SUT 4. Run the main -> npm run dc-up docker-compose 5. Run CLI -> purpleteam test 6. Once test has finished, check artefacts 7 . 3

Slide 78

Slide 78 text

As a consumer: 1. Run docker-compose-ui 2. Host Lambda functions 3. Run your SUT 4. Run the main -> npm run dc-up 5. Run CLI -> purpleteam test 6. Once test has finished, check artefacts docker-compose 7 . 4

Slide 79

Slide 79 text

As a consumer: 3. Run your SUT 5. Run CLI -> purpleteam test 6. Once test has finished, check artefacts 7 . 4

Slide 80

Slide 80 text

As a consumer: 1. Run your SUT 2. Run CLI -> purpleteam test 3. Once test has finished, check artefacts 7 . 5

Slide 81

Slide 81 text

ORCHESTRATOR ORCHESTRATOR 7 . 6

Slide 82

Slide 82 text

7 . 7

Slide 83

Slide 83 text

TESTERS TESTERS 7 . 8

Slide 84

Slide 84 text

TESTERS TESTERS app-scanner 7 . 8

Slide 85

Slide 85 text

TESTERS TESTERS app-scanner server-scanner 7 . 8

Slide 86

Slide 86 text

TESTERS TESTERS app-scanner server-scanner tls-checker 7 . 8

Slide 87

Slide 87 text

TESTERS TESTERS app-scanner server-scanner tls-checker Your tester here? 7 . 8

Slide 88

Slide 88 text

SLAVES SLAVES 7 . 9

Slide 89

Slide 89 text

7 . 10

Slide 90

Slide 90 text

Prod Dev 7 . 11

Slide 91

Slide 91 text

App Testing Slaves # docker-compose up --scale zap=2 version: "3.6" networks: compose_pt-net: external: true services: zap: image: owasp/zap2docker-stable networks: compose_pt-net: # Soft limit of 12 test sessions. ports: - "8080-8091:8080" 7 . 12

Slide 92

Slide 92 text

App Testing Slave helper (Selenium instance) (one for each App Testing Slave) version: "3.6" networks: compose_pt-net: external: true services: chrome: image: selenium/standalone-chrome networks: compose_pt-net: ports: - "4444-4455:4444" shm_size: 1G firefox: 7 . 13

Slide 93

Slide 93 text

CLI CLI 7 . 14

Slide 94

Slide 94 text

CLI CLI purpleteam 7 . 14

Slide 95

Slide 95 text

Notable dependencies: "blessed", "blessed-contrib", "chalk", "convict", "eventsource", "purpleteam-logger", "request", "request-promise", "request-promise-native", "sywac" 7 . 15

Slide 96

Slide 96 text

Notable dev dependencies: "code", "lab", "mocksee", "sinon" 7 . 16

Slide 97

Slide 97 text

7 . 17

Slide 98

Slide 98 text

about.js test.js testplan.js 7 . 18

Slide 99

Slide 99 text

7 . 19

Slide 100

Slide 100 text

PURPLETEAM IN ACTION PURPLETEAM IN ACTION 8 . 1

Slide 101

Slide 101 text

npm install -g purpleteam 8 . 2

Slide 102

Slide 102 text

npm install -g purpleteam Define SUT in the build user config file 8 . 2

Slide 103

Slide 103 text

{ "data": { "type": "testRun", "attributes": { "version": "0.1.0-alpha.1", "sutAuthentication": { "route": "/login", "usernameFieldLocater": "userName", "passwordFieldLocater": "password", "submit": "btn btn-danger", "expectedResponseFail": "Invalid" }, "sutIp": "pt-sut-cont", "sutPort": 4000, "sutProtocol": "http", 8 . 3

Slide 104

Slide 104 text

purpleteam test 8 . 4

Slide 105

Slide 105 text

8 . 5

Slide 106

Slide 106 text

CAN'T WAIT? CAN'T WAIT? 9

Slide 107

Slide 107 text

CAN'T WAIT? CAN'T WAIT? Help Build it  gitlab.com/purpleteam-labs 9

Slide 108

Slide 108 text

CAN'T WAIT? CAN'T WAIT? Help Build it  gitlab.com/purpleteam-labs Try old PoC  github.com/binarymist/NodeGoat/wiki/ 9