Seeing Red: Improving blue
teams through red teaming
Dave Hull
Tanium EDR Engineering
Slide 2
Slide 2 text
What is this?
Copyright 2015 Tanium Inc. All rights reserved.
2
Slide 3
Slide 3 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
3
Slide 4
Slide 4 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
4
Slide 5
Slide 5 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
5
Slide 6
Slide 6 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
6
Slide 7
Slide 7 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
7
Slide 8
Slide 8 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
8
Slide 9
Slide 9 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
9
Slide 10
Slide 10 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
10
Slide 11
Slide 11 text
Intro
Copyright 2015 Tanium Inc. All rights reserved.
11
Slide 12
Slide 12 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
12
Slide 13
Slide 13 text
Why red team?
Copyright 2015 Tanium Inc. All rights reserved.
13
Because it delivers a security incident.
Slide 14
Slide 14 text
Pen testing delivers… a nice report.
Copyright 2015 Tanium Inc. All rights reserved.
14
Slide 15
Slide 15 text
Why red team?
Because you will play like you practice.
Copyright 2015 Tanium Inc. All rights reserved.
15
Slide 16
Slide 16 text
Why red team?
Copyright 2015 Tanium Inc. All rights reserved.
16
Slide 17
Slide 17 text
“We run that play every day — end of every
practice,” [Phil] Booth said.
http://www.nytimes.com/2016/04/06/sports/ncaabasketball/villanova-national-
championship.html?_r=0
Slide 18
Slide 18 text
Why red team?
Copyright 2015 Tanium Inc. All rights reserved.
18
Slide 19
Slide 19 text
Why red team?
Because red teaming is quantifiable.
Copyright 2015 Tanium Inc. All rights reserved.
19
Slide 20
Slide 20 text
Why red team?
Mean-time-to-compromise.
Copyright 2015 Tanium Inc. All rights reserved.
20
Slide 21
Slide 21 text
Why red team?
Mean-time-to-detection.
Copyright 2015 Tanium Inc. All rights reserved.
21
Slide 22
Slide 22 text
Why red team?
Mean-time-to-recovery.
Copyright 2015 Tanium Inc. All rights reserved.
22
Slide 23
Slide 23 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
23
Slide 24
Slide 24 text
What is red teaming?
It is not threat modeling.
Copyright 2015 Tanium Inc. All rights reserved.
24
Slide 25
Slide 25 text
What is red teaming?
It is not vulnerability assessment.
Copyright 2015 Tanium Inc. All rights reserved.
25
Slide 26
Slide 26 text
What is red teaming?
It is not penetration testing.
Copyright 2015 Tanium Inc. All rights reserved.
26
Slide 27
Slide 27 text
What is red teaming?
Red teaming is different.
Copyright 2015 Tanium Inc. All rights reserved.
27
Slide 28
Slide 28 text
What is red teaming?
Some call it “adversary emulation.”
Copyright 2015 Tanium Inc. All rights reserved.
28
Slide 29
Slide 29 text
What is red teaming?
Some call it “a force-on-force engagement.”
Copyright 2015 Tanium Inc. All rights reserved.
29
Slide 30
Slide 30 text
Red teams:
Have mission objectives.
Copyright 2015 Tanium Inc. All rights reserved.
30
Slide 31
Slide 31 text
Red teams:
Have mission objectives.
Enterprise or domain admin?
Copyright 2015 Tanium Inc. All rights reserved.
31
Slide 32
Slide 32 text
Red teams:
Have mission objectives.
Customer pivot.
Copyright 2015 Tanium Inc. All rights reserved.
32
Slide 33
Slide 33 text
Red teams:
Have mission objectives.
IP theft.
Copyright 2015 Tanium Inc. All rights reserved.
33
Slide 34
Slide 34 text
Red teams:
Have mission objectives.
Burn it all down.
Copyright 2015 Tanium Inc. All rights reserved.
34
Slide 35
Slide 35 text
Red teams:
Have mission objectives.
Test incident response capabilities and procedures.
Copyright 2015 Tanium Inc. All rights reserved.
35
Slide 36
Slide 36 text
Red teams:
Have mission objectives.
Test incident response capabilities and procedures
of the organization... not just the blue team.
Copyright 2015 Tanium Inc. All rights reserved.
36
Slide 37
Slide 37 text
Who responds, if...
Copyright 2015 Tanium Inc. All rights reserved.
37
Slide 38
Slide 38 text
Who responds, if Brian Krebs is your IDS?
Not just the IR team.
Not just the security team.
Copyright 2015 Tanium Inc. All rights reserved.
38
Slide 39
Slide 39 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
39
Slide 40
Slide 40 text
Lesson learned
Outliers may be leads.
Copyright 2015 Tanium Inc. All rights reserved.
40
Slide 41
Slide 41 text
Outliers may be leads.
Copyright 2015 Tanium Inc. All rights reserved.
41
Slide 42
Slide 42 text
Outliers may be leads.
Copyright 2015 Tanium Inc. All rights reserved.
42
Slide 43
Slide 43 text
Do you even monoculture?
Copyright 2015 Tanium Inc. All rights reserved.
43
Slide 44
Slide 44 text
Dan Geer:
Copyright 2015 Tanium Inc. All rights reserved.
44
• "Internet security is quite possibly the most
intellectually challenging profession on the planet... for
two reasons... complexity... and rate of change [are] your
enemy.
Slide 45
Slide 45 text
Loathsome long tails...
Copyright 2015 Tanium Inc. All rights reserved.
45
Slide 46
Slide 46 text
“... everpresent everywhere...”
Copyright 2015 Tanium Inc. All rights reserved.
46
Slide 47
Slide 47 text
Build systems that automate
data collection, analysis and remediation.
Copyright 2015 Tanium Inc. All rights reserved.
47
Slide 48
Slide 48 text
Blue’s Prime Directive: Remediation
Copyright 2015 Tanium Inc. All rights reserved.
48
Slide 49
Slide 49 text
Remediation, like security, is a process not a product.
Copyright 2015 Tanium Inc. All rights reserved.
49
Slide 50
Slide 50 text
Investigate. Remediate. Repeat.
Copyright 2015 Tanium Inc. All rights reserved.
50
Slide 51
Slide 51 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
51
Slide 52
Slide 52 text
Who should be red teaming?
Any organization that may have a security incident.
Copyright 2015 Tanium Inc. All rights reserved.
52
Slide 53
Slide 53 text
Who should be red teaming?
Any organization with something worth protecting.
Copyright 2015 Tanium Inc. All rights reserved.
53
Slide 54
Slide 54 text
Who should be red teaming, practically speaking?
Organizations meeting the previous criteria and having:
Some monitoring.
Some defenses.
Some IR capabilities.
Copyright 2015 Tanium Inc. All rights reserved.
54
Slide 55
Slide 55 text
Who should be red teaming?
Probably an internal team, but not just the security team.
Copyright 2015 Tanium Inc. All rights reserved.
55
Slide 56
Slide 56 text
Lesson learned
Documentation is wrong.
Code comments are wrong.
Assumptions are wrong.
Copyright 2015 Tanium Inc. All rights reserved.
56
Slide 57
Slide 57 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
57
Slide 58
Slide 58 text
When should you red team?
Two, maybe three times a year.
Copyright 2015 Tanium Inc. All rights reserved.
58
Slide 59
Slide 59 text
Lesson learned
Avoid concurrent red team incidents.
Copyright 2015 Tanium Inc. All rights reserved.
59
Slide 60
Slide 60 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
60
Slide 61
Slide 61 text
Practicalities
Have Rules of Engagement.
Copyright 2015 Tanium Inc. All rights reserved.
61
Slide 62
Slide 62 text
Rules of engagement
Get approval from management and legal.
Copyright 2015 Tanium Inc. All rights reserved.
62
Slide 63
Slide 63 text
Rules of engagement
Copyright 2015 Tanium Inc. All rights reserved.
63
Slide 64
Slide 64 text
Rules of engagement
No accessing or tampering with customer data.
Copyright 2015 Tanium Inc. All rights reserved.
64
Slide 65
Slide 65 text
Rules of engagement
No accessing or tampering with real customer data.
Copyright 2015 Tanium Inc. All rights reserved.
65
Slide 66
Slide 66 text
Rules of engagement
No outages.
Copyright 2015 Tanium Inc. All rights reserved.
66
Slide 67
Slide 67 text
Rules of engagement
No weakening of existing
controls.
Copyright 2015 Tanium Inc. All rights reserved.
67
Slide 68
Slide 68 text
Rules of engagement
Give the red team access.
Copyright 2015 Tanium Inc. All rights reserved.
68
Slide 69
Slide 69 text
Rules of engagement
Give the red team source code.
Copyright 2015 Tanium Inc. All rights reserved.
69
Slide 70
Slide 70 text
Rules of engagement
Give the red team architecture diagrams.
Copyright 2015 Tanium Inc. All rights reserved.
70
Slide 71
Slide 71 text
Rules of engagement
Keep the blue team in the dark.
Copyright 2015 Tanium Inc. All rights reserved.
71
Slide 72
Slide 72 text
Rules of engagement – Don’t let blue do this
Copyright 2015 Tanium Inc. All rights reserved.
72
Slide 73
Slide 73 text
Rules of engagement
Real incidents trump red team incidents.
Copyright 2015 Tanium Inc. All rights reserved.
73
Slide 74
Slide 74 text
Rules of engagement
Red incidents are core hours only.
Copyright 2015 Tanium Inc. All rights reserved.
74
Slide 75
Slide 75 text
Rules of engagement
Red incidents are core hours only,
plus a little.
Copyright 2015 Tanium Inc. All rights reserved.
75
Slide 76
Slide 76 text
Rules of engagement
Cross team collaboration.
Copyright 2015 Tanium Inc. All rights reserved.
76
Slide 77
Slide 77 text
Rules of engagement
Establish a situation room.
Copyright 2015 Tanium Inc. All rights reserved.
77
Slide 78
Slide 78 text
Rules of engagement
Designate incident and investigative leads.
Copyright 2015 Tanium Inc. All rights reserved.
78
Slide 79
Slide 79 text
Rules of engagement
Delegate and PM.
Copyright 2015 Tanium Inc. All rights reserved.
79
Slide 80
Slide 80 text
Situation normal...
Investigate.
Copyright 2015 Tanium Inc. All rights reserved.
80
Slide 81
Slide 81 text
Situation normal, practice how you want to play
Document.
Copyright 2015 Tanium Inc. All rights reserved.
81
Slide 82
Slide 82 text
Situation normal, practice how you want to play
Report.
Copyright 2015 Tanium Inc. All rights reserved.
82
Slide 83
Slide 83 text
Situation normal, practice how you want to play
Copyright 2015 Tanium Inc. All rights reserved.
83
Slide 84
Slide 84 text
Situation normal, practice how you want to play
Plan for remediation.
Copyright 2015 Tanium Inc. All rights reserved.
84
Slide 85
Slide 85 text
Situation normal, practice how you want to play
Execute remediation.
Copyright 2015 Tanium Inc. All rights reserved.
85
Slide 86
Slide 86 text
Situation normal, practice how you want to play
Post remediation monitoring.
Copyright 2015 Tanium Inc. All rights reserved.
86
Slide 87
Slide 87 text
Take aways
Postmortems.
Copyright 2015 Tanium Inc. All rights reserved.
87
Slide 88
Slide 88 text
Postmortem: Who?
Stakeholders, blue team, red team.
Copyright 2015 Tanium Inc. All rights reserved.
88
Slide 89
Slide 89 text
Postmortem: What?
No blame games.
Copyright 2015 Tanium Inc. All rights reserved.
89
Slide 90
Slide 90 text
Postmortem: What?
But hold yourself accountable.
Copyright 2015 Tanium Inc. All rights reserved.
90
Slide 91
Slide 91 text
Postmortem: Story time.
Blue team goes first.
Copyright 2015 Tanium Inc. All rights reserved.
91
Slide 92
Slide 92 text
Postmortem: Tell all.
Copyright 2015 Tanium Inc. All rights reserved.
92
Slide 93
Slide 93 text
Postmortem: The facts.
Red team goes second.
Copyright 2015 Tanium Inc. All rights reserved.
93
Slide 94
Slide 94 text
Postmortem: Mind the gap.
Blue Red
Copyright 2015 Tanium Inc. All rights reserved.
94
Goal: close gap over time
Slide 95
Slide 95 text
Postmortem: Takeaways.
All teams get bugs, feature requests.
Copyright 2015 Tanium Inc. All rights reserved.
95
Slide 96
Slide 96 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
96
Slide 97
Slide 97 text
Lesson learned
No one runs as admin.
Copyright 2015 Tanium Inc. All rights reserved.
97
Slide 98
Slide 98 text
Lesson learned
Just-In-Time admin (JIT).
Copyright 2015 Tanium Inc. All rights reserved.
98
Slide 99
Slide 99 text
Lesson learned
Segment the network.
Copyright 2015 Tanium Inc. All rights reserved.
99
Slide 100
Slide 100 text
Lesson learned
Segment the accounts.
Copyright 2015 Tanium Inc. All rights reserved.
100
Slide 101
Slide 101 text
Lesson learned
Dedicated admin workstations.
Copyright 2015 Tanium Inc. All rights reserved.
101
Slide 102
Slide 102 text
Lesson learned
Zero human generated passwords.
Copyright 2015 Tanium Inc. All rights reserved.
102
Slide 103
Slide 103 text
Lesson learned
2FA everywhere.
Copyright 2015 Tanium Inc. All rights reserved.
103
Slide 104
Slide 104 text
Lesson learned
Don’t trust. Verify.
Copyright 2015 Tanium Inc. All rights reserved.
104
Slide 105
Slide 105 text
Agenda
• Teaser
• Why red teaming
• What is red teaming
• Highlights and lessons learned
• Who should be red teaming
• When
• Practicalities of red teaming
• Conclusion
Copyright 2015 Tanium Inc. All rights reserved.
105
Slide 106
Slide 106 text
Conclusion
Red teaming is hard.
Copyright 2015 Tanium Inc. All rights reserved.
106
Slide 107
Slide 107 text
Conclusion
Real incidents may be harder.
Copyright 2015 Tanium Inc. All rights reserved.
107
Slide 108
Slide 108 text
Conclusion
Practice how you want to play.
Copyright 2015 Tanium Inc. All rights reserved.
108