Security @ Slack

Monitoring for malicious activity and handling the resulting alerts is vital to the success of a defensive security program. Powerful, centralized logging is available to all of us, but it is only useful if we understand and take action on the data collected.

This talk will discuss tools everyone should consider using to monitor their infrastructure, including Elasticsearch, and the process by which users can create a reliable logging pipeline to handle data from thousands of hosts. Ryan and Nate will demonstrate how to scale these efforts by integrating security into a communication platform that helps users look at more data by delegating event management to the affected individuals directly.

Nate Brown l Developer l Slack
Ryan Huber l Security Developer l Slack

Transcript

1. Slack 9th March 2017 The Bad Things Happen When You’re Not Looking (Security @ Slack) Nate Brown, Senior Staff Security Engineer @nbrownus Ryan Huber, Manager of Security Operations @ryanhuber
2. About me (ryan)
3. About me (nate)
4. Slack
5. Security Operations team?
6. Do you Elasticsearch/Splunk?
7. Breaches
8. How does a company know when it has been hacked?
9. The company’s employees notice something strange.
10. A 3rd party contacts the company because they notice something strange (*Krebs)
11. Hacker(s) contact the company because they want them to notice something strange
12. They Don’t :(
13. Time
14. How do they get in?
15. Superhackers, right?
16. (NSA)
17. Exercise: Think like an attacker
18. Monitoring
19. kibana
22. Elastalert (or x-pack watcher)
24. AlertCenter
26. SecurityBot
27. Don’t be Netcool
28. Pipeline
29. rsyslog + relp
30. streamstash (or logstash)
31. elasticsearch (with kopf)
33. 4 Streamstash nodes ~25,000 events per second 28 elasticsearch nodes ~25TB of data online
34. Data Sources
35. kernel audit
36. auth logs (ssh, sudo, …)
37. Webserver (apache, nginx, etc)
38. Log (almost) all the things! mlogger or logger --size
39. So about auditd (Ryan’s favorite bit)
40. auditd
41. You already have it! (just like SELinux/apparmor)
42. (picture of logs from auditd)
43. audisp
44. So we wrote our own...
45. go-audit
46. syscalls
47. execve
48. listen
49. connect
50. ngrok?
51. Why not osquery/eBPF?
52. Finding Badness
53. Defender’s advantage
54. The hypothetical malicious insider
55. Off the shelf rulesets
56. ZERO DAYS != INVISIBILITY CLOAKS
57. Verification
58. Carbon Black, anecdotally
59. Canaries
61. Red Teaming
62. What if they got in anyway?
63. You have lots of forensic data!
64. You can rent some computers temporarily. Neat.
65. Security + Operations
66. Don’t let perfection be the enemy of good.
67. Try go-audit!
68. goo.gl/9D32SW @nbrownus @ryanhuber Questions?