The Nature Conservancy, the Elastic Stack, and Security Logs

How do you build a network of devices and log taps that monitor security at the world’s largest conservation non-profit? Can it be done at low budget scale across over 100 offices effectively? Can defending against attacks on a laptop in Pennsylvania help to influence river flow metric collection?

Nick and Daniel have begun a journey to answer these types of questions, and would like to share how they are leveraging the Elastic Stack and other open source tools to generate, pull, and aggregate data from remote offices and better manage their security risk. They’ll also look at how a small group of geeks are working to set an example for broader data collection at their organization.

Daniel Shirer l Information Security Analyst l The Nature Conservancy
Nick Waringa l Information Security and Risk Manager l The Nature Conservancy

Transcript

1. The Nature Conservancy Wednesday, March 8 2017 Sustainable Harvesting: How a Few Geeks Learned to Elastic Stack Logs Nick Waringa, Information Security and Risk Manager Dan Shirer, Information Security Analyst
2. • Who is TNC? • What is the Problem? • SIEM? • Architecture • Sensors Agenda 2 • Tool Use • Accomplishments • Where do we go from here?
3. 3 The Nature Conservancy (TNC) Cordie Diggins – Virginia Tech Doctoral Student Photo credit: © Patrick Cavan Brown
4. 4 Who are these guys? Nick (Analogy Android) Dan (Pun-fu Master) Sapling red spruce trees are planted in the newly tilled field of what was once a strip mining operation. Red spruce trees provide important habitat for West Virginia Northern flying squirrels. Photo credit: © Patrick Cavan Brown for The Nature Conservancy Mark Daniels Elastic Support
5. • No SIEM architecture to ingest logs • Best security logging: Client anti-malware software • Two v1 IDS devices at our central office (Security Onion) • All of our systems log… none of them log centrally What Problem Are We Trying to Solve 5 With limited visibility attacks are hard to predict, scope limit, and resolve.
6. 6 SIEMlessly Finding the Needle… A farmer in Minzhu Village hauling his rapeseed oil straw up to his farm house in Minzhu Village on the edge of Laohegou Nature Reserve, Pingwu County, Sichuan Province, China. Photo credit: © Nick Hall for The Nature Conservancy
7. A New Model… Sure, Why Not
8. Architecture Beats Filebeat Metricbeat Sensors Messaging Queue Nodes (2) Logstash Elasticsearch Kibana X-pack Instances (1) Master Nodes (2) Data Nodes (2) Syslog Log Sources "Auth logs, Firewall, Anti- Malware, Web Servers, Application Servers RabbitMQ Command and Control Threat Intel Critical Stack (Bro) Ansible Sensor Hardware Bro IDS Snort Emerging Threats (Snort) Syslog Aggregator Logstash Logstash Unifiedbeat Encrypted Communication Authentication Monitoring
9. 9 Let‘s Start Logging Ryan Adler, choker setter, in the logging yard at the Ellsworth Creek Preserve, WA Photo credit: © Chris Crisman
10. • They need to scale • Open Source is our friend • They need to be easy to manage • Threat Intelligence needs to be interchangeable • Signatures detections are great, but where possible we want traffic metadata Whoa… Wait a Minute, We Need The Logs 10 Step 1, Sensors Sensor Design Principles
11. Monitoring Sensor Load with Heatmaps
12. 12 filter { sleep { time => "2" # Sleep 2 seconds every => 75 # on every 75th event } } Low Bandwidth Troubleshooting https://www.elastic.co/guide/en/logstash/current/plugins-filters-sleep.html Just Sleep On It
13. 13 A Multi-Ewes Collection Tool Sheep used for weed and grass management grazing at the Fuller Star solar project in Lancaster, California. Photo credit: © Dave Lauridsen for The Nature Conservancy
14. Snort as a Layer 7 Monitor • Snort OpenAppID • Identifies applications out of monitored network stream • Logs at an aggregate level http://blog.snort.org/2014/03/firing-up-openappid.html
15. Bro IDS: Our Flight Recorder Jack Wallace & Corinne Diggins examine a West Virginia Northern flying squirrel and then attach a collar on for tracking purposes. Photo credit: © Patrick Cavan Brown for The Nature Conservancy
16. $cat http.log #separator \x09 … #fields ts uid id.orig_h id.orig_p id.resp_h … #types time string addr port addr port count … 1487691690.293045 C9TS1nwGRqoLCndnh 192.168.1.101 50370 … Perfect for Parsing and Pivoting Bro Logs
17. 17 dissect { mapping => { "message" => "%{ts} %{bro_uid} %{conn_src_ip} … } remove_field => [ "info_msg", "orig_filenames” ] } The sky’s delimit Hello Dissect https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html That’s a tab
18. 18 translate { dictionary_path => “/etc/logstash/MAC-to-PC-name.yml” field => mac_address destination => computer_name fallback => “-” } Answering the MAC vs PC question Dude, Where’s My PC? https://www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html MAC-to-PC-name.yml “aa:bb:cc:dd:ee:ff”: “HR-computer” “ff:ee:dd:cc:bb:aa”: “confroom-PC”
19. Global Sensor Map Our rollout will finish next year with over 100 offices instrumented and logging up to 32 TB of data.
20. Revelations • Tracking compromises cross-office • Operational metrics galore • Answering the question “What happened?” (without ducking under furniture) Things within Reach • Identifying user activity by machine, job description, role, area • Event tracking cradle to grave (From origination on the endpoint, to attack/data exfiltration) • Operational team onboarding… spreading the log love What Have We Accomplished?
21. • Can we move NSM closer to the host? • Bro/Packetbeat operating as an independent tamper- proof container on the client? • Home offices and the border falling apart demands we go where our users are. • Grid computing for off hours sensor computations? • How can we extend Elastic/Kibana? • Context menus to use data in external lookup sources? (Github #8042) • Process documentation and “breadcrumb” dashboards with Markup (Github #9432) • How about Maltego? Or other unique information gathering platforms. The Technical Stretch Bad Branch State Nature Preserve in Kentucky during Autumn. © The Nature Conservancy (Devan King)
22. Roadmap Beats Sensors Messaging Queue Logstash Elasticsearch Kibana Notification Syslog Log Sources RabbitMQ Router Netflow Solarwinds Command and Control External Threat Intel Prevention API driven log sources Python Unsupervised Machine Learning Syslog Aggregator Extend Analysis with 3rd Party Tools Reporting X-pack Internal Threat Intel
23. 23 We are a science and data driven organization. This project is the tip of the iceberg for us. Iceberg at LeConte Bay of Frederick Sound in Alaska in United States, North America. Photo credit: © Bill Kamin
24. • Can we use our skills to help our organization? • Using Elastic to eliminate data silos in specialized business applications? • “Sensors” in the field on preserves tracking river flow, animal migration, temperature… climate change? ๏ Can we fly a drone over for data pickup? • Can Prelert components help to enable good spending and science decisions? The Business Stretch Jordan Mitchell pilots a quad copter drone that carries a GoPro 3+ camera used to survey the bottom topography. George Raber assists with the tablet computer. Photo credit: © Tim Calver for The Nature Conservancy
25. 25 Questions… Visit us at the AMA
26. www.elastic.co
27. 27 Materials • Additional Icon sets - Elyounssi Wahib https://www.iconfinder.com/Wahib (CC Attribution 3.0 Unported) • https://www.ansible.com/ (logo use) • https://www.rabbitmq.com/ (logo use) • http://www.solarwinds.com/ (logo use) • https://www.criticalstack.com/ (logo use) • https://www.snort.org/ (logo use) • https://www.bro.org/ (logo use) • https://www.proofpoint.com/us/products/et-pro-ruleset (logo use) • https://www.python.org/ (logo use) • http://www.rsyslog.com/ (logo use) • https://www.paterva.com/web7/ (logo use) • https://www.zotac.com/ (Remote sensing device)
28. Except where otherwise noted, this work is licensed under http://creativecommons.org/licenses/by-nd/4.0/ Creative Commons and the double C in a circle are registered trademarks of Creative Commons in the United States and other countries. Third party marks and brands are the property of their respective holders. 28 Please attribute Elastic with a link to elastic.co