Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elastic{ON} 2018 - A Security Analytics Platform for Today

Elastic{ON} 2018 - A Security Analytics Platform for Today

Ever thought about building an end-to-end security analytics platform leveraging the Elastic Stack and X-Pack? Doing so offers opportunities like increasing team impact by having more data faster and gaining back time for threat hunting versus responding to alerts.

In this session, we'll explore how to analyze and correlate security data with a homegrown solution that’s fast and scalable.

Samir Bennacer | Senior Solution Architect| Elastic
Kevin Keeney |Cybersecurity Advocate | Elastic

Elastic Co

March 01, 2018
Tweet

More Decks by Elastic Co

Other Decks in Technology

Transcript

  1. Elastic
    Date: 01/March/2018
    A Security Analytics
    Platform for Today
    Kevin Keeney, Cybersecurity Advocate,
    Samir Bennacer, Senior Solutions Architect

    View Slide

  2. 2
    Attacks are inevitable

    View Slide

  3. Data Collection
    For effective security analysis

    View Slide

  4. • Collect all parts of the puzzle
    • Normalize for aggregation and correlation across sources
    • Enrich to extend attributes available for analysis
    • Index for immediate recall
    Foundation for Effective Security Analysis
    Collect Normalize Enrich Index

    View Slide

  5. Data Sources
    Domain
    Data
    Sources
    Timing Tools
    Network
    PCAP,
    Bro,
    NetFlow
    Real time, Packet-based
    Packetbeats, Logstash ( netflow
    module)
    Collect Normalize Enrich Index

    View Slide

  6. Data Sources
    Domain
    Data
    Sources
    Timing Tools
    Network
    PCAP,
    Bro,
    NetFlow
    Real time, Packet-based
    Packetbeats, Logstash ( netflow
    module)
    Application Logs Real-time, Event-based Filebeats, Logstash
    Collect Normalize Enrich Index

    View Slide

  7. Data Sources
    Domain
    Data
    Sources
    Timing Tools
    Network
    PCAP,
    Bro,
    NetFlow
    Real time, Packet-based
    Packetbeats, Logstash ( netflow
    module)
    Application Logs Real-time, Event-based Filebeats, Logstash
    Cloud Logs, API Real-time, Event-based Beats, Logstash
    Collect Normalize Enrich Index

    View Slide

  8. Data Sources
    Domain
    Data
    Sources
    Timing Tools
    Network
    PCAP,
    Bro,
    NetFlow
    Real time, Packet-based
    Packetbeats, Logstash ( netflow
    module)
    Application Logs Real-time, Event-based Filebeats, Logstash
    Cloud Logs, API Real-time, Event-based Beats, Logstash
    Host
    System
    State,
    Signature
    Alert
    Real-time, Asynchronous
    Auditbeats, Filebeats ( Osquery
    module),Winlogbeats
    Collect Normalize Enrich Index

    View Slide

  9. Data Sources
    Domain
    Data
    Sources
    Timing Tools
    Network
    PCAP,
    Bro,
    NetFlow
    Real time, Packet-based
    Packetbeats, Logstash ( netflow
    module)
    Application Logs Real-time, Event-based Filebeats, Logstash
    Cloud Logs, API Real-time, Event-based Beats, Logstash
    Host
    System
    State,
    Signature
    Alert
    Real-time, Asynchronous
    Auditbeats, Filebeats ( Osquery
    module),Winlogbeats
    Active Scanning User-driven, Asynchronous Vulnerability scanners
    Collect Normalize Enrich Index

    View Slide

  10. Event Agent
    Device Network Source Destination Service
    Threat GeoIp
    User
    Network Protocols… Various Services…
    Group 1 (Must be populated)
    Group 2 (Must be populated to the max extent practical where event message contains relevant fields.)
    Host
    Group 3 (should include Group 2 prefix and may include Group 3 prefix(es) in field names. Any Group 3 prefixes must not conflict with any defined ECS field name.)
    @timestamp
    ecs_version
    message
    File
    Error
    Elastic Common Schema Collect Normalize Enrich Index

    View Slide

  11. Logstash
    Inputs
    Beats


    JDBC


    TCP
    UDP
    HTTP
    Filters
    Extract Fields
    Geo Enrich
    Lookup Enrich
    DNS Lookups
    Pattern Matching
    ArcSight Codec

    Network / Security Data
    Syslog Servers
    Infra / App Data
    IoT / Sensors
    Persistent Disk Based
    Queues
    Normalization and Enrichment
    Beats
    Outputs
    Elasticsearch





    Kafka
    RabbitMQ
    RDBMS
    Centralized
    Configuration Management
    Elasticsearch
    Collect Normalize Enrich Index

    View Slide

  12. Threat
    intelligence
    Geo IP
    Information
    Other
    Information
    • Reputation information
    • IOCs
    • Vulnerability Data
    • TTPs
    • Physical Location
    • Country, State …
    • Postal Code
    • Geo Fence
    • Network Model
    • User information
    • Org Chart
    • DNS resolution
    Data Enrichment Collect Normalize Enrich Index

    View Slide

  13. A common use case is looking up ips from a spam/bot feed:
    filter {

    memcached {

    hosts => ["127.0.0.1:11211"]

    get => {

    "%{ip}" => "threat_src"

    }

    }

    }
    Recommend to read the blog https://www.elastic.co/blog/elasticsearch-data-enrichment-with-logstash-a-few-security-examples
    Example: Botip Lookup

    View Slide

  14. Alert Based detection
    Event Correlation

    View Slide

  15. ●Event correlation
    ●Cross-source Correlation
    ●Tiered Correlation
    ●Chained Correlation
    Alert Based detection

    View Slide

  16. Event correlation
    Logstash
    Elasticsearch
    X-pack
    X-pack Alerting
    Zoom

    View Slide

  17. Detecting Anomalies
    using ML

    View Slide

  18. What is Normal?
    When something behaves like itself
    Monday
    Tuesday
    Wednesday
    Thursday
    When something behaves like its peers

    View Slide

  19. high memory alerts
    -- server 1 -- server 2 -- server 3
    Host Behavior
    • Free disk space lower than average
    • Unusual log entries
    Network Behavior
    • Unusual connections between hosts
    • Higher than average data transfer
    Application Behavior
    • Service response time abnormally high
    • Dropped connections exceed normal
    When abnormal matters

    View Slide

  20. 20
    Understand
    Seasonality
    Reduce False
    Positives
    Avoid Manual
    Review and
    Revision
    The advantages of anomaly-driven alerting
    Identify
    Areas of
    Focus

    View Slide

  21. Getting Started – Machine Learning Recipes

    View Slide

  22. Threat Hunting

    View Slide

  23. Humans are more
    important than
    Hardware.

    View Slide

  24. Cyber is a human
    versus human conflict
    -dcode

    View Slide

  25. Know Thy Enemy
    Strategic
    Who & Why
    Tactical
    How & What

    View Slide

  26. Know Thy Self
    BlindSpots
    Culture
    Most Valuable Data
    Critical Systems

    View Slide

  27. Intelligence Operations

    View Slide

  28. What are you looking
    for?
    Hypothesis Investigation
    New
    Patterns
    and IOA
    IOCs
    Inform
    and
    Enrich
    Different data sets Identify the patterns Feed the IOCs back
    create new alerts to
    improve the speed of the
    detection
    Operations Intelligence
    Intelligence

    View Slide

  29. 29
    Pulling it all together…
    Understand who is
    your Adversary? 

    What is their
    Motivation ?
    What is the Impacts
    Of a successful attack?
    What are they
    targeting?

    View Slide

  30. Speed is king

    View Slide

  31. People are our most
    precious resource

    View Slide

  32. www.elastic.co

    View Slide

  33. Except where otherwise noted, this work is licensed under
    http://creativecommons.org/licenses/by-nd/4.0/
    Creative Commons and the double C in a circle are
    registered trademarks of Creative Commons in the United States and other countries.
    Third party marks and brands are the property of their respective holders.
    33
    Please attribute Elastic with a link to elastic.co

    View Slide