Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Community Intelligence & Open Source Tools

Community Intelligence & Open Source Tools

My presentation on building a Threat Intelligence pipeline for Cyber Threat Intelligence at the SANS CTI Summit.

Scott J. Roberts

February 03, 2016
Tweet

More Decks by Scott J. Roberts

Other Decks in Technology

Transcript

  1. Community Intelligence
    & Open Source Tools
    Building an Actionable Pipeline

    View Slide

  2. Intro

    View Slide

  3. Me:
    Scott J Roberts
    @sroberts
    Han Solo is my Spirit Animal

    View Slide

  4. What do CTI industry analysts say?
    "When it comes to eating @sroberts is a
    thought leader up & to the right on all
    quadrants!"
    ~ @rickhholland

    View Slide

  5. DFIRing Since 2006
    CTIing Since 2007
    Deving Since 2009

    View Slide

  6. View Slide

  7. View Slide

  8. The Problem
    We are spinning up considerable
    new telemetry using open source
    tools and we need to feed those
    tools with actionable intelligence.

    View Slide

  9. The Other Problem

    View Slide

  10. Pocket

    View Slide

  11. Chat

    View Slide

  12. Note Books

    View Slide

  13. And all the
    other
    sources...

    View Slide

  14. $$$$

    View Slide

  15. So I did what anyone with a little Python experience does
    I built my own...

    View Slide

  16. And I built my own again...
    And another time...
    In the end I built about 5 or 6...

    View Slide

  17. They all sorta sucked... !

    View Slide

  18. "I have not failed. I
    have found I've just
    found 10,000 ways
    that won't work."
    ~ Thomas Edison

    View Slide

  19. ‒ Direction

    View Slide

  20. Breath vs. Depth

    View Slide

  21. OSX, Linux, & GitHub
    centric threats

    View Slide

  22. ‒ Collection

    View Slide

  23. Twitter
    Email Lists
    Feeds
    Ongoing Incidents
    Manual

    View Slide

  24. ‒ Exploitation

    View Slide

  25. To Use a Technical Term
    Indicator Extraction
    sucks...

    View Slide

  26. But we did it anyway...1
    1 YOLO!!!

    View Slide

  27. Jager & Caçador 2
    2 Look it means hunter in Portuguese.

    View Slide

  28. View Slide

  29. Command
    $ pbpaste | cacador | jq '.[]'

    View Slide

  30. Output

    View Slide

  31. Tada!!!

    View Slide

  32. ‒ Analysis

    View Slide

  33. Threat Note

    View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. Enrichments
    Whois
    PassiveTotal
    Shodan
    VirusTotal

    View Slide

  40. View Slide

  41. View Slide

  42. Maltego

    View Slide

  43. Fast
    Incident
    Response

    View Slide

  44. ‒ Dissemination

    View Slide

  45. Now
    Manual

    View Slide

  46. Soon™
    osquery & Bro Intelligence
    Chat with Hubot
    Intelligence Reports
    Application Integration

    View Slide

  47. ‒ Feedback

    View Slide

  48. The Result

    View Slide

  49. View Slide

  50. The (REAL) Result
    A (somewhat) automated system
    providing centralized threat data
    & intelligence management made
    up of a single source of truth
    supported by purpose built
    collection, processing, and
    analysis integrations.

    View Slide

  51. Lessons

    View Slide

  52. This isn't easy
    But parts are.

    View Slide

  53. Threat Intel Tools Work
    When They're Integrated
    ~
    collection | analysis | dissemination

    View Slide

  54. High Value Investments
    Tool: Paterva Maltego ~ $760
    Service: PassiveTotal ~ $??
    Learning: Introducing Python ~
    $33

    View Slide

  55. Learn to Code

    View Slide

  56. Unix Philosophy
    Small is beautiful
    Make each program do one thing
    well
    Portability over efficiency
    Store data in flat files
    Make every program a filter

    View Slide

  57. Data formats matter less than format openness
    CSV & JSON

    View Slide

  58. Perfect
    Is the Enemy Of
    Good

    View Slide

  59. The Future:
    Scaling Up Collection & Storage
    Expanded Threat_Notes APIs &
    Integrations
    Reputation & Fuzzy Indicators

    View Slide

  60. Links
    github.com/defpoint/threatnote
    github.com/certsocietegenerale/FIR
    github.com/sroberts/jager
    github.com/sroberts/cacador
    github.com/kbandla/APTnotes
    github.com/armbues/iocparser
    github.com/ivanlei/threatbutt

    View Slide

  61. Thanks
    Threat Note: @brianwarehime
    FIR: @thomchop_
    APTNotes: @kbandla
    Jager: @kylemaxwell, @kbandla, & @deadbits

    View Slide

  62. Questions???
    ~
    @sroberts
    http://sroberts.github.io

    View Slide