osquery: approaching security the hacker way

osquery: approaching security the hacker way

osquery was released as an open source product by Facebook in October 2014. It is an instrumentation framework for Ubuntu, CentOS, and OS X. osquery makes low-level operating system analytics and monitoring both performant and intuitive.

This talk will walk through why we created osquery, how we use osquery at Facebook to improve our security how other companies currently take advantage of osquery, and how you can too!

We’ll outline some of the challenges and sensitivities we faced when developing osquery and planning its open source release, as well as how we overcame those issues. Additionally, we’ll spend some time talking about why we believe open source is critical to advancing the state of trusted, secure software. Security through obscurity is dead; this is the age of security through transparency.

Bc60a5fc6a131ea6cfa80e000b40c743?s=128

Mike Arpaia

May 28, 2015
Tweet

Transcript

  1. mike arpaia / facebook osquery: approaching security the hacker way

    @mikearpaia
  2. why is openness in security important?

  3. what's osquery? why is openness in security important?

  4. what's osquery? why is openness in security important? open source

    security design decisions
  5. how can we work together? what's osquery? why is openness

    in security important? open source security design decisions
  6. being open

  7. open source makes life easier

  8. no secrets are required we can be open in security

    •secrecy has stifled innovation •little software to help defend against modern attacks •reimplementing the same solutions, poorly •let's do the math
  9. visualizing attacks and the defenses against them attacker math 101

    popularized by Dino Dai Zovi and Dan Guido •attacker will take the least cost path through an attack graph •the juice has to be worth the squeeze https://www.trailofbits.com/resources/attacker_math_101_slides.pdf
  10. visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  11. visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  12. visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  13. visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle accomplish objective overcome obstacle overcome obstacle overcome obstacle
  14. we can alter the path of attackers by altering our

    defenses
  15. the future of security will be written in vim it's

    an engineering problem
  16. osquery

  17. an open platform for host instrumentation osquery if you need

    to collect low-level information from an operating system then you should use osquery •small footprint •rich capabilities •solves real world problems
  18. SQL for your infrastructure osquery use SQL queries to explore

    OS state •running processes •loaded kernel modules •active network connections •route table •firewall settings •installed software •much more
  19. osqueryi

  20. LaunchDaemons which run a binary at boot

  21. running processes

  22. processes listening on ports

  23. more tables are being written every day many tables are

    available •acpi_tables •arp_cache •crontab •file_events •kernel_info •listening_ports •logged_in_users •mounts •pci_devices •processes •routes •shell_history •smbios_tables •suid_bin •system_controls •usb_devices •users •groups •rpm_packages •apt_sources •deb_packages •homebrew_packages •kernel_modules •memory_map •shared_memory •browser_plugins •startup_items
  24. osqueryd

  25. daemon for low-level host monitoring osqueryd know how the results

    of a query change over time •schedule a query on your hosts via a config
 •the daemon takes care of periodically executing your queries •buffers results to disk and generates a log of state changes •logs results for aggregation and analytics •run your query in differential or snapshot mode
  26. event-based operating system introspection host event pub/sub stream subscribe to

    key OS events to create dynamically growing tables •subscribe to “publishers” •filesystem changes (inotify, FSEvents) •network setting changes (SCNetwork) •query the history of your host, as it evolves
  27. event-based file integrity logging file integrity monitoring use wildcards to

    monitor important files on your hosts •/bin/* •/Users/*/Downloads/**
  28. for config distribution and data infrastructure plugin system •simple plugin

    API •specify your plugins at runtime with a command-line flag filesystem http zookeeper configuration filesystem flume scribe logging
  29. design decisions

  30. lessons learned from MIDAS we knew that we could do

    better at •exposing capabilities safely •reducing engineering overhead •improving development culture
  31. the problem exposing capabilities •sharing a capability required sharing the

    whole module •few example modules were released •dramatically limited the utility of MIDAS
  32. the solution exposing capabilities SQL allows for simple, flexible analysis

    •tables allow us to give away the answers without giving away the questions •capabilities can be configured instead of developed
  33. the problem engineering overhead in MIDAS, capabilities were written in

    Python •"complex" code to solve simple problems •security people are not programmers
  34. the solution engineering overhead with SQL, asking a question doesn't

    require writing complex code •low-level operating system analytics with attention to UEX SELECT address, mac, count(mac) AS mac_count FROM arp_cache GROUP BY mac HAVING count(mac) > 1;
  35. the problem development culture open source code was not actively

    used internally •codebase began to drift •contributions weren't getting pulled in
  36. the solution development culture we do 100% of our engineering

    in the open, on GitHub
  37. fighting together

  38. and now our watch begins working together to stay safe

    many different industries are working together •offensive security researchers •defensive security professionals •entrepreneurs in infrastructure/security
  39. questions https://osquery.io