osquery: approaching security the hacker way

osquery: approaching security the hacker way

osquery was released as an open source product by Facebook in October 2014. It is an instrumentation framework for Ubuntu, CentOS, and OS X. osquery makes low-level operating system analytics and monitoring both performant and intuitive.

This talk will walk through why we created osquery, how we use osquery at Facebook to improve our security how other companies currently take advantage of osquery, and how you can too!

We’ll outline some of the challenges and sensitivities we faced when developing osquery and planning its open source release, as well as how we overcame those issues. Additionally, we’ll spend some time talking about why we believe open source is critical to advancing the state of trusted, secure software. Security through obscurity is dead; this is the age of security through transparency.

Bc60a5fc6a131ea6cfa80e000b40c743?s=128

Mike Arpaia

May 28, 2015
Tweet

Transcript

  1. 5.

    how can we work together? what's osquery? why is openness

    in security important? open source security design decisions
  2. 8.

    no secrets are required we can be open in security

    •secrecy has stifled innovation •little software to help defend against modern attacks •reimplementing the same solutions, poorly •let's do the math
  3. 9.

    visualizing attacks and the defenses against them attacker math 101

    popularized by Dino Dai Zovi and Dan Guido •attacker will take the least cost path through an attack graph •the juice has to be worth the squeeze https://www.trailofbits.com/resources/attacker_math_101_slides.pdf
  4. 10.

    visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  5. 11.

    visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  6. 12.

    visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle overcome obstacle accomplish objective
  7. 13.

    visualizing attacks and the defenses against them attack graphs start

    overcome obstacle overcome obstacle accomplish objective overcome obstacle overcome obstacle overcome obstacle
  8. 16.
  9. 17.

    an open platform for host instrumentation osquery if you need

    to collect low-level information from an operating system then you should use osquery •small footprint •rich capabilities •solves real world problems
  10. 18.

    SQL for your infrastructure osquery use SQL queries to explore

    OS state •running processes •loaded kernel modules •active network connections •route table •firewall settings •installed software •much more
  11. 19.
  12. 23.

    more tables are being written every day many tables are

    available •acpi_tables •arp_cache •crontab •file_events •kernel_info •listening_ports •logged_in_users •mounts •pci_devices •processes •routes •shell_history •smbios_tables •suid_bin •system_controls •usb_devices •users •groups •rpm_packages •apt_sources •deb_packages •homebrew_packages •kernel_modules •memory_map •shared_memory •browser_plugins •startup_items
  13. 24.
  14. 25.

    daemon for low-level host monitoring osqueryd know how the results

    of a query change over time •schedule a query on your hosts via a config
 •the daemon takes care of periodically executing your queries •buffers results to disk and generates a log of state changes •logs results for aggregation and analytics •run your query in differential or snapshot mode
  15. 26.

    event-based operating system introspection host event pub/sub stream subscribe to

    key OS events to create dynamically growing tables •subscribe to “publishers” •filesystem changes (inotify, FSEvents) •network setting changes (SCNetwork) •query the history of your host, as it evolves
  16. 27.

    event-based file integrity logging file integrity monitoring use wildcards to

    monitor important files on your hosts •/bin/* •/Users/*/Downloads/**
  17. 28.

    for config distribution and data infrastructure plugin system •simple plugin

    API •specify your plugins at runtime with a command-line flag filesystem http zookeeper configuration filesystem flume scribe logging
  18. 30.

    lessons learned from MIDAS we knew that we could do

    better at •exposing capabilities safely •reducing engineering overhead •improving development culture
  19. 31.

    the problem exposing capabilities •sharing a capability required sharing the

    whole module •few example modules were released •dramatically limited the utility of MIDAS
  20. 32.

    the solution exposing capabilities SQL allows for simple, flexible analysis

    •tables allow us to give away the answers without giving away the questions •capabilities can be configured instead of developed
  21. 33.

    the problem engineering overhead in MIDAS, capabilities were written in

    Python •"complex" code to solve simple problems •security people are not programmers
  22. 34.

    the solution engineering overhead with SQL, asking a question doesn't

    require writing complex code •low-level operating system analytics with attention to UEX SELECT address, mac, count(mac) AS mac_count FROM arp_cache GROUP BY mac HAVING count(mac) > 1;
  23. 35.

    the problem development culture open source code was not actively

    used internally •codebase began to drift •contributions weren't getting pulled in
  24. 38.

    and now our watch begins working together to stay safe

    many different industries are working together •offensive security researchers •defensive security professionals •entrepreneurs in infrastructure/security