OS X Operating System Security at Scale

A critical aspect of maintaining a robust infrastructure security posture is being able to ask low-level question of hosts in your environment. Even on a single host, performing operating system analytics can often be complex, error prone and computationally expensive. This presentation will examine how Facebook is attacking host instrumentation at scale. We’ll discuss how you can use the same software that we use, regardless of your scale or environment, for no cost at all.

Transcript

1. mike arpaia / facebook
   OS X Security at Scale
   ted reed / facebook
   

   View Slide

2. OS X security at Facebook
   production hardening
   client engineering intrusion detection
   
    
    
   

   View Slide

3. “detection” and “response”
   catch attackers
   •insider threats
   •espionage

   •external threats
   •APT
   •hacktivists
   •mass malware
   •the list is endless
   

   View Slide

4. defend enterprise and production infra
   single intrusion detection team
   •extract as much signal as possible
   •make high confidence decisions
   •harder for the more variable OS X client fleet

   •avoid duplication in production
   •ease burden for humans
   •apply the same intelligence feeds
   •reuse storage
   

   View Slide

5. mac and linux laptops
   focus on client machines
   developer

   laptop
   
   
   
   Most variable
   Largest attack surface
   ‘Highest’ risk
   

   View Slide

6. but it’s a hard problem
   “install to win”
   
   network-based IDS host-based IDS
   

   View Slide

7. but it’s a hard problem
   “install to win”
   network-based IDS host-based IDS
   your machine is cooked
   must be time for a new
   laptop
   do not install that again
   “install and pray”
   

   View Slide

8. we live in a windows centric world
   •more OS X laptops
   •most production infrastructure runs on Linux

   •few are instrumenting their OS X and Linux hosts
   •affordably
   •tailored to medium enterprises or large infrastructures
   •how would we solve that problem?
   but, times are changing
   

   View Slide

9. desired
   properties
   

   View Slide

10. performant
    easy
    flexible
    simple
    development
    deployable
    upgrades
    low maintenance
    user impact
    long uptime
    metrics
    configurable
    integrations
    compliance
    automation
    vulnerability
    management
    

    View Slide

11. osquery
    

    View Slide

12. SQL for your infrastructure
    osquery
    use SQL queries to explore OS state
    •running processes
    •loaded kernel modules
    •active network connections
    •route table
    •firewall settings
    •installed software
    •file modifications
    

    View Slide

13. why SQL?
    SELECT pid, name, uid FROM processes
    OS concepts are shared on Mac, Linux, and Windows
    the “concepts” have attributes:
    user ids, process ids, descriptors, ports, paths
    most developers and administrators know SQL
    

    View Slide

14. why SQL?
    SELECT pid, name, uid FROM processes
    [concept]
    

    View Slide

15. why SQL?
    SELECT pid, name, uid FROM processes
    [attributes] [concept]
    

    View Slide

16. why SQL?
    SELECT pid, name, uid FROM processes
    [constraints]
    WHERE uid != 0
    

    View Slide

17. why SQL?
    JOIN users ON processes.uid=users.uid
    SELECT pid, name, username FROM processes
    WHERE uid != 0 [join]
    [attribute]
    

    View Slide

18. more tables are being written every day
    many tables are available
    •acpi_tables
    •arp_cache
    •crontab
    •file_events
    •kernel_info
    •listening_ports
    •logged_in_users
    •mounts
    •pci_devices
    •processes
    •routes
    •shell_history
    •smbios_tables
    •suid_bin
    •system_controls
    •usb_devices
    •users
    •groups
    •rpm_packages
    •apt_sources
    •deb_packages
    •homebrew_packages
    •kernel_modules
    •memory_map
    •shared_memory
    •browser_plugins
    •startup_items
    

    View Slide

19. use simple tables, together
    osquery enables complex analysis
    by allowing users to join and
    aggregate across several simple
    tables
    •simple tables have many advantages
    •easier to write
    •easier to maintain
    •can be used in many contexts
    

    View Slide

20. osquery is much more
    than a security tool
    

    View Slide

21. osquery is much more
    than a security tool
    actually, literally…
    it is a family of tools
    

    View Slide

22. osqueryi
    

    View Slide

23. View Slide

24. LaunchDaemons which run a binary at boot
    

    View Slide

25. running processes
    

    View Slide

26. processes listening on ports
    

    View Slide

27. osqueryd
    

    View Slide

28. daemon for low-level host monitoring
    osqueryd
    know how the results of a query change over time
    •schedule a query on your hosts via a config

    •the daemon takes care of periodically executing your queries
    •buffers results to disk and generates a log of state changes
    •logs results for aggregation and analytics
    

    View Slide

29. event-based operating system introspection
    host eventing stream
    subscribe to key OS events to create dynamically growing tables
    •subscribe to “publishers”
    •filesystem changes (inotify, FSEvents)
    •network setting changes (SCNetwork)
    •application usages (NSNotificationCenter)
    •query the history of your host, as it evolves
    

    View Slide

30. for config distribution, data infrastructure and more
    plugin system
    •simple plugin API
    •specify your plugins at runtime with a command-line flag
    filesystem
    http
    zookeeper
    configuration
    filesystem
    flume
    scribe
    logging
    tls
    ldap
    oauth
    enrollment
    

    View Slide

31. how we config and log results
    facebook workflow
    1. osquery.pkg published automatically to https://osquery.io
    2. download weekly and update chef cookbook
    3. chef writes configuration and installs pkg
    1. newsyslog.d rotation file
    2. list of scheduled queries
    4. results written to /var/log/osqueryd.results.log
    5. splunk lightweight forwarder
    6. backend analytics
    

    View Slide

32. tables
    

    View Slide

33. creating tables is easy
    easily define what your tables “look like” in Python and use C++
    to implement what a full-table scan would return
    •the Python is used to generate faster C++ code transparently
    •you write a single C++ function which implements a full-table scan
    

    View Slide

34. table_name("time")
    schema([
    Column("hour", INTEGER),
    Column("minutes", INTEGER),
    Column("seconds",INTEGER),
    ])
    implementation("time@genTime")
    

    View Slide

35. namespace osquery {
    namespace tables {
    QueryData genTime(QueryContext& ctx) {
    QueryData results;
    struct tm* now = localtime(time(0));
    Row r;
    r["hour"] = INTEGER(now->tm_hour);
    r["minutes"] = INTEGER(now->tm_min);
    r["seconds"] = INTEGER(now->tm_sec);
    results.push_back(r);
    return results;
    }
    }
    }
    

    View Slide

36. https://osquery.io/tables
    browse all tables, columns, descriptions, and example queries
    

    View Slide

37. open source
    

    View Slide

38. all development happens in the open, on GitHub
    work on osquery with us
    the problem that osquery solves isn't unique to facebook
    •https://github.com/facebook/osquery
    •https://osquery.io
    •https://osquery.readthedocs.org
    this journey is 1% finished: get involved
    •we’re excited to take on future challenges in the open
    •let’s build together
    

    View Slide

39. View Slide

40. questions
    https://osquery.io
    

    View Slide