Stopping Attacks on Everyday Operating Systems Using Secure Monitoring

938bca9547ba1cac3e69d80efd67fe6b?s=47 Bryan Payne
February 01, 2010

Stopping Attacks on Everyday Operating Systems Using Secure Monitoring

The operating systems that most people use every day (e.g., Windows, Mac OS X, Linux) are not designed with security in mind. Yet, in today's highly connected world, security is more important than ever before. In this talk, I will describe how the computing community has reached this paradoxical state and how my research in secure host-based monitoring provides the foundation needed to start addressing it. The talk will start with a discussion of the Turret architecture, a secure and general architecture for monitoring running systems. Then I will describe a new security application that utilize Turret to enable security policies based on user intent. With the Turret architecture as a foundation, future research can focus on how to best utilize and deploy these techniques to enable the security that users need without limiting their work environment.

938bca9547ba1cac3e69d80efd67fe6b?s=128

Bryan Payne

February 01, 2010
Tweet

Transcript

  1. Stopping Attacks on Everyday Operating Systems Using Secure Monitoring Bryan

    D. Payne School of Computer Science Georgia Institute of Technology
  2. None
  3. None
  4. None
  5. None
  6. None
  7. None
  8. None
  9. None
  10. Sample Security Properties Property Supporting Mechanism Control file access Authentication,

    Access Control Prevent sensitive data leaks Information Flow Control, Firewalls Audit computer usage Secure Logging
  11. Sample Security Properties Property Supporting Mechanism Control file access Authentication,

    Access Control Prevent sensitive data leaks Information Flow Control, Firewalls Audit computer usage Secure Logging All rely on a secure operating system!
  12. Your OS Is Not Secure Usage Share for Various Operating

    Systems Based on Web Client Data Windows XP Windows Vista Windows 7 Windows 2000 Other Mac OS X Mac iPhone Linux Source: http://en.wikipedia.org/wiki/ Usage_share_of_operating_systems
  13. Your OS Is Not Secure Software Vulnerabilities ∝ SLOC 0

    75 150 225 300 1990 1995 2000 2005 2010 SLOC for Various Operating System Releases SLOC (in Millions) Release Year Windows Debian Mac OS X Source: http://en.wikipedia.org/wiki/ Source_lines_of_code
  14. Solution: Security Software? Antivirus Firewall Intrusion Detection Whitelisting

  15. Without a secure foundation, security applications are merely annoyances to

    an experienced attacker! Collapsed apartment building in Shanghai. Hardware Insecure OS Email Web Text AV Insecure OS makes applications vulnerable.
  16. Idea #1: Coprocessor Card • Easy installation • View host

    memory securely • Canʼt interpose on h/w events • Expensive (~$100 - $400)
  17. Idea #2: Virtualization • View host memory securely • Interpose

    on h/w events • Inexpensive (software only)
  18. My Research & Todayʼs Talk 2) Passive monitoring with XenAccess

    3) Active monitoring using protected hooks 4) Security applications utilizing monitoring architecture Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 1) Improving hypervisor-provided isolation
  19. My Research & Todayʼs Talk 2) Passive monitoring with XenAccess

    3) Active monitoring using protected hooks 4) Security applications utilizing monitoring architecture Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 1) Improving hypervisor-provided isolation
  20. Secure Monitoring What good is a security check that can

    be easily circumvented?
  21. Hypervisor User VM Security VM User Process XenAccess: Virtual Machine

    Introspection Security Application Mouse / Keyboard Network Disk User Process User Process OS Kernel Device Drivers BD Payne, M Carbone, W Lee. Secure and Flexible Monitoring of Virtual Machines. Proceedings of the Annual Computer Security Applications Conference, 2007. Virtual Machine Introspection
  22. XenAccess Process List in Security VM Process Explorer Output in

    User VM (Windows XP) Passive Monitoring Example
  23. 86 38 75 32 22135 39 31735 34 653 37

    79 32 Kernel VA Kernel Sym User VA 20 40 60 80 100 PV￿M PV￿H HVM￿M HVM￿H Time in Microseconds Memory Access Times
  24. ￿￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿￿

    ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ 0 1000 2000 3000 4000 0 1 2 3 4 5 6 Data size in bytes Time in microseconds ￿ HVM ￿ PV Memory Read Performance
  25. XenAccess Today 1700+ downloads Enabling Innovation - University research projects

    - Defense contractors - US military - Industrial research labs Community Involvement - Open source project - Patches from around the world http://www.xenaccess.org
  26. • Hypervisor provides protection and inter-VM comms • Memory protection

    used for hooks and trampoline, security relies on hypervisor being trusted Turret Architecture BD Payne, M Carbone, W Lee. Lares: An Architecture for Secure Active Monitoring Using Virtualization. Proceedings of the IEEE Symposium on Security and Privacy, 2008. Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk
  27. Hypervisor User VM Security VM Hooks User Processes ... Memory

    Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Turret Architecture • User VM is where user does regular work • Protected hooks transfer execution to trampoline • Trampoline transfers execution to Security VM, using an inter-VM communication channel
  28. Hypervisor User VM Security VM Hooks User Processes ... Memory

    Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Turret Architecture • Security VM is where security application runs • Receives event notifications from trampoline through the inter-VM communication channel • Uses introspection to enrich context information
  29. (all times are in micro-seconds) Traditional Hook 0 10 20

    30 40 Lares Hook 0 10 20 30 40 Performance Comparison
  30. • Bypass Hook (A1) • Modify event context (A2) •

    Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Libraries, OS and other dependencies Kernel or process execution flow Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook
  31. Libraries, OS and other dependencies Kernel or process execution flow

    Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Prevented by virtualization protections
  32. • A1.5 and A1.6 are protected via memory protections •

    A1.2 and A1.4 also protected via memory protections • A1.3 would require attacker to relocate all of the kernelʼs memory, which is difficult to do without detection • A1.1 protected by CPU IDTR IDT Syscall dispatcher Trampoline SSDT hook GDTR + GDT + Paging structures To Security VM A1.5 A1.4 A1.2 A1.1 A1.3 A1.6 “Bypass Hook” Attacks
  33. Libraries, OS and other dependencies Kernel or process execution flow

    Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks
  34. Libraries, OS and other dependencies Kernel or process execution flow

    Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Prevented by disabling interrupts in trampoline
  35. Application-Level Monitoring Effective monitoring requires finding and viewing the right

    things.
  36. Identifying User Network Traffic

  37. Identifying User Network Traffic

  38. Identifying User Network Traffic

  39. User Application h1 = Click h2 = Key(A) h3 =

    Key(D) h4 = Click h5 = Key(S) H Hi = {h2, h3, h5} content(Hi) = 'ADS' e1 = HTTP GET e2 = EMAIL 'ADS' e3 = HTTP GET content(e2) = 'ADS' Security Monitor E Using Hardware Events Hi 㱺 e content (Hi) = f (content (e)) [1] [2]
  40. Gyrus Framework Hypervisor Security Virtual Machine User VM Network-Based User

    Application User Kernel User Kernel Transparent Network Redirection Mouse / Keyboard Network Disk Transparent Proxy Enforcement Module Authorization Database User VM Device Model H/W Event 1 2 3 4 5 6 7 Authorization Definition Event Testing — Authorization Creation — Enforcement 1,2 3,4 5,6,7
  41. Hardware Event Interposition Memory Access Screen Scraping Supporting API For

    App Modules
  42. None
  43. None
  44. None
  45. Email: Outlook Express User VM Outlook Express Email Client User

    Kernel comctl32.dll win32k !"#$%&#'()$**+,"-'.'/012*34& !56"&,77*&#*5-' '89:;"<'////1:=/ !"#$%&#'()$**+,"-'.'/0=34:>=2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'////1/C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=/:H53& !56"&,77*&#*5-' '89:;"<'//////:> !"#$%&#'()$**+,"-'.'/0=3A*>5/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'/////&H1' 'DIEJKG !"#$%&#'()$**+,"-'.'/0=/:H5A2 !56"&,77*&#*5-' '89:;"<'/////4:2 !"#$%&#'()$**+,"-'.'/0=/:H5C> !56"&,77*&#*5-' '89:;"<'/////43> !"#$%&#'()$**+,"-'.'/0=3A*>*/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'//////4=' 'DLIEJKG !"#$%&#'()$**+,"-'.'/0=34:452 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'/////=H=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:C1/ !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:H&/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'/////=H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=34:C// !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=34:C2/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////C=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*/1/ !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////24=' 'DLO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B:2 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////4=' 'DLG !"#$%&#'()$**+,"-'.'/0=34:4&2 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'//////H1' 'DJMNG !"#$%&#'()$**+,"-'.'/0=3A*/// !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////=H1' 'DO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B32 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////H1' 'DG !"#$%&#'()$**+,"-'.'/0=3A*>C/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'/////24=' 'DLO?RJG !"#$%&#'()$**+,"-'.'/0=3A*HS2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////2C=' 'DLT)KQ?G !"#$%&#'()$**+,"-'.'/0=34:>A2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'//////H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*>// !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'/////=4=' 'DLP?)RG !"#$%&#'()$**+,"-'.'/0=3A*AS/ !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'//////C1' 'DP?)RG !"#$%&#'()$**+,"-'.'/0=3A*H*2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////4C1' 'DT)KQ?G !"#$%&#'()$**+,"-'.'/0=3A*>32 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////=H1' 'D)M)Q?G !"#$%&#'()$**+,"-'.'/0=3A*>4/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'//////C1' 'DO?RJG !"#$%&#'()$**+,"-'.'/0=3A*>:2 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////>C=' 'DL)M)Q?G !"#$%&#'()$**+,"-'.'/0=34:>5/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=/:H5>& !56"&,77*&#*5-' '89:;"<'//////B2 !"#$%&#'()$**+,"-'.'/0=34:>*/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:HB/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'//////C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=34:CC/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////41' 'D8EF)G mshtml.dll Event Testing Module - Mouse click on email send button Authorization Creation Module - Email from memory & screen - Reconstruct network output Enforcement Module - Validate outgoing emails
  46. Email: Outlook Express User VM Outlook Express Email Client User

    Kernel comctl32.dll win32k !"#$%&#'()$**+,"-'.'/012*34& !56"&,77*&#*5-' '89:;"<'////1:=/ !"#$%&#'()$**+,"-'.'/0=34:>=2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'////1/C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=/:H53& !56"&,77*&#*5-' '89:;"<'//////:> !"#$%&#'()$**+,"-'.'/0=3A*>5/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'/////&H1' 'DIEJKG !"#$%&#'()$**+,"-'.'/0=/:H5A2 !56"&,77*&#*5-' '89:;"<'/////4:2 !"#$%&#'()$**+,"-'.'/0=/:H5C> !56"&,77*&#*5-' '89:;"<'/////43> !"#$%&#'()$**+,"-'.'/0=3A*>*/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'//////4=' 'DLIEJKG !"#$%&#'()$**+,"-'.'/0=34:452 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'/////=H=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:C1/ !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:H&/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'/////=H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=34:C// !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=34:C2/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////C=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*/1/ !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////24=' 'DLO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B:2 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////4=' 'DLG !"#$%&#'()$**+,"-'.'/0=34:4&2 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'//////H1' 'DJMNG !"#$%&#'()$**+,"-'.'/0=3A*/// !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////=H1' 'DO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B32 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////H1' 'DG !"#$%&#'()$**+,"-'.'/0=3A*>C/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'/////24=' 'DLO?RJG !"#$%&#'()$**+,"-'.'/0=3A*HS2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////2C=' 'DLT)KQ?G !"#$%&#'()$**+,"-'.'/0=34:>A2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'//////H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*>// !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'/////=4=' 'DLP?)RG !"#$%&#'()$**+,"-'.'/0=3A*AS/ !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'//////C1' 'DP?)RG !"#$%&#'()$**+,"-'.'/0=3A*H*2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////4C1' 'DT)KQ?G !"#$%&#'()$**+,"-'.'/0=3A*>32 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////=H1' 'D)M)Q?G !"#$%&#'()$**+,"-'.'/0=3A*>4/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'//////C1' 'DO?RJG !"#$%&#'()$**+,"-'.'/0=3A*>:2 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////>C=' 'DL)M)Q?G !"#$%&#'()$**+,"-'.'/0=34:>5/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=/:H5>& !56"&,77*&#*5-' '89:;"<'//////B2 !"#$%&#'()$**+,"-'.'/0=34:>*/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:HB/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'//////C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=34:CC/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////41' 'D8EF)G mshtml.dll Event Testing Module - Mouse click on email send button Authorization Creation Module - Email from memory & screen - Reconstruct network output Enforcement Module - Validate outgoing emails User Interaction Mean StdDev Click, OE not running 1.8 ms 0.42 ms Click, no OE compose window 27.9 ms 0.99 ms Click in compose edit area 35.6 ms 0.52 ms Click in compose tool bar 167.5 ms 0.71 ms Click on send button 3520.6 ms 160.19 ms
  47. Related Work

  48. Historical / Systems Security Kernels (1974) Separation Kernels (Rushby 1981)

    GEMSOS (Gemini) Scomp (Honeywell) PSOS (Neumann et al. 1973) KSOS (Ford) VAX VMM (Karger et al. 1981) MILS (Alves-Foss et al. 2002) MASK (Martin et al. 2000) TCX (Irvine et al. 2003) VAX VMM (Karger et al. 1981) Hardware Trusted Operating System Untrusted Kernel Interfaces App App App Security Kernel - Tamperproof - Complete Mediation - Verifiable Hardware Separation Kernel App App App
  49. Security through VMI VMI Concept (Garfinkel and Rosenblum 2003) Intrusion

    Detection with VMI, all passive Livewire (Garfinkel and Rosenblum 2003) HyperSpector (Kourai et al. 2005) IntroVirt (Joshi et al. 2005) SBCFI (Petroni and Hicks 2007) Memory Forensics & Semantic Knowledge FATKit (Petroni et al. 2006) VIX (Hay and Nance 2008) BackTracker (King et al. 2005) “Digging for Data Structures” (Cozzie et al. 2008) “Robust signatures for kernel data structures” (Dolan-Gavitt et al. 2009)
  50. Future Work & Conclusions

  51. Platform Security Hardware-rooted security Reducing size of trusted computing base

    (TCB) Simplify deployment
  52. Application Support Hypervisor Hooks User Processes ... Memory Protector Network

    Traffic Trampoline Hardware Events Hook Events Mouse / Keyboard Network Disk Existing applications New applications for new threats
  53. Conclusions / Summary Hypervisor User VM Security VM Hooks User

    Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 • Secure and robust monitoring architecture • New security applications • Plans to further improve security, deployability
  54. Stopping Attacks on Everyday Operating Systems Using Secure Monitoring Bryan

    D. Payne School of Computer Science Georgia Institute of Technology
  55. None
  56. Our hypervisor modifications mark specified memory as read-only, forcing a

    trap on write Memory Protections User page table Virtual Physical User VM Is the PF listed as protected? Mark as read-only PTE propagation NO YES Memory Protection Policy Security VM Shadow page table Virtual Machine Hypervisor
  57. Is the write targeted at a protected region? Emulate the

    write Propagate exception to guest NO YES Page fault due to failed write Technique for byte-level memory protection Memory Protections
  58. None