Stopping Attacks on Everyday Operating Systems Using Secure Monitoring

The operating systems that most people use every day (e.g., Windows, Mac OS X, Linux) are not designed with security in mind. Yet, in today's highly connected world, security is more important than ever before. In this talk, I will describe how the computing community has reached this paradoxical state and how my research in secure host-based monitoring provides the foundation needed to start addressing it. The talk will start with a discussion of the Turret architecture, a secure and general architecture for monitoring running systems. Then I will describe a new security application that utilize Turret to enable security policies based on user intent. With the Turret architecture as a foundation, future research can focus on how to best utilize and deploy these techniques to enable the security that users need without limiting their work environment.

Transcript

1. Stopping Attacks on Everyday Operating Systems Using Secure Monitoring Bryan D. Payne School of Computer Science Georgia Institute of Technology
10. Sample Security Properties Property Supporting Mechanism Control file access Authentication, Access Control Prevent sensitive data leaks Information Flow Control, Firewalls Audit computer usage Secure Logging
11. Sample Security Properties Property Supporting Mechanism Control file access Authentication, Access Control Prevent sensitive data leaks Information Flow Control, Firewalls Audit computer usage Secure Logging All rely on a secure operating system!
12. Your OS Is Not Secure Usage Share for Various Operating Systems Based on Web Client Data Windows XP Windows Vista Windows 7 Windows 2000 Other Mac OS X Mac iPhone Linux Source: http://en.wikipedia.org/wiki/ Usage_share_of_operating_systems
13. Your OS Is Not Secure Software Vulnerabilities ∝ SLOC 0 75 150 225 300 1990 1995 2000 2005 2010 SLOC for Various Operating System Releases SLOC (in Millions) Release Year Windows Debian Mac OS X Source: http://en.wikipedia.org/wiki/ Source_lines_of_code
14. Solution: Security Software? Antivirus Firewall Intrusion Detection Whitelisting
15. Without a secure foundation, security applications are merely annoyances to an experienced attacker! Collapsed apartment building in Shanghai. Hardware Insecure OS Email Web Text AV Insecure OS makes applications vulnerable.
16. Idea #1: Coprocessor Card • Easy installation • View host memory securely • Canʼt interpose on h/w events • Expensive (~$100 - $400)
17. Idea #2: Virtualization • View host memory securely • Interpose on h/w events • Inexpensive (software only)
18. My Research & Todayʼs Talk 2) Passive monitoring with XenAccess 3) Active monitoring using protected hooks 4) Security applications utilizing monitoring architecture Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 1) Improving hypervisor-provided isolation
19. My Research & Todayʼs Talk 2) Passive monitoring with XenAccess 3) Active monitoring using protected hooks 4) Security applications utilizing monitoring architecture Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 1) Improving hypervisor-provided isolation
20. Secure Monitoring What good is a security check that can be easily circumvented?
21. Hypervisor User VM Security VM User Process XenAccess: Virtual Machine Introspection Security Application Mouse / Keyboard Network Disk User Process User Process OS Kernel Device Drivers BD Payne, M Carbone, W Lee. Secure and Flexible Monitoring of Virtual Machines. Proceedings of the Annual Computer Security Applications Conference, 2007. Virtual Machine Introspection
22. XenAccess Process List in Security VM Process Explorer Output in User VM (Windows XP) Passive Monitoring Example
23. 86 38 75 32 22135 39 31735 34 653 37 79 32 Kernel VA Kernel Sym User VA 20 40 60 80 100 PV￿M PV￿H HVM￿M HVM￿H Time in Microseconds Memory Access Times
24. ￿￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ 0 1000 2000 3000 4000 0 1 2 3 4 5 6 Data size in bytes Time in microseconds ￿ HVM ￿ PV Memory Read Performance
25. XenAccess Today 1700+ downloads Enabling Innovation - University research projects - Defense contractors - US military - Industrial research labs Community Involvement - Open source project - Patches from around the world http://www.xenaccess.org
26. • Hypervisor provides protection and inter-VM comms • Memory protection used for hooks and trampoline, security relies on hypervisor being trusted Turret Architecture BD Payne, M Carbone, W Lee. Lares: An Architecture for Secure Active Monitoring Using Virtualization. Proceedings of the IEEE Symposium on Security and Privacy, 2008. Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk
27. Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Turret Architecture • User VM is where user does regular work • Protected hooks transfer execution to trampoline • Trampoline transfers execution to Security VM, using an inter-VM communication channel
28. Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Turret Architecture • Security VM is where security application runs • Receives event notifications from trampoline through the inter-VM communication channel • Uses introspection to enrich context information
29. (all times are in micro-seconds) Traditional Hook 0 10 20 30 40 Lares Hook 0 10 20 30 40 Performance Comparison
30. • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Libraries, OS and other dependencies Kernel or process execution flow Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook
31. Libraries, OS and other dependencies Kernel or process execution flow Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Prevented by virtualization protections
32. • A1.5 and A1.6 are protected via memory protections • A1.2 and A1.4 also protected via memory protections • A1.3 would require attacker to relocate all of the kernelʼs memory, which is difficult to do without detection • A1.1 protected by CPU IDTR IDT Syscall dispatcher Trampoline SSDT hook GDTR + GDT + Paging structures To Security VM A1.5 A1.4 A1.2 A1.1 A1.3 A1.6 “Bypass Hook” Attacks
33. Libraries, OS and other dependencies Kernel or process execution flow Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks
34. Libraries, OS and other dependencies Kernel or process execution flow Event occurred A1 Event handling finished Security Application Notification Resume/response A2 A4 A3 A5 Hook • Bypass Hook (A1) • Modify event context (A2) • Tamper with security application (A3) • Tamper with dependencies (A4) • Tamper with response (A5) Active Monitoring Attacks Prevented by disabling interrupts in trampoline
35. Application-Level Monitoring Effective monitoring requires finding and viewing the right things.
36. Identifying User Network Traffic
37. Identifying User Network Traffic
38. Identifying User Network Traffic
39. User Application h1 = Click h2 = Key(A) h3 = Key(D) h4 = Click h5 = Key(S) H Hi = {h2, h3, h5} content(Hi) = 'ADS' e1 = HTTP GET e2 = EMAIL 'ADS' e3 = HTTP GET content(e2) = 'ADS' Security Monitor E Using Hardware Events Hi 㱺 e content (Hi) = f (content (e)) [1] [2]
40. Gyrus Framework Hypervisor Security Virtual Machine User VM Network-Based User Application User Kernel User Kernel Transparent Network Redirection Mouse / Keyboard Network Disk Transparent Proxy Enforcement Module Authorization Database User VM Device Model H/W Event 1 2 3 4 5 6 7 Authorization Definition Event Testing — Authorization Creation — Enforcement 1,2 3,4 5,6,7
41. Hardware Event Interposition Memory Access Screen Scraping Supporting API For App Modules
45. Email: Outlook Express User VM Outlook Express Email Client User Kernel comctl32.dll win32k !"#$%&#'()$**+,"-'.'/012*34& !56"&,77*&#*5-' '89:;"<'////1:=/ !"#$%&#'()$**+,"-'.'/0=34:>=2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'////1/C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=/:H53& !56"&,77*&#*5-' '89:;"<'//////:> !"#$%&#'()$**+,"-'.'/0=3A*>5/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'/////&H1' 'DIEJKG !"#$%&#'()$**+,"-'.'/0=/:H5A2 !56"&,77*&#*5-' '89:;"<'/////4:2 !"#$%&#'()$**+,"-'.'/0=/:H5C> !56"&,77*&#*5-' '89:;"<'/////43> !"#$%&#'()$**+,"-'.'/0=3A*>*/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'//////4=' 'DLIEJKG !"#$%&#'()$**+,"-'.'/0=34:452 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'/////=H=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:C1/ !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:H&/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'/////=H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=34:C// !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=34:C2/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////C=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*/1/ !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////24=' 'DLO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B:2 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////4=' 'DLG !"#$%&#'()$**+,"-'.'/0=34:4&2 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'//////H1' 'DJMNG !"#$%&#'()$**+,"-'.'/0=3A*/// !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////=H1' 'DO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B32 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////H1' 'DG !"#$%&#'()$**+,"-'.'/0=3A*>C/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'/////24=' 'DLO?RJG !"#$%&#'()$**+,"-'.'/0=3A*HS2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////2C=' 'DLT)KQ?G !"#$%&#'()$**+,"-'.'/0=34:>A2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'//////H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*>// !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'/////=4=' 'DLP?)RG !"#$%&#'()$**+,"-'.'/0=3A*AS/ !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'//////C1' 'DP?)RG !"#$%&#'()$**+,"-'.'/0=3A*H*2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////4C1' 'DT)KQ?G !"#$%&#'()$**+,"-'.'/0=3A*>32 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////=H1' 'D)M)Q?G !"#$%&#'()$**+,"-'.'/0=3A*>4/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'//////C1' 'DO?RJG !"#$%&#'()$**+,"-'.'/0=3A*>:2 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////>C=' 'DL)M)Q?G !"#$%&#'()$**+,"-'.'/0=34:>5/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=/:H5>& !56"&,77*&#*5-' '89:;"<'//////B2 !"#$%&#'()$**+,"-'.'/0=34:>*/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:HB/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'//////C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=34:CC/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////41' 'D8EF)G mshtml.dll Event Testing Module - Mouse click on email send button Authorization Creation Module - Email from memory & screen - Reconstruct network output Enforcement Module - Validate outgoing emails
46. Email: Outlook Express User VM Outlook Express Email Client User Kernel comctl32.dll win32k !"#$%&#'()$**+,"-'.'/012*34& !56"&,77*&#*5-' '89:;"<'////1:=/ !"#$%&#'()$**+,"-'.'/0=34:>=2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'////1/C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=/:H53& !56"&,77*&#*5-' '89:;"<'//////:> !"#$%&#'()$**+,"-'.'/0=3A*>5/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'/////&H1' 'DIEJKG !"#$%&#'()$**+,"-'.'/0=/:H5A2 !56"&,77*&#*5-' '89:;"<'/////4:2 !"#$%&#'()$**+,"-'.'/0=/:H5C> !56"&,77*&#*5-' '89:;"<'/////43> !"#$%&#'()$**+,"-'.'/0=3A*>*/ !"#$%&#'(?9*@*7#-'.'/0=3A=H2/' '89:;"<'//////4=' 'DLIEJKG !"#$%&#'()$**+,"-'.'/0=34:452 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'/////=H=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:C1/ !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:H&/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'/////=H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=34:C// !"#$%&#'(?9*@*7#-'.'/0=34&::/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=34:C2/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////C=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*/1/ !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////24=' 'DLO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B:2 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////4=' 'DLG !"#$%&#'()$**+,"-'.'/0=34:4&2 !"#$%&#'(?9*@*7#-'.'/0=34&3H/' '89:;"<'//////H1' 'DJMNG !"#$%&#'()$**+,"-'.'/0=3A*/// !"#$%&#'(?9*@*7#-'.'/0=3A=45/' '89:;"<'/////=H1' 'DO)PQG !"#$%&#'()$**+,"-'.'/0=3=&B32 !"#$%&#'(?9*@*7#-'.'/05=4=2' '89:;"<'//////H1' 'DG !"#$%&#'()$**+,"-'.'/0=3A*>C/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'/////24=' 'DLO?RJG !"#$%&#'()$**+,"-'.'/0=3A*HS2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////2C=' 'DLT)KQ?G !"#$%&#'()$**+,"-'.'/0=34:>A2 !"#$%&#'(?9*@*7#-'.'/0=3A5B&/' '89:;"<'//////H=' 'DL8EF)G !"#$%&#'()$**+,"-'.'/0=3A*>// !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'/////=4=' 'DLP?)RG !"#$%&#'()$**+,"-'.'/0=3A*AS/ !"#$%&#'(?9*@*7#-'.'/0=3A===/' '89:;"<'//////C1' 'DP?)RG !"#$%&#'()$**+,"-'.'/0=3A*H*2 !"#$%&#'(?9*@*7#-'.'/0=3A1A/2' '89:;"<'/////4C1' 'DT)KQ?G !"#$%&#'()$**+,"-'.'/0=3A*>32 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////=H1' 'D)M)Q?G !"#$%&#'()$**+,"-'.'/0=3A*>4/ !"#$%&#'(?9*@*7#-'.'/0=3A=4>/' '89:;"<'//////C1' 'DO?RJG !"#$%&#'()$**+,"-'.'/0=3A*>:2 !"#$%&#'(?9*@*7#-'.'/0=3A=HH/' '89:;"<'/////>C=' 'DL)M)Q?G !"#$%&#'()$**+,"-'.'/0=34:>5/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////41' 'DJMNG !"#$%&#'()$**+,"-'.'/0=/:H5>& !56"&,77*&#*5-' '89:;"<'//////B2 !"#$%&#'()$**+,"-'.'/0=34:>*/ !"#$%&#'(?9*@*7#-'.'/0=3A542/' '89:;"<'//////4=' 'DLJMNG !"#$%&#'()$**+,"-'.'/0=34:HB/ !"#$%&#'(?9*@*7#-'.'/0=34&32/' '89:;"<'//////C1' 'D8EF)G !"#$%&#'()$**+,"-'.'/0=34:CC/ !"#$%&#'(?9*@*7#-'.'/0=34&:>/' '89:;"<'//////41' 'D8EF)G mshtml.dll Event Testing Module - Mouse click on email send button Authorization Creation Module - Email from memory & screen - Reconstruct network output Enforcement Module - Validate outgoing emails User Interaction Mean StdDev Click, OE not running 1.8 ms 0.42 ms Click, no OE compose window 27.9 ms 0.99 ms Click in compose edit area 35.6 ms 0.52 ms Click in compose tool bar 167.5 ms 0.71 ms Click on send button 3520.6 ms 160.19 ms
47. Related Work
48. Historical / Systems Security Kernels (1974) Separation Kernels (Rushby 1981) GEMSOS (Gemini) Scomp (Honeywell) PSOS (Neumann et al. 1973) KSOS (Ford) VAX VMM (Karger et al. 1981) MILS (Alves-Foss et al. 2002) MASK (Martin et al. 2000) TCX (Irvine et al. 2003) VAX VMM (Karger et al. 1981) Hardware Trusted Operating System Untrusted Kernel Interfaces App App App Security Kernel - Tamperproof - Complete Mediation - Verifiable Hardware Separation Kernel App App App
49. Security through VMI VMI Concept (Garfinkel and Rosenblum 2003) Intrusion Detection with VMI, all passive Livewire (Garfinkel and Rosenblum 2003) HyperSpector (Kourai et al. 2005) IntroVirt (Joshi et al. 2005) SBCFI (Petroni and Hicks 2007) Memory Forensics & Semantic Knowledge FATKit (Petroni et al. 2006) VIX (Hay and Nance 2008) BackTracker (King et al. 2005) “Digging for Data Structures” (Cozzie et al. 2008) “Robust signatures for kernel data structures” (Dolan-Gavitt et al. 2009)
50. Future Work & Conclusions
51. Platform Security Hardware-rooted security Reducing size of trusted computing base (TCB) Simplify deployment
52. Application Support Hypervisor Hooks User Processes ... Memory Protector Network Traffic Trampoline Hardware Events Hook Events Mouse / Keyboard Network Disk Existing applications New applications for new threats
53. Conclusions / Summary Hypervisor User VM Security VM Hooks User Processes ... Memory Protector Virtual Machine Introspection (XenAccess) Network Traffic Trampoline Hardware Events Hook Events Security Application Mouse / Keyboard Network Disk Mandatory Access Control 4 1 2 3 • Secure and robust monitoring architecture • New security applications • Plans to further improve security, deployability
54. Stopping Attacks on Everyday Operating Systems Using Secure Monitoring Bryan D. Payne School of Computer Science Georgia Institute of Technology
56. Our hypervisor modifications mark specified memory as read-only, forcing a trap on write Memory Protections User page table Virtual Physical User VM Is the PF listed as protected? Mark as read-only PTE propagation NO YES Memory Protection Policy Security VM Shadow page table Virtual Machine Hypervisor
57. Is the write targeted at a protected region? Emulate the write Propagate exception to guest NO YES Page fault due to failed write Technique for byte-level memory protection Memory Protections