Virtual Machine Introspection: From Lab to Reality

938bca9547ba1cac3e69d80efd67fe6b?s=47 Bryan Payne
September 25, 2009

Virtual Machine Introspection: From Lab to Reality

Virtual machine introspection, a technique for observing the runtime state between virtual machines, emerged as a useful security tool in 2003. However, much work had to be done before the technique could be used for building viable applications. This talk looks at many of the advances made related to virtual machine introspection in the past six years including the open source XenAccess Library, techniques for performing runtime memory analysis, and interesting security applications. It concludes with a discussion of the feasibility of deploying an introspection-based security architecture on desktop systems.

938bca9547ba1cac3e69d80efd67fe6b?s=128

Bryan Payne

September 25, 2009
Tweet

Transcript

  1. Virtual Machine Introspection: From Lab To Reality Bryan D. Payne

    / bdpayne@cc.gatech.edu Georgia Tech Information Security Center
  2. Operating System Processes Users

  3. TPM VT-x multi-core Hardware / Firmware

  4. TPM VT-x multi-core Thin Hypervisor

  5. TPM VT-x multi-core Security VM

  6. TPM VT-x multi-core Virtual Machine Introspection (VMI)

  7. TPM VT-x multi-core Ideal Architecture

  8. TPM VT-x multi-core Current Architecture Xen Security VM (Domain 0)

    Desktop or Server VM (User Domain) x86
  9. XenAccess VMI Library VMI Security Pros and Cons Memory Analysis

    Techniques Interesting Applications
  10. TPM VT-x multi-core Xen Security VM (Domain 0) Desktop or

    Server VM (User Domain) x86 XenAccess Monitor App BD Payne, M Carbone, W Lee. Secure and Flexible Monitoring of Virtual Machines. Proceedings of the Annual Computer Security Applications Conference. 2007.
  11. XenAccess Library XenControl Library XenStore Library Security VM Kernel +

    Drivers Xen Hypervisor Monitor App #1 Monitor App #2 Monitor App #3 Memory Analysis Software Architecture
  12. User VM Page Directory Page Table Kernel Data 4 Security

    VM Security Application XenAccess System.map 1 2 6 3 5 Example Usage Scenario
  13. 86 38 75 32 22135 39 31735 34 653 37

    79 32 Kernel VA Kernel Sym User VA 20 40 60 80 100 PV￿M PV￿H HVM￿M HVM￿H Time in Microseconds Memory Access Times
  14. ￿￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿￿

    ￿ ￿ ￿ ￿ ￿ ￿ ￿ ￿ 0 1000 2000 3000 4000 0 1 2 3 4 5 6 Data size in bytes Time in microseconds ￿ HVM ￿ PV Memory Read Performance
  15. XenAccess Impact 1700+ downloads Enabling Innovation - University research projects

    - Defense contractors - US military - Industrial research labs Community Involvement - Open source project - Patches from around the world http://www.xenaccess.org
  16. XenAccess VMI Library VMI Security Pros and Cons Memory Analysis

    Techniques Interesting Applications
  17. Controlled isolation - Security VM sees what it needs -

    User VM is isolated Small attack surface - Limited to hypervisor interface Tamperproof - Depending on assurance level of TCB Verifiable (?) Security Benefits of VMI
  18. Security Concerns with VMI Semantic Gap - Difficult to obtain

    high-level data - Data semantics based on untrusted code Untrusted Data Structures - Page tables, data, etc may be modified Performance
  19. XenAccess VMI Library VMI Security Pros and Cons Memory Analysis

    Techniques Interesting Applications
  20. None
  21. EPROCESS data structure starts here 1. Locate Data

  22. 1. Locate Data 2. Parse Data

  23. 1. Locate Data 2. Parse Data 3. Interpret Data Directory

    Table Base Address Process ID is 1924 Pointer to Next Process Process Name is explorer.exe
  24. 1. Hard code the location 2. Use ad hoc heuristics

    Locating Data In Memory EPROCESS_list_head = 0x82bcbb98 EPROCESS_pid_offset = 0x84 “EPROCESS starts with 0x03001b00...” A Schuster. Searching for processes and threads in Microsoft Windows memory dumps. Proceedings of the Digital Forensics Research Workshop. 2006.
  25. Our Idea Use machine learning to build model of data

    structure Goals Model general enough to work across OS versions Model precise enough to limit false positives Procedure compatible with VMI abstractions Locating Data In Memory Ground Truth Memory Memory Memory Memory Algorithm Data Model
  26. Algorithm Selection struct _EPROCESS{ UChar type; UChar absolute; UChar size;

    UChar inserted; Int4B signal_state; Ptr32 waitlist_flink; Ptr32 waitlist_blink; Ptr32 profilelist_flink; Ptr32 profilelist_blink; Uint4B directory_table_base; Uint2B limit_low; Uint2B base_low; ... }; 03 00 1b 00 00 00 00 00 88 b4 9e 82 88 b4 9e 82 90 b4 9e 82 90 b4 9e 82 e0 01 74 07 7a cf 00 00 ... 2. Hidden States 1. Output Sequence 3. Future states depend only on the present state, not past states
  27. Algorithm Choice: Hidden Markov Model Algorithm Selection struct _EPROCESS{ UChar

    type; UChar absolute; UChar size; UChar inserted; Int4B signal_state; Ptr32 waitlist_flink; Ptr32 waitlist_blink; Ptr32 profilelist_flink; Ptr32 profilelist_blink; Uint4B directory_table_base; Uint2B limit_low; Uint2B base_low; ... }; 03 00 1b 00 00 00 00 00 88 b4 9e 82 88 b4 9e 82 90 b4 9e 82 90 b4 9e 82 e0 01 74 07 7a cf 00 00 ... 2. Hidden States 1. Output Sequence 3. Future states depend only on the present state, not past states
  28. HMM Training struct struct k-means clustering Initial HMM struct struct

    Baum-Welch algorithm Trained HMM • Find parameters of HMM using sample data • k-means clustering sets initial values • Baum-Welch iterates until it converges
  29. Building Datasets 80 02 8F 13 AB 13 CA 13

    OE 10 23 1E 0A 22 Instance #1 0x0 0x8 80 02 8F 13 AB 13 CA 13 OE 10 23 1E 0A 22 0x0 0x8 Instance #2 80 02 8F 13 AB 13 CA 13 OE 10 23 1E 0A 22 0x0 0x8 Instance #3 1. Sliding window “slices” memory into instances 3. If training, label instances 80 02 8F 13 8F 13 AB 13 AB 13 CA 13 N P N 2. If training, get ground truth
  30. Integration with VMI Raw Memory Dump Memory Slicer Data Structure

    Locations Labeled Instances Train HMM HMM Classifier HMM Classification Memory from VMI Memory Slicer Post Processing Data Location DB Evaluation Training • Use classifier to find locations at runtime • Store results in local database cache • Invalidate or flush cache as locations change
  31. HMM Classification 1 10 100 1000 10000 100000 0 0.2

    0.4 0.6 0.8 1 • HMM provides probability for each instance • Determine threshold value for classification
  32. HMM Classification 1 10 100 1000 10000 100000 0 0.2

    0.4 0.6 0.8 1 • HMM provides probability for each instance • Determine threshold value for classification EPROCESS structs random data
  33. Preliminary Results HMM Classification Accuracy False Positive False Negative EPROCESS

    97.891% 2.112% 0.000% ETHREAD 98.666% 1.337% 0.068% FILE_OBJECT 96.606% 3.401% 0.000% Training datasets - Windows 2000 SP4 - Windows XP SP2 Evaluation datasets - Windows 2000 SP0 - Windows 2000 SP4 - Windows XP SP2
  34. Future Plans More Datasets - Every version and service pack

    of Windows - Several Linux kernel versions More Data Structures - Test many data structures to ensure generality - Various sizes and data types within structures Test System Limits - How data structure size affects accuracy - Performance with VMI integration
  35. XenAccess VMI Library VMI Security Pros and Cons Memory Analysis

    Techniques Interesting Applications
  36. XenAccess Process List Example Output in dom0 Process Explorer Output

    inside Windows XP
  37. XenAccess Process List Example Output in dom0 Process Explorer Output

    inside Windows XP
  38. Physical Memory XenAccess Introspection Library PyXa Interface FUSE Interface Memoryze

    Volatility Other file- based tools Runtime Analysis Using Virtual Machine Introspection Physical Memory Dump Live Access Snapshot File On Disk Other file- based tools Memoryze Volatility Forensic Memory Analysis Forensic Memory Analysis Tools • Forensic tools can be used with introspection • Helps bridge semantic gap for some OSes • Provides platform for rapid tool development
  39. John The Ripper (password cracker) Volatility PyXa XenAccess + +

    Passwords
  40. None
  41. None
  42. None
  43. Malware Detection Kernel Hook Detection User Hook Detection Extract keys

    and passphrases View Windows registry List kernel drivers View process code and data Kill a process Firewall network traffic Other Applications
  44. TPM VT-x multi-core Desktop Deployment

  45. Conclusions VMI is powerful and useful for many applications Foundation

    exists to support continued research Exciting opportunities for new research - memory analysis - desktop deployment - integration with h/w security - new applications
  46. Virtual Machine Introspection: From Lab To Reality Bryan D. Payne

    / bdpayne@cc.gatech.edu Georgia Tech Information Security Center