Secure and Flexible Monitoring of Virtual Machines Bryan D. Payne - Martim D. P. de A. Carbone - Wenke Lee School of Computer Science Georgia Institute of Technology 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Protecting Software Monitors Kernel User User OS Kernel User Kernel User User VM Monitoring VM Virtual Machine Monitor Moving monitors to a different virtual machine provides protection through isolation and reduced dependency on untrusted software. 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Related Research Intrusion Detection Systems Livewire, Garfinkel et al, NDSS 2003 Hyperspector, Kourai et al, VEE 2005 IntroVirt, Joshi et al, SOSP 2005 Workload Isolation NetTop, Meushaw et al, US DoD research Terra, Garfinkel et al, SOSP 2003 Forensics & Debugging ReVirt, Dunlap et al, OSDI 2002 System Monitoring Antfarm, Jones et al, USENIX 2006 Honeypot Monitoring, Asrigo et al, VEE 2006 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions Previous research focused on the applications, but creating a secure and flexible monitoring architecture is non-trivial • Six requirements for secure and flexible monitoring using introspection in a virtualized environment • The XenAccess architecture that shows these requirements are obtainable • Our implementation of XenAccess, which is freely available for other researchers to use 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions & Related Research Xen Background Information Architecture & Implementation Example Applications Performance Analysis Conclusions 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Background: Xen VMM • One control domain + Multiple user domains • Xen provides m2p and p2m tables to convert between physical and machine addresses Kernel User PV Domain (User VM) Kernel User Domain 0 (Monitoring VM) Kernel User HVM Domain (User VM) ... XenoLinux XenoLinux Windows or Linux Machine address Physical address Virtual address (kernel) Virtual address (user) Front-End Driver Back-End Driver Device Driver Xen Virtual Machine Monitor I/O Device 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Xen Blktap I/O Model • Split driver model sends I/O through domain 0 • Blktap provides user space access to block level read and write information to/from disk 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions & Related Research Xen Background Information Architecture & Implementation Example Applications Performance Analysis Conclusions 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Six criteria for a useful monitoring architecture: 1. No superfluous modifications to the VMM 2. No modifications to the VM or the target OS 3. Small performance impact 4. Rapid development of new monitors 5. Ability to monitor any data on target OS 6. Target OS cannot tamper with monitors Monitoring Criteria 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 XenAccess Architecture Xen VMM Monitor VM User VM User Kernel User Kernel XenAccess Blktap BE Driver FE Driver Disk Driver XenCtrl XenAccess Library XenControl Library BlkTap Arch XenStore Library Domain 0 Kernel + Drivers Xen VMM Monitor App #1 Monitor App #2 Monitor App #3 BlkTap 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Memory Introspection xa_init(), xa_destroy() xa_access_kernel_symbol() xa_access_virtual_address() xa_access_user_virtual_address() 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Memory Introspection Details • HVM Support - Different m2p mapping; otherwise same as PV - XenAccess works with both PV and HVM - Recent versions of XenAccess support Windows • Improving Performance - Walking another domain’s page table is costly - LRU cache provides acceptable performance - Frequent cache hits expected in most apps • Bridging the Semantic Gap 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Disk Monitoring xadisk_init(), xadisk_destroy() xadisk_set_watch() xadisk_unset_watch() xadisk_activate() xadisk_deactivate() 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Disk Monitoring Details • Asynchronous operation - Disk events queued for processing by monitor - Data sent to disk device driver • Inference Engine - Changes inferred; current versus previous state - Requires knowledge of filesystem • Limitations - Only file/direction creation/deletion - File read/write, object renaming, file truncating, and more possible using this architecture 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions & Related Research Xen Background Information Architecture & Implementation Example Applications Performance Analysis Conclusions 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Memory Monitor Example XenAccess Process List Example Output in dom0 Process Explorer Output inside Windows XP 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Memory Monitor Example XenAccess Process List Example Output in dom0 Process Explorer Output inside Windows XP 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Disk Monitor Example XenAccess Disk Monitor Example Output in dom0 Disk Activity Generated in User VM 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Disk Monitor Example XenAccess Disk Monitor Example Output in dom0 Disk Activity Generated in User VM 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions & Related Research Xen Background Information Architecture & Implementation Example Applications Performance Analysis Conclusions 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Memory Monitor Performance 0 10 20 30 40 50 60 70 80 90 Virtual Address Kernel Symbol User Address Time in microseconds 21213 6881 541 PV-M PV-H HVM-M HVM-H 0 1 2 3 4 5 4000 3000 2000 1000 500 100 Time in microseconds Data size in bytes fc6-pv fc6-hvm Time To Access Memory • Acceptable for most apps • Caching very useful Time To Read Memory • Very small, page is mapped into local address space • Similar for PV and HVM 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Disk Monitor Performance 0 500 1000 1500 2000 10 50 100 200 500 1000 2000 Time in milliseconds Number of Files Created Mode 1 Mode 2 Mode 3 Mode 1: Monitoring with watch points enabled Mode 2: Monitoring with watch points disabled Mode 3: Tapdisk driver with no monitoring 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Our Contributions & Related Research Xen Background Information Architecture & Implementation Example Applications Performance Analysis Conclusions 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 No superfluous modifications to the VMM No modifications to the VM or the target OS Small performance impact Rapid development of new monitors Ability to monitor any data on target OS Target OS cannot tamper with monitors Monitoring Criteria Revisited 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 • Code discussed in the paper available online • Memory introspection has more features - Support for Windows VMs - Improved support for HVM domains - Improved cache utilization - Expanded API with better documentation • Disk introspection available as proof of concept • Available at http://xenaccess.sourceforge.net XenAccess Today 

Secure and Flexible Monitoring of Virtual Machines, ACSAC 2007 Summary • Prior work focused on the applications of introspection; not how to implement it • We proposed six requirements for secure and flexible monitoring using introspection • Our XenAccess architecture demonstrated that these requirements are obtainable • Our implementation of XenAccess is freely available for other researchers to use 

Questions? http://xenaccess.sourceforge.net