Upgrade to Pro — share decks privately, control downloads, hide ads and more …

An Introduction to Virtual Machine Introspection Using LibVMI

Bryan Payne
December 09, 2014

An Introduction to Virtual Machine Introspection Using LibVMI

MMF Workshop at ACSAC 2014, December 2014. This talk will provide a brief introduction to LibVMI. LibVMI provides an API for performing memory analysis tasks on running virtual machines. We will explore how LibVMI provides a common API for interfacing with Linux and Windows VMs running on both Xen and KVM/Qemu. And we will look at the abstractions available ranging from memory access based on physical addresses, virtual addresses, or kernel symbols to event driven runtime memory analysis. Finally, we will discuss different ways of using LibVMI including as a C library, a Python library, and as a Volatility address space plugin. Along the way we will discuss how LibVMI has worked to address the performance and semantic gap concerns that often arise when programming with virtual machine introspection. LibVMI is actively maintained, and freely available on GitHub under the LGPL license (https://github.com/libvmi/libvmi). http://www.acsac.org/2014/workshops/mmf/

Bryan Payne

December 09, 2014
Tweet

More Decks by Bryan Payne

Other Decks in Technology

Transcript

  1. © 2014 Nebula, Inc. All rights reserved.
    (cloud) Computing for the Enterprise
    An Introduction to Virtual Machine
    Introspection Using LibVMI
    Bryan  D.  Payne  
    [email protected]

    View Slide

  2. © 2014 Nebula, Inc. All rights reserved.
    Virtual Machine Introspection
    Memory  Analysis  (at  runtime)  
    Events  
    -­‐ Register  changes  
    -­‐ Memory  read  /  write  /  execute  
    -­‐ Memory  mapped  I/O  
    -­‐ Exceptions  (e.g.,  page  faults)

    View Slide

  3. © 2014 Nebula, Inc. All rights reserved.
    Use Cases
    Forensics  
    System-­‐level  debugging  and  analysis  
    Runtime  security  
    Timeline  or  trend  analysis  
    Debugging  
    Other  ideas?

    View Slide

  4. © 2014 Nebula, Inc. All rights reserved.
    Hard Problems
    Semantic  Gap  
    Performance  
    Platform  Support

    View Slide

  5. © 2014 Nebula, Inc. All rights reserved.
    LibVMI
    Goal:  Make  VMI  more  accessible  to  programmers  
    -­‐ Backends:  KVM/QEMU,  Xen,  Raw  snapshot  files  
    -­‐ Operating  Systems:  Linux,  Windows  
    -­‐ Architectures:  x86  (32-­‐bit,  PAE,  64-­‐bit),  ARM  
    -­‐ API:  read/write  memory,  memory  events,  
    translations,  cache  management
    LibVMI
    (C language API)
    KVM Xen Other VMM
    Memory
    Snapshot
    patch

    View Slide

  6. © 2014 Nebula, Inc. All rights reserved.
    LIBVMI INTERNALS

    View Slide

  7. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Internals
    Initialization  of  LibVMI  
    Runtime  guest  introspection  
    -­‐ Memory  access  (read  /  write)  
    -­‐ Register  access  
    -­‐ Memory  events  
    -­‐ Address  translation  
    -­‐ Symbol  resolution  
    -­‐ Cache  management  
    Hypervisor-­‐level  support  

    View Slide

  8. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)

    View Slide

  9. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)
    From
    VMM
    and/or OS

    View Slide

  10. © 2014 Nebula, Inc. All rights reserved.
    Config file: libvmi.conf
    winxpsp2 {
    ostype = "Windows";
    win_tasks = 0x88;
    win_pdbase = 0x18;
    win_pid = 0x84;
    win_kdvb = 0x80544ce0;
    }
    win7sp1x64 {
    ostype = "Windows";
    win_tasks = 0x188;
    win_pdbase = 0x28;
    win_pid = 0x180;
    win_kdvb = 0xfffff800027f10a0;
    }

    View Slide

  11. © 2014 Nebula, Inc. All rights reserved.
    Finding the Kernel
    1.
    0x0
    0x5A4D 0x00004550
    0x4d7000
    ...
    MZ Header Image NT Sig
    ntoskrnl.exe
    Export Table Name
    base+0x3c
    Scan up from physical address 0x0
    2.
    3.

    View Slide

  12. © 2014 Nebula, Inc. All rights reserved.
    Finding the Kernel
    1.
    0x0
    0x5A4D 0x00004550
    0x4d7000
    ...
    MZ Header Image NT Sig
    ntoskrnl.exe
    Export Table Name
    base+0x3c
    Scan up from physical address 0x0
    0x5A4D 0x00004550
    0x4d7000
    MZ Header Image NT Sig
    ntoskrnl.exe
    Export Table Name
    base+0x3c
    Scan down from virtual address in IDTR
    IDTR
    2.
    3.

    View Slide

  13. © 2014 Nebula, Inc. All rights reserved.
    Finding the Kernel
    1.
    0x0
    0x5A4D 0x00004550
    0x4d7000
    ...
    MZ Header Image NT Sig
    ntoskrnl.exe
    Export Table Name
    base+0x3c
    Scan up from physical address 0x0
    0x5A4D 0x00004550
    0x4d7000
    MZ Header Image NT Sig
    ntoskrnl.exe
    Export Table Name
    base+0x3c
    Scan down from virtual address in IDTR
    IDTR
    2.
    3. _KDDEBUGGER_DATA64[KernBase]

    View Slide

  14. © 2014 Nebula, Inc. All rights reserved.
    Where Is _KDDEBUGGER_DATA64?
    May  take  longer  than  just  finding  kernel  directly  
    Symbol  access  makes  it  all  worthwhile  
    -­‐ KernBase  
    -­‐ PsLoadedModuleList  
    -­‐ 125+  symbols
    0x0
    "\x00\xf8\xFF\xFFKDBG"
    0x??????
    ...
    KDBG Signature
    Scan up from physical address 0x0

    View Slide

  15. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)

    View Slide

  16. © 2014 Nebula, Inc. All rights reserved.
    Symbol Resolution
    _KDDEBUGGER_DATA64  
    Kernel  PE  Export  Table  
    -­‐ Start  with  kernel  base  location  
    -­‐ Parse  kernel  PE  header  
    -­‐ RVA  to  export  table  in  optional  head  data  dir

    View Slide

  17. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)

    View Slide

  18. © 2014 Nebula, Inc. All rights reserved.
    Page Directory
    CR3,  or  Search  for  “System”  EPROCESS  struct
    0x0
    0x1b0003 [dtb vaddr]
    base
    ...
    Header
    System
    base+0x18
    Scan up from physical address 0x0
    DirectoryTableBase ImageFileName
    base+0x174
    nt!_EPROCESS
    +0x000 Pcb : _KPROCESS
    +0x000 Header : _DISPATCHER_HEADER
    +0x010 ProfileListHead : _LIST_ENTRY
    +0x018 DirectoryTableBase : [2] Uint4B
    +0x020 LdtDescriptor : _KGDTENTRY
    +0x028 Int21Descriptor : _KIDTENTRY
    +0x030 IopmOffset : Uint2B
    ...
    +0x000 Count : Uint4B
    +0x000 Ptr : Ptr32 Void
    +0x084 UniqueProcessId : Ptr32 Void
    +0x088 ActiveProcessLinks : _LIST_ENTRY
    +0x000 Flink : Ptr32 _LIST_ENTRY
    +0x004 Blink : Ptr32 _LIST_ENTRY
    ...

    View Slide

  19. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)

    View Slide

  20. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)
    Using symbol from
    _KDDEBUGGER_DATA64
    (PsActiveProcessHead)

    View Slide

  21. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Initialization
    Find  VM  (Xen,  KVM,  etc)  
    Read  config  file  
    Memory  layout  and  size  
    Find  kernel  base  location  
    Init  symbol  resolution  
    (exports,  debug  info,  etc)  
    Find  page  directory  
    location  (CR3  /  kpgd)  
    Find  kernel  process  list
    vmi_init(…)
    vmi_init_complete(…)

    View Slide

  22. © 2014 Nebula, Inc. All rights reserved.
    LibVMI Runtime
    Read  /  Write  Functions  
    -­‐ Starting  from  Kernel  Symbol,  Vaddr,  or  Paddr  
    -­‐ Specify  length  to  read  
    -­‐ Read  a  string  (ASCII  or  UNICODE)  
    Address  Translation  Functions  
    -­‐ Kernel  or  User  Vaddr  to  Paddr  
    -­‐ Kernel  symbol  to  Vaddr  
    Convenience  Functions  
    -­‐ Pause  /  Resume,  Memory  size,  CPU  Registers  
    -­‐ LibVMI  cache  manipulation

    View Slide

  23. © 2014 Nebula, Inc. All rights reserved.
    Read Example (vmi_read_ksym)
    resolve  symbol translate  to  paddr read  from  VMM
    handle  page  wraps

    View Slide

  24. © 2014 Nebula, Inc. All rights reserved.
    Page-level Cache
    Page Cache Hash Table
    Hash = Paddr
    Hash-1
    Hash-2
    Hash-3
    Hash-n
    ...
    Handle/Buf-A
    Handle/Buf-B
    Handle/Buf-C
    Handle/Buf-n
    Memory
    Request
    Handle or
    Buffer
    Read
    Memory Buffer
    In Cache?
    Yes
    No
    VMI Application
    Hypervisor / VMM
    LibVMI
    Page Cache LRU List
    if (lru is full)
    remove 1/2 most stale
    Fresh
    Stale
    ...
    Notify

    View Slide

  25. © 2014 Nebula, Inc. All rights reserved.
    Virtual To Physical Cache
    Paddr
    Translate
    Vaddr Paddr
    Hash-1
    Hash-2
    Hash-3
    Hash-n
    ...
    Handle/Buf-A
    Handle/Buf-B
    Handle/Buf-C
    Handle/Buf-n
    In Cache?
    V2P Cache
    Yes
    No
    VMI Application
    Hypervisor / VMM
    LibVMI
    Hash = CityHash(va << 64 | cr3)
    Walk Guest
    Page Tables
    Memory
    Reads
    Valid?
    Yes
    No

    View Slide

  26. © 2014 Nebula, Inc. All rights reserved.
    Cache Summary
    Page-­‐level  data  
    Virtual  address  to  Physical  address  
    Process  ID  to  Directory  Table  Base  
    Kernel  Symbol  to  Virtual  address

    View Slide

  27. © 2014 Nebula, Inc. All rights reserved.
    Cache Performance
    No Cache
    Page Only
    Addr Only
    All Cache
    time in microseconds
    1 10 100 1000 10000
    6
    6
    6
    50
    6
    6
    123
    1331
    vmi_translate_ksym2v vmi_translate_kv2p
    System  configuration:  Xen  4.1.1,  Dual  Intel  Xeon  X5675,  24G  RAM,  Windows  XP  VM  
    Times  shown  are  for  cache  hits,  when  possible

    View Slide

  28. © 2014 Nebula, Inc. All rights reserved.
    Cache Performance
    System  configuration:  Xen  4.1.1,  Dual  Intel  Xeon  X5675,  24G  RAM,  Windows  XP  VM  
    Times  shown  are  for  cache  hits,  when  possible
    No Cache
    Page Only
    Addr Only
    All Cache
    time in microseconds
    1 10 100
    4
    33
    5
    28
    5
    48
    6
    42
    vmi_read_pa (1875 x 4 bytes)
    vmi_read_pa (1 x 7.5k bytes)

    View Slide

  29. © 2014 Nebula, Inc. All rights reserved.
    ADDITIONAL FEATURES

    View Slide

  30. © 2014 Nebula, Inc. All rights reserved.
    Events (Xen)
    Pause  guest  and  transfer  control  to  callback  
    function  in  your  application  
    Memory  r/w/x  events  on  defined  regions  
    Register  r/w  events  on  CR0/CR3/CR4/MSR  regs  
    Interrupt  events  
    Single  step  through  instructions

    View Slide

  31. © 2014 Nebula, Inc. All rights reserved.
    Shared Memory Snapshots (KVM)
    Requires  custom  patch  for  Qemu-­‐KVM  
    Transparently  creates  a  guest  snapshot  
    Guest  continues  running  
    VMI  app  gets  direct  memory  access  
    VMI  app  can  refresh  snapshot  at  will

    View Slide

  32. © 2014 Nebula, Inc. All rights reserved.
    Rekall Profiles (Windows)
    Use  Rekall  tool  to  generate  Windows  profiles  
    Profiles  replace  the  need  to  provide  offsets  in  
    the  libvmi.conf  file  
    Especially  useful  for  Windows  8,  where  KDBG  is  
    typically  not  accessible

    View Slide

  33. © 2014 Nebula, Inc. All rights reserved.
    USING LIBVMI

    View Slide

  34. © 2014 Nebula, Inc. All rights reserved.

    View Slide

  35. © 2014 Nebula, Inc. All rights reserved.
    LibVMI
    (C language API)
    pyvmi
    (Python language wrapper for LibVMI)
    KVM Xen Other VMM
    Memory
    Snapshot
    patch

    View Slide

  36. © 2014 Nebula, Inc. All rights reserved.

    View Slide

  37. © 2014 Nebula, Inc. All rights reserved.
    LibVMI
    (C language API)
    pyvmi
    (Python language wrapper for LibVMI)
    Volatility
    (memory analysis framework)
    pyvmi address
    space plugin
    plugin
    plugin
    plugin
    plugin
    plugin
    Runtime analysis
    capabilities augment
    Volatility's rich
    memory analysis.
    ...
    KVM Xen Other VMM
    Memory
    Snapshot
    patch

    View Slide

  38. © 2014 Nebula, Inc. All rights reserved.
    Development  
    https://github.com/libvmi/libvmi  
    Discussion  
    https://groups.google.com/d/forum/vmitools

    View Slide

  39. © 2014 Nebula, Inc. All rights reserved.
    (cloud) Computing for the Enterprise
    An Introduction to Virtual Machine
    Introspection Using LibVMI
    Bryan  D.  Payne  
    [email protected]

    View Slide