Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Windows Kernel Heap: Segment heap in windows kernel Part 1

Angelboy
December 07, 2020

Windows Kernel Heap: Segment heap in windows kernel Part 1

Windows Kernel Heap: Segment heap in windows kernel Part 1
Segment heap internals

Angelboy

December 07, 2020
Tweet

Other Decks in Research

Transcript

  1. Windows Kernel Heap
    Part 1: Segment heap in windows kernel
    [email protected] @scwuaptx
    Angelboy

    View Slide

  2. • Angelboy

    • Researcher at DEVCORE

    • CTF Player

    • HITCON / 217

    • Chroot

    • Co-founder of pwnable.tw

    • Speaker

    • HITB GSEC 2018/AVTokyo 2018/VXCON
    Whoami

    View Slide

  3. Kernel Pool
    • Non Paged Pool

    • Memory that will not be page out

    • Paged Pool

    • Memory that will be page out

    • Session Paged Pool

    • Memory that will be page out

    • Each session is independent

    View Slide

  4. Memory Allocator in kernel
    • ExAllocatePool/ExAllocatePoolWithTag

    • ExFreePool/ExFreePoolWithTag

    • Nt Heap

    • RtlpAllocateHeap

    • RtlpFreeHeap

    View Slide

  5. Memory Allocator in kernel
    • ExAllocatePoolWithTag/ExFreePoolWithTag

    • Before 19H1 (1903)

    • Kernel Pool Allocator

    • From 19H1 ~ Now

    • Segment heap

    • Note

    • Segment heap in kernel is very similar to segment heap in userland

    • Some size and config are difference

    • We will focus on kernel in this slide

    • Windows 10 20H2

    View Slide

  6. Memory Allocator in kernel
    • Segment Heap Overview

    • Frontend Allocation

    • Low FragmentationHeap

    • Variable Size Allocation

    • Backend Allocation

    • Segment Allocation

    • Large Block Allocation

    View Slide

  7. Memory Allocator in kernel
    • Segment Heap Overview
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator

    View Slide

  8. Memory Allocator in kernel
    • Segment Heap Overview
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator
    Enable LFH and Size < 0x200

    View Slide

  9. Memory Allocator in kernel
    • Segment Heap Overview
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator
    Size <= 0xfe0

    (0xfe0 < Size <= 0x20000
    and
    (size & 0xfff) is not zero )

    View Slide

  10. Memory Allocator in kernel
    • Segment Heap Overview
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator
    0x20000 < Size
    < SegContexts[1].MaxAllocationSize

    (0x7f0000)

    View Slide

  11. Memory Allocator in kernel
    • Segment Heap Overview
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator
    Size > 0x7f0000

    View Slide

  12. Memory Allocator in kernel
    • If FrontEnd's memory is

    insufficient,

    it will be allocated

    from BackEnd to

    FrontEnd first
    ntoskrnl.exe
    ExAllocatePool
    WithTag
    ntoskrnl.exe
    ExAllocateHeap
    Pool
    LFH
    RtlpHpLfh
    ContextAllocate
    VS Allocation
    RtlpHpVsContext
    AllocateInternal
    Segment Allocation
    RtlpHpSegAlloc
    Block Allocation
    RtlpHpLarge
    Alloc
    FrontEnd allocator
    Backend Allocator

    View Slide

  13. Memory Allocator in kernel
    • Important structure

    • _EX_POOL_HEAP_MANAGER_STATE

    • _SEGMENT_HEAP

    • _RTLP_HP_HEAP_GLOBALS

    View Slide

  14. Memory Allocator in kernel
    • nt!ExPoolState (_EX_POOL_HEAP_MANAGER_STATE)

    • The core structure in the Kernel pool memory allocator

    • Whether it is Page Pool or Non page pool, etc. are managed by this
    structure

    View Slide

  15. Memory Allocator in kernel
    • ExPoolState
    (_EX_POOL_HEAP_MANAGER_STATE)

    • HeapManager
    (_RTLP_HP_HEAP_MANAGER )

    • Store some Global variables, some
    Metadata, etc.

    • NumberOfPool

    • Number of Pool Node

    • Default 1
    NumberOfPool (8bytes)
    HeapManager (0x38d0bytes)
    PoolNode[64] (0x20c0*64)

    0x0
    0x38d0
    0x3900
    0x86900

    View Slide

  16. Memory Allocator in kernel
    • ExPoolState
    (_EX_POOL_HEAP_MANAGER_STATE)

    • PoolNode (_EX_HEAP_POOL_NODE)

    • Each node will have four Heaps
    corresponding to different
    segment heaps, such as Paged/
    Nonpaged pool, etc.
    NumberOfPool (8bytes)
    HeapManager (0x38d0bytes)
    PoolNode[64] (0x20c0*64)

    0x0
    0x38d0
    0x3900
    0x86900

    View Slide

  17. Memory Allocator in kernel
    • _SEGMENT_HEAP

    • The core structure in the segment heap

    • Each Pool will have a _SEGMENT_HEAP structure. When allocating
    memory, it will decide which HEAP to allocate according to the pool type.

    • In ExPoolState

    • These are four segment heap will be created and initialized when the
    system is turned on

    • ExPoolState.PoolNode[0].Heap[0-4]

    View Slide

  18. Memory Allocator in kernel
    • _SEGMENT_HEAP

    • ExPoolState.PoolNode[0].Heap[0] -> NonPagedPool

    • ExPoolState.PoolNode[0].Heap[1] -> NonPagedPoolNx

    • ExPoolState.PoolNode[0].Heap[2] -> PagedPool

    View Slide

  19. Memory Allocator in kernel
    • _SEGMENT_HEAP

    • EnvHandle (RTL_HP_ENV_HANDLE)

    • environment handle of segment heap

    • Signature

    • Signature of segment heap

    • It always 0xddeeddee

    • AllocatedBase

    • Point to the end of the entire structure of
    _SEGMENT_HEAP, used to allocate the structure
    required by LFH Allocator, after allocation, it will point
    to the end of the allocated structure


    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP
    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)
    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    0x0 EnvHandle (10bytes)

    UserContext (8bytes)


    0x28

    View Slide

  20. Memory Allocator in kernel
    • _SEGMENT_HEAP

    • SegContexts (_HEAP_SEG_CONTEXT)

    • The core structure of the back-end
    manager

    • Divided into two segcontexts
    according to size

    • 0x20000 < Size <= 0x7f000

    • 0x7f000 < Size <= 0x7f0000


    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP
    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)
    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    0x0 EnvHandle (10bytes)

    UserContext (8bytes)


    0x28

    View Slide

  21. Memory Allocator in kernel
    • _SEGMENT_HEAP

    • VsContext (_HEAP_VS_CONTEXT)

    • The core structure of front-end VS
    Allocator

    • LfhContext (_HEAP_LFH_CONTEXT)

    • The core structure of the front-end
    LowFragmentationHeap Allocator


    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP
    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)
    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    0x0 EnvHandle (10bytes)

    UserContext (8bytes)


    0x28

    View Slide

  22. Memory Allocator in kernel
    • nt!RtlpHpHeapGlobals (_RTLP_HP_HEAP_GLOBALS)

    • In Segment Heap, many fields, values, and functions are mostly
    encoded

    • This structure will be used to store related key and other information

    View Slide

  23. Memory Allocator in kernel
    • RtlpHpHeapGlobals
    (_RTLP_HP_HEAP_GLOBALS)

    • HeapKey

    • Used for VS Allocator, Segment
    Allocator

    • LfhKey

    • Used for LowFragmentationHeap

    • Both are 8 byte random value
    HeapKey (8bytes)
    LfhKey (8bytes)
    _RTLP_HP_HEAP_GLOBALS

    0x0
    0x8

    View Slide

  24. Memory Allocator in kernel
    • Segment Heap

    • Frontend Allocation

    • Low FragmentationHeap

    • Variable Size Allocation

    • Backend Allocation

    • Segment Allocation

    • Large Block Allocation

    View Slide

  25. Low FragmentationHeap
    • Size

    • In the case of default

    • Size <= 0x200 and LFH is enabled for this size

    • The activation timing is very similar to that of NtHeap. It will be
    activated after 18 consecutive allocations of the same size block

    View Slide

  26. Low FragmentationHeap
    • Data Structure

    • Memory allocation mechanism

    View Slide

  27. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • The core structure of the LFH Allocator

    • Used to manage the memory allocated by LFH, record all the
    information and structure of LFH in the heap

    View Slide

  28. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • BackendCtx
    (_HEAP_SEG_CONTEXT)

    • Point to the Backend allocator
    used by the LFH

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[129]

    View Slide

  29. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Callbacks
    (_HEAP_SUBALLOCATOR_CALLBACKS)

    • Call back function table (use to allocate/
    release subsegments)

    • Allocate

    • Free

    • Commit

    • Decommit

    • ExtendContext

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[129]

    View Slide

  30. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Callbacks
    (_HEAP_SUBALLOCATOR_CALLBACKS)

    • Function pointer 會被 encode 過

    • The value of callbacks will be xored
    by

    • RtlpHpHeapGlobals.HeapKey

    • Address of LfhContext

    • Function pointer

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[129]

    View Slide

  31. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Config (_RTL_HP_LFH_CONFIG)

    • Used to indicate the attributes of
    the LFH Allocator

    • It will be used to determine
    whether the size of allocation is
    within the scope of LFH allocator

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[129]

    View Slide

  32. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Config (_RTL_HP_LFH_CONFIG)

    • MaxBlockSize

    • The size of the max block in LFH

    • WitholdPageCrossingBlocks

    • Are there any cross-page blocks

    • DisableRandomization

    • Whether to disable randomization of
    LFH
    MaxBlockSize (2bytes)
    WitholdPageCrossingBlocks (1bit)
    _RTL_HP_LFH_CONFIG
    0x0
    0x2
    DisableRandomization (1bit)
    0x2

    View Slide

  33. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Buckets (_HEAP_LFH_BUCKET)

    • Bucket array, each buckets
    corresponds to blocks in a
    specific size range

    • When LFH is enabled, it will
    point to the
    _HEAP_LFH_BUCKET structure

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[129]

    View Slide

  34. Low FragmentationHeap
    • LfhContext (_HEAP_LFH_CONTEXT)

    • Buckets (_HEAP_LFH_BUCKET)

    • If it is not enabled, it will be used to
    indicate the number of times the
    block of the size is allocated.

    • The last 2 bytes are always 1

    • The next 2 byte is the number of
    allocations

    • Each allocation will add 0x21

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[0]

    0x0021 0x0001
    Buckets[0]

    View Slide

  35. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • The bucket and its related structure will be allocated only when the LFH
    is enabled

    • LFH Allocator is use the structure to manage the block corresponding
    to the Size

    View Slide

  36. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • State
    (_HEAP_LFH_SUBSEGMENT_OW
    NER)

    • Used to indicate the status of the
    Buckets

    • This structure is used to manage
    the memory pool of LFH

    State (0x38bytes)
    TotalBlockCount (8bytes)
    _HEAP_LFH_BUCKET
    0x0
    0x38
    0x40 TotalSubsegmentCount (8bytes)
    ReciprocalBlockSize (8bytes)

    AffinitySlots[]
    0x48
    0x60

    View Slide

  37. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • State
    (_HEAP_LFH_SUBSEGMENT_OW
    NER)

    • BucketIndex

    • The index of the bucket
    IsBuckets (1bit)
    _HEAP_LFH_SUBSEGMENT_OWNER
    0x0
    0x1 BucketIndex (1byte)

    0x8
    0x18

    AvailableSubsegmentCount (8B)
    AvailableSubsegmentList (8B)
    FullSubsegmentList (8bytes)
    0x20

    View Slide

  38. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • State
    (_HEAP_LFH_SUBSEGMENT_OW
    NER)

    • AvailableSubsegmentCount

    • Number of available
    subsegments
    IsBuckets (1bit)
    _HEAP_LFH_SUBSEGMENT_OWNER
    0x0
    0x1 BucketIndex (1byte)

    0x8
    0x18

    AvailableSubsegmentCount (8B)
    AvailableSubsegmentList (8B)
    FullSubsegmentList (8bytes)
    0x20

    View Slide

  39. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • State
    (_HEAP_LFH_SUBSEGMENT_OWNER)

    • AvailableSubsegmentList
    (_LIST_ENTRY)

    • Point to the next available
    subsegment in the bucket

    • FullSubsegmentList (_LIST_ENTRY)

    • Point to the next used up
    subsegment in the bucket
    IsBuckets (1bit)
    _HEAP_LFH_SUBSEGMENT_OWNER
    0x0
    0x1 BucketIndex (1byte)

    0x8
    0x18

    AvailableSubsegmentCount (8B)
    AvailableSubsegmentList (8B)
    FullSubsegmentList (8bytes)
    0x20

    View Slide

  40. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • AffinitySlots
    (_HEAP_LFH_AFFINITY_SLOT)

    • The main structure used to
    manage the memory pool using
    by LFH

    • There is only one by default

    State (0x38bytes)
    TotalBlockCount (8bytes)
    _HEAP_LFH_BUCKET
    0x0
    0x38
    0x40 TotalSubsegmentCount (8bytes)
    ReciprocalBlockSize (8bytes)

    *AffinitySlots[]
    0x48
    0x60

    View Slide

  41. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • AffinitySlots
    (_HEAP_LFH_AFFINITY_SLOT)

    • State
    (_HEAP_LFH_SUBSEGMENT_O
    WNER)

    • Same structure as State in
    Bucket, but this one is mainly
    used to manage subsegment
    State (0x38bytes)
    ActiveSubsegment (8bytes)
    _HEAP_LFH_AFFINITY_SLOT
    0x0
    0x38

    View Slide

  42. Low FragmentationHeap
    • Buckets (_HEAP_LFH_BUCKET)

    • AffinitySlots
    (_HEAP_LFH_AFFINITY_SLOT)

    • ActiveSubsegment
    (_HEAP_LFH_FAST_REF)

    • Point to the subsegment being
    used

    • The lowest 12 bits indicate how
    many blocks are available in the
    subsegment
    State (0x38bytes)
    ActiveSubsegment (8bytes)
    _HEAP_LFH_AFFINITY_SLOT
    0x0
    0x38

    View Slide

  43. Low FragmentationHeap
    • LFH Subsegments (_HEAP_LFH_SUBSEGMENT)

    • The memory pool of LFH is very similar in structure to UserBlock in
    NtHeap, but each block does not have header and other metadata.

    • Once there is not enough memory, it will take subsegment from
    Buckets->State.availableSubsegmentList first, if it has no available
    subsegment , it will allocate from backend allocator for a new
    subsegment

    • Mainly managed by buckets->AffinitySlots

    View Slide

  44. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • ListEntry(_LIST_ENTRY)

    • Flink

    • Point to the next full/available LFH
    subsegment

    • Blink

    • Point to the previous full/available
    LFH subsegment
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  45. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • Owner
    (_HEAP_LFH_SUBSEGMENT_OWN
    ER)

    • Point to the structure that
    manages the subsegment

    • Point back to the
    AffinitySlots.State of the bucket
    to which it belongs
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  46. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • FreeCount

    • The number of times the block is
    released

    • Not the number of freed blocks
    of the subsegment
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  47. Low FragmentationHeap
    • LFH Subsegments (_HEAP_LFH_SUBSEGMENT)

    • BlockCount

    • the total number of block in the
    subsegment

    • FreeHint

    • The index of block that the last allocated

    • If the index of block that the last freed
    larger than freehint. It will be updated to the
    index

    • Used in allocated algorithm
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  48. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • Location

    • Indicates the location of the
    subsegment

    • 0 : AvailableSubsegmentList

    • 1 : FullSubsegmentList

    • 2 : The entire subsegment will be
    returned to back-end soon
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  49. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • BlockOffsets
    (_HEAP_LFH_SUBSEGMENT_ENC
    ODED_OFFSETS)

    • Used to indicate the block size
    of the subsegment and the offset
    of the first block in the
    subsegment
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  50. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • BlockSize

    • The size of block in the subsegment

    • The size is the original size

    • FirstBlockOffset

    • The offset of the first block

    • FirstBlock = subsegment +
    FirstBlockOffset
    BlockSize (2bytes)
    _HEAP_LFH_SUBSEGMENT_ENCODED_OFFSETS
    0x0
    FirstBlockOffset (2bytes)
    0x2

    View Slide

  51. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • BlockOffsets
    (_HEAP_LFH_SUBSEGMENT_ENC
    ODED_OFFSETS)

    • The value will be encoded

    • EncodedData =
    RtlpHpHeapGlobals.LfhKey^Blo
    ckOffsets^(subsegment >> 12)
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  52. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • BlockBitmap

    • Indicates the usage status of the
    block in the subsegment, and
    every two bits corresponds to a
    block

    • bit 0 : is_busy bit

    • bit 1 : unused bytes
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  53. Low FragmentationHeap
    • LFH Subsegments
    (_HEAP_LFH_SUBSEGMENT)

    • Block

    • The allocated memory return to user.

    • If the unused byte in the
    corresponding BlockBitmap is 1,
    then the last two bytes of the block
    are used to represent unused bytes

    • If there is only 1 byte, it will be
    recorded as 0x8000
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    _HEAP_LFH_SUBSEGMENT
    0x0
    0x10
    0x20
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    0x22
    0x24
    0x26
    0x28
    0x30

    View Slide

  54. Low FragmentationHeap

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[0]
    Buckets[1]

    0x88
    _HEAP_LFH_BUCKET

    State (0x38bytes)
    TotalBlockCount
    TotalSubsegmentCount
    ReciprocalBlockSize

    AffinitySlots[]
    State
    ActiveSubsegment
    _HEAP_LFH_AFFINITY_SLOT
    IsBuckets
    BucketIndex


    AvailableSubsegmentCount
    AvailableSubsegmentList
    FullSubsegmentList
    _HEAP_LFH_SUBSEGMENT_OWNE
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    _HEAP_LFH_SUBSEGMENT

    View Slide

  55. Low FragmentationHeap
    • Data Structure

    • Memory allocation mechanism

    View Slide

  56. Low FragmentationHeap
    • Allocate

    • When allocating memory smaller than LfhContext-
    >Config.MaxBlockSize, it will first check whether the corresponding
    bucket is LFH enabled

    • bucket index = RtlpLfhBucketIndexMap[needbytes+0xf]

    • Check : Buckets[index]->State & 1 == 1 (disable)

    • If it’s not enabled, it will update(RtlpHpLfhBucketUpdateStats) the
    usage times corresponding bucket.

    View Slide

  57. Low FragmentationHeap
    • Allocate

    • When allocating memory smaller than LfhContext->Config.MaxBlockSize,
    it will first check whether the corresponding bucket is LFH enabled

    • The buckets[idx] will record the number of times when it is not enabled

    • When we allocated a chunk : the corresponding buckets[idx] +=
    0x210000

    • Once (buckets[idx] >> 16) & 0x1f> 0x10 or (buckets[idx] >> 16)>
    0xff00 it will enable LFH (RtlpHpLfhBucketActivate)

    View Slide

  58. Low FragmentationHeap
    • Allocate

    • About enable LFH (RtlpHpLfhBucketActivate)

    • At the beginning, it will allocate the required structure of the bucket
    (RtlpHpHeapExtendContext)

    • Bucket, owner, affinityslot …

    • The allocation method is directly allocated from _SEGMENG_HEAP-
    >AllocatedBase (usually pointing to the end of the segment heap structure)

    • After allocation, the bucket related structure will be initialized by
    RtlpHpLfhBucketInitialize

    View Slide

  59. Low FragmentationHeap
    _SEGMENT_HEAP


    Signature (4bytes)

    AllocatedBase (8bytes)

    LfhContext (0x4c0bytes)
    0x10
    0xe8
    0x340
    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT


    Buckets[0]
    Buckets[1]

    View Slide

  60. Low FragmentationHeap
    _SEGMENT_HEAP


    Signature (4bytes)

    AllocatedBase (8bytes)

    LfhContext (0x4c0bytes)
    0x10
    0xe8
    0x340
    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT


    Buckets[0]
    Buckets[1]

    State (0x38bytes)
    **AffinitySlots

    State
    ActiveSubsegment
    0x800
    0xa40
    AffinitySlots[0]
    0x8e0
    0x860
    _HEAP_LFH_BUCKET
    _HEAP_LFH_AFFINITY_SLOT
    *_HEAP_LFH_AFFINITY_SLOT

    View Slide

  61. Low FragmentationHeap
    • Allocate

    • After LFH is enabled, LFH will be used as long as the block of the size is allocated

    • Implementation function is nt!RtlpHpLfhSlotAllocate

    • Next, it will check if there is an available block in ActiveSubsegment

    • The check is to take the lowest 12 bits of ActiveSubsegment (representing
    the number of blocks that can be allocated), and if there are available block,
    it will allocate from the subsegment

    • If not, it will confirm whether the subsegment really has no block that can be
    allocated

    View Slide

  62. Low FragmentationHeap

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[0]
    Buckets[1]

    0x88
    _HEAP_LFH_BUCKET

    State (0x38bytes)
    TotalBlockCount
    TotalSubsegmentCount
    ReciprocalBlockSize

    AffinitySlots[]
    State
    ActiveSubsegment
    _HEAP_LFH_AFFINITY_SLOT
    IsBuckets
    BucketIndex


    AvailableSubsegmentCount
    AvailableSubsegmentList
    FullSubsegmentList
    _HEAP_LFH_SUBSEGMENT_OWNE
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    _HEAP_LFH_SUBSEGMENT
    Find AffinitySlots

    View Slide

  63. Low FragmentationHeap

    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT
    0x0
    0x8
    0x3c

    0x80

    Buckets[0]
    Buckets[1]

    0x88
    _HEAP_LFH_BUCKET

    State (0x38bytes)
    TotalBlockCount
    TotalSubsegmentCount
    ReciprocalBlockSize

    AffinitySlots[]
    State
    ActiveSubsegment
    _HEAP_LFH_AFFINITY_SLOT
    IsBuckets
    BucketIndex


    AvailableSubsegmentCount
    AvailableSubsegmentList
    FullSubsegmentList
    _HEAP_LFH_SUBSEGMENT_OWNE
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    _HEAP_LFH_SUBSEGMENT
    Check ActiveSubsegment
    (ActiveSubsegment & 0xfff) > 0

    View Slide

  64. • Allocate

    • Which block will be taken?

    • In the case that the subsegment have available blocks, it will be similar to the LFH
    block in NtHeap, it would take the value of RtlpLowFragHeapRandomData[x] first.

    • Next time it will retrieve the value from RtlpLowFragHeapRandomData[x+1]

    • x is 1 byte ,x = rand() % 256 after 256 rounds

    • RtlpLowFragHeapRandomData is a 256-bytes array filled with random value

    • The range of random value is 0x0 - 0x7f
    Low FragmentationHeap

    View Slide

  65. Low FragmentationHeap
    • Allocate

    • Next, it will retrieve value of Bitmap[Index] according to the value of
    subsegment->FreeHint

    • Index = (2*freehint) >> 6

    • After retrieving the value of the Bitmap, it will look for the bit
    corresponding first allocated block

    • Then adding the random index, it is the block to be allocated

    View Slide

  66. Low FragmentationHeap
    • Allocate

    • If the block is allocated, it will tack the next nearest block

    • Finally, after updating the bit map and unused byte, it will return the
    memory block

    View Slide

  67. Low FragmentationHeap
    • Get an index

    • Searchwidth =
    RtlpSearchWidth[Bucketindex];

    • randval =
    RtlpLowFragHeapRandomData[x]

    • Bitmap = BlockBitmap[(2*freehint)
    >> 6]
    _HEAP_LFH_SUBSEGMENT
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[0]
    Block
    Block

    BlockBitmap[1]

    View Slide

  68. Low FragmentationHeap
    • Get an index

    • Firstnotfreeidx = first not free block
    in Bitmap

    • val = (searchwidth *randval >> 7) &
    0x1FFFFFE

    • blockindex = (firstnotfreeidx + val)
    & 0x3f
    _HEAP_LFH_SUBSEGMENT
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[0]
    Block
    Block

    BlockBitmap[1]

    View Slide

  69. Low FragmentationHeap
    • Check if the BusyBitmap correspond
    to index is 0

    • If it is not zero, it will take the next
    nearest block

    • If it is zero, it will set the
    corresponding BlockBitmap, and
    confirm whether there is unused
    byte
    _HEAP_LFH_SUBSEGMENT
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[0]
    Block
    Block

    BlockBitmap[1]

    View Slide

  70. Low FragmentationHeap
    • If there are unused byte, set
    unusedbyte at the end of the block

    • Return the block
    _HEAP_LFH_SUBSEGMENT
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[0]
    Block
    Block

    BlockBitmap[1]

    View Slide

  71. Low FragmentationHeap
    • Allocate

    • If the lowest 12 bit of the ActiveSubsegment is 0, it would check
    whether subsegment->FreeCount is greater than 1

    • if the value is greater than 1 (indicating that the subsegment still
    have allocatable blocks), it will update the lowest 12 bit at the
    ActiveSubsegment

    • If the subsegment has no block that can be allocated, it will be filled
    from Buckets[idx]->State.AvailableSubsegmentList

    View Slide

  72. Low FragmentationHeap
    • Allocate

    • In no subsegment available, It would allocate a new subsegment from the
    back-end allocator and initialize the subsegment
    (RtlpHpLfhSubsegmentCreate)

    • It will initialize BlockOffsets, BitMap, etc.

    • After initialization, it will add to AffinitySlots->State.AvailableSubsegmentList,
    and point subsegment->Owner back to AffinitySlots->State

    • AffinitySlots->ActiveSubsegment is also set to this subsegment and the
    lowest 12 bit is initialized

    View Slide

  73. Low FragmentationHeap
    _SEGMENT_HEAP


    AllocatedBase (8bytes)

    LfhContext (0x4c0bytes)
    0xe8
    0x340
    BackendCtx (8bytes)
    Callbacks (0x28bytes)
    Config (4bytes)
    _HEAP_LFH_CONTEXT


    Buckets[0]
    Buckets[1]

    State (0x38bytes)
    **AffinitySlots

    State
    ActiveSubsegment
    0x800
    0xa40
    AffinitySlots[0]
    0x8e0
    0x860
    _HEAP_LFH_BUCKET
    _HEAP_LFH_AFFINITY_SLOT
    *_HEAP_LFH_AFFINITY_SLOT
    ListEntry (10bytes)
    Owner (8bytes)
    FreeCount (2bytes)
    BlockCount (2bytes)
    FreeHint (2bytes)
    Location (1byte)

    BlockOffsets (4bytes)


    BlockBitmap[]
    Block

    IsBuckets
    BucketIndex


    AvailableSubsegmentCount
    AvailableSubsegmentList
    FullSubsegmentList
    _HEAP_LFH_SUBSEGMENT_OWNER
    _HEAP_LFH_SUBSEGMENT
    SegContexts (0x180bytes)
    VsContext (0xc0ytes)
    0x280
    0x100

    View Slide

  74. Low FragmentationHeap
    • Free

    • Main implementation function is nt!RtlpHpLfhSubsegmentFreeBlock

    • At the beginning, it will take subsegment.BlockOffsets first and decode it

    • According to the content of BlockOffsets, it will retrieve block size and
    firstblockoffset

    • According to the previous info, the index of block can be calculated

    • Clear the corresponding bitmap and set FreeCount = FreeCount + 1

    View Slide

  75. Low FragmentationHeap
    • Free

    • If FreeCount == BlockCount-1 means that all the blocks of the
    subsegment are released, the subsegment will be removed from the
    AvaliableSubsegmentList

    • There will also be double linked list checks when removing here

    • It will release subsegment to backend Allocator

    View Slide

  76. Memory Allocator in kernel
    • Segment Heap

    • Frontend Allocation

    • Low FragmentationHeap

    • Variable Size Allocation

    • Backend Allocation

    • Segment Allocation

    • Large Block Allocation

    View Slide

  77. Variable Size Allocation
    • Size

    • Size <= 0x200 and not enable LFH

    • 0x200 < Size <= 0xfe0

    • 0xfe0 < Size <= 0x20000 and (Size & 0xfff) != 0

    View Slide

  78. Variable Size Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  79. Variable Size Allocation
    • Chunk

    • The basic structure of memory allocator

    • In front of the chunk in VS allocation, there are some related metadata
    that record information of the chunk

    • _HEAP_VS_CHUNK_HEADER

    • _HEAP_VS_CHUNK_FREE_HEADER

    • Similar to back-end allocator in NtHeap

    View Slide

  80. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • MemoryCost

    • Only used when freed
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)

    View Slide

  81. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • UnsafeSize

    • The size of chunk

    • The value is right shift by 4 bits

    • UnasfePrevSize

    • The size of previous chunk

    • The value is right shift by 4 bits
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)

    View Slide

  82. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • Allocated

    • Indicates whether the chunk is
    allocated

    • The value is 1, if it is allocated
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)

    View Slide

  83. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • Chunk header will be encoded

    • Encoded header is xor with

    • Chunk header

    • Chunk address

    • RtlpHpHeapGlobals.HeapKey
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)
    Encode

    View Slide

  84. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • EncodedSegmentPageOffset

    • Indicates the index of pages of
    the chunk in the VS subsegment

    • It is used to find the VS
    subsegment

    • It will also be encoded.
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)

    View Slide

  85. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • EncodedSegmentPageOffset is xor
    with

    • (int8)Chunk address

    • (int8)RtlpHpHeapGlobals.HeapKe
    y

    • SegmentPageOffset
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)
    Encode

    View Slide

  86. Variable Size Allocation
    • _HEAP_VS_CHUNK_HEADER

    • UnusedBytes

    • Indicates whether the allocated
    chunk has unused memory
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    EncodedSegmentPageOffset (1byte)
    Inused
    UnusedBytes (1bit)
    SkipDuringWalk (1bit)
    Spare (22 bit)

    View Slide

  87. Variable Size Allocation
    • _HEAP_VS_CHUNK_FREE_HEADER

    • MemoryCost

    • Indicates how many pages of
    memory need to be committed
    when the chunk is allocated
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    Freed
    Node (18byte)

    View Slide

  88. Variable Size Allocation
    • _HEAP_VS_CHUNK_FREE_HEADER

    • 8 byte header is in the same as
    allocated chunk

    • Node (_RTL_BALANCED_NODE)

    • Node of rbtree and Freed chunk
    will be stored in a rbtree structure
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    Freed
    Node (18byte)

    View Slide

  89. Variable Size Allocation
    • _HEAP_VS_CHUNK_FREE_HEADER

    • Node (RTL_BALANCED_NODE)

    • Children

    • Left/Right

    • The left and right subtrees of the
    Node

    • ParentValue

    • Parent node of this node
    User Data
    MemoryCost (2byte)
    UnsafeSize (2byte)
    UnsafePrevSize (2byte)
    Allocated (1byte)
    Padding (1byte)
    Freed
    Left (8byte)
    Right (8byte)
    ParentValue (8byte)

    View Slide

  90. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • The core structure of the VS Allocator

    • Used to manage the memory allocated by VS allocator, and record all
    the information and structure of VS allocator in the heap

    View Slide

  91. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • Lock

    • Just for lock

    • LockType (_RTLP_HP_LOCK_TYPE)

    • The type of Lock can be divided
    into

    • Paged/NonPaged/TypeMax
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  92. Variable Size Allocation
    • FreeChunkTree (_RTL_RB_TREE)

    • In VS Allocation, after free a chunk, the chunk will be placed in the
    FreeChunkTree of the heap, and the chunk will be inserted in the
    FreeChunkTree according to the size.

    • If the size of chunk is larger than the node, it will be placed in right subtree

    otherwise, will be placed in left subtree

    • If there is no larger chunk than the chunk, the right subtree is NULL, and
    the other side is also

    • There will be a node check when taken out of the tree

    View Slide

  93. Variable Size Allocation
    • FreeChunkTree (_RTL_RB_TREE)

    • Root

    • Point to the root of the rbtree

    • Encoded

    • Indicates whether the root has been
    encoded (default disable)

    • About encode :

    • EncodedRoot = Root ^ FreeChunkTree
    Encoded
    Root
    FreeChunkTree
    0x0
    0x8

    View Slide

  94. Variable Size Allocation
    • FreeChunkTree (_RTL_RB_TREE)
    Left
    Right
    Chunk header (0x420)
    Parent (NULL)
    Left
    Right
    Chunk header (0x210)
    Parent
    Left
    Right
    Chunk header (0x510)
    Parent
    Encoded
    Root
    FreeChunkTree

    View Slide

  95. Variable Size Allocation
    • DelayFreeContext (_HEAP_VS_DELAY_FREE_CONTEXT)

    • When enable delay free (default in kernel but disable in usermode) and size of
    chunk < 0x1000, after Free a chunk, it will not be free immediately, but will be
    added to a singly linked list called DelayFreeContext, until the number of chunks
    in the linked list exceeds 0x20, the chunks in the linked list will be freed at one
    time

    • Next pointer will be put at the beginning of user data

    • FILO

    • If you want to check whether delay free is enable, you can check VsContext-
    >Config

    View Slide

  96. Variable Size Allocation
    • DelayFreeContext
    (_HEAP_VS_DELAY_FREE_CONTEXT)

    • Depth

    • The number of chunks in the Linked list

    • NextEntry

    • Point to next chunk

    • At this time chunk is still Allocated
    Depth (2byte)
    NextEntry (8byte)
    Sequence (6byte)
    DelayFreeContext
    Listhead

    View Slide

  97. Variable Size Allocation
    • DelayFreeContext (_HEAP_VS_DELAY_FREE_CONTEXT)
    Depth (2byte)
    NextEntry (8byte)
    Sequence (6byte)
    DelayFreeContext
    Next
    Chunk header (16 byte)
    Null
    Chunk header (16 byte)
    Listhead

    View Slide

  98. Variable Size Allocation
    • VS Subsegment (_HEAP_VS_SUBSEGMENT)

    • The memory pool of VS allocation

    • Once there is not enough for allocation, it will allocated from the
    backend allocator for a new subsegment

    • Each subsegment will be linked in a linked list

    View Slide


  99. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • ListEntry(_LIST_ENTRY)

    • Flink

    • Point to next subsegment

    • Blink

    • Point to previous subsegment
    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  100. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • ListEntry(_LIST_ENTRY)

    • The value would be encoded

    • Flink/Blink will xor with following
    value

    • Address of ListEntry

    • Address of next/prev
    subsegment

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  101. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • ListEntry(_LIST_ENTRY)

    • There is also double linked list
    check here

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  102. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • CommitBitmap

    • Indicates the status of page
    commit in the subsegment, and
    the page starts from the beginning
    of the subsegment

    • CommitLock

    • Lock used when Commit

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  103. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • Size

    • Size of VS subsegment

    • The value is right shift by 4 bits

    • Signature

    • Signature for verification, make sure
    that the subsegment is found when
    free

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  104. Variable Size Allocation
    • VS Subsegment
    (_HEAP_VS_SUBSEGMENT)

    • Behind the VS subsegment header
    is the memory pool of VS Allocator.
    At the beginning, the memory pool
    of subsegment is a large chunk

    • Split when allocated

    • Coalesce to-be-freed block with
    neighbors

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    0x0
    0x10
    0x18
    0x20
    0x22
    Chunk header
    0x30
    Chunk header
    Chunk header

    View Slide

  105. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • BackendCtx

    • Point to the Backend allocator
    (_HEAP_SEG_CONTEXT)
    structure used by the VS
    Allocator
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  106. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • Callbacks
    (_HEAP_SUBALLOCATOR_CALLBACKS)

    • Call back function table (Use to allocate/free
    subsegment)

    • Allocate

    • Free

    • Commit

    • Decommit

    • ExtendContext
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  107. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • Callbacks
    (_HEAP_SUBALLOCATOR_CALLBACKS)

    • Function pointer will be encoded

    • Callbacks will xor with following
    value

    • RtlpHpHeapGlobals.HeapKey

    • VsContext address

    • Function pointer
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  108. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • Config (_RTL_HP_VS_CONFIG)

    • Used to represent the attributes of
    the Vs Allocator

    • PageAlignLargeAllocs (default 1
    in kernel)

    • FullDecommit

    • EnableDelayFree (default 1 in
    kernel)
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  109. Variable Size Allocation
    • VsContext (_HEAP_VS_CONTEXT)

    • Config (_RTL_HP_VS_CONFIG)

    • Used to represent the attributes of
    the Vs Allocator

    • PageAlignLargeAllocs (default 0
    in user mode)

    • FullDecommit

    • EnableDelayFree (default 0 in
    usermode)
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0

    View Slide

  110. Variable Size Allocation
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0
    Encoded
    Root
    VS Subsegment
    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    Signature (15bit)
    FullCommit (1bit)
    Free Chunk header (0x300)
    Node
    Chunk header
    Chunk (0x230)
    Node
    Chunk (0x430)
    Node
    NextEntry

    View Slide

  111. Variable Size Allocation
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0
    Blink
    Flink
    VS Subsegment
    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    Signature (15bit)
    FullCommit (1bit)
    Free Chunk header (0x300)
    Node
    Chunk header
    ListEntry
    Subsegm
    ent
    NextEntry

    View Slide

  112. Variable Size Allocation
    LockType
    VsContext
    Lock
    FreeChunkTree
    SubsegmentList

    DelayFreeContext

    BackendCtx
    Callbacks
    Config
    0x0
    0x8
    0x10
    0x20
    0x40
    0x80
    0x88
    0xb0
    NextEntry
    Depth
    VS Subsegment
    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    Signature (15bit)
    FullCommit (1bit)
    Free Chunk header (0x300)
    Node
    Chunk header
    NextEntry

    View Slide

  113. Variable Size Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  114. Variable Size Allocation
    • Allocate

    • Main implementation function is nt!RtlpHpVsContextAllocateInternal

    • It will calculate the required chunk size at the beginning

    • Then it will find a suitable chunk from FreeChunkTree in VsContext

    • Start searching from the root, when the required chunk is larger
    than the node, continue searching from the right subtree until it is
    found or is NULL

    View Slide

  115. Variable Size Allocation
    • Allocate

    • If the chunk that can be allocated is not found, a subsegment will be
    allocated and the subsegment will be added to the VsContext, and
    then start searching from FreeChunkTree again

    • It will create a large chunk when it initialize the subsegment.

    • RtlpHpVsSubsegmentCreate

    • Request memory (RtlpHpSegVsAllocate) from backend, and
    create a new subsegment, the minimum size is (0x10000)

    View Slide

  116. Variable Size Allocation
    • Allocate

    • RtlpHpVsContextAddSubsegment

    • Add subsegment to VsContext

    • It will determinate how to split the subsegment according to whether to enable
    PageAlignLargeAllocs

    • If PageAlignLargeAllocs is set, the subsegment will be split into two chunks, one
    is the first chunk behind the subsegment structure, and the second is the page
    alignment, the user data of the chunk is aligned, and both chunks are added to
    FreeChunkTree

    • If not, the entire subsegment will be treated as a large chunk and added to
    FreeChunkTree

    View Slide

  117. Variable Size Allocation

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    Chunk header (size:0xffd0)

    ListEntry
    CommitBitmap
    CommitLock
    Size (2byte)
    VS Subsegment
    Signature (15bit)
    FullCommit (1bit)
    Chunk header (size:0xfb0)
    Chunk header (size:0xf020)
    0xffff..2030 0xffff..2030
    0xffff..2fe0
    Split
    Page

    alignment

    View Slide

  118. Variable Size Allocation
    • Allocate

    • If the found size of chunk size larger then request size, the chunk will be split, and the
    remaining chunks will be re-added to FreeChunkTree

    • RtlpHpVsChunkSplit

    • It will remove the chunk out of FreeChunkTree, and split the chunk, and re-added
    re-added FreeChunkTree as a new Freed chunk.

    • This will also be split according to whether the page alignment is or not. If the size
    of the chunk to be split exceeds 1 page, it split remainder chunk into two pieces
    according to the page

    • If request size < chunk size, unused byte will be recorded at the last 2 bytes of chunk

    View Slide

  119. Variable Size Allocation

    Chunk header (size:0x11020)

    Chunk header (size:0x3920)
    0xffff..3fe0 0xffff..3fe0
    Split
    allocate

    0x3920
    Chunk header (size:0x6e0)
    0xffff..7900
    0xffff..8fe0
    Chunk header (size:0xd020)
    Remainder

    View Slide

  120. Variable Size Allocation
    • Free

    • Main implementation function is nt!RtlpHpVsContextFree

    • The signature and Allocated bytes of the subsegment will be verified
    at first

    • Subsegment->Size ^ Subsegment->Signature ^ 0x2BED == 0

    • BSOD if verification fails

    View Slide

  121. Variable Size Allocation
    • Free

    • Next, it will check if the number of chunks in VsContext-
    >DelayFreeContext is greater than 0x20

    • VsContext->DelayFreeContext.Depth > 0x20

    • If it is less than 0x20, the chunk will be inserted at the beginning of the
    linked list and then return

    • VsContext->DelayFreeContext.NextEntry

    View Slide

  122. Variable Size Allocation
    • Free

    • If DelayFreeContext> 0x20, then the chunks in the linked list will be
    freed one by one

    • When it free a chunk, EncodedSegmentPageOffset will be used to
    find the VS subsegment of the chunk, and verify the Allocated byte
    and segment signature

    View Slide

  123. Variable Size Allocation
    • Free

    • Next, it will check whether the front and next chunks can be merged.
    If VsContext->Config.flag has enable PageAlignLargeAllocs, then one
    more chunk will be check.

    The merged chunk will be moved out of FreeChunkTree first. There is
    a tree structure check. After the merge, update prev_size and Size

    • RtlpHpVsChunkCoalesce

    • After merging, if address of chunk+0x20 is the beginning of page, the
    chunk will be page aligned and split into two pieces

    View Slide

  124. Variable Size Allocation
    • FreeChunkTree check
    Left
    Right
    Chunk header (0x420)
    Parent
    Left
    Right
    Chunk header (0x210)
    Parent
    Left
    Right
    Chunk header (0x510)
    Parent

    L R
    R->Parent->Right == R
    L->Parent->Left == L
    P->Left->Parent == P
    P->Right->Parent == P

    View Slide

  125. Variable Size Allocation
    • Free

    • If the size of the merged chunk is exactly equal to the size of the
    subsegment, the entire subsegment will be moved out of the linked list.

    • There is also a double linked list check here and it will release the
    subsegment to the backend allocator

    • If it is not equal, it will calculate MemoryCost and Segmentpageoffset of
    the chunk and then encode it.

    • Finally, look for a suitable position in FreeChunkTree to insert

    View Slide

  126. Pool Header
    • When the size of allocation <= 0xfe0

    • Both LFH and VS Allocator will allocate additional 0x10 byte (64bit)

    • It is used to store the Pool Header. Pool Header is used for Pool Allocator before 19H1 .

    • Now it is basically used in a few cases such as CacheAligned, PoolQuota and PoolTrack

    • The header contains

    • Size

    • PreviousSize

    • Pool type

    • PoolIndex

    View Slide

  127. Pool Header
    • When the size of allocation <= 0xfe0
    Chunk header (16 byte)
    _POOL_HEADER (16 byte)
    _POOL_HEADER (16 byte)
    LFH VS
    Return ptr

    View Slide

  128. Pool Header
    • Pool Header (_POOL_HEADER)

    • PreviousSize

    • Used in the CacheAligned case,
    indicating the offset between the
    previous Pool header and the
    header

    • Pool Index

    • Useless in Segment Heap
    PoolIndex (1byte)
    PreviousSize (1byte)
    PoolType (1byte)
    BlockSize (1byte)
    PoolTag (4 bytes)
    ProcessBilled (8 bytes)
    0x0
    0x1
    0x2
    0x3
    0x4
    0x8
    Pool header

    View Slide

  129. Pool Header
    • Pool Header (_POOL_HEADER)

    • BlockSize

    • The size of the block

    • The value is right shift by 4 bits

    • PoolType

    • The pooltype of the block
    PoolIndex (1byte)
    PreviousSize (1byte)
    PoolType (1byte)
    BlockSize (1byte)
    PoolTag (4 bytes)
    ProcessBilled (8 bytes)
    0x0
    0x1
    0x2
    0x3
    0x4
    0x8
    Pool header

    View Slide

  130. Pool Header
    • Pool Header (_POOL_HEADER)

    • PoolTag

    • The tag string is filled in when the
    block is allocated

    • When you use ExAllocatePoolWithTag,
    you can specify the pooltag

    • ProcessBilled

    • Used for PoolQuota and CacheAligned
    PoolIndex (1byte)
    PreviousSize (1byte)
    PoolType (1byte)
    BlockSize (1byte)
    PoolTag (4 bytes)
    ProcessBilled (8 bytes)
    0x0
    0x1
    0x2
    0x3
    0x4
    0x8
    Pool header

    View Slide

  131. Dynamic Lookaside
    • If the size of allocation 0x201
    • It will not be freed immediately after free. The block will be added to a
    Dynamic Lookaside

    • Depending on the size of the block, it will be added to different linked lists

    • The size is taken from Pool Header

    • When it allocate, it will first take block from Dynamic Lookaside first

    • It will enter the normal allocation if it is not found

    View Slide

  132. Dynamic Lookaside
    • _SEGMENT_HEAP

    • UserContext
    (_RTL_DYNAMIC_LOOKASIDE)

    • The core structure of the dynamic
    lookaside


    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP
    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)
    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    0x0 EnvHandle (10bytes)

    UserContext (8bytes)


    0x28

    View Slide

  133. Dynamic Lookaside
    • UserContext
    (_RTL_DYNAMIC_LOOKASIDE)

    • EnabledBucketBitmap

    • A bitmap is used to indicate which
    buckets have enable lookaside

    • BucketCount

    • The total number of buckets in
    lookaside
    EnabledBucketBitmap (8bytes)
    BucketCount (4bytes)
    ActiveBucketCount (4bytes)
    Buckets[64] (0x40bytes)
    0x0
    0x8
    0xc
    0x40
    _RTL_DYNAMIC_LOOKASIDE

    View Slide

  134. Dynamic Lookaside
    • UserContext
    (_RTL_DYNAMIC_LOOKASIDE)

    • ActiveBucketCount

    • The number of buckets with enable
    lookaside

    • Buckets (_RTL_LOOKASIDE)

    • Used to manage the structure of
    different sizes of lookaside, which is
    where the lookaside are
    EnabledBucketBitmap (8bytes)
    BucketCount (4bytes)
    ActiveBucketCount (4bytes)
    Buckets[64] (0x40bytes)
    0x0
    0x8
    0xc
    0x40
    _RTL_DYNAMIC_LOOKASIDE

    View Slide

  135. Dynamic Lookaside
    • Buckets (_RTL_LOOKASIDE)

    • ListHead (_SLIST_HEADER)

    • The structure of the head of the Singly
    linked list contains the length of the
    linked list, and the linked list itself,
    which is common in windows kernel

    • Depth

    • The number of chunks that can be
    stored in the bucket
    ListHead (16bytes)
    Depth (2bytes)

    0x0
    0x10
    _RTL_LOOKASIDE

    View Slide

  136. Dynamic Lookaside
    • _SLIST_HEADER

    • Depth

    • The number of nodes in the linked list

    • NextEntry

    • Point to next node

    • In the Dynamic Lookaside, it points to
    the Userdata of the freed chunk,
    behind the pool header
    Depth (2bytes)

    NextEntry (8 byte)
    0x0
    0x8
    _SLIST_HEADER

    View Slide

  137. Dynamic Lookaside


    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP
    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)
    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    0x0 EnvHandle (10bytes)

    UserContext (8bytes)


    0x28
    EnabledBucketBitmap (8bytes)
    BucketCount (4bytes)
    ActiveBucketCount (4bytes)
    Buckets[0] (0x40bytes)
    _RTL_DYNAMIC_LOOKASIDE
    Depth (2bytes)

    _RTL_LOOKASIDE
    Depth (2bytes)

    NextEntry (8 byte)
    Chunk header (16 byte)
    _POOL_HEADER (16 byte)
    VS chunk

    Buckets[1] (0x40bytes)

    View Slide

  138. Memory Allocator in kernel
    • Segment Heap

    • Frontend Allocation

    • Low FragmentationHeap

    • Variable Size Allocation

    • Backend Allocation

    • Segment Allocation

    • Large Block Allocation

    View Slide

  139. Segment Allocation
    • Size (in the kernel)

    • Size is a multiple of page

    • Size <= 0x7f0000

    • Size is not a multiple of page

    • 0x20000 < Size <= 0x7f0000

    • It’s a bit different in userland.

    View Slide

  140. Segment Allocation
    • it can be divided into two categories. The difference between the two is
    that the block units are different.

    • 0x20000 < Size <= 0x7f000

    • 0x1000 as the unit of block

    • 0x7f000 < Size <= 0x7f0000

    • 0x10000 as the unit of block

    • Both unit are called page here

    View Slide

  141. Segment Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  142. Segment Allocation
    • Page

    • In the case of size <= 0x7f000, the
    allocation unit of Segment Allocation

    • For example, if you allocate 0x1450
    bytes, it would allocate 2 page

    • 0x7f000 < Size <= 0x7f0000

    • It will use 0x10000 as the allocation unit

    • In the following slides, I will use 0x1000 as
    an example
    Page
    Page
    Page
    Page
    0x1000
    0x10000
    Pages

    View Slide

  143. Segment Allocation
    • Block (chunk)

    • Consists of single or multiple pages

    • The memory block allocated by
    Segment Allocatior

    • As shown in the figure, the block
    is composed of 3 pages
    Page
    Page
    Page
    Page
    0x1000
    Block

    View Slide

  144. Segment Allocation
    • Page Segment (_HEAP_PAGE_SEGMENT)

    • memory pool of segment allocation

    • Once there is no available page segment, a new page segment will be
    allocated from the system(MmAllocatePoolMemory) , but only the
    required structure will be allocated at the beginning

    • Each page segment will be inserted into a double linked list

    View Slide

  145. Segment Allocation
    • _HEAP_PAGE_SEGMENT

    • ListEntry (_LIST_ENTRY)

    • Flink

    • Point to the next page segment
    in the Linked list

    • Blink

    • Point to the previous page
    segment in the Linked list

    ListEntry
    Signature
    _HEAP_PAGE_SEGMENT


    DescArray[256]
    0x0
    0x10
    0x40
    Pages
    0x2000 

    0x10000

    View Slide

  146. Segment Allocation
    • _HEAP_PAGE_SEGMENT

    • Signature

    • The signature of the page segment is used
    to verify the page segment

    • Signature will xor with following value

    • Address of page segment

    • Address of Segcontext

    • RtlpHpHeapGlobals

    • 0xA2E64EADA2E64EAD

    ListEntry
    Signature
    _HEAP_PAGE_SEGMENT


    DescArray[256]
    0x0
    0x10
    0x40
    Pages
    0x2000
    0xA2E64EADA2E64EAD

    View Slide

  147. • _HEAP_PAGE_SEGMENT

    • DescArray
    (_HEAP_PAGE_RANGE_DESCRIPTOR)

    • Page range descriptor array

    • Each element corresponds to each
    page one by one

    • Pages

    • Memory pool of segment allocator
    Segment Allocation

    ListEntry
    Signature
    _HEAP_PAGE_SEGMENT


    DescArray[256]
    0x0
    0x10
    0x40
    Pages
    0x2000
    0xA2E64EADA2E64EAD

    View Slide

  148. Segment Allocation
    • Page range descriptor (_HEAP_PAGE_RANGE_DESCRIPTOR)

    • Descriptor for page

    • Indicates the status (Allocated or Freed) and information of each page
    in the page segment (whether the page is the beginning of a block, size
    of block, etc.)

    • It can be divided into allocated and freed

    • The page range descriptor in the freed state will be stored in
    FreePageRanges which is a rbtree structure

    View Slide

  149. Segment Allocation

    ListEntry
    Signature
    _HEAP_PAGE_SEGMENT


    DescArray[0x02]
    0x0
    0x10
    0x20
    0x2000
    DescArray[0x03]

    Page
    Page

    0x3000

    0x40

    View Slide

  150. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • TreeSignature

    • Signature of page range descriptor

    • The values is always 0xccddccdd

    • Only at the beginning of Block

    • UnusedBytes

    • Unusedbytes in an allocated block
    Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19
    Key (4 bytes)
    0x1c

    View Slide

  151. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • RangeFlag

    • Used to indicate the page status

    • Bit 1 : allocated bit

    • Bit 2 : block header bit

    • Bit 3 : Committed

    • LFH : RangeFlag & 0xc = 8

    • VS : RangeFlag & 0xc = 0xc
    Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19
    Key (4 bytes)
    0x1c

    View Slide

  152. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • CommitedPageCount

    • Indicates the number of pages
    committed in the corresponding
    page
    Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19
    Key (4 bytes)
    0x1c

    View Slide

  153. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • Key(_HEAP_DESCRIPTOR_KEY)

    • Store the size of the page
    corresponding to the page
    descriptor and the number of
    committed pages
    Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19
    Key (4 bytes)
    0x1c

    View Slide

  154. • _HEAP_DESCRIPTOR_KEY

    • EncodedCommittedPageCount

    • ~EncodedCommittedPageCount is the
    number of pages committed in the block

    • Only used in block header

    • UniCount

    • The size of Block

    • Value is page count
    Segment Allocation
    EncodedCommittedPageCount (2bytes)
    _HEAP_DESCRIPTOR_KEY
    0x0
    0xA2E64EADA2E64EAD
    LargePageCost (1bytes)
    0x2
    UnitCount (1bytes)
    0x3

    View Slide

  155. Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19
    Key (4 bytes)
    0x1c

    EncodedCommittedPageCount (2bytes)
    _HEAP_DESCRIPTOR_KEY
    LargePageCost (1bytes)
    UnitCount (1bytes)
    0x1c
    0x1f

    View Slide

  156. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • UnitOffset

    • If page is not block header

    • The UnitCount field will be
    called UnitOffset

    • Indicates the offset of the page
    in the block
    Segment Allocation
    TreeSignature
    UnusedBytes
    Allocated and not header

    RangeFlag (1byte)
    0x0
    0x4
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19

    0x1c


    UnitOffset (1byte)
    0x1f

    View Slide

  157. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • TreeNode
    (_RTL_BALANCED_NODE)

    • The node in the FreePageRanges
    Segment Allocation
    TreeNode (0x18 bytes)
    Freed
    RangeFlag (1byte)
    0x0
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19

    0x1c Key (4 bytes)

    View Slide

  158. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • TreeNode (_RTL_BALANCED_NODE)

    • Left

    • Point to a page descriptor that block size is smaller
    than its.

    • Right

    • Point to a page descriptor that block size is greater
    than its.

    • ParentValue

    • Point to parent node

    • The lowest 1 bit will determine whether to encode
    Parent
    Segment Allocation
    TreeNode
    Freed
    RangeFlag (1byte)
    0x0
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19

    Left
    Right
    ParentValue
    0x8
    0x10
    Key (4 bytes)

    View Slide

  159. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • Key(_HEAP_DESCRIPTOR_KEY)

    • Same as the case of allocated
    Segment Allocation
    TreeNode (0x18 bytes)
    Freed
    RangeFlag (1byte)
    0x0
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19

    Key (4 bytes)

    View Slide

  160. • _HEAP_PAGE_RANGE_DESCRIPTOR

    • UnitOffset

    • If page is not block header

    • The UnitCount field will be called
    UnitOffset

    • Indicates the offset of the page in
    the block
    Segment Allocation
    TreeNode (0x18 bytes)
    Freed and not header
    RangeFlag (1byte)
    0x0
    0xA2E64EADA2E64EAD
    0x18
    CommittedPageCount (1byte)
    0x19


    0x1c

    UnitOffset (1byte)
    0x1f

    View Slide

  161. Segment Allocation
    • SegContexts (_HEAP_SEG_CONTEXT)

    • The core structure of the Segment allocation

    • Used to manage the memory allocated by segment allocator, and
    record all the information and structure of segment allocator in the heap

    • There are two SegContexts in each Heap

    • Size <= 0x7f000

    • 0x7f000 < Size < 0x7f0000

    View Slide

  162. • SegContexts (_HEAP_SEG_CONTEXT)

    • SegmentMask

    • A mask used to find Page Segment

    • Page segment = block ptr &
    SegmentMask

    • UnitShift

    • Used to calculate the index of the page
    descriptor

    • Index = block ptr >> UnitShit
    Segment Allocation
    SegmentMask (8bytes)
    UnitShift (1bytes)
    SegContext
    PagesPerUnitShift (1byte)
    FirstDescriptorIndex (1byte)
    0x0
    0x8
    0xA2E64EADA2E64EAD
    0x9

    0xa
    VsContext (8bytes)
    0x18 LfhContext (8bytes)

    Heap (8bytes)
    0x38
    0x20
    SegmentListHead (10bytes)

    0x48

    FreePageRanges (10bytes)
    0x60

    View Slide

  163. • SegContexts (_HEAP_SEG_CONTEXT)

    • PagePerUnitShift

    • (0x1 << PagePerUnitShift) indicates the size of a
    page in the SegContext

    • The size of page unit is (0x1 <<
    PagePerUnitShift)*0x1000

    • If the value is zero

    • Page unit is 0x1000

    • FirstDescriptorIndex

    • The index of the first Page Descriptor in the
    SegContext
    Segment Allocation
    SegmentMask (8bytes)
    UnitShift (1bytes)
    SegContext
    PagesPerUnitShift (1byte)
    FirstDescriptorIndex (1byte)
    0x0
    0x8
    0xA2E64EADA2E64EAD
    0x9

    0xa
    VsContext (8bytes)
    0x18 LfhContext (8bytes)

    Heap (8bytes)
    0x38
    0x20
    SegmentListHead (10bytes)

    0x48

    FreePageRanges (10bytes)
    0x60

    View Slide

  164. • SegContexts (_HEAP_SEG_CONTEXT)

    • LfhContext
    (_HEAP_LFH_CONTEXT)

    • Point to the LFH allocator in the
    segment heap

    • VsContext (_HEAP_VS_CONTEXT)

    • Point to the VS allocator in the
    segment heap
    Segment Allocation
    SegmentMask (8bytes)
    UnitShift (1bytes)
    SegContext
    PagesPerUnitShift (1byte)
    FirstDescriptorIndex (1byte)
    0x0
    0x8
    0xA2E64EADA2E64EAD
    0x9

    0xa
    VsContext (8bytes)
    0x18 LfhContext (8bytes)

    Heap (8bytes)
    0x38
    0x20
    SegmentListHead (10bytes)

    0x48

    FreePageRanges (10bytes)
    0x60

    View Slide

  165. • SegContexts (_HEAP_SEG_CONTEXT)

    • Heap(_SEGMENT_HEAP)

    • Point to the segment heap to which it
    belongs

    • SegmentListHead (_LIST_ENTRY)

    • Point to the page segment in the
    segment allocator

    • The linked list is a double linked list
    with integrity check
    Segment Allocation
    SegmentMask (8bytes)
    UnitShift (1bytes)
    SegContext
    PagesPerUnitShift (1byte)
    FirstDescriptorIndex (1byte)
    0x0
    0x8
    0xA2E64EADA2E64EAD
    0x9

    0xa
    VsContext (8bytes)
    0x18 LfhContext (8bytes)

    Heap (8bytes)
    0x38
    0x20
    SegmentListHead (10bytes)

    0x48

    FreePageRanges (10bytes)
    0x60

    View Slide

  166. Segment Allocation
    • FreePageRanges (_RTL_RB_TREE)

    • In Segment Allocation, after releasing a block, the page descriptor of the block
    will be inserted into the FreePageRanges of the SegContext according to the
    size

    • If the block size is greater than the node, the page descriptor will be inserted
    into the right subtree, otherwise will be will be inserted into left subtree.

    • If there is no greater than the page descriptor, the right subtree is NULL and
    the other side is also

    • There will be a node check when the node is taken out of the tree

    View Slide

  167. • SegContexts (_HEAP_SEG_CONTEXT)

    • FreePageRanges (_RTL_RB_TREE)

    • Root

    • Point to the root of the rbtree

    • Encoded

    • Indicates whether the root has been
    encoded (default disable)

    • About encode

    • EncodedRoot = Root ^ FreeChunkTree
    Segment Allocation
    0xA2E64EADA2E64EAD
    Encoded
    Root
    FreePageRanges
    0x0
    0x8

    View Slide

  168. Segment Allocation
    0x0
    0x8
    0x9
    0xa
    0x18
    0x38
    0x20
    0x48
    SegmentMask
    UnitShift (0xc)
    SegContext
    PagesPerUnitShift (0x0)
    FirstDescriptorIndex (0x2)

    VsContext
    LfhContext

    Heap
    SegmentListHead


    FreePageRanges
    0x60
    Page Segment

    ListEntry
    Signature


    DescArray[0x02]
    DescArray[0x03]

    Page 0x2
    Page 0x3

    Freed Block
    Blink
    Flink
    Encoded
    Root
    TreeNode
    RangeFlag (0x2)

    UnitCount (0x2)
    0x2000
    Page Descriptor

    View Slide

  169. Segment Allocation

    _HEAP_PAGE_SEGMENT
    0x0
    0x40
    Block
    0x2000
    DescArray[0x02]
    DescArray[0x03]
    0x60

    _HEAP_VS_SUBSEGMENT
    Chunk header
    Chunk header
    Chunk header

    _HEAP_PAGE_SEGMENT
    0x0
    0x40
    Block
    0x2000
    DescArray[0x02]
    DescArray[0x03]
    0x60

    _HEAP_LFH_SUBSEGMENT
    LFH Block
    LFH Block

    View Slide

  170. Segment Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  171. Segment Allocation
    • Allocate

    • The allocation is based on page as the unit to allocate, and divided into

    • Size <= 0x7f000

    • Page uses 0x1000 bytes as a unit

    • 0x7f000 < Size < 0x7f0000

    • Page uses 0x10000 bytes as a unit

    View Slide

  172. Segment Allocation
    • Allocate

    • The allocation is based on page as the unit to allocate

    • For example, if it allocate 0x1337 bytes, segment allocation will
    allocate 0x2000, which is 2 page units, and the extra memory
    0x2000-0x1337 will be recorded in unused byte

    View Slide

  173. Segment Allocation
    • Allocate

    • Main implementation function is nt!RtlpHpSegAlloc

    • It will use RtlpHpSegPageRangeAllocate to get freed page descriptor
    or create a new page descriptor

    • First, it will search from FreePageRanges.

    • Start searching from the root, when the required block is larger
    than the node, continue searching from the right subtree until it
    is found or is NULL

    View Slide

  174. Segment Allocation
    • Allocate

    • If no suitable page descriptor is found, a new page segment will be allocated and the
    first page descriptor of the page segment will be initialized, and then this page
    descriptor will be used for allocated. The page segment will be inserted into the
    SegmentListHead.

    • In fact, it only allocated the memory required for the page segment and page
    descriptor structure, and the block part is not allocated at first

    • RtlpHpSegSegmentAllocate : Allocate page segment

    • RtlpHpSegSegmentInitialize : Initialize the first page descriptor

    • RtlpHpSegHeapAddSegment : insert into SegmentListHead

    View Slide

  175. Segment Allocation
    • Allocate

    • If it found a suitable or created a page descriptor, the page descriptor
    will be removed from FreePageRanges

    • When the required number of pages is smaller than the block,
    splitting will be done.

    • The page descriptor corresponding to the remainder block will be
    inserted into FreePageRanges

    View Slide

  176. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First

    View Slide

  177. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Found node
    Allocate 0x1337

    View Slide

  178. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Remove node
    Allocate 0x1337

    View Slide

  179. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Request size

    < block size
    Allocate 0x1337

    View Slide

  180. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Allocate 0x1337
    Split Block

    and update

    page
    descriptor 

    of next

    block

    View Slide

  181. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0x2)
    CommittedPageCount (1)
    UnitOffset (0x0)
    First
    Split Block

    and update

    page
    descriptor 

    of next

    block
    Allocate 0x1337
    Block
    First

    View Slide

  182. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    Unused
    RangeFlag (0x2)
    CommittedPageCount (1)
    UnitOffset (0x0)
    First
    Insert 

    page descriptor

    of next block 

    to 

    FreePageRange
    Allocate 0x1337
    Block
    First

    View Slide

  183. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    NULL
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeNode
    RangeFlag (0x2)
    CommittedPageCount (1)
    UnitOffset (0x0)
    First
    Update 

    page 

    descriptor

    of allocated

    block
    Allocate 0x1337
    Block
    First

    View Slide

  184. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Block
    TreeSignature
    RangeFlag (0x3)

    UnitSize (0x2)
    Unused
    RangeFlag (1)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeNode
    RangeFlag (0x2)
    CommittedPageCount (1)
    UnitOffset (0x0)
    First | Allocated
    Allocate 0x1337
    Block
    First
    Allocated

    View Slide

  185. Segment Allocation
    • Allocate

    • After setting the page descriptor, it will check whether all pages in the block
    are committed

    • It will add up the CommittedPageCount of all page descriptors in the block

    • If the block needs committed, it will commit memory to the specified VA

    • RtlpHpSegMgrCommit ->RtlpHpAllocVA->MmAllocatePoolMemory

    • Then update the CommittedPagecount of all page descriptors
    corresponding to the block

    View Slide

  186. Segment Allocation
    • Allocate

    • Finally, it will return the block that correspond to the page descriptor.

    • Block = (Page descriptor & SegmentMask) + ((index of Page
    descriptor) << SegContext->Unitshift)

    View Slide

  187. Segment Allocation
    • Free

    • At the beginning, it will verify whether the page segment where the ptr is located is
    legal

    • RtlpHpSegDescriptorValidate

    • Verify: Page segment->signature

    • 0xA2E64EADA2E64EAD == Page segment^SegContext^
    RtlpHpHeapGlobals^signature

    • Verify that the page descriptor of the page where the ptr is located is Allocated

    • Double Free check

    View Slide

  188. Segment Allocation
    • Free

    • Remark

    • from free pointer to page segment

    • Free pointer & segment mask

    • Page descriptor = 

    page_segment + 0x20 * ((free pointer - page segment) >> segcontext->UnitShift)

    • _HEAP_SEG_CONTEXT =

    (page segment)^(page segment-
    >Signature)^0xA2E64EADA2E64EAD^RtlpHpHeapGlobals.HeapKey


    View Slide

  189. Segment Allocation
    • Free

    • Next, we will see if the free pointer is at the beginning of block

    • If free pointer is not at the beginning of the block, it will check the
    RangeFlag of the Page descriptor to determine whether to use VS
    Allocator or Lfh Allocator to release the memory

    • If free pointer is not at the beginning of the block, it means that the
    free pointer is managed by segment allocation, and
    RtlpHpSegPageRangeShrink will be used

    View Slide

  190. Segment Allocation
    • Free

    • Then the Allocated bit of page descriptor correspond to block will be cleared, and it will check
    whether the previous and following blocks are Freed. If it is Freed, it will be merged

    • RtlpHpSegPageRangeCoalesce

    • The way to find the previous block is to check whether the page descriptor of previous page
    is at the beginning of the block. If it is not the beginning, it will use the UnitOffset of the page
    descriptor of previous page to calculate the page descriptor of the previous block.

    • The following is calculated using the UnitCount of the current page descriptor

    • Determine whether the RangeFlag of the Page descriptor at the beginning of the block is
    Allocated

    • It use page descriptor of the block at the beginning to check whether the block is allocated.

    View Slide

  191. Segment Allocation
    • Free

    • If the previous block is Free

    • Remove the Page descriptor of the previous block from
    FreePageRanges

    • Clear the first bit of the RangeFlag of the Page descriptor of the Block
    which we want to free

    • Update the UnitCount of the Page descriptor of the previous block

    • Update UnitOffset of the last page descriptor of Block after merge

    View Slide

  192. Segment Allocation
    • Free

    • If the following block is Free

    • Remove the page descriptor of following block from FreePageRanges

    • Clear the first bit of the RangeFlag of the Page descriptor of the
    following block

    • Update the UnitCount of the Page descriptor of the block which we want
    to free

    • Update UnitOffset of the last page descriptor of Block after merge

    View Slide

  193. Segment Allocation
    • Free

    • Finally, inserted it to FreePageRanges according to the block size

    View Slide

  194. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated

    View Slide

  195. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Check 

    Signature

    of Page

    segment

    View Slide

  196. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Check 

    Allocated

    bit

    View Slide

  197. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Check allocated
    and first bit of 

    Prev page

    descriptor

    View Slide

  198. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Use UnitOffset 

    to find the first

    page descriptor

    of prev block

    View Slide

  199. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Check Allocated

    bit of prev block

    View Slide

  200. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Remove node

    View Slide

  201. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeSignature
    RangeFlag (0x2)

    UnitSize (0x2)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Block
    First | Allocated
    Update

    size of prev

    block

    View Slide

  202. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeSignature
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x3)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    First | Allocated
    Clean Allcated

    and first bit 

    of page 

    descriptor

    View Slide

  203. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeSignature
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x0)
    CommittedPageCount (1)
    UnitSize (0x1)
    First
    Free 0x4000
    Set the last

    page descriptor

    of the free block

    View Slide

  204. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeSignature
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Free 0x4000
    Insert

    the page 

    descriptor to

    FreePageRages

    View Slide

  205. Segment Allocation

    _HEAP_PAGE_SEGMENT
    DescArray[0x02]
    DescArray[0x03]

    Page 2
    Page 3
    Page 4
    Page segment
    DescArray[0x04]
    0x0
    0x40
    0x60
    0x80
    0x2000
    0x3000
    0x4000

    Free Block
    TreeNode
    RangeFlag (0x2)

    UnitSize (0x3)
    Unused
    RangeFlag (0)
    CommittedPageCount (1)
    UnitOffset (0x1)
    TreeSignature
    RangeFlag (0x0)
    CommittedPageCount (1)
    UnitOffset (0x2)
    First
    Free 0x4000

    View Slide

  206. Memory Allocator in kernel
    • Segment Heap

    • Frontend Allocation

    • Low FragmentationHeap

    • Variable Size Allocation

    • Backend Allocation

    • Segment Allocation

    • Large Block Allocation

    View Slide

  207. Large Block Allocation
    • Size

    • Size > 0x7f0000

    View Slide

  208. Large Block Allocation
    • Only use rbtree to manage. Compared to other Allocations, it is much
    simpler. In fact, it almost directly allocate for a large block of memory from
    the system and stores it with rbtree.

    • Release is also removed from rbtree and returned to the system directly.

    View Slide

  209. Large Block Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  210. Large Block Allocation
    • _SEGMENT_HEAP

    • LargeAllocMetadata (_RTL_RB_TREE)

    • Used to manage the allocated
    memory by Large Allocation

    • It is also a rbree structure, and the
    address of allocated memory is used
    as KEY


    Dfa

    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP

    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)

    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    EnvHandle (10bytes)
    0x0

    View Slide

  211. Large Block Allocation
    • _SEGMENT_HEAP

    • LargeReservedPages

    • Number of pages reserved for Large
    Block Allocation

    • LargeCommittedPages

    • Number of committed pages for
    Large Allocation


    Dfa

    Signature (4bytes)
    LargeAllocMetadata (10bytes)
    _SEGMENT_HEAP

    LargeReservedPages (8bytes)
    LargeCommittedPages (8bytes)

    AllocatedBase (8bytes)
    SegContexts (0x180bytes)

    VsContext (0xc0bytes)
    LfhContext (0x4c0bytes)
    0x10
    0x48
    0x58
    0x60
    0xe8
    0x100
    0x280
    0x340
    EnvHandle (10bytes)
    0x0

    View Slide

  212. Large Block Allocation
    • _HEAP_LARGE_ALLOC_DATA

    • TreeNode (_RTL_BALANCED_NODE)

    • Left

    • Point to a node whose VirtualAddress is smaller
    than the node

    • Right

    • Point to a Node whose VirtualAddress is greater
    than the Node

    • ParentValue is pointed to parent node

    • The lowest 1 bit will determine whether to encode
    Parent node
    VirtualAddress (8 bytes)

    _HEAP_LARGE_ALLOC_DATA
    0x0
    0x18
    AllocatedPages (52 bit)
    0x20:12
    Left
    Right
    ParentValue

    View Slide

  213. Large Block Allocation
    • _HEAP_LARGE_ALLOC_DATA

    • VirtualAddress

    • The lowest 16 bits as Unusedbytes

    • Address of large block
    VirtualAddress (8 bytes)

    _HEAP_LARGE_ALLOC_DATA
    0x0
    0x18
    AllocatedPages (52 bit)
    0x20:12
    TreeNode (0x18 bytes)

    View Slide

  214. Large Block Allocation
    • _HEAP_LARGE_ALLOC_DATA

    • AllocatedPages

    • The number of allocated pages
    VirtualAddress (8 bytes)

    _HEAP_LARGE_ALLOC_DATA
    0x0
    0x18
    AllocatedPages (52 bit)
    0x20:12
    TreeNode (0x18 bytes)

    View Slide

  215. Large Block Allocation
    • Data Structure

    • Memory allocation mechanism

    View Slide

  216. Large Block Allocation
    • Allocate

    • The allocation is based on page as the unit.

    • Main implementation function is nt!RtlpHpMetadataAlloc

    • At the beginning, it will allocate memory for storing Large block Metadata
    (_HEAP_LARGE_ALLOC_DATA)

    • RtlpHpMetadataHeapCtxGet & RtlpHpMetadataHeapStart

    • It will determine which Heap to allocate from according to the value of
    Segment heap->EnvHandle

    • ExPoolState->HeapManager.MetadataHeaps[idx]

    View Slide

  217. Large Block Allocation
    • Allocate

    • The allocation is based on page as the unit.

    • Main implementation function is nt!RtlpHpMetadataAlloc

    • Next, it will use RtlpHpAllocVA to allocated memory, and then store
    the VirtualAddress in Metadata

    • Finally, insert the Metadtata into Segment heap->LargeAllocMetadata

    View Slide

  218. Large Block Allocation
    • Free

    • Main implementation function is (RtlpHpLargeFree)

    • First find the node correspond to the free ptr from Segment heap-
    >LargeAllocMetadata, and remove the node

    • Use RtlpHpFreeVA to release memory

    • Finally release the memory storing Metadata (RtlpHpMetadataFree) of
    the released memory

    View Slide

  219. Reference
    • Segment Heap

    • https://www.blackhat.com/docs/us-16/materials/us-16-Yason-
    Windows-10-Segment-Heap-Internals.pdf

    • https://www.sstic.org/media/SSTIC2020/SSTIC-actes/
    pool_overflow_exploitation_since_windows_10_19h1/SSTIC2020-
    Article-pool_overflow_exploitation_since_windows_10_19h1-
    bayet_fariello.pdf

    View Slide

  220. About Part 2
    • We will talk about window kernel pool exploit

    • Way to spray and defragment in different heap

    • Metadata corruption in segment heap

    • …etc

    View Slide