Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managed Runtime Systems: Lecture 03 - Memory Ma...

zakkak
February 14, 2018

Managed Runtime Systems: Lecture 03 - Memory Management

zakkak

February 14, 2018
Tweet

More Decks by zakkak

Other Decks in Programming

Transcript

  1. Managed Runtime Systems Memory Management Foivos Zakkak https://foivos.zakkak.net Except where

    otherwise noted, this presentation is licensed under the Creative Commons Attribution 4.0 International License. Third party marks and brands are the property of their respective holders.
  2. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 2 Acknowledgments The following slides

    are based on the corresponding slides of Mario Wolczko about Memory Management: ▪ Memory management part 1 ▪ Memory management part 2 ▪ Memory management debugging hints
  3. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 4 Static Memory Allocation ▪

    Binding of name to memory address at compile/link time ▪ All sizes are fixed, i.e., known at compile time ▪ No stack allocation ▪ Used in early FORTRAN, BASIC, and various languages for embedded/real-time systems
  4. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 5 Static Memory Allocation ▪

    Pros – No runtime overheads for allocation/de-allocation – Memory requirements known at compile time – No failures due to lack of memory ▪ Cons – Need to allocate and keep the maximum possible memory footprint for the whole program execution – No recursion due to lack of stack allocation
  5. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 6 Dynamic Allocation (on the

    stack) ▪ Languages (at least most of them) are based on procedures ▪ LIFO/Depth-First invocation order (in most cases) – See Scheme and Smalltalk for counter-examples ▪ Memory used by procedures can be managed as a stack – Allocate on invocation – Release on return ▪ Hardware support (SP register, call/ret instr.) since the 1960s
  6. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 7 Dynamic Allocation (on the

    stack) ▪ Pros – Low runtime overheads – Bump stack pointer for allocation – Bulk de-allocation on return – No memory leaks ▪ Cons – Names passed as parameters, the deeper a procedure is invoked the more parameters it gets – Cannot return memory to previous procedures – Data-lifetime equals the procedure’s lifetime – Can’t handle complex data structures like graphs
  7. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 8 Dynamic Allocation (on the

    heap) ▪ Arbitrary requests for memory segments ▪ Allocations may fail ▪ Fast allocation vs Fast de-allocation trade-offs ▪ Heap may not be contiguous
  8. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 9 Dynamic Allocation (on the

    heap) ▪ Pros – Arbitrary allocation sizes – Allocation and de-allocation from different procedures – Handling of complex data structures ▪ Cons – Noticeable runtime overheads – Need to perform a de-allocation for each allocation ▪ See region-based allocation for an enhancement – Memory leaks are possible
  9. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 16 Heap Allocation: Free-list variations

    ▪ First-Fit: – Search free-list from the beginning, peek first block that fits – Add the remaining of the block (if any) as a new block to the list – May result in a number of small blocks at beginning of free-list ▪ Best-Fit: – Search free-list from the beginning, peek the block that best fits – Add the remaining of the block (if any) as a new block to the list – Reduces fragmentation – Slow since we need to traverse the whole free-list ▪ Next-Fit: – Search from where we stopped last time, peek the block that best fits – Add the remaining of the block (if any) as a new block to the list – Might increase fragmentation – Often faster than First-fit
  10. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 37 Fragmentation ▪ Fragmentation is

    the phenomenon of not being able to use parts of memory because of inefficient management ▪ A heavily fragmented system may have plenty of free memory, but chopped in small blocks that don’t fit a new request
  11. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 38 Fragmentation Categorization ▪ Internal

    Fragmentation is the result of allocating larger chunks than actually needed (often due to alignment restrictions) ▪ External Fragmentation is the result of constantly splitting free blocks, resulting in multiple small non-contiguous free blocks that cannot be used
  12. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 39 Free-list Allocation Main Overheads

    1) Loop over free-blocks, even over obvious non-matches 2) For each block check if it fits 3) Split the block if it’s bigger
  13. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 40 Single-size Free-lists 1) A

    set of free-lists, one for each size (for a set of common sizes) 2) A generic free-list for the rest 3) Always peek the first block from the list of the desired size 4) If empty take a block from the generic one and split it
  14. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 44 Garbage Collection 1) Ease

    programming, no need to argue about object lifetimes 2) Eliminate errors due to dangling pointers 3) Take care of the previous issues 4) Still possible to leak memory!
  15. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 45 How does it work?

    public static void main(String args[]) { List <String > lines= Files.readAllLines(Paths.get(args[0]), Charset.defaultCharset()); int nLines= lines.size(); // reclaim lines?? System.out.println(nLines); } public static void main(String args[]) { List <String > lines= Files.readAllLines(Paths.get(args[0]), Charset.defaultCharset()); int nLines= lines.size(); // reclaim lines?? System.out.println(nLines); }
  16. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 47 Liveness An object is

    dead when it is no longer needed “But, VMs (and compilers) have severely limited crystal balls” – Mario Wolczko
  17. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 48 Liveness in the Real

    World ▪ An object is dead when it is no longer reachable – Reachable is an object that can be reached by following pointers starting from the system’s roots – The system’s roots are all the variables in scope (of all threads) – Requires traversal of stacks and globals
  18. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 49 Reachability Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
  19. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 50 Reference Counting ▪ Keep

    a reference counter per object ▪ Increment it when a reference to that object is assigned to a variable ▪ Decrement it when a reference to that object is overwritten ▪ If the counter is zero, the object can be reclaimed
  20. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 51 Reference Counting Roots: •

    Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E C 0 1 1 1 3 1 2 1
  21. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 52 Reference Counting Roots: •

    Static variables • Java stack frame • Native stack frame • JNI • etc. A F C E D H G A F H D G E 0 1 1 3 1 2 1 B 0
  22. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 53 Reference Counting Roots: •

    Static variables • Java stack frame • Native stack frame • JNI • etc. A F E D H G A F H D G E 0 1 1 1 2 1 B 0 C 2
  23. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 54 Reference Counting Drawbacks 1)

    Reference counting has to be performed on all variables (stack, global, and heap) 2) References in an activation record have to be decremented before de-allocating the frame upon return 3) Decrementing the reference counter often incurs a cache miss 4) Decrementing the reference counter always incurs a write 5) Concurrent threads might contend on the reference counter 6) Cannot reclaim cycles
  24. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 55 Reference Counting Cycle Roots:

    • Static variables • Java stack frame • Native stack frame • JNI • etc. F E D H G F 1 1 1 1 2
  25. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 56 Reference Counting with Delayed

    Reclamation ▪ Avoid unbound recursion ▪ Reduce the number of pauses for Garbage Collection
  26. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 57 Tracing Collection: Mark-n-Sweep ▪

    Mark: Follow the system’s roots and mark reachable objects ▪ Sweep: Reclaim unmarked objects at the end
  27. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 58 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G
  28. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 59 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F
  29. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 60 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H
  30. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 61 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G
  31. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 62 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
  32. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 63 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
  33. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 64 Mark-n-sweep Roots: • Static

    variables • Java stack frame • Native stack frame • JNI • etc. A F E D H G A F H D G E B C
  34. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 65 Mark Implementations ▪ Recursive:

    Worst case each object creates an activation (i.e., marking a single linked-list) ▪ Work queue: Each object creates a new node in the queue – Dominant approach in parallel garbage collectors
  35. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 67 Sweep with Free-list ▪

    Add reclaimed chunks to a free-list ▪ Requires parsing the whole heap to find the non-marked objects – Possible (using the object headers), but inefficient ▪ Coalescing adjacent free blocks
  36. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 68 Compacting Sweep ▪ Move

    live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
  37. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 69 Compacting Sweep ▪ Move

    live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
  38. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 70 Compacting Sweep ▪ Move

    live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
  39. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 71 Compacting Sweep with Forwarding

    Pointers Add an extra field in each object’s header – the forwarding pointer 1) Compute forwarding pointers 2) Update all pointers using the forwarding pointers 3) Move the objects
  40. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 72 Compacting Sweep with Temporary

    Table ▪ Instead of using a forwarding pointer replace actual header with pointer to a temporary table entry ▪ Each temporary table entry holds a header and a forwarding location
  41. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 73 Compacting Sweep with Threading

    1) Replace the object header with a pointer to a list 2) This starts from the object and goes through all the fields that reference it 3) The last field in the list contains the initial content of the object header 4) When the object is moved the list is traversed to update the corresponding fields
  42. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 74 Copying Collection  Combination

    of Trace and Compaction  Split heap memory in from and to semi-spaces  Copy live objects on trace to to semi-space  Leave forwarding pointers in from semi-space  At the end, from becomes to and vice versa
  43. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 75 Copying Collection ▪ Pros

    – Bump allocation – Traverse only live objects – Can be used by parallel GCs – Increase locality? ▪ Cons – Requires twice the memory (at least during collection) – Copies the whole heap in each collection