Managed Runtime Systems: Lecture 03 - Memory Management

Transcript

1. Managed Runtime Systems Memory Management Foivos Zakkak https://foivos.zakkak.net Except where otherwise noted, this presentation is licensed under the Creative Commons Attribution 4.0 International License. Third party marks and brands are the property of their respective holders.
2. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 2 Acknowledgments The following slides are based on the corresponding slides of Mario Wolczko about Memory Management: ▪ Memory management part 1 ▪ Memory management part 2 ▪ Memory management debugging hints
3. Static vs Dynamic Allocation
4. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 4 Static Memory Allocation ▪ Binding of name to memory address at compile/link time ▪ All sizes are fixed, i.e., known at compile time ▪ No stack allocation ▪ Used in early FORTRAN, BASIC, and various languages for embedded/real-time systems
5. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 5 Static Memory Allocation ▪ Pros – No runtime overheads for allocation/de-allocation – Memory requirements known at compile time – No failures due to lack of memory ▪ Cons – Need to allocate and keep the maximum possible memory footprint for the whole program execution – No recursion due to lack of stack allocation
6. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 6 Dynamic Allocation (on the stack) ▪ Languages (at least most of them) are based on procedures ▪ LIFO/Depth-First invocation order (in most cases) – See Scheme and Smalltalk for counter-examples ▪ Memory used by procedures can be managed as a stack – Allocate on invocation – Release on return ▪ Hardware support (SP register, call/ret instr.) since the 1960s
7. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 7 Dynamic Allocation (on the stack) ▪ Pros – Low runtime overheads – Bump stack pointer for allocation – Bulk de-allocation on return – No memory leaks ▪ Cons – Names passed as parameters, the deeper a procedure is invoked the more parameters it gets – Cannot return memory to previous procedures – Data-lifetime equals the procedure’s lifetime – Can’t handle complex data structures like graphs
8. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 8 Dynamic Allocation (on the heap) ▪ Arbitrary requests for memory segments ▪ Allocations may fail ▪ Fast allocation vs Fast de-allocation trade-offs ▪ Heap may not be contiguous
9. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 9 Dynamic Allocation (on the heap) ▪ Pros – Arbitrary allocation sizes – Allocation and de-allocation from different procedures – Handling of complex data structures ▪ Cons – Noticeable runtime overheads – Need to perform a de-allocation for each allocation ▪ See region-based allocation for an enhancement – Memory leaks are possible
10. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 10 Heap Allocation Free
11. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 11 Heap Allocation Free
12. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 12 Heap Allocation Free
13. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 13 Heap Allocation Free
14. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 14 Heap Allocation Free
15. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 15 Heap Allocation Free ? ?
16. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 16 Heap Allocation: Free-list variations ▪ First-Fit: – Search free-list from the beginning, peek first block that fits – Add the remaining of the block (if any) as a new block to the list – May result in a number of small blocks at beginning of free-list ▪ Best-Fit: – Search free-list from the beginning, peek the block that best fits – Add the remaining of the block (if any) as a new block to the list – Reduces fragmentation – Slow since we need to traverse the whole free-list ▪ Next-Fit: – Search from where we stopped last time, peek the block that best fits – Add the remaining of the block (if any) as a new block to the list – Might increase fragmentation – Often faster than First-fit
17. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 17 First-Fit Head Request
18. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 18 First-Fit Head Request
19. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 19 First-Fit Head Request
20. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 20 First-Fit Head Request
21. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 21 Best-Fit Head Request
22. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 22 Best-Fit Head Request
23. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 23 Best-Fit Head Request
24. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 24 Best-Fit Head Request
25. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 25 Best-Fit Head Request
26. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 26 Best-Fit Head Request
27. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 27 Best-Fit Head Request
28. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 28 Next-Fit Head Request
29. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 29 Next-Fit Head Request
30. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 30 Next-Fit Head Request
31. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 31 Next-Fit Head Request
32. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 32 Next-Fit Head Request Request
33. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 33 Next-Fit Head Request Request
34. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 34 Next-Fit Head Request Request
35. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 35 Next-Fit Head Request Request
36. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 36 Next-Fit Head Request Request
37. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 37 Fragmentation ▪ Fragmentation is the phenomenon of not being able to use parts of memory because of inefficient management ▪ A heavily fragmented system may have plenty of free memory, but chopped in small blocks that don’t fit a new request
38. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 38 Fragmentation Categorization ▪ Internal Fragmentation is the result of allocating larger chunks than actually needed (often due to alignment restrictions) ▪ External Fragmentation is the result of constantly splitting free blocks, resulting in multiple small non-contiguous free blocks that cannot be used
39. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 39 Free-list Allocation Main Overheads 1) Loop over free-blocks, even over obvious non-matches 2) For each block check if it fits 3) Split the block if it’s bigger
40. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 40 Single-size Free-lists 1) A set of free-lists, one for each size (for a set of common sizes) 2) A generic free-list for the rest 3) Always peek the first block from the list of the desired size 4) If empty take a block from the generic one and split it
41. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 41 Single-size Free-lists Head1 Head2 Head3 Heads
42. What about multi-threaded applications???
43. Garbage Collection
44. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 44 Garbage Collection 1) Ease programming, no need to argue about object lifetimes 2) Eliminate errors due to dangling pointers 3) Take care of the previous issues 4) Still possible to leak memory!
45. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 45 How does it work? public static void main(String args[]) { List <String > lines= Files.readAllLines(Paths.get(args[0]), Charset.defaultCharset()); int nLines= lines.size(); // reclaim lines?? System.out.println(nLines); } public static void main(String args[]) { List <String > lines= Files.readAllLines(Paths.get(args[0]), Charset.defaultCharset()); int nLines= lines.size(); // reclaim lines?? System.out.println(nLines); }
46. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 46 Liveness An object is dead when it is no longer needed
47. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 47 Liveness An object is dead when it is no longer needed “But, VMs (and compilers) have severely limited crystal balls” – Mario Wolczko
48. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 48 Liveness in the Real World ▪ An object is dead when it is no longer reachable – Reachable is an object that can be reached by following pointers starting from the system’s roots – The system’s roots are all the variables in scope (of all threads) – Requires traversal of stacks and globals
49. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 49 Reachability Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
50. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 50 Reference Counting ▪ Keep a reference counter per object ▪ Increment it when a reference to that object is assigned to a variable ▪ Decrement it when a reference to that object is overwritten ▪ If the counter is zero, the object can be reclaimed
51. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 51 Reference Counting Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E C 0 1 1 1 3 1 2 1
52. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 52 Reference Counting Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F C E D H G A F H D G E 0 1 1 3 1 2 1 B 0
53. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 53 Reference Counting Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F E D H G A F H D G E 0 1 1 1 2 1 B 0 C 2
54. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 54 Reference Counting Drawbacks 1) Reference counting has to be performed on all variables (stack, global, and heap) 2) References in an activation record have to be decremented before de-allocating the frame upon return 3) Decrementing the reference counter often incurs a cache miss 4) Decrementing the reference counter always incurs a write 5) Concurrent threads might contend on the reference counter 6) Cannot reclaim cycles
55. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 55 Reference Counting Cycle Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. F E D H G F 1 1 1 1 2
56. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 56 Reference Counting with Delayed Reclamation ▪ Avoid unbound recursion ▪ Reduce the number of pauses for Garbage Collection
57. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 57 Tracing Collection: Mark-n-Sweep ▪ Mark: Follow the system’s roots and mark reachable objects ▪ Sweep: Reclaim unmarked objects at the end
58. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 58 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G
59. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 59 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F
60. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 60 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H
61. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 61 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G
62. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 62 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
63. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 63 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F B C E D H G A F H D G E
64. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 64 Mark-n-sweep Roots: • Static variables • Java stack frame • Native stack frame • JNI • etc. A F E D H G A F H D G E B C
65. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 65 Mark Implementations ▪ Recursive: Worst case each object creates an activation (i.e., marking a single linked-list) ▪ Work queue: Each object creates a new node in the queue – Dominant approach in parallel garbage collectors
66. Sweep Implementations
67. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 67 Sweep with Free-list ▪ Add reclaimed chunks to a free-list ▪ Requires parsing the whole heap to find the non-marked objects – Possible (using the object headers), but inefficient ▪ Coalescing adjacent free blocks
68. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 68 Compacting Sweep ▪ Move live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
69. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 69 Compacting Sweep ▪ Move live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
70. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 70 Compacting Sweep ▪ Move live objects to consecutive memory addresses – Moving objects breaks references though ▪ Create a contiguous large free space after the live objects ▪ Run on every collection or when heap is heavily fragmented
71. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 71 Compacting Sweep with Forwarding Pointers Add an extra field in each object’s header – the forwarding pointer 1) Compute forwarding pointers 2) Update all pointers using the forwarding pointers 3) Move the objects
72. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 72 Compacting Sweep with Temporary Table ▪ Instead of using a forwarding pointer replace actual header with pointer to a temporary table entry ▪ Each temporary table entry holds a header and a forwarding location
73. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 73 Compacting Sweep with Threading 1) Replace the object header with a pointer to a list 2) This starts from the object and goes through all the fields that reference it 3) The last field in the list contains the initial content of the object header 4) When the object is moved the list is traversed to update the corresponding fields
74. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 74 Copying Collection  Combination of Trace and Compaction  Split heap memory in from and to semi-spaces  Copy live objects on trace to to semi-space  Leave forwarding pointers in from semi-space  At the end, from becomes to and vice versa
75. Managed Runtime Systems CC-BY https://Foivos.Zakkak.net 75 Copying Collection ▪ Pros – Bump allocation – Traverse only live objects – Can be used by parallel GCs – Increase locality? ▪ Cons – Requires twice the memory (at least during collection) – Copies the whole heap in each collection