Fundamentals of Memory Management in Go: Learning Through the History

24/10/2025 Slides for the presentation at Go West Conference in Lehi, UT.

Transcript

1. Takuto Nagami @logica0419 Fundamentals of Memory Management in Go: Learning Through the History
2. Who is new or struggles to learn memory management?
3. I did too!!! Memory management is actually difficult...
4. Why memory management matters? • For better performance • Who takes care of Go itself? • Memory management is fun to learn!
5. Example case about performance Single memory allocation change -> 57% lower CPU, 99% lower memory!!
6. Sessions are difficult...
7. Sessions are difficult...
8. Sessions are difficult...
9. Sessions are difficult...
10. Sessions are difficult...
11. Sessions are difficult... Hard for beginners! but why are they?
12. Learning process: Why, What and How When you want to add a new feature,
13. Learning process: Why, What and How When you want to add a new feature, Why Why do you need it?
14. Learning process: Why, What and How Why Why do you need it? What What is it? When you want to add a new feature,
15. Learning process: Why, What and How Why Why do you need it? What What is it? How How will you implement it? When you want to add a new feature,
16. When you want to add a new feature, Learning process: Why, What and How Why Why do you need it? What What is it? How How will you implement it? Typical sessions focus on this!
17. Learning process: Why, What and How Why Why do you need it? What What is it? How How will you implement it? When you want to add a new feature,
18. I’m here to cover “Why”s and “What”s of memory management! Please keep this in mind!
19. Pre-requirements: what go provides • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
20. Pre-requirements: what go provides Your code go build • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
21. Pre-requirements: what go provides Your code binary go build • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
22. Pre-requirements: what go provides Your code binary go build compiler’s duty • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
23. Pre-requirements: what go provides Your code binary go build compiler’s duty (execute binary) • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
24. Pre-requirements: what go provides Your code binary go build compiler’s duty (execute binary) runtime’s duty • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
25. Pre-requirements: what go provides Your code binary go build compiler’s duty (execute binary) runtime’s duty go run • Compiler ◦ Converts your code into binary • Runtime ◦ Runs together with your code during execution
26. What memory is: Ancient memory management
27. What is memory? Memory = RAM (Random Access Memory)
28. What is memory? Memory = RAM (Random Access Memory) ↑ Storage (HDD/SSD)
29. What is memory? Memory = RAM (Random Access Memory) ↑ Storage (HDD/SSD)
30. Memory visualization . . . • Key-value store ◦ Key: Memory address ◦ Value: 1 byte for each OS splits RAM to prepare virtual memory for each process! (Learn “virtual memory” for more) 0x0000 0x0001 0x0002 0x0003
31. Read/Write with assembly . . . . . . 0x0000 0x0001 0x0002 0x0003 {address}
32. Read/Write with assembly . . . . . . Write 0x0000 0x0001 0x0002 0x0003 {address} 1 1
33. Read/Write with assembly . . . . . . Read 0x0000 0x0001 0x0002 0x0003 {address} 1 1 %eax (CPU register)
34. 【Pain】Intuitive, but painful... • Programmers have to remember all the addresses 🤯
35. Stack and Heap: Era of variables
36. High-level languages arose • 1957: Fortran was released ◦ “First” high-level programming language • COBOL, BASIC, PASCAL, C... and eventually Go
37. Abstraction of data: Variables . . . (Variables)
38. Abstraction of data: Variables . . . . . . . . . . . {a addr} 12 a (Variables) 12
39. Abstraction of data: Variables a (Variables) 12 {a addr} {b addr} b 34 . . . . . . . . . 12 34
40. Abstraction of data: Variables a (Variables) 12 b 34 {a addr} {b addr} . . . . . . . . . 12 34 Programmers recognize only these!
41. Programmers don't need to know the address 🙌 But languages need to manage them 😨
42. We want some rules to place them! {a addr} ? {b addr} ? . . . . . . . . . 12 34
43. Function calling
44. Function calling main()
45. Function calling Call main()
46. Function calling Call main() a()
47. Function calling Call main() a() Call
48. Function calling Call main() a() Call b()
49. Function calling Call main() a() Return
50. Function calling Call main() a()
51. Function calling Return main()
52. Function calling main()
53. Function calling Program exits
54. Data structure of function calling Last In, First Out (LIFO)...? main() a() Call main() a() Return
55. Data structure of function calling First In, First Out (FIFO)...? main() a() Call main() a() Return Yes! That’s STACK !!
56. . . . Stack memory area
57. . . . main() stack frame Stack memory area {main() s.f. addr}
58. Stack frame • Representation of a “function” in memory ◦ Each frame = scope • Structure is defined in ABI ◦ Local variables ◦ (Some of) arguments ◦ Position of the previous function stack frame ◦ Position where previous function stopped ◦ etc... a() stack frame • call on line 10 • return to main() 5 7
59. . . . main() stack frame Stack memory area {main() s.f. addr}
60. . . . main() stack frame 5 Stack memory area {main() s.f. addr} {v addr}
61. . . . main() stack frame • call on line 3 5 Stack memory area {main() s.f. addr} {v addr}
62. . . . main() stack frame • call on line 3 5 a() stack frame Stack memory area {a() s.f. addr} {main() s.f. addr} {v addr}
63. . . . main() stack frame • call on line 3 5 a() stack frame • return to main() Stack memory area {a() s.f. addr} {main() s.f. addr} {v addr}
64. . . . main() stack frame • call on line 3 5 a() stack frame • return to main() 5 Stack memory area {a() s.f. addr} {arg addr} {main() s.f. addr} {v addr}
65. . . . main() stack frame • call on line 3 5 a() stack frame • return to main() 5 arg+2= 7 Stack memory area {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr}
66. . . . main() stack frame • call on line 3 5 a() stack frame • return to main() 5 7 Stack memory area {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr}
67. . . . main() stack frame • call on line 3 5 a() stack frame • return to main() 5 7 Stack memory area {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr}
68. . . . main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() 5 7 Stack memory area {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr}
69. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame
70. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a()
71. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {arg addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a() 7
72. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {arg addr} {b addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a() 7 1
73. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {arg addr} {b addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a() 7 1
74. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {arg addr} {b addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a() 7 1 arg+b= 8 (CPU register)
75. . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • call on line 10 • return to main() {b() s.f. addr} {arg addr} {b addr} {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 b() stack frame • return to a() 7 1 8 (CPU register)
76. . . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • return to main() {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 8 (CPU register)
77. . . . Stack memory area main() stack frame • call on line 3 5 a() stack frame • return to main() {a() s.f. addr} {arg addr} {a addr} {main() s.f. addr} {v addr} 5 7 8 (CPU register)
78. . . . Stack memory area main() stack frame 5 {main() s.f. addr} {v addr} 8 (CPU register)
79. . . . Stack memory area main() stack frame 8 {main() s.f. addr} {v addr} 8 (CPU register)
80. . . . Stack memory area main() stack frame 8 {main() s.f. addr} {v addr}
81. . . . Stack memory area Program exits
82. Stack is not a silver bullet... • Can’t handle unknown data size (maps/slices) ◦ Stack allocation is compiler’s duty • Sometimes can’t share variables between functions
83. Which is also called “heap memory area” The new concept: Dynamic Memory Allocation
84. {main() s.f. addr} Heap memory area . . . . . . . main() stack frame
85. Heap memory area . . . . . . . [] {slice_h addr} {main() s.f. addr} main() stack frame
86. Heap memory area . . . . . . . [] {slice_h addr} {main() s.f. addr} main() stack frame Address is determined by runtime
87. Heap memory area . . . . . . . [] {slice_h addr} {main() s.f. addr} main() stack frame main() doesn’t know where the slice is...
88. . . . . . . . [] {slice_h addr} {main() s.f. addr} {slice addr} main() stack frame Heap memory area {slice_h addr}
89. . . . . . . . [] {slice_h addr} {main() s.f. addr} {slice addr} main() stack frame Heap memory area This is pointer!! {slice_h addr}
90. . . . . . . . [] {slice_h addr} {main() s.f. addr} {slice addr} {num addr} main() stack frame Heap memory area 5 This is pointer!! {slice_h addr}
91. It’s not limited to a heap reference Let’s understand what pointers do with another example!
92. Pointer arg VS Value arg Could you explain the difference?
93. . . . main() stack frame • call on line 3 5 Value arg {main() s.f. addr} {num addr}
94. . . . main() stack frame • call on line 3 5 Value arg f() stack frame • call on line 3 {f() s.f. addr} {main() s.f. addr} {num addr}
95. . . . main() stack frame • call on line 3 5 Value arg f() stack frame • call on line 3 5 {f() s.f. addr} {num addr} {main() s.f. addr} {num addr}
96. {f() s.f. addr} {num addr} {main() s.f. addr} {num addr} . . . main() stack frame • call on line 3 5 Value arg f() stack frame • call on line 3 10
97. {f() s.f. addr} {num addr} {main() s.f. addr} {num addr} . . . main() stack frame • call on line 3 5 Value arg f() stack frame • call on line 3 10 Doesn’t affect to num in main() func!
98. . . . Pointer arg main() stack frame • call on line 3 5 {main() s.f. addr} {num addr}
99. . . . Pointer arg main() stack frame • call on line 3 5 f() stack frame • call on line 3 {f() s.f. addr} {main() s.f. addr} {num addr}
100. . . . main() stack frame • call on line 3 f() stack frame • call on line 3 Pointer arg 5 {num addr} {f() s.f. addr} {num addr} {main() s.f. addr} {num addr}
101. . . . main() stack frame • call on line 3 f() stack frame • call on line 3 Pointer arg {f() s.f. addr} {num addr} {main() s.f. addr} {num addr} 10 {num addr}
102. . . . {f() s.f. addr} {num addr} {main() s.f. addr} {num addr} main() stack frame • call on line 3 10 f() stack frame • call on line 3 Pointer arg {num addr} Does affect to num in main() func!
103. Combination of stack and heap enables the general-purpose memory management!
104. Heap management in early era . . . . . . . {main() s.f. addr} main() stack frame Especially in C • func malloc(int size) pointer • func free(pointer memory) (Conceptual pseudocode)
105. Heap management in early era Especially in C • func malloc(int size) pointer • func free(pointer memory) (Conceptual pseudocode) . . . . . . . ~~~ {some addr} {main() s.f. addr} main() stack frame {some addr}
106. Heap management in early era . . . . . . . {main() s.f. addr} main() stack frame {some addr} Especially in C • func malloc(int size) pointer • func free(pointer memory) (Conceptual pseudocode)
107. 【Pain】What if we forget free()? • The memory usage infinitely increases ◦ Memory Leak • Eventually, OOM kill
108. Garbage collection: Automatic heap management
109. Garbage collection (GC) • Cleans up heap variables that are no longer in use • Executed by runtime
110. Heap Area Stack Area Mark and sweep algorithm main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
111. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
112. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
113. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
114. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
115. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
116. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
117. Heap Area Stack Area Mark phase main() stack frame a() stack frame b() stack frame a b c e {a addr} d {b addr} {var1 addr} var1 {d addr}
118. Heap Area Stack Area main() stack frame a() stack frame b() stack frame a b e {a addr} d {b addr} {var1 addr} var1 {d addr} Sweep phase c
119. Heap Area Stack Area main() stack frame a() stack frame b() stack frame a b e {a addr} d {b addr} {var1 addr} var1 {d addr} Sweep phase
120. Very simple, but hard to optimize • Marking concurrently to other goroutines ◦ Tri-color marking algorithm etc... • Adjusting triggers • Green Tea!!!
121. Very simple, but hard to optimize • Can be optimized more with complex algorithm • Go doesn’t = philosophy of symplicity
122. 【Pain】GC is great, but expensive • Placing much data in the heap = high GC cost ◦ Especially marking
123. Most GC languages determine by type • Old Java: all objects are in heap • Python: all objects are in heap • JS: all non-primitives are in heap
124. • Old Java: all objects are in heap • Python: all objects are in heap • JS: all non-primitives are in heap Most GC languages determine by type Non-primitive types tend to be in heap!
125. Heap escape 【Go specific】
126. Heap escape • Put as much data in the stack as possible • Specific data “escapes” to the heap
127. Do we know which variable escapes?
128. by Jacob Walker Do we know which variable escapes?
129. There are some common cases... • Returning pointers • Interface-type variables • map, channel, string, most of slice
130. Only the compiler & runtime know... • Many are decided by compiler • Rest are decided by runtime
131. Heap escape analysis • Compiler analyzes variables ◦ It can output the analysis log go {run/build/test} -gcflags="-m" **.go
132. Benchmarking • Actual allocation can be checked by benchmarking go test -bench {package} -benchmem
133. Example (escape)
134. Example (escape)
135. Example (escape)
136. Example (no escape)
137. Example (no escape)
138. Example (no escape)
139. Example (no escape in log, but escape)
140. Example (no escape in log, but escape)
141. Example (no escape in log, but escape)
142. Object management in heap • Variables in heap (objects) varies... int (8 bytes) struct (dynamically sized) pointer (8 bytes)
143. … … Object management in heap • Objects of the same size are grouped into span int (8 bytes) struct (8 bytes) struct struct (32 bytes) struct struct int int span 1 (for 8 bytes) span 2 (for 8 bytes) span 3 (for 32 bytes) int int
144. Object management in heap Spans are grouped into larger units • mcentral ↓ • arena ↓ • mheap
145. Zero allocation • Zero allocation = no heap escape ◦ Allocates data to stack; it doesn’t mean “zero”
146. Zero allocation libraries zerolog: zero allocation logging
147. Zero allocation libraries fasthttp: (partially) zero allocation http server
148. Zero allocation libraries go-reflect: zero allocation reflection
149. Zero allocation isn't a silver bullet... • No heap = involves argument copying ◦ Big copy -> huge calculation • Zero allocation isn't justice ◦ Measure the effect with benchmark main() • call on line 3 Var foo() • call on line 3 Var bar() • call on line 3 Var
150. Understand heap escape and utilize zero allocation libraries wisely!
151. Stack overflow • Language decides border between heap and stack ◦ e.g., in C on Linux, stack size is 8MB . . . . . Heap var main() stack frame
152. Stack overflow • Going over the stack limit with nested function
153. Stack overflow • Going over the stack limit with nested function → Stack overflow!!
154. 【Pain】Stack may explode... • Heap escaping increases stack usage ◦ Makes stack overflow more likely
155. 【Pain】Heap also may explode... • Preparing a huge stack area sounds good... • More stack area = Less heap area ◦ ✅ Fewer stack overflows ◦ ❌ Higher risk of OOM kill 😭 . . . . . Heap var main() stack frame 󰷺
156. Flexible stack 【Go specific】
157. Go has flexible stack • Go creates a stack for each goroutine • Each stack grows and shrinks on demand ◦ Initialized with 4kB
158. Rough idea main goroutine main() stack frame a() stack frame a2() stack frame a3() stack frame a4() stack frame m2() stack frame b() stack frame a goroutine b goroutine
159. How is it implemented? • Originally, Go used segmented stacks • Now, Go uses stack copying (optimized way)
160. Stack copying . . . main() stack frame m2() stack frame main goroutine’s stack
161. Stack copying . . . main() stack frame m2() stack frame m3() stack frame
162. Stack copying . . . main() stack frame m2() stack frame x2 size New main goroutine’s stack
163. Stack copying . . . main() stack frame m2() stack frame main() stack frame m2() stack frame
164. Stack copying . . . main() stack frame m2() stack frame
165. Stack copying . . . main() stack frame m2() stack frame m3() stack frame Finally able to execute m3()
166. Stack copying . . . main() stack frame m2() stack frame
167. Stack copying . . . main() stack frame m2() stack frame
168. Flexible stack → No stack overflow?
169. No stack overflow? Stack overflow still exists in Go!
170. Maximum stack size • Stack of each goroutine has limit of 1GB (default) ◦ If stack exceeds limit, stack overflow happends
171. Modifying the max size • Override with runtime/debug.SetMaxStack()
172. Flexible size stack per goroutine makes stack in Go special! This is what makes goroutine special, too
173. Explanation of the example case
174. 【Reminder】Example case https://developers.cyberagent.co.jp/blog/archives/54653/ Single memory allocation change -> 57% lower CPU, 99% lower memory!!
175. Bad performance: too many escaping • Periodical spike in CPU consumption • App seems to have many short-living heap objects ◦ GC likely to be triggered many times • Bottleneck: Filtering function
176. Filtering function: before
177. Filtering function: before
178. Filtering function: before
179. Filtering function: before This allocation made a lot of garbage!
180. Filtering function: after
181. Filtering function: after Reusing the slice
182. Benchmarking • Benchmarking the both funtions ◦ Using list of 1-100 as the original slice ◦ Filter: is number even
183. x2 faster! Benchmarking • Benchmarking the both funtions ◦ Using list of 1-100 as the original slice ◦ Filter: is number even
184. x2 faster! Benchmarking • Benchmarking the both funtions ◦ Using list of 1-100 as the original slice ◦ Filter: is number even Caused by only a single allocation
185. Further refactoring (article) • Using iterator makes the logic safer and simpler
186. As a result... • As the whole workload ◦ 57% lower CPU!! ◦ 99% lower memory!!
187. Conclusion
188. Conclusion • Memory management has been developed to satisfy programmers’ needs • Knowledge about memory management is power • This session is just a start ◦ Let’s dive deeply into each concept!
189. Appendix: What to see next?
190. Garbage collection logics Garbage Collection Semantics @ GopherCon SG 2019
191. Heap escape condition and analysis Escape Analysis in Go: Understanding and Optimizing Memory Allocation @ Go Conference 2023 Online
192. Memory optimizations with pprof Memory Management in Go: The good, the bad and the ugly @ GopherCon UK 2023
193. How Go allocation works with OS Understanding Go's Memory Allocator @ GopherCon UK 2018 ⚠Advanced
194. Memory design over concurrency The Go Memory Model @ GoSF Meetup
195. Thank you! Let’s have a nice conference!