Untangling the web: Memory management in Chrome’s web platform implementation

Major browsers these days are composed of a myriad of different internal or external components or libraries that provide capabilities to their respective web platform implementation. Chrome is no different here as requests for presenting a webpage are forwarded to the Blink rendering engine. Blink maintains an implementation of the document object model (DOM) that represents a webpage and delegates specific tasks to subcomponents, the building blocks of the platform. E.g., execution of JavaScript (JS), often used to manipulate the DOM implemented in Blink, is carried out by the high-performance virtual machine V8. Similarly, Blink delegates to other subcomponents for styling, layouting, painting, and compositing of a webpage. Unsurprisingly, many of these building blocks come with their own customized memory management resulting in a potpourri of constraints, APIs, and interactions. This talk will provide a high-level overview of the memory architecture used to manage the DOM and JS, including the different object models and cherry-pick specific challenges that arise when multiple garbage collectors collaborate on providing effective, efficient, and safe reclamation of unused memory in such a heterogeneous environment.

Transcript

1. Untangling the web: Memory management in Chrome’s web platform Michael Lippautz Google Dynamic Languages Symposium Keynote Athens, Greece, Oct, 2019 Parts of the JavaScript object graph when running TypeScript benchmark from the Octane benchmark suite
2. Google Thanks to the >2B Chrome users out there for keeping us busy with interesting problems to solve The work presented was done in collaboration with multiple Chromium teams and specifically Ulan Degenbaev and Hannes Payer
3. Google What we work on Garbage collection and memory management in Chrome’s rendering engine Blink and its JavaScript virtual machine V8
4. Google Rule of thumb: Ship incrementally where possible • Chrome is moving fast. No branches as they would hard to merge. • Develop on master with keeping backwards compatibility ◦ E.g., have temporary C++ types during transition from reference counting to tracing GC ▪ RawPtrWillBeMember / RefPtrWillBeRawPtr ▪ WillBeRefCountedGarbageCollected • Switch features one-by-one providing immediate benefit ◦ E.g., concurrent marking in V8 was enabled with bailouts for scenarios that wouldn’t work yet
5. Google Rule of thumb: Ship incrementally where possible “Changing the engine of a F1 car during the race”
6. Google We publish as well Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102. Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Garbage collection as a joint venture. Commun. ACM 62, 6 (May 2019), 36-41. Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages. Ulan Degenbaev, Jochen Eisinger, Manfred Ernst, Ross McIlroy, and Hannes Payer. 2016. Idle time garbage collection scheduling. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 570-583. Daniel Clifford, Hannes Payer, Michael Stanton, and Ben L. Titzer. 2015. Memento mori: dynamic allocation-site-based optimizations. In Proceedings of the 2015 International Symposium on Memory Management (ISMM '15). ACM, New York, NY, USA, 105-117. Daniel Clifford, Hannes Payer, Michael Starzinger, and Ben L. Titzer. 2014. Allocation folding based on dominance. In Proceedings of the 2014 international symposium on Memory management (ISMM '14). ACM, New York, NY, USA, 15-24.
7. Google Talk covers most recent work Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102. Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Garbage collection as a joint venture. Commun. ACM 62, 6 (May 2019), 36-41. Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages. Ulan Degenbaev, Jochen Eisinger, Manfred Ernst, Ross McIlroy, and Hannes Payer. 2016. Idle time garbage collection scheduling. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 570-583. Daniel Clifford, Hannes Payer, Michael Stanton, and Ben L. Titzer. 2015. Memento mori: dynamic allocation-site-based optimizations. In Proceedings of the 2015 International Symposium on Memory Management (ISMM '15). ACM, New York, NY, USA, 105-117. Daniel Clifford, Hannes Payer, Michael Starzinger, and Ben L. Titzer. 2014. Allocation folding based on dominance. In Proceedings of the 2014 international symposium on Memory management (ISMM '14). ACM, New York, NY, USA, 15-24.
8. Google Chrome architecture Browser mojo IPC separate process
9. Google Chrome architecture Blink (Renderer) Browser mojo IPC
10. Google Blink (Renderer) Blink (Renderer) Chrome architecture Browser mojo IPC data flow
11. Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync
12. Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust
13. Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust changing state JavaScript / C++ rendering state C++
14. Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust changing state JavaScript / C++ rendering state C++ interaction between dynamic and static language
15. Google Changing state: DOM • Document Object Model (DOM) • Cross-platform language-independent representation of HTML • Web Interface Definition Language (IDL) • Encoded in C++ and picked up by the rendering pipeline Photo: Birger Eriksson / CC-BY-SA-3.0
16. Google Changing state: JavaScript • Lingua franca of the web • Used to interact with the DOM • Untrusted code executed in an isolated environment, the virtual machine of a browser <script> function createDiv() { let newDiv = document.createElement("div"); document.body.appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script>
17. Google other Chrome architecture other V8 (JS VM) other Blink (Renderer) V8 (JS VM) Browser Blink (Renderer) mojo IPC
18. Google V8 Changing state Blink
19. Google V8 Changing state • Bindings layer glues JS to C++ • Hard API boundary to V8 Bindings
20. Google Problem • Software split in components with hard boundaries • Ways to compose arbitrary object graphs across components • No static ownership of memory • Potentially differently managed environments A B object cross-component reference component boundary, e.g. API
21. Google Problem • Software split in components with hard boundaries • Ways to compose arbitrary object graphs across components • No static ownership of memory • Potentially differently managed environments A B root Unknown references create roots into environment
22. Google Problem • Software split in components with hard boundaries • Ways to compose arbitrary object graphs across components • No static ownership of memory • Potentially differently managed environments A B root Unknown references create roots into environment
23. Google Problem • Software split in components with hard boundaries • Ways to compose arbitrary object graphs across components • No static ownership of memory • Potentially differently managed environments A B c.f. reference counting cycles Unknown references create roots into environment
24. Google The bond • JavaScript ⇔ DOM <script> document; </script> V8 Blink
25. Google The bond • JavaScript ⇔ DOM • Objects come in halves <script> document; </script> document blink::HTMLDocument V8 Blink for JS, e.g. properties, elements for DOM, e.g. addEventListener
26. Google The bond • JavaScript ⇔ DOM • Objects come in halves • Reference each other <script> document; </script> document blink::HTMLDocument V8 Blink
27. Google The bond • JavaScript ⇔ DOM • Objects come in halves • Reference each other <script> document; </script> document blink::HTMLDocument V8 Blink
28. Google Is this a problem in practice?
29. Google Example Mixing server-side and client-side rendering <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html>
30. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink
31. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink blink::HTMLDocument
32. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink blink::HTMLDocument Code (createDiv)
33. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Code (createDiv)
34. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener Code (createDiv) closure
35. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement Code (createDiv) closure
36. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement Code (createDiv) closure
37. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) newDiv closure "div”
38. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
39. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
40. Google Example <!DOCTYPE html> <head><script> function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); </script></head> <body> <span></span> </body> </html> V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body closure "div” newDiv
41. Google Modern frameworks
42. Google Cross-component garbage collection Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages.
43. Google Cross-component garbage collection (CC GC) Effective, efficient, and safe garbage collection across component boundaries • Tracing garbage collectors in V8 and Blink for JS and C++, respectively • Renderer garbage collections ◦ Start at C++ and JS root sets ◦ At component boundary: Delegate processing of object to specialized GC • Allows specialized GCs to have different optimizations as long as they use tracing and rely on same invariants
44. Google Interlude: Tri-color marking • white: not discovered yet • grey: discovered, pushed onto the worklist • black: fully processed roots
45. Google CC GC • Start at roots V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
46. Google CC GC • Start at roots • Delegate processing to specialized GC V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
47. Google CC GC • Start at roots • Delegate processing to specialized GC • Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
48. Google CC GC • Start at roots • Delegate processing to specialized GC • Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
49. Google CC GC • Start at roots • Delegate processing to specialized GC • Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
50. Google CC GC • Start at roots • Delegate processing to specialized GC • Continue in components where there’s work • Fixed-point over “special objects”, e.g. ephemerons V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div”
51. Google CC GC • Start at roots • Delegate processing to specialized GC • Continue in components where there’s work • Fixed-point over “special objects”, e.g. ephemerons • Delegate reclamation to specialized GC V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” reclaim
52. Google Tooling • Basic capabilities in both worlds: ◦ Objects ▪ Identity ▪ Naming ◦ Edges
53. Google Tooling • Basic capabilities in both worlds: ◦ Objects ▪ Identity ▪ Naming ◦ Edges C++ JS JS
54. Google Cross-component GC V8 and Blink: Garbage collection architecture V8 Generational • Young: Parallel scavenge • Old: Mark-Sweep-Compact Mark-Sweep-Compact • Concurrent marking • Concurrent sweeping • Parallel compaction in atomic pause Some optimizations • Idle-time garbage collection • Allocation folding/pretenuring • Shape changes • ... Blink Non-generational • Mark-Sweep-Some-Compact Mark-Sweep-Some-Compact • Incremental marking • Concurrent sweeping • Sequential sliding window compaction for backing stores in atomic pause Some optimizations • Idle-time garbage collection • Promptly freeing of backing stores • …
55. Google Concurrent marking of shape-changing objects Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102.
56. Google Concurrent marking of shape-changing objects Motivation • JavaScript becomes more and more popular ◦ O(GiB) live memory ◦ O(s) marking time on the main thread • Even low end devices come with >1 core these days Problem • Traditional literature does not cover shape changing objects
57. Google V8’s object model header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1: header • type • size • layout HiddenClass 0: header • type • size • layout MetaHiddenClass 0:
58. Google V8’s object model: Field types header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1:
59. Google V8’s object model: Field types header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged <= tagged <= tagged Field types: • tagged: ◦ references ◦ unboxed integers • untagged: raw bytes
60. Google V8’s object model: Value tagging header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged 0 0 1 0 0x123 0 1 2 3 other bits 1 0 1 0 0x123 <= integer <= reference tag bit
61. Google V8’s object model: Value tagging header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged 0 0 1 0 0x123 0 1 2 3 other bits 1 0 1 0 0x123 <= small integer <= reference tag bit (SMI)
62. Google V8’s object model: Field types header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1: header • type • size • layout HiddenClass 0: + is tagged? yes • reference • SMI no • raw bytes
63. Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; Simplified example. Requires more JavaScript voodoo in practice.
64. Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass1 out-of-object backing stores space for in-object properties
65. Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: reference 4: header 3.14 0: 1: HiddenClass2 HiddenClass1
66. Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: 3.14 4: header 3.14 0: 1: HiddenClass2 HiddenClass1 HiddenClass3
67. Google Double unboxing changes the shape of an object header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass3 3.14 4: header 3.14 0: 1: header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass2 reference 4:
68. Google Traditional marking round Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v)
69. Google Traditional marking round Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v) FollowReference(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist
70. Google Traditional marking round The main invariant (strong tri-color): no black to white references maintained in V8 and Blink by a write-barrier (Dijkstra): // After store v ↦ obj[x] WriteBarrier(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist
71. Google Safe and unsafe shape changes • There exist safe shape changes that traditional marking can handle ◦ E.g.: Appending a field • Unsafe shape changes require special handling ◦ E.g.: unboxed doubles Invariant (reference safety) The value passed to FollowReference() is tagged Invariant (worklist items) Each worklist item is a reference to an object More unsafe shape changes and invariants in the paper
72. Google What can go wrong with double unboxing Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v) FollowReference(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist
73. Google Lock-based algorithm Mutator: UnsafeChange_L(obj, ...): lock(obj) UnsafeChange(obj, ...) unlock(obj) Concurrent marker: VisitObject_L(obj): lock(obj) VisitObject(obj) unlock(obj)
74. Google Wait-free snapshotting algorithm Mutator: UnsafeChange_S(obj, ...): CAS (white ↦ gray) ↦ Markbits(obj) if CAS (gray ↦ black) ↦ Markbits(obj): VisitObject(obj) UnsafeChange(obj, ...) Concurrent marker: VisitObject_S(obj): snapshot = TakeSnapshot(obj) if CAS (gray ↦ black) ↦ Markbits(obj): for v in snapshot: FollowReference(v)
75. Google Evaluation • Chrome version M72 (Oct 2018) • Chrome’s Catapult benchmarking framework to measure on real webpages • Configurations: ◦ Baseline: Incremental marking ◦ Concurrent marking without unsafe shape changes (N) ◦ Concurrent marking with snapshotting (S) ◦ Concurrent marking with per-word locking (LW) ◦ Concurrent marking with per-page locking (LP) Double unboxing is unsafe
76. Google Evaluation: Marking time on main thread
77. Google Evaluation: Total marking time (all threads)
78. Google Concurrent marking of shape-changing objects • Shape changes may appear as performance optimizations in sequential VMs without concurrent GC • There exist safe and unsafe shape changes (see paper) • Safe shape changes can be handled by traditional marking algorithms • Unsafe shape changes require special handling ◦ E.g., locking, snapshotting
79. Google Final thoughts Binding different environments together can be a real problem in practice (see all web browsers) ◦ For Chrome: CC-GC was a viable option but restricting references across components may provide better tradeoffs for other systems Introducing concurrent GC earlier in the design of the VM may prevent complexity ◦ For Chrome: User base is large and maintaining optimizations for peak performance is required
80. Untangling the web: Memory management in Chrome’s web platform Michael Lippautz Google mlippautz@google.com https://research.google.com/pubs/MichaelLippautz.html Thanks / Q&A