Untangling the web: Memory management in Chrome’s web platform Michael Lippautz Google Dynamic Languages Symposium Keynote Athens, Greece, Oct, 2019 Parts of the JavaScript object graph when running TypeScript benchmark from the Octane benchmark suite 

Google Thanks to the >2B Chrome users out there for keeping us busy with interesting problems to solve The work presented was done in collaboration with multiple Chromium teams and specifically Ulan Degenbaev and Hannes Payer 

Google What we work on Garbage collection and memory management in Chrome’s rendering engine Blink and its JavaScript virtual machine V8 

Google Rule of thumb: Ship incrementally where possible ● Chrome is moving fast. No branches as they would hard to merge. ● Develop on master with keeping backwards compatibility ○ E.g., have temporary C++ types during transition from reference counting to tracing GC ■ RawPtrWillBeMember / RefPtrWillBeRawPtr ■ WillBeRefCountedGarbageCollected ● Switch features one-by-one providing immediate benefit ○ E.g., concurrent marking in V8 was enabled with bailouts for scenarios that wouldn’t work yet 

Google Rule of thumb: Ship incrementally where possible “Changing the engine of a F1 car during the race” 

Google We publish as well Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102. Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Garbage collection as a joint venture. Commun. ACM 62, 6 (May 2019), 36-41. Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages. Ulan Degenbaev, Jochen Eisinger, Manfred Ernst, Ross McIlroy, and Hannes Payer. 2016. Idle time garbage collection scheduling. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 570-583. Daniel Clifford, Hannes Payer, Michael Stanton, and Ben L. Titzer. 2015. Memento mori: dynamic allocation-site-based optimizations. In Proceedings of the 2015 International Symposium on Memory Management (ISMM '15). ACM, New York, NY, USA, 105-117. Daniel Clifford, Hannes Payer, Michael Starzinger, and Ben L. Titzer. 2014. Allocation folding based on dominance. In Proceedings of the 2014 international symposium on Memory management (ISMM '14). ACM, New York, NY, USA, 15-24. 

Google Talk covers most recent work Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102. Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Garbage collection as a joint venture. Commun. ACM 62, 6 (May 2019), 36-41. Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages. Ulan Degenbaev, Jochen Eisinger, Manfred Ernst, Ross McIlroy, and Hannes Payer. 2016. Idle time garbage collection scheduling. In Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '16). ACM, New York, NY, USA, 570-583. Daniel Clifford, Hannes Payer, Michael Stanton, and Ben L. Titzer. 2015. Memento mori: dynamic allocation-site-based optimizations. In Proceedings of the 2015 International Symposium on Memory Management (ISMM '15). ACM, New York, NY, USA, 105-117. Daniel Clifford, Hannes Payer, Michael Starzinger, and Ben L. Titzer. 2014. Allocation folding based on dominance. In Proceedings of the 2014 international symposium on Memory management (ISMM '14). ACM, New York, NY, USA, 15-24. 

Google Chrome architecture Browser mojo IPC separate process 

Google Chrome architecture Blink (Renderer) Browser mojo IPC 

Google Blink (Renderer) Blink (Renderer) Chrome architecture Browser mojo IPC data flow 

Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 

Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust 

Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust changing state JavaScript / C++ rendering state C++ 

Google Interlude: Rendering 101 input events, callbacks, script style layout compositor setup paint raster compositor vsync vsync 16ms or bust changing state JavaScript / C++ rendering state C++ interaction between dynamic and static language 

Google Changing state: DOM ● Document Object Model (DOM) ● Cross-platform language-independent representation of HTML ● Web Interface Definition Language (IDL) ● Encoded in C++ and picked up by the rendering pipeline Photo: Birger Eriksson / CC-BY-SA-3.0 

Google Changing state: JavaScript ● Lingua franca of the web ● Used to interact with the DOM ● Untrusted code executed in an isolated environment, the virtual machine of a browser function createDiv() { let newDiv = document.createElement("div"); document.body.appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); 

Google other Chrome architecture other V8 (JS VM) other Blink (Renderer) V8 (JS VM) Browser Blink (Renderer) mojo IPC 

Google V8 Changing state Blink 

Google V8 Changing state ● Bindings layer glues JS to C++ ● Hard API boundary to V8 Bindings 

Google Problem ● Software split in components with hard boundaries ● Ways to compose arbitrary object graphs across components ● No static ownership of memory ● Potentially differently managed environments A B object cross-component reference component boundary, e.g. API 

Google Problem ● Software split in components with hard boundaries ● Ways to compose arbitrary object graphs across components ● No static ownership of memory ● Potentially differently managed environments A B root Unknown references create roots into environment 

Google Problem ● Software split in components with hard boundaries ● Ways to compose arbitrary object graphs across components ● No static ownership of memory ● Potentially differently managed environments A B root Unknown references create roots into environment 

Google Problem ● Software split in components with hard boundaries ● Ways to compose arbitrary object graphs across components ● No static ownership of memory ● Potentially differently managed environments A B c.f. reference counting cycles Unknown references create roots into environment 

Google The bond ● JavaScript ⇔ DOM document; V8 Blink 

Google The bond ● JavaScript ⇔ DOM ● Objects come in halves document; document blink::HTMLDocument V8 Blink for JS, e.g. properties, elements for DOM, e.g. addEventListener 

Google The bond ● JavaScript ⇔ DOM ● Objects come in halves ● Reference each other document; document blink::HTMLDocument V8 Blink 

Google The bond ● JavaScript ⇔ DOM ● Objects come in halves ● Reference each other document; document blink::HTMLDocument V8 Blink 

Google Is this a problem in practice? 

Google Example Mixing server-side and client-side rendering function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink blink::HTMLDocument 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink blink::HTMLDocument Code (createDiv) 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Code (createDiv) 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener Code (createDiv) closure 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement Code (createDiv) closure 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement Code (createDiv) closure 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) newDiv closure "div” 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google Example function createDiv() { let newDiv = document.createElement("div"); document.body .appendChild(newDiv); } document.addEventListener( "DOMContentLoaded", createDiv); V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body closure "div” newDiv 

Google Modern frameworks 

Google Cross-component garbage collection Ulan Degenbaev, Jochen Eisinger, Kentaro Hara, Marcel Hlopko, Michael Lippautz, and Hannes Payer. 2018. Cross-Component Garbage Collection. Proc. ACM Program. Lang. 2, OOPSLA, Article 151 (November 2018), 24 pages. 

Google Cross-component garbage collection (CC GC) Effective, efficient, and safe garbage collection across component boundaries ● Tracing garbage collectors in V8 and Blink for JS and C++, respectively ● Renderer garbage collections ○ Start at C++ and JS root sets ○ At component boundary: Delegate processing of object to specialized GC ● Allows specialized GCs to have different optimizations as long as they use tracing and rely on same invariants 

Google Interlude: Tri-color marking ● white: not discovered yet ● grey: discovered, pushed onto the worklist ● black: fully processed roots 

Google CC GC ● Start at roots V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC ● Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC ● Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC ● Continue in components where there’s work V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC ● Continue in components where there’s work ● Fixed-point over “special objects”, e.g. ephemerons V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” 

Google CC GC ● Start at roots ● Delegate processing to specialized GC ● Continue in components where there’s work ● Fixed-point over “special objects”, e.g. ephemerons ● Delegate reclamation to specialized GC V8 Blink document blink::HTMLDocument Hashmap "DOMContentLoaded” blink::EventListener blink::HTMLBodyElement blink::HTMLSpanElement blink::HTMLDivElement Code (createDiv) body newDiv closure "div” reclaim 

Google Tooling ● Basic capabilities in both worlds: ○ Objects ■ Identity ■ Naming ○ Edges 

Google Tooling ● Basic capabilities in both worlds: ○ Objects ■ Identity ■ Naming ○ Edges C++ JS JS 

Google Cross-component GC V8 and Blink: Garbage collection architecture V8 Generational ● Young: Parallel scavenge ● Old: Mark-Sweep-Compact Mark-Sweep-Compact ● Concurrent marking ● Concurrent sweeping ● Parallel compaction in atomic pause Some optimizations ● Idle-time garbage collection ● Allocation folding/pretenuring ● Shape changes ● ... Blink Non-generational ● Mark-Sweep-Some-Compact Mark-Sweep-Some-Compact ● Incremental marking ● Concurrent sweeping ● Sequential sliding window compaction for backing stores in atomic pause Some optimizations ● Idle-time garbage collection ● Promptly freeing of backing stores ● … 

Google Concurrent marking of shape-changing objects Ulan Degenbaev, Michael Lippautz, and Hannes Payer. 2019. Concurrent marking of shape-changing objects. In Proceedings of the 2019 ACM SIGPLAN International Symposium on Memory Management (ISMM 2019). ACM, New York, NY, USA, 89-102. 

Google Concurrent marking of shape-changing objects Motivation ● JavaScript becomes more and more popular ○ O(GiB) live memory ○ O(s) marking time on the main thread ● Even low end devices come with >1 core these days Problem ● Traditional literature does not cover shape changing objects 

Google V8’s object model header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1: header ● type ● size ● layout HiddenClass 0: header ● type ● size ● layout MetaHiddenClass 0: 

Google V8’s object model: Field types header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1: 

Google V8’s object model: Field types header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged <= tagged <= tagged Field types: ● tagged: ○ references ○ unboxed integers ● untagged: raw bytes 

Google V8’s object model: Value tagging header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged 0 0 1 0 0x123 0 1 2 3 other bits 1 0 1 0 0x123 <= integer <= reference tag bit 

Google V8’s object model: Value tagging header integer raw bytes ... reference Object 0: 1: 2: ... n + 1: <= untagged 0 0 1 0 0x123 0 1 2 3 other bits 1 0 1 0 0x123 <= small integer <= reference tag bit (SMI) 

Google V8’s object model: Field types header word 1 word 2 ... word n Object 0: 1: 2: ... n + 1: header ● type ● size ● layout HiddenClass 0: + is tagged? yes ● reference ● SMI no ● raw bytes 

Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; Simplified example. Requires more JavaScript voodoo in practice. 

Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass1 out-of-object backing stores space for in-object properties 

Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: reference 4: header 3.14 0: 1: HiddenClass2 HiddenClass1 

Google Runtime optimization: Double unboxing let point = {}; point.x = 3.14; header properties elements slack area JSObject 0: 1: 2: 3: 3.14 4: header 3.14 0: 1: HiddenClass2 HiddenClass1 HiddenClass3 

Google Double unboxing changes the shape of an object header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass3 3.14 4: header 3.14 0: 1: header properties elements slack area JSObject 0: 1: 2: 3: HiddenClass2 reference 4: 

Google Traditional marking round Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v) 

Google Traditional marking round Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v) FollowReference(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist 

Google Traditional marking round The main invariant (strong tri-color): no black to white references maintained in V8 and Blink by a write-barrier (Dijkstra): // After store v ↦ obj[x] WriteBarrier(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist 

Google Safe and unsafe shape changes ● There exist safe shape changes that traditional marking can handle ○ E.g.: Appending a field ● Unsafe shape changes require special handling ○ E.g.: unboxed doubles Invariant (reference safety) The value passed to FollowReference() is tagged Invariant (worklist items) Each worklist item is a reference to an object More unsafe shape changes and invariants in the paper 

Google What can go wrong with double unboxing Visit(obj): store black ↦ Markbits(obj) load class ↤ obj[0] for f in TaggedFields(obj, class): load v ↤ obj[f] FollowReference(v) FollowReference(v): if HasRefTag(v) and CAS (white ↦ gray) ↦ Markbits(v): push v ↦ Worklist 

Google Lock-based algorithm Mutator: UnsafeChange_L(obj, ...): lock(obj) UnsafeChange(obj, ...) unlock(obj) Concurrent marker: VisitObject_L(obj): lock(obj) VisitObject(obj) unlock(obj) 

Google Wait-free snapshotting algorithm Mutator: UnsafeChange_S(obj, ...): CAS (white ↦ gray) ↦ Markbits(obj) if CAS (gray ↦ black) ↦ Markbits(obj): VisitObject(obj) UnsafeChange(obj, ...) Concurrent marker: VisitObject_S(obj): snapshot = TakeSnapshot(obj) if CAS (gray ↦ black) ↦ Markbits(obj): for v in snapshot: FollowReference(v) 

Google Evaluation ● Chrome version M72 (Oct 2018) ● Chrome’s Catapult benchmarking framework to measure on real webpages ● Configurations: ○ Baseline: Incremental marking ○ Concurrent marking without unsafe shape changes (N) ○ Concurrent marking with snapshotting (S) ○ Concurrent marking with per-word locking (LW) ○ Concurrent marking with per-page locking (LP) Double unboxing is unsafe 

Google Evaluation: Marking time on main thread 

Google Evaluation: Total marking time (all threads) 

Google Concurrent marking of shape-changing objects ● Shape changes may appear as performance optimizations in sequential VMs without concurrent GC ● There exist safe and unsafe shape changes (see paper) ● Safe shape changes can be handled by traditional marking algorithms ● Unsafe shape changes require special handling ○ E.g., locking, snapshotting 

Google Final thoughts Binding different environments together can be a real problem in practice (see all web browsers) ○ For Chrome: CC-GC was a viable option but restricting references across components may provide better tradeoffs for other systems Introducing concurrent GC earlier in the design of the VM may prevent complexity ○ For Chrome: User base is large and maintaining optimizations for peak performance is required 

Untangling the web: Memory management in Chrome’s web platform Michael Lippautz Google [email protected] https://research.google.com/pubs/MichaelLippautz.html Thanks / Q&A