Code Caching for JS developers

Video: https://skillsmatter.com/skillscasts/12178-code-caching-for-js-developers

Everybody knows that the fastest way to do something is to not do it at all, and V8 is no exception. After all, why parse and compile JavaScript that we’ve already parsed and compiled before?

But the caching behaviour of a JavaScript engine can be a bit of a mystery. What even gets cached, and why? Who decides when it is cached and where is it stored — and how many caches even are there? What if I use a service worker?

This talk will demystify code caching in V8, describing how code caching interacts with the disk, how in-memory code is re-used, how compiled scripts are serialized, and most importantly, how JS developers can benefit from these optimizations.

Transcript

1. Code caching for JS Developers Leszek Swirski – V8, Google @leszekswirski
2. <script src="script.js">
3. Parse/Compile source code script <script src="script.js">
4. Parse/Compile source code Execute script <script src="script.js">
7. V8 Isolate Cache Chrome Resource Cache Code caching = +
8. V8 Isolate Cache Chrome Resource Cache Code caching = +
9. Parse/Compile source code Execute V8 Isolate Cache script <script src="script.js">
10. Parse/Compile source code Execute V8 Isolate Cache script <script src="script.js"> Cold run ❄
11. 0011010 0100110 1100101 Parse/Compile source code Execute V8 Isolate Cache script <script src="script.js"> Cold run ❄
12. 0011010 0100110 1100101 Parse/Compile source code Execute script V8 Isolate Cache <script src="script.js"> Cold run ❄
13. 0011010 0100110 1100101 Parse/Compile source code Execute script V8 Isolate Cache Isolate cache source code <script src="script.js"> Cold run ❄
14. 0011010 0100110 1100101 Parse/Compile source code Execute script V8 Isolate Cache Isolate cache source code Cold run ❄ <script src="script.js">
15. 0011010 0100110 1100101 Parse/Compile source code V8 Isolate Cache Isolate cache source code <script src="script.js"> Warm run ☀
16. 0011010 0100110 1100101 Parse/Compile source code V8 Isolate Cache Isolate cache source code <script src="script.js"> Warm run ☀
17. 0011010 0100110 1100101 source code script V8 Isolate Cache Isolate cache source code Lookup <script src="script.js"> Warm run ☀
18. 0011010 0100110 1100101 Parse/Compile source code script <script src="script.js"> V8 Isolate Cache Isolate cache source code Execute Warm run ☀
19. 0011010 0100110 1100101 source code script <script src="script.js"> V8 Isolate Cache Isolate cache source code Execute Cache hit rate: 80% Lookup Warm run ☀
20. V8 Isolate Cache Isolate cache source code source code 0011010 0100110 1100101 Isolate cache source code source code <script src="script.js"> <script src="script.js">
21. V8 Isolate Cache Isolate cache source code source code 0011010 0100110 1100101 Isolate cache source code source code script <script src="script.js"> <script src="script.js"> Lookup
22. V8 Isolate Cache Isolate cache source code Parse/Compile source code 0011010 0100110 1100101 Isolate cache source code source code script script <script src="script.js"> <script src="script.js"> Lookup
23. V8 Isolate Cache Chrome Resource Cache Code caching = +
24. <script src="script.js"> Chrome Resource Cache Resource cache URL Metadata Response
25. <script src="script.js"> Chrome Resource Cache Resource cache URL Metadata Response script.js [data] Cold run ❄
26. <script src="script.js"> Chrome Resource Cache Resource cache URL Metadata Response script.js [data] Warm run ☀
27. Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js">
28. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> Cold run ❄
29. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Cold run ❄
30. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Timestamp Cold run ❄
31. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Timestamp Warm run ☀
32. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Timestamp Warm run ☀
33. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Timestamp Warm run ☀ ≥72 hours
34. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> script.js [data] Timestamp Warm run ☀ ≥72 hours
35. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] <72 hours Timestamp
36. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response Serialize <script src="script.js"> Warm run ☀ script.js [data] <72 hours Timestamp
37. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response Serialize <script src="script.js"> Warm run ☀ script.js [data] <72 hours Timestamp
38. Parse/Compile source code Execute Chrome Resource Cache Resource cache URL Metadata Response Serialize <script src="script.js"> Warm run ☀ script.js [data] <72 hours Code cache
39. Chrome Resource Cache Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache
40. Chrome Resource Cache Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache
41. code cache Chrome Resource Cache Resource cache URL Metadata Response Deserialize Hot run <script src="script.js"> script.js [data] Code cache
42. code cache Chrome Resource Cache Resource cache URL Metadata Response Deserialize Hot run <script src="script.js"> script.js [data] Code cache
43. code cache Execute Chrome Resource Cache Resource cache URL Metadata Response Deserialize Hot run <script src="script.js"> script.js [data] Code cache
44. code cache Execute Deserialize <script src="script.js"> Chrome Resource Cache Hot run Resource cache URL Metadata Response script.js [data] Code cache Cache hit rate: 80%*
45. code cache Execute Deserialize <script src="script.js"> Chrome Resource Cache Resource cache URL Metadata Response Hot run script.js [data] Code cache Cache hit rate: 80%* * of cacheable external scripts
46. Code cache Cache Chrome Resource Cache Network cache URL Response Hot run <script src="script.js"> script.js [data] URL script.js Code cache
47. Code cache Cache Chrome Resource Cache Network cache URL Response Hot run <script src="script.js"> script.js [data] URL script.js Code cache
48. Code cache Cache Chrome Resource Cache Network cache URL Response Hot run <script src="script.js"> script.js [data] URL script.js Code cache
49. Code caching Cold run ❄ Warm run ☀ Hot runs Use isolate cache Compile Compile + Serialize Deserialize In-memory (same tab*) On-disk * actually “same V8 isolate”, which means same renderer process and same thread
50. Code caching? Cold run ❄ Warm run ☀ Hot runs Use isolate cache Compile + Serialize? Deserialize? In-memory (same tab*) On-disk * actually “same V8 isolate”, which means same renderer process and same thread
52. Code caching for JS Developers
53. What should JS developers do?
54. Nothing.
56. Nothing.
57. Nothing. don’t change code
58. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache
59. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache HTTP GET script.js
60. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache HTTP 304 Not Modified
61. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache HTTP GET script.js
62. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache HTTP 200 [data2]
63. don’t change code Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data2] Code cache HTTP 200 [data2]
64. don’t change code Resource cache URL Metadata Response <script src="script.js"> script.js [data2] HTTP 200 [data2] Cold run ❄
65. Nothing. don’t change code don’t change URLs
66. don’t change URLs Resource cache URL Metadata Response Hot run <script src="script.js"> script.js [data] Code cache
67. don’t change URLs Resource cache URL Metadata Response Hot run <script src="script.js?my_cool_param=1"> script.js [data] Code cache
68. don’t change URLs Resource cache URL Metadata Response Hot run <script src="script.js?my_cool_param=1"> script.js [data] Code cache Cold run ❄ script.js?… [data]
69. Nothing. don’t change code don’t change URLs don’t change behaviour
70. Parse/Compile source code Execute don’t change behaviour Resource cache URL Metadata Response Serialize <script src="script.js"> Warm run ☀ script.js [data] Timestamp
71. Serialize? Execute? Parse/Compile source code don’t change behaviour Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp
72. function foo() { // Lotsa code } button.onclick = foo
73. function foo() { // Lotsa code } button.onclick = foo
74. Parse/Compile source code don’t change behaviour Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script skipped inner functions Execute Serialize
75. function foo() { // Lotsa code } foo()
76. function foo() { // Lotsa code } foo()
77. function foo() { // Lotsa code } foo()
78. Parse/Compile source code don’t change behaviour Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script skipped Execute parsing inner functions
79. Parse/Compile source code don’t change behaviour Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script skipped Execute Serialize parsing inner functions
80. Parse/Compile source code don’t change behaviour Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script skipped Serialize Execute parsing inner functions
81. code cache Execute Resource cache URL Metadata Response Deserialize Hot run <script src="script.js"> script.js [data] Code cache don’t change behaviour parsing inner functions
82. function foo() { // Lotsa code } function bar() { // Lotsa code } if (Math.random() > 0.5) { foo() } else { bar() }
83. Nothing. don’t change code don’t change URLs don’t change behaviour
84. What else should JS developers do?
85. Something.
86. function foo() { // Lotsa code } function bar() { // Lotsa code } if (random() > 0.5) { foo() } else { bar() } 10 WAYS TO INCREASE CODE CACHING NUMBER 4 WILL SHOCK YOU!
87. Something.
88. Something. avoid inline scripts
89. <!-- page1.html --> <script> function foo() { // Lotsa code } foo() </script> <!-- page2.html --> <script> function foo() { // Lotsa code } foo() </script>
90. Resource cache [data] [data] Response Metadata URL page1.html Code cache page2.html Code cache ...
91. <!-- page1.html --> <script src="foo.js"></script> <script> foo() </script> // foo.js function foo() { // Lotsa code } <!-- page2.html --> <script src="foo.js"></script> <script> foo() </script>
92. Resource cache [data] [data] Response Metadata URL page1.html page2.html foo.js [data] Code cache
93. Something. avoid inline scripts split out libraries from code
94. <!-- page1.html --> <script src="page1.js"></script> <!-- page2.html --> <script src="page2.js"></script>
95. <!-- page1.html --> <script src="page1.js"></script> <!-- page2.html --> <script src="page2.js"></script> // page1.js (function() { // lodash ... root._ = lodash() }).call(this) (function page1code(){ // ... })() // page2.js (function() { // lodash ... root._ = lodash() }).call(this) (function page2code(){ // ... })()
96. Resource cache [data] [data] Response Metadata URL page1.js Code cache page2.js Code cache
97. <!-- page1.html --> <script src="lib.js"></script> <script src="page1.js"></script> <!-- page2.html --> <script src="lib.js"></script> <script src="page2.js"></script> // page1.js (function page1code(){ // ... })() // page2.js (function page2code(){ // ... })() // lib.js (function() { // lodash ... root._ = lodash() }).call(this)
98. Resource cache [data] [data] Response Metadata URL page1.js Code cache page2.js Code cache lib.js [data] Code cache
99. <!-- index.html --> <script src="main.js"></script> // main.js (function() { // lodash ... root._ = lodash() }).call(this) (function main(){ // ... })()
100. Resource cache [data] Response Metadata URL main.js Code cache
101. Resource cache [data] Response Metadata URL main.js Code cache HTTP 200 [data2]
102. Resource cache [data2] Response Metadata URL main.js
103. <!-- index.html --> <script src="lib.js"></script> <script src="main.js"></script> // main.js (function main(){ // ... })() // lib.js (function() { // lodash ... root._ = lodash() }).call(this)
104. Resource cache [data] [data] Response Metadata URL main.js Code cache lib.js Code cache
105. Resource cache [data] [data] Response Metadata URL main.js Code cache lib.js Code cache HTTP 200 [data2]
106. Resource cache [data2] Response Metadata URL main.js [data] lib.js Code cache
107. Something. avoid inline scripts split out libraries from code group small files together
108. // foo.js (function foo(){ // 800b })() // bar.js (function bar(){ // 800b })() < 1Kb < 1Kb Dear reader, originally this slide said 1Mb. What I meant was a much more reasonable 1Kb
109. // foobar.js (function foo(){ // 800b })() (function bar(){ // 800b })() >1Kb Dear reader, originally this slide said 1Mb. What I meant was a much more reasonable 1Kb
110. Something. avoid inline scripts split out libraries from code group small files together use service worker caches
111. // main.js navigator.serviceWorker.register('/sw.js') // sw.js self.addEventListener('fetch', event=>{ async function cachedFetch(event) { const cache = await caches.open(cacheName) let response = await cache.match(event.request) if (response) return response response = await fetch(event.request) cache.put(event.request, response.clone()) return response } event.respondWith(cachedFetch(event)) }) self.addEventListener('install', event=>{ async function buildCache() { const cache = await caches.open(cacheName) return cache.addAll([ '/main.css', '/main.js', '/offline.html' ]) } event.waitUntil(buildCache()) })
112. Parse/Compile source code Service Worker cache Resource cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script skipped Execute Serialize
113. Parse/Compile source code Service Worker cache Service worker cache URL Metadata Response <script src="script.js"> Warm run ☀ script.js [data] Timestamp top-level script Execute Serialize skipped
114. Parse/Compile source code Service Worker cache Service worker cache URL Metadata Response <script src="script.js"> script.js [data] top-level script Execute Serialize skipped Cold run ❄
115. Parse/Compile source code Service Worker cache Service worker cache URL Metadata Response <script src="script.js"> script.js [data] top-level script Execute Serialize compiled Cold run ❄
116. Something. avoid inline scripts split out libraries from code group small files together use service worker caches Nothing. don’t change code don’t change URLs don’t change behaviour What should JS developers do?
117. Profiling
118. chrome://tracing
119. chrome://tracing
120. chrome://tracing
121. chrome://tracing
122. chrome://tracing
123. chrome://tracing
124. chrome://tracing v8.compile
125. chrome://tracing v8.compile
126. Cold run ❄ chrome://tracing Parse/Compile
127. Warm run ☀ chrome://tracing Parse/Compile Serialize
128. Hot run chrome://tracing Deserialize
129. Tracing is tricky trace in a new profile google-chrome --user-data-dir=$(mktemp -d) don’t have other tabs open don’t install extensions
130. What about Node?
131. 2018-07-04, Version 10.6.0 Notable changes: * build: * Node.js should now be about 60% faster to startup than the previous version, thanks to the use V8's code cache feature for core modules.
132. What about my Node modules?
133. const vm = require('vm') const script = new vm.Script(code) script.runInThisContext()
134. const vm = require('vm') const script = new vm.Script(code, { cachedData: cachedDataBuffer }) script.runInThisContext()
135. const vm = require('vm') const script = new vm.Script(code, { cachedData: cachedDataBuffer }) script.runInThisContext() const cachedDataBuffer = script.createCachedData()
136. npm install v8-compile-cache h/t Andres Suarez a.k.a. @zertosh
137. Code caching for JS Developers Leszek Swirski Cold run ❄ Warm run ☀ Hot runs Use isolate cache Compile Compile + Serialize Deserialize In-memory On-disk Nothing. don’t change code don’t change URLs don’t change behaviour Something. avoid inline scripts split out libraries from code group small files together use service worker caches Profile. chrome://tracing What should JS developers do?