Taking Out The Garbage (And How You Can Help)

The GNOME desktop is based on a Javascript engine, and there are some long-running and much-discussed problems around Javascript's garbage collection. This past year I've been working on a solution to the garbage collection problems in GJS, an initiative nicknamed "taking out the garbage". This talk will have a short technical portion about taking out the garbage, as well as some of the cool things I've learned by working more closely with Mozilla.

I will also talk about new features introduced in GJS, such as upgrading the Javascript engine to Firefox 68, and how they affect our four audiences: users, app developers, GNOME Shell developers, and shell extension developers.

Finally, there has never been a better time to get started contributing to GJS than right now, and I will talk about some projects that you can do to help make things fabulous for Javascript developers in GNOME.

Transcript

1. Taking Out The Garbage (And How You Can Help) Philip Chimento — ptomato, @therealptomato GUADEC Thessaloniki, August 25, 2019
2. Introduction — Contents of this talk • What's new in GJS?
3. Introduction — Contents of this talk • What's new in GJS? • We need some help! Here's what you can do
4. Introduction — Contents of this talk • What's new in GJS? • We need some help! Here's what you can do • Oh yeah, what's happening with that garbage collection thing everyone was talking about?
5. What's new in GJS?
6. What's new in GJS in 3.32 and 3.34? • Big Hammer • Docs site restored • Crash fixes • Gio.Settings • Sysprof • GTK 4
7. Docs site restored gjs-docs.gnome.org • Thanks to Meg Ford for moving this to GNOME infrastructure • Primarily useful for app developers • We added Mutter docs, when St gets fixed it should become more useful for shell developers and shell extension developers as well
8. Crash fixes • A lot of crash fixes merged this cycle • Rule: "Some C APIs may not be supported, but at least it shouldn't be possible to crash the shell just by writing JS code" • Primarily useful for users • 10 crashing bugs still open
9. Gio.Settings overrides • Gio.Settings overrides that don't abort • Also in the spirit of "it shouldn't be possible to crash the shell" • Primarily useful for shell extension developers • Thanks to Cinnamon developers for making it possible to contribute these back! const {Gio} = imports.gi; const settings = new Gio.Settings();
10. Sysprof • Better profiler integration • "Just works" • Including garbage collection timing marks • Useful for all four audiences (users indirectly!) • Thanks to Christian Hergert
11. GTK 4 • Now possible to write GTK 4 applications with GJS • Primarily useful for app developers • Thanks to Florian Müllner
12. See appendix for features that SpiderMonkey 68 will add or remove
13. How you can help
14. There has never been a better time to start contributing to GJS than right now.
15. Well-defined open projects • Better console interpreter output (issue #107) • Promise event source (issue #1) • Make the debugger more useful (issue #207 and #208) • Write a migration tool for SpiderMonkey 68 (issue #277) • Add a number of missing features from gobject-introspection
16. Better console interpreter output • What's the benefit? ◦ ◦ Readability ◦ Developer experience • What's the work? ◦ Evaluate prior art in other environments ◦ Pick the best solution and port it ◦ No C++ needed! • GitLab issue GJS Node
17. Better console interpreter output • What's the benefit? ◦ ◦ Readability ◦ Developer experience • What's the work? ◦ Evaluate prior art in other environments ◦ Pick the best solution and port it ◦ No C++ needed! • GitLab issue GJS Node
18. Promise event source • What's the benefit? ◦ Using Promises for async operations is a great way to write intuitive code, but... ◦ Promises resolve in an idle handler, so it can take a relatively long time for your Promise to resolve after the operation finishes ◦ Using Promises shouldn't be slower than using the equivalent callback-based operations • What's the work? ◦ Create a GLib event source to resolve pending promises ◦ Described in more detail in the GitLab issue • GitLab issue
19. Make the debugger more useful • What's the benefit? ◦ The debugger is quite bare-bones. It doesn't handle multiple source files very well, so it isn't as useful as it could be for debugging most apps • What's the work? ◦ Add more information to the existing backtrace command to make it more useful ◦ Keep track of source files loaded in the debugger so you can list them and switch from one to the other ◦ (Probably) no C++ needed! • GitLab issues: issue #207 and #208 ◦ and any other features you can think of ◦ use the debugger and figure out what's missing!
20. Write a migration tool • What's the benefit? ◦ We're upgrading to SpiderMonkey 68 in GNOME 3.36. Mozilla often removes nonstandard Mozilla-specific extensions and people are often surprised when their code breaks ◦ A migration tool like last year's for SpiderMonkey 60 would be well received: run it in one pass over your app or extension's source code and see the likely breakage • What's the work? ◦ Adapt the code of moz60tool to flag things that are being removed in SpiderMonkey 68 ◦ No C++ needed! • GitLab issue
21. Improve introspection • What's the benefit? ◦ Missing functionality from GObject introspection test suite • What's the work? ◦ Depends! Find out where a compliance test is failing, and get it to pass • GitLab issues: #44, #74, #88, #106, #267, #269, #272, #273, #275, #276
22. Less-well-defined open projects • Become a code reviewer for GJS • Improve gjs-docs • Create a good workflow for new GJS contributors (JHBuild is frustrating) • Create a benchmark CI job (issue #119) • Write a contributors bot that categorizes merge requests, suggests reviewers, backports merge requests to stable branches, and thanks people • Adopt a stale merge request
23. Taking Out the Garbage
25. What happened? March 2018 — Problem Statement Georges Stavracas (feaneron) comes up with a plausible explanation for GNOME Shell memory hogging: the problem nicknamed "tardy sweep". Evergreen meme, thanks Andy Holmes
26. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); }
27. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null;
28. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc();
29. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc();
30. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc(); System.gc();
31. The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc(); System.gc(); System.gc();
32. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer • Workaround in GJS • Georges, Daniel van Vugt, Marco Trevisan (Treviño) and Carlos Garnacho contributed • Nicknamed the "Big Hammer" because it's unsubtle and can lower performance • Detects when an extra GC is needed and runs it in ~10 seconds • Plan is to solve it for real and remove the workaround, but at this point we don't know how Image by Steve Buissinne licensed from Pixabay
33. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer September 2018 — A Visit to Mozilla • I met with some people from the SpiderMonkey team at Mozilla Toronto • The problem came up in the conversation • I learned Firefox has the same problem with DOM objects • Solution suggested by Nika Layzell of the Gecko team
34. "Toggle ref" vs. "Expando object" JS wrapper with state GObject refcount >1 C reference Prevent GC JS wrapper with state GObject refcount 1 C reference JS object with state GObject JS wrapper C reference Prevent GC Wrapper treats this object's properties like its own
35. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring • Start writing code for the fix • Realize a large refactor is needed By Gaétan Veillette - Own work, CC BY-SA 4.0
36. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck • First proof of concept doesn't work • I'm also busy with other things Image by PublicDomainPictures licensed from Pixabay
37. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck May 2019 — Proof of Concept Patch • Works with custom SpiderMonkey built with Clang • Testing blocked on SpiderMonkey build bug with GCC • Fixed by Marco "Deathspresso" by bjornery is licensed under CC BY-NC-ND 2.0
38. Taking out the garbage vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); }
39. Taking out the garbage vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null;
40. Taking out the garbage vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc();
41. What happened next? March 2018 — Problem Statement April 2018 — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck May 2019 — Proof of Concept Patch July 2019 — Retarget to GNOME 3.36 • Works as expected in a controlled test • Still too much real-life testing to do • Preliminary runs of GNOME Shell show more work needed Image by Gerhild Klinkow licensed from Pixabay
42. Some terrible situations: "the tardy sweep lite" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); this._window._button = button; // !! } // ... this._window = null; System.gc(); System.gc();
43. Some terrible situations: "the eternal selfie" vfunc_activate() { this._window = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); button._button = button; // !!!!!!! } // ... this._window = null; System.gc();
45. A cool thing • https://gitlab.gnome.org/ptomato/garbage • Test your own configurations and make your own diagrams
46. Also a cool thing • Visiting Mozilla really helped build a relationship between the SpiderMonkey developers and GNOME • Positive for us because now they know what we need from their JS engine • During one of these visits / mini-hackfests we started SpiderMonkey Embedding Examples
47. Thanks to... • GJS contributors from 3.32 and 3.34 • Endless paid for some of this work If you saw a project that looks interesting to you, let's do some hacking! I'm here during the unconference days. All screenshots were taken by me. Screenshots and presentation licensed under CC BY-ND 4.0.
48. Questions?
49. Appendix: SpiderMonkey 68 features (planned, GNOME 3.36)
50. New String features ES standard methods for trimming spaces Replaces nonstandard trimLeft, trimRight ES stage 4 proposal (standard ready) String.matchAll() Easier access to regex capture groups ES stage 4 proposal (standard ready) gjs> ' foo '.trimStart() "foo " gjs> ' foo '.trimEnd() " foo" gjs> ' foo '.trimLeft() gjs> for (let match of 'test1test2'.matchAll(/test(\d)/g) print(match); test1,1 test2,2 YES NO
51. New Array features Array.flat() Well known from lodash and friends ES stage 4 proposal (standard ready) Array.flatMap() A sort of reverse Array.filter()! You can add elements while iterating functional-style ES stage 4 proposal (standard ready) gjs> [1, 2, [3, 4]].flat() 1,2,3,4 function dashComponents(strings) { return strings.flatMap(s => s.split('-')); } gjs> dashComponents(['foo-bar', 'a-b-c']) foo,bar,a,b,c
52. New Object features Object.fromEntries() Create an Object from iterable key-value pairs ES stage 4 proposal (standard ready) gjs> JSON.stringify( .... Object.fromEntries(['a', 1], ['b', 2])) "{"a":1,"b":2}"
53. New JS environment features globalThis Ever find it weird that GJS's global object is named "window"? It should now be accessible via "globalThis" ES stage 3 proposal (candidate) Intl.RelativeTimeFormat Can be useful for user interfaces gjs> globalThis [object GjsGlobal] gjs> let rtf1 = new Intl.RelativeTimeFormat( .... 'en', {style: 'narrow'}); gjs> rtf1.format(-1, 'day') "1 day ago" gjs> let rtf2 = new Intl.RelativeTimeFormat( .... 'es', {numeric: 'auto'}); gjs> rtf2.format(2, 'day') "pasado mañana"
54. New JS language features BigInt ES stage 3 proposal (candidate) Also has two new TypedArray types gjs> Number.MAX_SAFE_INTEGER 9007199254740991 gjs> 9007199254740992 === 9007199254740993 true gjs> BigInt(Number.MAX_SAFE_INTEGER) 9007199254740991n gjs> 9007199254740992n === 9007199254740993n false gjs> let s = new BigInt64Array([-1n, -2n]) gjs> let u = new BigUint64Array([1n, 2n])
55. String generics — nonstandard way of calling String methods on a non-string object. These were only implemented by Mozilla and will now be removed. (Using them already logs a warning.) Note: Array generics won't be removed yet, but they will be in the future so still don't use them! Removals gjs> let num = 15; gjs> String.endsWith(num, 5) typein:2:1 warning: String.endsWith is deprecated; use String.prototype.endsWith instead true gjs> String.prototype.endsWith.call(num, 5) true gjs> String(num).endsWith(5) true gjs> `${num}`.endsWith(5) true NO YES YES YES