Taking Out The Garbage (And How You Can Help)

Taking Out The Garbage (And How You Can Help)

The GNOME desktop is based on a Javascript engine, and there are some long-running and much-discussed problems around Javascript's garbage collection. This past year I've been working on a solution to the garbage collection problems in GJS, an initiative nicknamed "taking out the garbage". This talk will have a short technical portion about taking out the garbage, as well as some of the cool things I've learned by working more closely with Mozilla.

I will also talk about new features introduced in GJS, such as upgrading the Javascript engine to Firefox 68, and how they affect our four audiences: users, app developers, GNOME Shell developers, and shell extension developers.

Finally, there has never been a better time to get started contributing to GJS than right now, and I will talk about some projects that you can do to help make things fabulous for Javascript developers in GNOME.

B2014c8170f4d16a313cfa79071fd861?s=128

Philip Chimento

August 25, 2019
Tweet

Transcript

  1. 1.

    Taking Out The Garbage (And How You Can Help) Philip

    Chimento — ptomato, @therealptomato GUADEC Thessaloniki, August 25, 2019
  2. 3.

    Introduction — Contents of this talk • What's new in

    GJS? • We need some help! Here's what you can do
  3. 4.

    Introduction — Contents of this talk • What's new in

    GJS? • We need some help! Here's what you can do • Oh yeah, what's happening with that garbage collection thing everyone was talking about?
  4. 6.

    What's new in GJS in 3.32 and 3.34? • Big

    Hammer • Docs site restored • Crash fixes • Gio.Settings • Sysprof • GTK 4
  5. 7.

    Docs site restored gjs-docs.gnome.org • Thanks to Meg Ford for

    moving this to GNOME infrastructure • Primarily useful for app developers • We added Mutter docs, when St gets fixed it should become more useful for shell developers and shell extension developers as well
  6. 8.

    Crash fixes • A lot of crash fixes merged this

    cycle • Rule: "Some C APIs may not be supported, but at least it shouldn't be possible to crash the shell just by writing JS code" • Primarily useful for users • 10 crashing bugs still open
  7. 9.

    Gio.Settings overrides • Gio.Settings overrides that don't abort • Also

    in the spirit of "it shouldn't be possible to crash the shell" • Primarily useful for shell extension developers • Thanks to Cinnamon developers for making it possible to contribute these back! const {Gio} = imports.gi; const settings = new Gio.Settings();
  8. 10.

    Sysprof • Better profiler integration • "Just works" • Including

    garbage collection timing marks • Useful for all four audiences (users indirectly!) • Thanks to Christian Hergert
  9. 11.

    GTK 4 • Now possible to write GTK 4 applications

    with GJS • Primarily useful for app developers • Thanks to Florian Müllner
  10. 15.

    Well-defined open projects • Better console interpreter output (issue #107)

    • Promise event source (issue #1) • Make the debugger more useful (issue #207 and #208) • Write a migration tool for SpiderMonkey 68 (issue #277) • Add a number of missing features from gobject-introspection
  11. 16.

    Better console interpreter output • What's the benefit? ◦ ◦

    Readability ◦ Developer experience • What's the work? ◦ Evaluate prior art in other environments ◦ Pick the best solution and port it ◦ No C++ needed! • GitLab issue GJS Node
  12. 17.

    Better console interpreter output • What's the benefit? ◦ ◦

    Readability ◦ Developer experience • What's the work? ◦ Evaluate prior art in other environments ◦ Pick the best solution and port it ◦ No C++ needed! • GitLab issue GJS Node
  13. 18.

    Promise event source • What's the benefit? ◦ Using Promises

    for async operations is a great way to write intuitive code, but... ◦ Promises resolve in an idle handler, so it can take a relatively long time for your Promise to resolve after the operation finishes ◦ Using Promises shouldn't be slower than using the equivalent callback-based operations • What's the work? ◦ Create a GLib event source to resolve pending promises ◦ Described in more detail in the GitLab issue • GitLab issue
  14. 19.

    Make the debugger more useful • What's the benefit? ◦

    The debugger is quite bare-bones. It doesn't handle multiple source files very well, so it isn't as useful as it could be for debugging most apps • What's the work? ◦ Add more information to the existing backtrace command to make it more useful ◦ Keep track of source files loaded in the debugger so you can list them and switch from one to the other ◦ (Probably) no C++ needed! • GitLab issues: issue #207 and #208 ◦ and any other features you can think of ◦ use the debugger and figure out what's missing!
  15. 20.

    Write a migration tool • What's the benefit? ◦ We're

    upgrading to SpiderMonkey 68 in GNOME 3.36. Mozilla often removes nonstandard Mozilla-specific extensions and people are often surprised when their code breaks ◦ A migration tool like last year's for SpiderMonkey 60 would be well received: run it in one pass over your app or extension's source code and see the likely breakage • What's the work? ◦ Adapt the code of moz60tool to flag things that are being removed in SpiderMonkey 68 ◦ No C++ needed! • GitLab issue
  16. 21.

    Improve introspection • What's the benefit? ◦ Missing functionality from

    GObject introspection test suite • What's the work? ◦ Depends! Find out where a compliance test is failing, and get it to pass • GitLab issues: #44, #74, #88, #106, #267, #269, #272, #273, #275, #276
  17. 22.

    Less-well-defined open projects • Become a code reviewer for GJS

    • Improve gjs-docs • Create a good workflow for new GJS contributors (JHBuild is frustrating) • Create a benchmark CI job (issue #119) • Write a contributors bot that categorizes merge requests, suggests reviewers, backports merge requests to stable branches, and thanks people • Adopt a stale merge request
  18. 24.
  19. 25.

    What happened? March 2018 — Problem Statement Georges Stavracas (feaneron)

    comes up with a plausible explanation for GNOME Shell memory hogging: the problem nicknamed "tardy sweep". Evergreen meme, thanks Andy Holmes
  20. 26.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); }
  21. 27.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null;
  22. 28.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc();
  23. 29.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc();
  24. 30.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc(); System.gc();
  25. 31.

    The "tardy sweep" vfunc_activate() { this._window = new Garbage.Window(); const

    label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc(); System.gc(); System.gc(); System.gc();
  26. 32.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer • Workaround in GJS • Georges, Daniel van Vugt, Marco Trevisan (Treviño) and Carlos Garnacho contributed • Nicknamed the "Big Hammer" because it's unsubtle and can lower performance • Detects when an extra GC is needed and runs it in ~10 seconds • Plan is to solve it for real and remove the workaround, but at this point we don't know how Image by Steve Buissinne licensed from Pixabay
  27. 33.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer September 2018 — A Visit to Mozilla • I met with some people from the SpiderMonkey team at Mozilla Toronto • The problem came up in the conversation • I learned Firefox has the same problem with DOM objects • Solution suggested by Nika Layzell of the Gecko team
  28. 34.

    "Toggle ref" vs. "Expando object" JS wrapper with state GObject

    refcount >1 C reference Prevent GC JS wrapper with state GObject refcount 1 C reference JS object with state GObject JS wrapper C reference Prevent GC Wrapper treats this object's properties like its own
  29. 35.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring • Start writing code for the fix • Realize a large refactor is needed By Gaétan Veillette - Own work, CC BY-SA 4.0
  30. 36.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck • First proof of concept doesn't work • I'm also busy with other things Image by PublicDomainPictures licensed from Pixabay
  31. 37.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck May 2019 — Proof of Concept Patch • Works with custom SpiderMonkey built with Clang • Testing blocked on SpiderMonkey build bug with GCC • Fixed by Marco "Deathspresso" by bjornery is licensed under CC BY-NC-ND 2.0
  32. 38.

    Taking out the garbage vfunc_activate() { this._window = new Garbage.Window();

    const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); }
  33. 39.

    Taking out the garbage vfunc_activate() { this._window = new Garbage.Window();

    const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null;
  34. 40.

    Taking out the garbage vfunc_activate() { this._window = new Garbage.Window();

    const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); } // ... this._window = null; System.gc();
  35. 41.

    What happened next? March 2018 — Problem Statement April 2018

    — Big Hammer September 2018 — A Visit to Mozilla October 2018 to January 2019 — Refactoring January 2019 to April 2019 — Stuck May 2019 — Proof of Concept Patch July 2019 — Retarget to GNOME 3.36 • Works as expected in a controlled test • Still too much real-life testing to do • Preliminary runs of GNOME Shell show more work needed Image by Gerhild Klinkow licensed from Pixabay
  36. 42.

    Some terrible situations: "the tardy sweep lite" vfunc_activate() { this._window

    = new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); this._window._button = button; // !! } // ... this._window = null; System.gc(); System.gc();
  37. 43.

    Some terrible situations: "the eternal selfie" vfunc_activate() { this._window =

    new Garbage.Window(); const label2 = new Garbage.Label2(); const button = new Garbage.Button(); const box2 = new Garbage.Box2(); box2.add(label2); box2.add(button); const label1 = new Garbage.Label1(); const box1 = new Garbage.Box1(); box1.add(label1); box1.add(box2); this._window.add(box1); button._button = button; // !!!!!!! } // ... this._window = null; System.gc();
  38. 44.
  39. 46.

    Also a cool thing • Visiting Mozilla really helped build

    a relationship between the SpiderMonkey developers and GNOME • Positive for us because now they know what we need from their JS engine • During one of these visits / mini-hackfests we started SpiderMonkey Embedding Examples
  40. 47.

    Thanks to... • GJS contributors from 3.32 and 3.34 •

    Endless paid for some of this work If you saw a project that looks interesting to you, let's do some hacking! I'm here during the unconference days. All screenshots were taken by me. Screenshots and presentation licensed under CC BY-ND 4.0.
  41. 50.

    New String features ES standard methods for trimming spaces Replaces

    nonstandard trimLeft, trimRight ES stage 4 proposal (standard ready) String.matchAll() Easier access to regex capture groups ES stage 4 proposal (standard ready) gjs> ' foo '.trimStart() "foo " gjs> ' foo '.trimEnd() " foo" gjs> ' foo '.trimLeft() gjs> for (let match of 'test1test2'.matchAll(/test(\d)/g) print(match); test1,1 test2,2 YES NO
  42. 51.

    New Array features Array.flat() Well known from lodash and friends

    ES stage 4 proposal (standard ready) Array.flatMap() A sort of reverse Array.filter()! You can add elements while iterating functional-style ES stage 4 proposal (standard ready) gjs> [1, 2, [3, 4]].flat() 1,2,3,4 function dashComponents(strings) { return strings.flatMap(s => s.split('-')); } gjs> dashComponents(['foo-bar', 'a-b-c']) foo,bar,a,b,c
  43. 52.

    New Object features Object.fromEntries() Create an Object from iterable key-value

    pairs ES stage 4 proposal (standard ready) gjs> JSON.stringify( .... Object.fromEntries(['a', 1], ['b', 2])) "{"a":1,"b":2}"
  44. 53.

    New JS environment features globalThis Ever find it weird that

    GJS's global object is named "window"? It should now be accessible via "globalThis" ES stage 3 proposal (candidate) Intl.RelativeTimeFormat Can be useful for user interfaces gjs> globalThis [object GjsGlobal] gjs> let rtf1 = new Intl.RelativeTimeFormat( .... 'en', {style: 'narrow'}); gjs> rtf1.format(-1, 'day') "1 day ago" gjs> let rtf2 = new Intl.RelativeTimeFormat( .... 'es', {numeric: 'auto'}); gjs> rtf2.format(2, 'day') "pasado mañana"
  45. 54.

    New JS language features BigInt ES stage 3 proposal (candidate)

    Also has two new TypedArray types gjs> Number.MAX_SAFE_INTEGER 9007199254740991 gjs> 9007199254740992 === 9007199254740993 true gjs> BigInt(Number.MAX_SAFE_INTEGER) 9007199254740991n gjs> 9007199254740992n === 9007199254740993n false gjs> let s = new BigInt64Array([-1n, -2n]) gjs> let u = new BigUint64Array([1n, 2n])
  46. 55.

    String generics — nonstandard way of calling String methods on

    a non-string object. These were only implemented by Mozilla and will now be removed. (Using them already logs a warning.) Note: Array generics won't be removed yet, but they will be in the future so still don't use them! Removals gjs> let num = 15; gjs> String.endsWith(num, 5) typein:2:1 warning: String.endsWith is deprecated; use String.prototype.endsWith instead true gjs> String.prototype.endsWith.call(num, 5) true gjs> String(num).endsWith(5) true gjs> `${num}`.endsWith(5) true NO YES YES YES