Taking Out The Garbage (And How You Can Help)

The GNOME desktop is based on a Javascript engine, and there are some long-running and much-discussed problems around Javascript's garbage collection. This past year I've been working on a solution to the garbage collection problems in GJS, an initiative nicknamed "taking out the garbage". This talk will have a short technical portion about taking out the garbage, as well as some of the cool things I've learned by working more closely with Mozilla.

I will also talk about new features introduced in GJS, such as upgrading the Javascript engine to Firefox 68, and how they affect our four audiences: users, app developers, GNOME Shell developers, and shell extension developers.

Finally, there has never been a better time to get started contributing to GJS than right now, and I will talk about some projects that you can do to help make things fabulous for Javascript developers in GNOME.

Transcript

1. Taking Out The Garbage
   (And How You Can Help)
   Philip Chimento — ptomato, @therealptomato
   GUADEC Thessaloniki, August 25, 2019
   

   View Slide

2. Introduction — Contents of this talk
   ● What's new in GJS?
   

   View Slide

3. Introduction — Contents of this talk
   ● What's new in GJS?
   ● We need some help! Here's what you can do
   

   View Slide

4. Introduction — Contents of this talk
   ● What's new in GJS?
   ● We need some help! Here's what you can do
   ● Oh yeah, what's happening with that garbage collection thing everyone was
   talking about?
   

   View Slide

5. What's new in GJS?
   

   View Slide

6. What's new in GJS in 3.32 and 3.34?
   ● Big Hammer
   ● Docs site restored
   ● Crash fixes
   ● Gio.Settings
   ● Sysprof
   ● GTK 4
   

   View Slide

7. Docs site restored
   gjs-docs.gnome.org
   ● Thanks to Meg Ford for moving this to
   GNOME infrastructure
   ● Primarily useful for app developers
   ● We added Mutter docs, when St gets fixed it
   should become more useful for shell
   developers and shell extension developers as
   well
   

   View Slide

8. Crash fixes
   ● A lot of crash fixes merged this cycle
   ● Rule: "Some C APIs may not be supported, but at least it shouldn't be possible
   to crash the shell just by writing JS code"
   ● Primarily useful for users
   ● 10 crashing bugs still open
   

   View Slide

9. Gio.Settings overrides
   ● Gio.Settings overrides that don't abort
   ● Also in the spirit of "it shouldn't be possible to crash the shell"
   ● Primarily useful for shell extension developers
   ● Thanks to Cinnamon developers for making it possible to contribute these
   back!
   const {Gio} = imports.gi;
   const settings = new Gio.Settings();
   

   View Slide

10. Sysprof
    ● Better profiler integration
    ● "Just works"
    ● Including garbage collection
    timing marks
    ● Useful for all four audiences
    (users indirectly!)
    ● Thanks to Christian Hergert
    

    View Slide

11. GTK 4
    ● Now possible to write GTK 4 applications with GJS
    ● Primarily useful for app developers
    ● Thanks to Florian Müllner
    

    View Slide

12. See appendix for features that SpiderMonkey 68 will add or remove
    

    View Slide

13. How you can help
    

    View Slide

14. There has never been
    a better time to start
    contributing to GJS
    than right now.
    

    View Slide

15. Well-defined open projects
    ● Better console interpreter output (issue #107)
    ● Promise event source (issue #1)
    ● Make the debugger more useful (issue #207 and #208)
    ● Write a migration tool for SpiderMonkey 68 (issue #277)
    ● Add a number of missing features from gobject-introspection
    

    View Slide

16. Better console
    interpreter output
    ● What's the benefit?
    ○
    ○ Readability
    ○ Developer experience
    ● What's the work?
    ○ Evaluate prior art in
    other environments
    ○ Pick the best solution
    and port it
    ○ No C++ needed!
    ● GitLab issue
    GJS
    Node
    

    View Slide

17. Better console
    interpreter output
    ● What's the benefit?
    ○
    ○ Readability
    ○ Developer experience
    ● What's the work?
    ○ Evaluate prior art in
    other environments
    ○ Pick the best solution
    and port it
    ○ No C++ needed!
    ● GitLab issue
    GJS
    Node
    

    View Slide

18. Promise event source
    ● What's the benefit?
    ○ Using Promises for async operations is a great way to write intuitive code, but...
    ○ Promises resolve in an idle handler, so it can take a relatively long time for your Promise to
    resolve after the operation finishes
    ○ Using Promises shouldn't be slower than using the equivalent callback-based operations
    ● What's the work?
    ○ Create a GLib event source to resolve pending promises
    ○ Described in more detail in the GitLab issue
    ● GitLab issue
    

    View Slide

19. Make the debugger more useful
    ● What's the benefit?
    ○ The debugger is quite bare-bones. It doesn't handle multiple source files very well, so it isn't as
    useful as it could be for debugging most apps
    ● What's the work?
    ○ Add more information to the existing backtrace command to make it more useful
    ○ Keep track of source files loaded in the debugger so you can list them and switch from one to
    the other
    ○ (Probably) no C++ needed!
    ● GitLab issues: issue #207 and #208
    ○ and any other features you can think of
    ○ use the debugger and figure out what's missing!
    

    View Slide

20. Write a migration tool
    ● What's the benefit?
    ○ We're upgrading to SpiderMonkey 68 in GNOME 3.36. Mozilla often removes nonstandard
    Mozilla-specific extensions and people are often surprised when their code breaks
    ○ A migration tool like last year's for SpiderMonkey 60 would be well received: run it in one pass
    over your app or extension's source code and see the likely breakage
    ● What's the work?
    ○ Adapt the code of moz60tool to flag things that are being removed in SpiderMonkey 68
    ○ No C++ needed!
    ● GitLab issue
    

    View Slide

21. Improve introspection
    ● What's the benefit?
    ○ Missing functionality from GObject introspection test suite
    ● What's the work?
    ○ Depends! Find out where a compliance test is failing, and get it to pass
    ● GitLab issues: #44, #74, #88, #106, #267, #269, #272, #273, #275, #276
    

    View Slide

22. Less-well-defined open projects
    ● Become a code reviewer for GJS
    ● Improve gjs-docs
    ● Create a good workflow for new GJS contributors (JHBuild is frustrating)
    ● Create a benchmark CI job (issue #119)
    ● Write a contributors bot that categorizes merge requests, suggests reviewers,
    backports merge requests to stable branches, and thanks people
    ● Adopt a stale merge request
    

    View Slide

23. Taking Out the Garbage
    

    View Slide

24. View Slide

25. What happened?
    March 2018 — Problem Statement
    Georges Stavracas (feaneron) comes up with a
    plausible explanation for GNOME Shell memory
    hogging: the problem nicknamed "tardy sweep".
    Evergreen meme, thanks Andy Holmes
    

    View Slide

26. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    

    View Slide

27. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    

    View Slide

28. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    

    View Slide

29. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    

    View Slide

30. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    System.gc();
    

    View Slide

31. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    System.gc();
    System.gc();
    

    View Slide

32. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    ● Workaround in GJS
    ● Georges, Daniel van Vugt, Marco Trevisan
    (Treviño) and Carlos Garnacho contributed
    ● Nicknamed the "Big Hammer" because it's
    unsubtle and can lower performance
    ● Detects when an extra GC is needed and
    runs it in ~10 seconds
    ● Plan is to solve it for real and remove the
    workaround, but at this point we don't
    know how Image by Steve Buissinne licensed from Pixabay
    

    View Slide

33. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    ● I met with some people from the
    SpiderMonkey team at Mozilla Toronto
    ● The problem came up in the conversation
    ● I learned Firefox has the same problem
    with DOM objects
    ● Solution suggested by Nika Layzell of the
    Gecko team
    

    View Slide

34. "Toggle ref" vs. "Expando object"
    JS
    wrapper
    with
    state
    GObject
    refcount >1
    C reference
    Prevent GC
    JS
    wrapper
    with
    state
    GObject
    refcount 1
    C reference
    JS
    object
    with
    state
    GObject JS
    wrapper
    C
    reference
    Prevent GC
    Wrapper treats this object's
    properties like its own
    

    View Slide

35. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    ● Start writing code for the fix
    ● Realize a large refactor is needed
    By Gaétan Veillette - Own work, CC BY-SA 4.0
    

    View Slide

36. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    ● First proof of concept doesn't work
    ● I'm also busy with other things
    Image by PublicDomainPictures licensed from Pixabay
    

    View Slide

37. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    May 2019 — Proof of Concept Patch
    ● Works with custom SpiderMonkey built
    with Clang
    ● Testing blocked on SpiderMonkey build
    bug with GCC
    ● Fixed by Marco
    "Deathspresso" by bjornery is licensed under CC BY-NC-ND 2.0
    

    View Slide

38. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    

    View Slide

39. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    

    View Slide

40. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    

    View Slide

41. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    May 2019 — Proof of Concept Patch
    July 2019 — Retarget to GNOME 3.36
    ● Works as expected in a controlled test
    ● Still too much real-life testing to do
    ● Preliminary runs of GNOME Shell show
    more work needed
    Image by Gerhild Klinkow licensed from Pixabay
    

    View Slide

42. Some terrible situations: "the tardy sweep lite"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    this._window._button = button; // !!
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    

    View Slide

43. Some terrible situations: "the eternal selfie"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    button._button = button; // !!!!!!!
    }
    // ...
    this._window = null;
    System.gc();
    

    View Slide

44. View Slide

45. A cool thing
    ● https://gitlab.gnome.org/ptomato/garbage
    ● Test your own configurations and make your own diagrams
    

    View Slide

46. Also a cool thing
    ● Visiting Mozilla really helped build a relationship between the SpiderMonkey
    developers and GNOME
    ● Positive for us because now they know what we need from their JS engine
    ● During one of these visits / mini-hackfests we started SpiderMonkey
    Embedding Examples
    

    View Slide

47. Thanks to...
    ● GJS contributors from 3.32 and
    3.34
    ● Endless paid for some of this
    work
    If you saw a project that looks
    interesting to you, let's do some
    hacking! I'm here during the
    unconference days.
    All screenshots were taken by me.
    Screenshots and presentation
    licensed under CC BY-ND 4.0.
    

    View Slide

48. Questions?
    

    View Slide

49. Appendix:
    SpiderMonkey 68
    features (planned,
    GNOME 3.36)
    

    View Slide

50. New String features
    ES standard methods for trimming spaces
    Replaces nonstandard trimLeft, trimRight
    ES stage 4 proposal (standard ready)
    String.matchAll()
    Easier access to regex capture groups
    ES stage 4 proposal (standard ready)
    gjs> ' foo '.trimStart()
    "foo "
    gjs> ' foo '.trimEnd()
    " foo"
    gjs> ' foo '.trimLeft()
    gjs> for (let match of
    'test1test2'.matchAll(/test(\d)/g)
    print(match);
    test1,1
    test2,2
    YES
    NO
    

    View Slide

51. New Array features
    Array.flat()
    Well known from lodash and friends
    ES stage 4 proposal (standard ready)
    Array.flatMap()
    A sort of reverse Array.filter()! You can add
    elements while iterating functional-style
    ES stage 4 proposal (standard ready)
    gjs> [1, 2, [3, 4]].flat()
    1,2,3,4
    function dashComponents(strings) {
    return strings.flatMap(s => s.split('-'));
    }
    gjs> dashComponents(['foo-bar', 'a-b-c'])
    foo,bar,a,b,c
    

    View Slide

52. New Object features
    Object.fromEntries()
    Create an Object from iterable key-value pairs
    ES stage 4 proposal (standard ready)
    gjs> JSON.stringify(
    .... Object.fromEntries(['a', 1], ['b', 2]))
    "{"a":1,"b":2}"
    

    View Slide

53. New JS environment features
    globalThis
    Ever find it weird that GJS's global object is
    named "window"? It should now be accessible via
    "globalThis"
    ES stage 3 proposal (candidate)
    Intl.RelativeTimeFormat
    Can be useful for user interfaces
    gjs> globalThis
    [object GjsGlobal]
    gjs> let rtf1 = new Intl.RelativeTimeFormat(
    .... 'en', {style: 'narrow'});
    gjs> rtf1.format(-1, 'day')
    "1 day ago"
    gjs> let rtf2 = new Intl.RelativeTimeFormat(
    .... 'es', {numeric: 'auto'});
    gjs> rtf2.format(2, 'day')
    "pasado mañana"
    

    View Slide

54. New JS language features
    BigInt
    ES stage 3 proposal (candidate)
    Also has two new TypedArray types
    gjs> Number.MAX_SAFE_INTEGER
    9007199254740991
    gjs> 9007199254740992 === 9007199254740993
    true
    gjs> BigInt(Number.MAX_SAFE_INTEGER)
    9007199254740991n
    gjs> 9007199254740992n === 9007199254740993n
    false
    gjs> let s = new BigInt64Array([-1n, -2n])
    gjs> let u = new BigUint64Array([1n, 2n])
    

    View Slide

55. String generics — nonstandard way of calling
    String methods on a non-string object. These
    were only implemented by Mozilla and will now
    be removed. (Using them already logs a
    warning.)
    Note: Array generics won't be removed yet, but
    they will be in the future so still don't use them!
    Removals
    gjs> let num = 15;
    gjs> String.endsWith(num, 5)
    typein:2:1 warning: String.endsWith is
    deprecated; use String.prototype.endsWith
    instead
    true
    gjs> String.prototype.endsWith.call(num, 5)
    true
    gjs> String(num).endsWith(5)
    true
    gjs> `${num}`.endsWith(5)
    true
    NO
    YES
    YES
    YES
    

    View Slide