Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Taking Out The Garbage (And How You Can Help)

Taking Out The Garbage (And How You Can Help)

The GNOME desktop is based on a Javascript engine, and there are some long-running and much-discussed problems around Javascript's garbage collection. This past year I've been working on a solution to the garbage collection problems in GJS, an initiative nicknamed "taking out the garbage". This talk will have a short technical portion about taking out the garbage, as well as some of the cool things I've learned by working more closely with Mozilla.

I will also talk about new features introduced in GJS, such as upgrading the Javascript engine to Firefox 68, and how they affect our four audiences: users, app developers, GNOME Shell developers, and shell extension developers.

Finally, there has never been a better time to get started contributing to GJS than right now, and I will talk about some projects that you can do to help make things fabulous for Javascript developers in GNOME.

Philip Chimento

August 25, 2019
Tweet

More Decks by Philip Chimento

Other Decks in Programming

Transcript

  1. Taking Out The Garbage
    (And How You Can Help)
    Philip Chimento — ptomato, @therealptomato
    GUADEC Thessaloniki, August 25, 2019

    View full-size slide

  2. Introduction — Contents of this talk
    ● What's new in GJS?

    View full-size slide

  3. Introduction — Contents of this talk
    ● What's new in GJS?
    ● We need some help! Here's what you can do

    View full-size slide

  4. Introduction — Contents of this talk
    ● What's new in GJS?
    ● We need some help! Here's what you can do
    ● Oh yeah, what's happening with that garbage collection thing everyone was
    talking about?

    View full-size slide

  5. What's new in GJS?

    View full-size slide

  6. What's new in GJS in 3.32 and 3.34?
    ● Big Hammer
    ● Docs site restored
    ● Crash fixes
    ● Gio.Settings
    ● Sysprof
    ● GTK 4

    View full-size slide

  7. Docs site restored
    gjs-docs.gnome.org
    ● Thanks to Meg Ford for moving this to
    GNOME infrastructure
    ● Primarily useful for app developers
    ● We added Mutter docs, when St gets fixed it
    should become more useful for shell
    developers and shell extension developers as
    well

    View full-size slide

  8. Crash fixes
    ● A lot of crash fixes merged this cycle
    ● Rule: "Some C APIs may not be supported, but at least it shouldn't be possible
    to crash the shell just by writing JS code"
    ● Primarily useful for users
    ● 10 crashing bugs still open

    View full-size slide

  9. Gio.Settings overrides
    ● Gio.Settings overrides that don't abort
    ● Also in the spirit of "it shouldn't be possible to crash the shell"
    ● Primarily useful for shell extension developers
    ● Thanks to Cinnamon developers for making it possible to contribute these
    back!
    const {Gio} = imports.gi;
    const settings = new Gio.Settings();

    View full-size slide

  10. Sysprof
    ● Better profiler integration
    ● "Just works"
    ● Including garbage collection
    timing marks
    ● Useful for all four audiences
    (users indirectly!)
    ● Thanks to Christian Hergert

    View full-size slide

  11. GTK 4
    ● Now possible to write GTK 4 applications with GJS
    ● Primarily useful for app developers
    ● Thanks to Florian Müllner

    View full-size slide

  12. See appendix for features that SpiderMonkey 68 will add or remove

    View full-size slide

  13. How you can help

    View full-size slide

  14. There has never been
    a better time to start
    contributing to GJS
    than right now.

    View full-size slide

  15. Well-defined open projects
    ● Better console interpreter output (issue #107)
    ● Promise event source (issue #1)
    ● Make the debugger more useful (issue #207 and #208)
    ● Write a migration tool for SpiderMonkey 68 (issue #277)
    ● Add a number of missing features from gobject-introspection

    View full-size slide

  16. Better console
    interpreter output
    ● What's the benefit?

    ○ Readability
    ○ Developer experience
    ● What's the work?
    ○ Evaluate prior art in
    other environments
    ○ Pick the best solution
    and port it
    ○ No C++ needed!
    ● GitLab issue
    GJS
    Node

    View full-size slide

  17. Better console
    interpreter output
    ● What's the benefit?

    ○ Readability
    ○ Developer experience
    ● What's the work?
    ○ Evaluate prior art in
    other environments
    ○ Pick the best solution
    and port it
    ○ No C++ needed!
    ● GitLab issue
    GJS
    Node

    View full-size slide

  18. Promise event source
    ● What's the benefit?
    ○ Using Promises for async operations is a great way to write intuitive code, but...
    ○ Promises resolve in an idle handler, so it can take a relatively long time for your Promise to
    resolve after the operation finishes
    ○ Using Promises shouldn't be slower than using the equivalent callback-based operations
    ● What's the work?
    ○ Create a GLib event source to resolve pending promises
    ○ Described in more detail in the GitLab issue
    ● GitLab issue

    View full-size slide

  19. Make the debugger more useful
    ● What's the benefit?
    ○ The debugger is quite bare-bones. It doesn't handle multiple source files very well, so it isn't as
    useful as it could be for debugging most apps
    ● What's the work?
    ○ Add more information to the existing backtrace command to make it more useful
    ○ Keep track of source files loaded in the debugger so you can list them and switch from one to
    the other
    ○ (Probably) no C++ needed!
    ● GitLab issues: issue #207 and #208
    ○ and any other features you can think of
    ○ use the debugger and figure out what's missing!

    View full-size slide

  20. Write a migration tool
    ● What's the benefit?
    ○ We're upgrading to SpiderMonkey 68 in GNOME 3.36. Mozilla often removes nonstandard
    Mozilla-specific extensions and people are often surprised when their code breaks
    ○ A migration tool like last year's for SpiderMonkey 60 would be well received: run it in one pass
    over your app or extension's source code and see the likely breakage
    ● What's the work?
    ○ Adapt the code of moz60tool to flag things that are being removed in SpiderMonkey 68
    ○ No C++ needed!
    ● GitLab issue

    View full-size slide

  21. Improve introspection
    ● What's the benefit?
    ○ Missing functionality from GObject introspection test suite
    ● What's the work?
    ○ Depends! Find out where a compliance test is failing, and get it to pass
    ● GitLab issues: #44, #74, #88, #106, #267, #269, #272, #273, #275, #276

    View full-size slide

  22. Less-well-defined open projects
    ● Become a code reviewer for GJS
    ● Improve gjs-docs
    ● Create a good workflow for new GJS contributors (JHBuild is frustrating)
    ● Create a benchmark CI job (issue #119)
    ● Write a contributors bot that categorizes merge requests, suggests reviewers,
    backports merge requests to stable branches, and thanks people
    ● Adopt a stale merge request

    View full-size slide

  23. Taking Out the Garbage

    View full-size slide

  24. What happened?
    March 2018 — Problem Statement
    Georges Stavracas (feaneron) comes up with a
    plausible explanation for GNOME Shell memory
    hogging: the problem nicknamed "tardy sweep".
    Evergreen meme, thanks Andy Holmes

    View full-size slide

  25. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }

    View full-size slide

  26. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;

    View full-size slide

  27. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();

    View full-size slide

  28. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();

    View full-size slide

  29. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    System.gc();

    View full-size slide

  30. The "tardy sweep"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();
    System.gc();
    System.gc();

    View full-size slide

  31. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    ● Workaround in GJS
    ● Georges, Daniel van Vugt, Marco Trevisan
    (Treviño) and Carlos Garnacho contributed
    ● Nicknamed the "Big Hammer" because it's
    unsubtle and can lower performance
    ● Detects when an extra GC is needed and
    runs it in ~10 seconds
    ● Plan is to solve it for real and remove the
    workaround, but at this point we don't
    know how Image by Steve Buissinne licensed from Pixabay

    View full-size slide

  32. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    ● I met with some people from the
    SpiderMonkey team at Mozilla Toronto
    ● The problem came up in the conversation
    ● I learned Firefox has the same problem
    with DOM objects
    ● Solution suggested by Nika Layzell of the
    Gecko team

    View full-size slide

  33. "Toggle ref" vs. "Expando object"
    JS
    wrapper
    with
    state
    GObject
    refcount >1
    C reference
    Prevent GC
    JS
    wrapper
    with
    state
    GObject
    refcount 1
    C reference
    JS
    object
    with
    state
    GObject JS
    wrapper
    C
    reference
    Prevent GC
    Wrapper treats this object's
    properties like its own

    View full-size slide

  34. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    ● Start writing code for the fix
    ● Realize a large refactor is needed
    By Gaétan Veillette - Own work, CC BY-SA 4.0

    View full-size slide

  35. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    ● First proof of concept doesn't work
    ● I'm also busy with other things
    Image by PublicDomainPictures licensed from Pixabay

    View full-size slide

  36. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    May 2019 — Proof of Concept Patch
    ● Works with custom SpiderMonkey built
    with Clang
    ● Testing blocked on SpiderMonkey build
    bug with GCC
    ● Fixed by Marco
    "Deathspresso" by bjornery is licensed under CC BY-NC-ND 2.0

    View full-size slide

  37. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }

    View full-size slide

  38. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;

    View full-size slide

  39. Taking out the garbage
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    }
    // ...
    this._window = null;
    System.gc();

    View full-size slide

  40. What happened next?
    March 2018 — Problem Statement
    April 2018 — Big Hammer
    September 2018 — A Visit to Mozilla
    October 2018 to January 2019 — Refactoring
    January 2019 to April 2019 — Stuck
    May 2019 — Proof of Concept Patch
    July 2019 — Retarget to GNOME 3.36
    ● Works as expected in a controlled test
    ● Still too much real-life testing to do
    ● Preliminary runs of GNOME Shell show
    more work needed
    Image by Gerhild Klinkow licensed from Pixabay

    View full-size slide

  41. Some terrible situations: "the tardy sweep lite"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    this._window._button = button; // !!
    }
    // ...
    this._window = null;
    System.gc();
    System.gc();

    View full-size slide

  42. Some terrible situations: "the eternal selfie"
    vfunc_activate() {
    this._window = new Garbage.Window();
    const label2 = new Garbage.Label2();
    const button = new Garbage.Button();
    const box2 = new Garbage.Box2();
    box2.add(label2);
    box2.add(button);
    const label1 = new Garbage.Label1();
    const box1 = new Garbage.Box1();
    box1.add(label1);
    box1.add(box2);
    this._window.add(box1);
    button._button = button; // !!!!!!!
    }
    // ...
    this._window = null;
    System.gc();

    View full-size slide

  43. A cool thing
    ● https://gitlab.gnome.org/ptomato/garbage
    ● Test your own configurations and make your own diagrams

    View full-size slide

  44. Also a cool thing
    ● Visiting Mozilla really helped build a relationship between the SpiderMonkey
    developers and GNOME
    ● Positive for us because now they know what we need from their JS engine
    ● During one of these visits / mini-hackfests we started SpiderMonkey
    Embedding Examples

    View full-size slide

  45. Thanks to...
    ● GJS contributors from 3.32 and
    3.34
    ● Endless paid for some of this
    work
    If you saw a project that looks
    interesting to you, let's do some
    hacking! I'm here during the
    unconference days.
    All screenshots were taken by me.
    Screenshots and presentation
    licensed under CC BY-ND 4.0.

    View full-size slide

  46. Appendix:
    SpiderMonkey 68
    features (planned,
    GNOME 3.36)

    View full-size slide

  47. New String features
    ES standard methods for trimming spaces
    Replaces nonstandard trimLeft, trimRight
    ES stage 4 proposal (standard ready)
    String.matchAll()
    Easier access to regex capture groups
    ES stage 4 proposal (standard ready)
    gjs> ' foo '.trimStart()
    "foo "
    gjs> ' foo '.trimEnd()
    " foo"
    gjs> ' foo '.trimLeft()
    gjs> for (let match of
    'test1test2'.matchAll(/test(\d)/g)
    print(match);
    test1,1
    test2,2
    YES
    NO

    View full-size slide

  48. New Array features
    Array.flat()
    Well known from lodash and friends
    ES stage 4 proposal (standard ready)
    Array.flatMap()
    A sort of reverse Array.filter()! You can add
    elements while iterating functional-style
    ES stage 4 proposal (standard ready)
    gjs> [1, 2, [3, 4]].flat()
    1,2,3,4
    function dashComponents(strings) {
    return strings.flatMap(s => s.split('-'));
    }
    gjs> dashComponents(['foo-bar', 'a-b-c'])
    foo,bar,a,b,c

    View full-size slide

  49. New Object features
    Object.fromEntries()
    Create an Object from iterable key-value pairs
    ES stage 4 proposal (standard ready)
    gjs> JSON.stringify(
    .... Object.fromEntries(['a', 1], ['b', 2]))
    "{"a":1,"b":2}"

    View full-size slide

  50. New JS environment features
    globalThis
    Ever find it weird that GJS's global object is
    named "window"? It should now be accessible via
    "globalThis"
    ES stage 3 proposal (candidate)
    Intl.RelativeTimeFormat
    Can be useful for user interfaces
    gjs> globalThis
    [object GjsGlobal]
    gjs> let rtf1 = new Intl.RelativeTimeFormat(
    .... 'en', {style: 'narrow'});
    gjs> rtf1.format(-1, 'day')
    "1 day ago"
    gjs> let rtf2 = new Intl.RelativeTimeFormat(
    .... 'es', {numeric: 'auto'});
    gjs> rtf2.format(2, 'day')
    "pasado mañana"

    View full-size slide

  51. New JS language features
    BigInt
    ES stage 3 proposal (candidate)
    Also has two new TypedArray types
    gjs> Number.MAX_SAFE_INTEGER
    9007199254740991
    gjs> 9007199254740992 === 9007199254740993
    true
    gjs> BigInt(Number.MAX_SAFE_INTEGER)
    9007199254740991n
    gjs> 9007199254740992n === 9007199254740993n
    false
    gjs> let s = new BigInt64Array([-1n, -2n])
    gjs> let u = new BigUint64Array([1n, 2n])

    View full-size slide

  52. String generics — nonstandard way of calling
    String methods on a non-string object. These
    were only implemented by Mozilla and will now
    be removed. (Using them already logs a
    warning.)
    Note: Array generics won't be removed yet, but
    they will be in the future so still don't use them!
    Removals
    gjs> let num = 15;
    gjs> String.endsWith(num, 5)
    typein:2:1 warning: String.endsWith is
    deprecated; use String.prototype.endsWith
    instead
    true
    gjs> String.prototype.endsWith.call(num, 5)
    true
    gjs> String(num).endsWith(5)
    true
    gjs> `${num}`.endsWith(5)
    true
    NO
    YES
    YES
    YES

    View full-size slide