Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JavaScript in GNOME 2018

JavaScript in GNOME 2018

This talk is about all the improvements made in GNOME's JavaScript platform in the past year. We've made many strides: developer experience, especially for new contributors; new JavaScript language features; and performance improvements, especially in memory usage. I'll talk about the improvements and how they affect the four audiences: users, app developers, GNOME Shell developers, and shell extension developers. I'll also talk about some projects that we need your help with!

Philip Chimento

July 07, 2018
Tweet

More Decks by Philip Chimento

Other Decks in Programming

Transcript

  1. Javascript in GNOME 2018
    Philip Chimento — ptomato, @therealptomato
    GUADEC, July 7, 2017

    View full-size slide

  2. Introduction — Contents of this talk
    ● Social progress
    ● Technical progress
    ● How you can help

    View full-size slide

  3. Introduction — The Four Audiences
    ● Users
    ● App developers
    ● Shell developers
    ● Shell extension developers

    View full-size slide

  4. Social Progress

    View full-size slide

  5. Social — We switched to GitLab!

    View full-size slide

  6. Switching to GitLab
    ● Many more first-time and drive-by contributors
    ● Maintainer's patch review workflow much easier
    ● How did we live without continuous integration?
    ● Someone could look into a merge request bot

    View full-size slide

  7. Social — Continuous Integration

    View full-size slide

  8. Continuous Integration
    ● We run linters
    ● We build and run tests with GCC and Clang
    ● Contributors get feedback on their merge requests within a few minutes
    ● "Thorough tests" run only at scheduled intervals on master, or manually
    ● Thanks to Claudio André

    View full-size slide

  9. Social — Linters
    ● Linters are good, they remove tedium and turnaround time from code review
    ● Can be frustrating for drive-by contributors or new contributors who haven't
    yet internalized the code style rules
    ● Linter configurations still need some tweaking; thanks to Avi Zajac (llzes) for
    doing some work on this
    ● There are no good C++ linters compared to what we're used to in JS
    ● In general still a work in progress

    View full-size slide

  10. Social — Deployment
    ● Code coverage report published automatically:
    https://gnome.pages.gitlab.gnome.org/gjs/
    ● Flatpak package with GJS interpreter created when manually requested at
    end of CI pipeline
    ● Thanks to Claudio André for working on this
    ● Let Claudio know what would be useful for you as an app developer or shell
    extension developer

    View full-size slide

  11. Social — Documentation site

    View full-size slide

  12. Documentation site
    ● http://devdocs.baznga.org/
    ● GSoC project of Evan Welsh (rockon999)
    ● Improvements to automatically generated documentation
    ● Sample apps with best practices
    ● Everaldo Canuto continues to host the website for free, thanks!

    View full-size slide

  13. Social — Review backlog
    ● Before migrating to GitLab, I turned all the stale patches from the last 10
    years into merge requests
    ● Most of them are merged now
    ● There's still a backlog
    ● Thanks to Cosimo Cecchi for reviewing almost every line that I write
    ● Want to help? Adopt a stale merge request

    View full-size slide

  14. Social — Press
    ● Due to the "tardy sweep problem" (more later) we were indirectly discussed
    on OMG!Ubuntu, Reddit /r/linux, and a few other Linux-watcher sites
    ● Reactions ranged from constructive to toxically negative or comically
    uninformed
    ● Thanks to Sri Ramkrishna and Andy Holmes for being unflappable positive
    voices

    View full-size slide

  15. Effect on the community
    ● One surprising side effect is that the big press got the GNOME community
    thinking about how to fix the tardy sweep problem in GJS and the shell
    ● Generated a lot of ideas that I couldn't have thought of myself

    View full-size slide

  16. Social — Collaboration with other projects
    ● Mozilla: We've been collaborating with the SpiderMonkey team to make
    SpiderMonkey 60 work as a standalone JS engine without any downstream
    patches
    ● We're continuing to badger Mozilla to get a good tarball release process in
    place for standalone SpiderMonkey
    ● Cinnamon: At least one developer has interest in re-unifying CJS and GJS for
    a future Cinnamon release, we've merged some of their patches

    View full-size slide

  17. Technical Progress

    View full-size slide

  18. Technical — Language & Platform
    ● Javascript engine upgraded to SpiderMonkey 60 (GNOME 3.29.4 or .90)
    ● Fewer JS language changes
    ● Affects app developers, shell developers, and shell extension developers
    ● Not much change for users

    View full-size slide

  19. Async iteration
    There's now support in the language for
    "asynchronous iterators" as per the latest ES
    standard.
    https://jsbin.com/folotu/edit?js,console
    /t Jake Archibald for the example
    async function* asyncRandomNumbers() {
    const url = 'https://www.random.org/...';
    while (true) {
    const response = await fetch(url);
    const text = await response.text();
    yield Number(text);
    }
    }
    async function example() {
    for await (const num of asyncRandomNumbers()) {
    console.log(num);
    if (num > 0.95) break;
    }
    }

    View full-size slide

  20. Rest operator in object destructuring
    As per the latest ES standard, you can now
    unpack certain properties from an object and
    use the rest operator to catch the remaining
    ones.
    gjs> var {a, b, ...cd} =
    .... {a: 1, b: 2, c: 3, d: 4};
    gjs> a
    1
    gjs> b
    2
    gjs> JSON.stringify(cd)
    "{"c":3,"d":4}"

    View full-size slide

  21. Spread operator in object literals
    Also new in the ES standard, you can merge
    objects or add properties to them using the
    spread operator.
    gjs> const obj1 = {a: 1, b: 2};
    gjs> const obj2 = {c: 3, d: 4};
    gjs> const merged = {...obj1, ...obj2};
    gjs> JSON.stringify(merged)
    "{"a":1,"b":2,"c":3,"d":4}"

    View full-size slide

  22. Anonymous catch
    Some days you just don't care about errors.*
    Then why bother binding a name to them?
    *Of course, all GNOME code always handles errors properly
    try {
    dangerousOperation();
    } catch (e) {
    void e; // make linter happy
    print("Error, but I don't care");
    }
    try {
    dangerousOperation();
    } catch {
    print("Error, but I don't care");
    }
    NO
    YES

    View full-size slide

  23. Finally, a finally method
    Available in many popular Promise libraries, now
    part of built-in Promises.
    (Since you can have try...catch...finally with
    await expressions, it makes sense that it must
    be possible with Promises too)
    load(file)
    .then(text => {
    print(text);
    })
    .catch(error => {
    logError(error, "Oh no!");
    })
    .finally(() => {
    mainLoop.quit();
    });

    View full-size slide

  24. Removals! Nonstandard Mozilla syntax
    Conditional catch
    For-each-in
    Legacy lambda syntax
    Legacy iterators, array comprehensions,
    generator comprehensions
    catch (e if e.code === 23) {...}
    catch (e) { if (e.code === 23) {...} }
    for each (let a in obj) {...}
    for (let a of Object.values(obj)) {...}
    for each (let a in arr) {...}
    for (let a of arr) {...}
    arr.map(function (x) x * x);
    arr.map(x => x * x);
    [for (x of iterable) if (cond(x)) expr(x)]
    iterable.filter(cond).map(expr)
    YES
    NO
    YES
    NO
    YES
    NO
    YES
    NO
    YES
    NO

    View full-size slide

  25. ByteArray / Uint8Array
    ● Uint8Array is Javascript's way of handling
    byte arrays
    ● ByteArray is GJS's custom solution,
    pre-dating Uint8Array
    ● ByteArray would have had to be rewritten
    for SpiderMonkey 60 anyway
    ● ByteArray now wrapper around Uint8Array
    ● Some introspected functions now return
    Uint8Array instead of ByteArray, so if you
    don't want to port your code, there is one
    thing you have to change to keep it
    working
    const ba =
    functionReturningSomeBytes();
    const ba = new ByteArray.ByteArray(
    functionReturningSomeBytes());
    NO
    YES

    View full-size slide

  26. ByteArray / Uint8Array
    ● If you do want to port your code, the main
    difference is that Uint8Array's length is
    fixed
    ● Assigning past the end of ByteArray
    lengthens the array
    ● Assigning past the end of Uint8Array is
    ignored
    ● Length of Uint8Array must be set at
    creation time
    const a = ByteArray.fromArray([1, 2, 3]);
    a[3] = 4;
    let a = Uint8Array.from([1, 2, 3]);
    a = Uint8Array.of(...a, 4);
    const a = new ByteArray.ByteArray();
    a[0] = 255;
    const a = new Uint8Array(1);
    a[0] = 255;
    NO
    NO
    YES
    YES

    View full-size slide

  27. ES6 Modules
    ● Not certain to land in 3.30, otherwise 3.32
    ● Thanks to djrenren
    ● Follow along at merge request 150
    ● Porting has to happen top-down! ES6
    modules can't be imported inside code
    that's imported via the old imports object
    import main from 'js/app';
    main(ARGV);

    View full-size slide

  28. GIO Async operations
    ● Outreachy project of Avi Zajac (llzes)
    ● Intended to land in 3.32
    ● If you pass a callback to GIO async
    methods, you'll get the C-style callback API
    as usual
    ● If you omit the callback, you'll get a
    Promise
    ● Promises work transparently with async
    functions and await expressions
    ● Support for annotations in
    gobject-introspection to benefit other
    languages
    try {
    let [contents] =
    await file.load_contents_async(null);
    print(contents);
    let info = await file.query_info_async(
    'standard::*',
    Gio.FileQueryInfoFlags.NONE,
    GLib.PRIORITY_DEFAULT, null);
    print(info.get_size(), 'bytes');
    } catch (e) {
    logError(e, 'Oh no!');
    } finally {
    loop.quit();
    }

    View full-size slide

  29. Technical — Performance
    ● A few notable performance improvements
    ● Including one that was widely ridiculed on Reddit
    ● Mostly affects users

    View full-size slide

  30. "Tardy Sweep" problem
    ● In JS, object references are directional, so
    an entire tree of garbage nodes can be
    collected at once
    ● In GObject, references are only counted
    and not directional
    ● This does not play nicely with JS's
    garbage collector

    View full-size slide

  31. JS references

    View full-size slide

  32. JS references






    View full-size slide

  33. GObject references

    View full-size slide

  34. GObject references



    View full-size slide

  35. GObject references


    View full-size slide

  36. "Tardy Sweep" problem
    ● Mitigated by the "Big Hammer" patch
    thanks to Georges Stavracas (feaneron),
    scheduling more GCs than strictly
    necessary
    ● Also mitigated by the "Toggle toggle refs"
    patch from Giovanni Campagna long ago,
    now merged
    ● Still a problem though!
    ● Thanks to Georges for the graphs and the
    technical explanation on his blog
    ● Thanks to Andy Holmes for coining the
    name, the meme, and running interference
    on /r/linux

    View full-size slide

  37. Memory per object instance
    ● Each GObject that's exposed to JS has a
    proxy JS object
    ● JS objects are highly optimized in the
    engine, our custom state for them is not
    ● We shared one struct between instances
    (one per GObject instance) and prototypes
    (one per GObject class), many fields
    unnecessary for one or the other
    ● Thanks to Carlos Garnacho for starting
    this process
    BEFORE
    ● Instances 128 bytes (+32 if any JS state)
    ● Prototypes 288 bytes
    AFTER
    ● Instances 88 bytes (+32 if any JS state)
    ● Prototypes 192 bytes
    POSSIBLE FUTURE
    ● Instances always 56 bytes

    View full-size slide

  38. Argument cache
    ● Speed up calling introspected C functions from JS by caching instructions for
    how to convert their arguments from JS values to C data
    ● Another patch by Giovanni Campagna from years ago, never reviewed
    ● Issue #70
    ● Planned to land in 3.32

    View full-size slide

  39. Technical — Developer tools
    ● We've seen a few major improvements to the GJS developer experience
    ● Doesn't directly affect users
    ● Affects app developers and shell extension developers
    ● To a lesser degree it affects shell developers

    View full-size slide

  40. Profiler
    ● Since 3.28
    ● Run your program with GJS_ENABLE_PROFILER=1 in the environment
    ● Or run the gjs interpreter with --profile
    ● Option to use SIGUSR2 to start and stop the profiler
    ● Linux-only for now
    ● Thanks to Christian Hergert for writing the initial implementation, and to Endless for sponsoring my subsequent work

    View full-size slide

  41. JS Heap dumps
    ● Since 3.28
    ● Run your program with
    GJS_DEBUG_HEAP_OUTPUT=some_name
    in the environment
    ● Send SIGUSR1 to dump the JS heap
    ● Or call System.dumpHeap('some_name')
    in your program
    ● Use heapgraph.py (in gjs/tools/) to
    visualize all the objects
    ● Thanks to Andy Holmes, Tommi
    Komulainen (10 years old patch!), Juan
    Pablo Ugarte

    View full-size slide

  42. Debugger
    ● Working hard to land it in 3.30
    ● Run the gjs interpreter with -d
    ● GDB-like commands

    View full-size slide

  43. How you can help

    View full-size slide

  44. How you can help
    ● On the threshold of becoming a thriving, self-sustaining project
    ● We need more people working on it for that to happen
    ● After 1.5 years I know a lot about the GJS code base
    ● I'm not planning to go anywhere yet, so now is the time to spread that
    knowledge to other people
    ● Too many things still to do, for the number of maintainers we have

    View full-size slide

  45. Well-defined open projects
    ● Two possible approaches for reducing the memory used per object instance
    (issue #175, #176)
    ● GjsPromiseSource to clean up and streamline async operations (issue #1)
    ● Better console interpreter output (issue #107)
    ● About 10 bite-size newcomer-friendly issues, some should really go in 3.30,
    so start now!

    View full-size slide

  46. Less-well-defined open projects
    ● Bring some modern C++ expertise to the project (I'm winging it a lot)
    ● Write "Code Hospitable" documentation
    ● Create a good developer workflow (JHBuild is frustrating, Flatpak/Flapjack
    are not so useful for testing the shell and extensions, BuildStream is not
    mature and takes tons of disk space)
    ● Write a contributors bot that categorizes merge requests, suggests reviewers,
    backports MRs to stable branches, and thanks people
    ● Investigate XPCOM, may have a good solution for the tardy sweep problem
    ● Take some stale merge requests under your wing

    View full-size slide

  47. Thanks!
    Let's do some hacking! We'll be
    working on GJS during the
    unconference days
    Also here's Shady Javascript Cat from
    last year, because I like cat pictures
    (public domain)
    All screenshots were taken by me.
    Screenshots and presentation
    licensed under CC BY-ND 4.0.

    View full-size slide