JavaScript in GNOME 2018

This talk is about all the improvements made in GNOME's JavaScript platform in the past year. We've made many strides: developer experience, especially for new contributors; new JavaScript language features; and performance improvements, especially in memory usage. I'll talk about the improvements and how they affect the four audiences: users, app developers, GNOME Shell developers, and shell extension developers. I'll also talk about some projects that we need your help with!

Transcript

1. Javascript in GNOME 2018 Philip Chimento — ptomato, @therealptomato GUADEC, July 7, 2017
2. Introduction — Contents of this talk • Social progress • Technical progress • How you can help
3. Introduction — The Four Audiences • Users • App developers • Shell developers • Shell extension developers
4. Social Progress
5. Social — We switched to GitLab!
6. Switching to GitLab • Many more first-time and drive-by contributors • Maintainer's patch review workflow much easier • How did we live without continuous integration? • Someone could look into a merge request bot
7. Social — Continuous Integration
8. Continuous Integration • We run linters • We build and run tests with GCC and Clang • Contributors get feedback on their merge requests within a few minutes • "Thorough tests" run only at scheduled intervals on master, or manually • Thanks to Claudio André
9. Social — Linters • Linters are good, they remove tedium and turnaround time from code review • Can be frustrating for drive-by contributors or new contributors who haven't yet internalized the code style rules • Linter configurations still need some tweaking; thanks to Avi Zajac (llzes) for doing some work on this • There are no good C++ linters compared to what we're used to in JS • In general still a work in progress
10. Social — Deployment • Code coverage report published automatically: https://gnome.pages.gitlab.gnome.org/gjs/ • Flatpak package with GJS interpreter created when manually requested at end of CI pipeline • Thanks to Claudio André for working on this • Let Claudio know what would be useful for you as an app developer or shell extension developer
11. Social — Documentation site
12. Documentation site • http://devdocs.baznga.org/ • GSoC project of Evan Welsh (rockon999) • Improvements to automatically generated documentation • Sample apps with best practices • Everaldo Canuto continues to host the website for free, thanks!
13. Social — Review backlog • Before migrating to GitLab, I turned all the stale patches from the last 10 years into merge requests • Most of them are merged now • There's still a backlog • Thanks to Cosimo Cecchi for reviewing almost every line that I write • Want to help? Adopt a stale merge request
14. Social — Press • Due to the "tardy sweep problem" (more later) we were indirectly discussed on OMG!Ubuntu, Reddit /r/linux, and a few other Linux-watcher sites • Reactions ranged from constructive to toxically negative or comically uninformed • Thanks to Sri Ramkrishna and Andy Holmes for being unflappable positive voices
16. Effect on the community • One surprising side effect is that the big press got the GNOME community thinking about how to fix the tardy sweep problem in GJS and the shell • Generated a lot of ideas that I couldn't have thought of myself
17. Social — Collaboration with other projects • Mozilla: We've been collaborating with the SpiderMonkey team to make SpiderMonkey 60 work as a standalone JS engine without any downstream patches • We're continuing to badger Mozilla to get a good tarball release process in place for standalone SpiderMonkey • Cinnamon: At least one developer has interest in re-unifying CJS and GJS for a future Cinnamon release, we've merged some of their patches
18. Technical Progress
19. Technical — Language & Platform • Javascript engine upgraded to SpiderMonkey 60 (GNOME 3.29.4 or .90) • Fewer JS language changes • Affects app developers, shell developers, and shell extension developers • Not much change for users
20. Async iteration There's now support in the language for "asynchronous iterators" as per the latest ES standard. https://jsbin.com/folotu/edit?js,console /t Jake Archibald for the example async function* asyncRandomNumbers() { const url = 'https://www.random.org/...'; while (true) { const response = await fetch(url); const text = await response.text(); yield Number(text); } } async function example() { for await (const num of asyncRandomNumbers()) { console.log(num); if (num > 0.95) break; } }
21. Rest operator in object destructuring As per the latest ES standard, you can now unpack certain properties from an object and use the rest operator to catch the remaining ones. gjs> var {a, b, ...cd} = .... {a: 1, b: 2, c: 3, d: 4}; gjs> a 1 gjs> b 2 gjs> JSON.stringify(cd) "{"c":3,"d":4}"
22. Spread operator in object literals Also new in the ES standard, you can merge objects or add properties to them using the spread operator. gjs> const obj1 = {a: 1, b: 2}; gjs> const obj2 = {c: 3, d: 4}; gjs> const merged = {...obj1, ...obj2}; gjs> JSON.stringify(merged) "{"a":1,"b":2,"c":3,"d":4}"
23. Anonymous catch Some days you just don't care about errors.* Then why bother binding a name to them? *Of course, all GNOME code always handles errors properly try { dangerousOperation(); } catch (e) { void e; // make linter happy print("Error, but I don't care"); } try { dangerousOperation(); } catch { print("Error, but I don't care"); } NO YES
24. Finally, a finally method Available in many popular Promise libraries, now part of built-in Promises. (Since you can have try...catch...finally with await expressions, it makes sense that it must be possible with Promises too) load(file) .then(text => { print(text); }) .catch(error => { logError(error, "Oh no!"); }) .finally(() => { mainLoop.quit(); });
25. Removals! Nonstandard Mozilla syntax Conditional catch For-each-in Legacy lambda syntax Legacy iterators, array comprehensions, generator comprehensions catch (e if e.code === 23) {...} catch (e) { if (e.code === 23) {...} } for each (let a in obj) {...} for (let a of Object.values(obj)) {...} for each (let a in arr) {...} for (let a of arr) {...} arr.map(function (x) x * x); arr.map(x => x * x); [for (x of iterable) if (cond(x)) expr(x)] iterable.filter(cond).map(expr) YES NO YES NO YES NO YES NO YES NO
27. ByteArray / Uint8Array • Uint8Array is Javascript's way of handling byte arrays • ByteArray is GJS's custom solution, pre-dating Uint8Array • ByteArray would have had to be rewritten for SpiderMonkey 60 anyway • ByteArray now wrapper around Uint8Array • Some introspected functions now return Uint8Array instead of ByteArray, so if you don't want to port your code, there is one thing you have to change to keep it working const ba = functionReturningSomeBytes(); const ba = new ByteArray.ByteArray( functionReturningSomeBytes()); NO YES
28. ByteArray / Uint8Array • If you do want to port your code, the main difference is that Uint8Array's length is fixed • Assigning past the end of ByteArray lengthens the array • Assigning past the end of Uint8Array is ignored • Length of Uint8Array must be set at creation time const a = ByteArray.fromArray([1, 2, 3]); a[3] = 4; let a = Uint8Array.from([1, 2, 3]); a = Uint8Array.of(...a, 4); const a = new ByteArray.ByteArray(); a[0] = 255; const a = new Uint8Array(1); a[0] = 255; NO NO YES YES
29. ES6 Modules • Not certain to land in 3.30, otherwise 3.32 • Thanks to djrenren • Follow along at merge request 150 • Porting has to happen top-down! ES6 modules can't be imported inside code that's imported via the old imports object import main from 'js/app'; main(ARGV);
30. GIO Async operations • Outreachy project of Avi Zajac (llzes) • Intended to land in 3.32 • If you pass a callback to GIO async methods, you'll get the C-style callback API as usual • If you omit the callback, you'll get a Promise • Promises work transparently with async functions and await expressions • Support for annotations in gobject-introspection to benefit other languages try { let [contents] = await file.load_contents_async(null); print(contents); let info = await file.query_info_async( 'standard::*', Gio.FileQueryInfoFlags.NONE, GLib.PRIORITY_DEFAULT, null); print(info.get_size(), 'bytes'); } catch (e) { logError(e, 'Oh no!'); } finally { loop.quit(); }
31. Technical — Performance • A few notable performance improvements • Including one that was widely ridiculed on Reddit • Mostly affects users
32. "Tardy Sweep" problem • In JS, object references are directional, so an entire tree of garbage nodes can be collected at once • In GObject, references are only counted and not directional • This does not play nicely with JS's garbage collector
33. JS references
34. JS references ♻ ♻ ♻ ♻ ♻ ♻
35. GObject references ♻
36. GObject references ♻ ♻ ♻
37. GObject references ♻ ♻
38. "Tardy Sweep" problem • Mitigated by the "Big Hammer" patch thanks to Georges Stavracas (feaneron), scheduling more GCs than strictly necessary • Also mitigated by the "Toggle toggle refs" patch from Giovanni Campagna long ago, now merged • Still a problem though! • Thanks to Georges for the graphs and the technical explanation on his blog • Thanks to Andy Holmes for coining the name, the meme, and running interference on /r/linux
39. Memory per object instance • Each GObject that's exposed to JS has a proxy JS object • JS objects are highly optimized in the engine, our custom state for them is not • We shared one struct between instances (one per GObject instance) and prototypes (one per GObject class), many fields unnecessary for one or the other • Thanks to Carlos Garnacho for starting this process BEFORE • Instances 128 bytes (+32 if any JS state) • Prototypes 288 bytes AFTER • Instances 88 bytes (+32 if any JS state) • Prototypes 192 bytes POSSIBLE FUTURE • Instances always 56 bytes
40. Argument cache • Speed up calling introspected C functions from JS by caching instructions for how to convert their arguments from JS values to C data • Another patch by Giovanni Campagna from years ago, never reviewed • Issue #70 • Planned to land in 3.32
41. Technical — Developer tools • We've seen a few major improvements to the GJS developer experience • Doesn't directly affect users • Affects app developers and shell extension developers • To a lesser degree it affects shell developers
42. Profiler • Since 3.28 • Run your program with GJS_ENABLE_PROFILER=1 in the environment • Or run the gjs interpreter with --profile • Option to use SIGUSR2 to start and stop the profiler • Linux-only for now • Thanks to Christian Hergert for writing the initial implementation, and to Endless for sponsoring my subsequent work
43. JS Heap dumps • Since 3.28 • Run your program with GJS_DEBUG_HEAP_OUTPUT=some_name in the environment • Send SIGUSR1 to dump the JS heap • Or call System.dumpHeap('some_name') in your program • Use heapgraph.py (in gjs/tools/) to visualize all the objects • Thanks to Andy Holmes, Tommi Komulainen (10 years old patch!), Juan Pablo Ugarte
44. Debugger • Working hard to land it in 3.30 • Run the gjs interpreter with -d • GDB-like commands
45. How you can help
46. How you can help • On the threshold of becoming a thriving, self-sustaining project • We need more people working on it for that to happen • After 1.5 years I know a lot about the GJS code base • I'm not planning to go anywhere yet, so now is the time to spread that knowledge to other people • Too many things still to do, for the number of maintainers we have
47. Well-defined open projects • Two possible approaches for reducing the memory used per object instance (issue #175, #176) • GjsPromiseSource to clean up and streamline async operations (issue #1) • Better console interpreter output (issue #107) • About 10 bite-size newcomer-friendly issues, some should really go in 3.30, so start now!
48. Less-well-defined open projects • Bring some modern C++ expertise to the project (I'm winging it a lot) • Write "Code Hospitable" documentation • Create a good developer workflow (JHBuild is frustrating, Flatpak/Flapjack are not so useful for testing the shell and extensions, BuildStream is not mature and takes tons of disk space) • Write a contributors bot that categorizes merge requests, suggests reviewers, backports MRs to stable branches, and thanks people • Investigate XPCOM, may have a good solution for the tardy sweep problem • Take some stale merge requests under your wing
49. Thanks! Let's do some hacking! We'll be working on GJS during the unconference days Also here's Shady Javascript Cat from last year, because I like cat pictures (public domain) All screenshots were taken by me. Screenshots and presentation licensed under CC BY-ND 4.0.
50. Questions?