JS at Urban Airship

JS at Urban Airship

Talk given for a "Lunch and Learn" for Engineering at Urban Airship.

57f33120143ad72783fc64dbff4a2a67?s=128

Chris Dickinson

September 16, 2013
Tweet

Transcript

  1. { JS at Monday, September 16, 13

  2. {Some History. Monday, September 16, 13

  3. { SOME HISTORY: • airship is a bit quirky as

    far as Django projects go • but as regards JS, it was totally orthodox Monday, September 16, 13
  4. { SOME HISTORY: • “JS is just another static asset”

    • a pretty common attitude at the time Monday, September 16, 13
  5. { SOME HISTORY: Monday, September 16, 13

  6. { SOME HISTORY: Monday, September 16, 13

  7. { SOME HISTORY: Monday, September 16, 13

  8. { SOME HISTORY: Monday, September 16, 13

  9. { SOME HISTORY: Monday, September 16, 13

  10. { SOME HISTORY: Monday, September 16, 13

  11. { SOME HISTORY: Monday, September 16, 13

  12. { SOME HISTORY: Monday, September 16, 13

  13. { SOME HISTORY: Monday, September 16, 13

  14. { SOME HISTORY: Monday, September 16, 13

  15. { SOME HISTORY: • 3 major products were using a

    lot of JS • Rich Push Composer • Reports • Push Composer (not to mention various other pages) Monday, September 16, 13
  16. { SOME HISTORY: None of them were sharing code, even

    amongst themselves! Monday, September 16, 13
  17. { SOME HISTORY: WELL, that’s not entirely true. Monday, September

    16, 13
  18. { SOME HISTORY: WELL, that’s not entirely true. Monday, September

    16, 13
  19. { SOME HISTORY: WELL, that’s not entirely true. Monday, September

    16, 13
  20. { SOME HISTORY: • But the code worked • The

    site worked Monday, September 16, 13
  21. { SOME HISTORY: • I was the only dedicated JS

    developer for a long time. • There was no way I could replace all the legacy code in one fell swoop. Monday, September 16, 13
  22. { SOME HISTORY: The solution had to grow with airship.

    Monday, September 16, 13
  23. { SOME HISTORY: The solution had gradually replace the legacy

    code within airship. Monday, September 16, 13
  24. { SOME HISTORY: The solution had to scale beyond a

    single dedicated JS developer on airship. Monday, September 16, 13
  25. {We had to Modularize, Organize & Standardize our JS. Monday,

    September 16, 13
  26. {We had to Modularize, Organize & Standardize our JS. Monday,

    September 16, 13
  27. { MODULARIZE: • Modular JavaScript • Based on CommonJS •

    One module to a JS file. Monday, September 16, 13
  28. { MODULARIZE: • Modules may export any value • Modules

    may import the value exported by any other module Monday, September 16, 13
  29. { MODULARIZE: • Non-exported values cannot be affected by any

    other module. • Separation of concerns! Monday, September 16, 13
  30. { // import a value from another file var other

    = require('./path/to/file') MODULARIZE: Monday, September 16, 13
  31. { // commonjs: module.exports = 'any js value' // require.js:

    return 'any js value' MODULARIZE: Monday, September 16, 13
  32. { // require.js: define(function(require) { var other = require('./path') return

    'any js value' }) MODULARIZE: Monday, September 16, 13
  33. { MODULARIZE: • In general: Modules export functions that do

    one thing. • Functions should only take the bare minimum of information necessary to do their jobs. Monday, September 16, 13
  34. { MODULARIZE: Importantly: JS require’s should be static. Monday, September

    16, 13
  35. { MODULARIZE: Which is to say: everything that your app

    needs to work should be statically analyzable. Monday, September 16, 13
  36. { MODULARIZE: • Modules should be 200-300LOC maximum. Monday, September

    16, 13
  37. { MODULARIZE: • Break up big modules into small modules.

    Monday, September 16, 13
  38. { MODULARIZE: • Modules are broken up by moving module.js

    to module/ index.js • Then slicing discrete logic out into new modules in that directory. Monday, September 16, 13
  39. { $ wc -l airship/static/js/page/message.js 620 airship/static/js/page/message.js MODULARIZE: Monday, September

    16, 13
  40. {$ for i in airship/static/js/page/message/*.js; do wc -l $i; done

    96 airship/static/js/page/message/build_state.js 202 airship/static/js/page/message/index.js 149 airship/static/js/page/message/is_valid.js 58 airship/static/js/page/message/make_scheduled.js 30 airship/static/js/page/message/send.js 32 airship/static/js/page/message/setup_advanced.js 184 airship/static/js/page/message/setup_audience.js 77 airship/static/js/page/message/setup_draft.js 132 airship/static/js/page/message/setup_message.js 299 airship/static/js/page/message/setup_richpush.js 59 airship/static/js/page/message/setup_schedule.js 41 airship/static/js/page/message/setup_send.js MODULARIZE: Monday, September 16, 13
  41. {$ for i in airship/static/js/page/message/*.js; do wc -l $i; done

    96 airship/static/js/page/message/build_state.js 202 airship/static/js/page/message/index.js 149 airship/static/js/page/message/is_valid.js 58 airship/static/js/page/message/make_scheduled.js 30 airship/static/js/page/message/send.js 32 airship/static/js/page/message/setup_advanced.js 184 airship/static/js/page/message/setup_audience.js 77 airship/static/js/page/message/setup_draft.js 132 airship/static/js/page/message/setup_message.js 299 airship/static/js/page/message/setup_richpush.js 59 airship/static/js/page/message/setup_schedule.js 41 airship/static/js/page/message/setup_send.js MODULARIZE: Monday, September 16, 13
  42. { MODULARIZE: This system slowly but surely grew to encompass

    the entire JS codebase. Monday, September 16, 13
  43. { MODULARIZE: The Web team plans JS features in terms

    of what modules we can reuse, and what modules need to be created. Monday, September 16, 13
  44. {We had to Modularize, Organize & Standardize our JS. Monday,

    September 16, 13
  45. { ORGANIZE: • Modules are split up into directories based

    on the scope of the concerns Monday, September 16, 13
  46. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Monday, September 16, 13
  47. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: General purpose utility modules -- parse querystrings, zip tuples, etc. Monday, September 16, 13
  48. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: More complex modules. These often create objects with multiple methods. Monday, September 16, 13
  49. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: They’re expected to only deal with native JS objects -- no DOM objects allowed! Monday, September 16, 13
  50. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Modules that deal with DOM manipulation. Usually take a single element. Should not touch anything outside of that element. Monday, September 16, 13
  51. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: These modules are usually concerned with setting initial values, and emitting new user values over time. Monday, September 16, 13
  52. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: The entry point directories. Monday, September 16, 13
  53. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: These tie together modules from the widgets/ directory with templates from the templates/ directory. Monday, September 16, 13
  54. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Any information that Django should provide is pulled in these modules and provided to submodules. Monday, September 16, 13
  55. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: This includes: •URLs for XHRs •Available permissions •Model data Monday, September 16, 13
  56. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: The preferred method to pass data is by HTML5 data-attribute on a named container element. Monday, September 16, 13
  57. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Behaviors run on every page of the site. They should be as minimal as possible. Monday, September 16, 13
  58. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Page modules run only for a single view and are specified by Django via the @js('module') view decorator. Monday, September 16, 13
  59. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Often these modules are split up into subdirectories -- such as Segments and the Push Composer. Monday, September 16, 13
  60. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: This directory contains everything necessary for Require.JS to load templates. When we switch to browserify, it will be going away. Monday, September 16, 13
  61. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Our templates directory -- this is where all of our plate templates live. Monday, September 16, 13
  62. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: Plate is a Django- template-language-alike in JS. (It can use include, extend, and friends). Monday, September 16, 13
  63. { utils/ modules/ widgets/ behaviors/ page/ plugins/ templates/ thirdparty/ TOPIC:

    ORGANIZE: And this is where all of the thirdparty code lives. NB: It is not guaranteed to be wrapped in a module. Monday, September 16, 13
  64. {We had to Modularize, Organize & Standardize our JS. Monday,

    September 16, 13
  65. { STANDARDIZE: We created a style guide. Optimized for context

    switching between JS and Python. • No semicolons • Comma first • Opinionated http://git.io/h5ADLQ http://git.io/lDyAaA STYLE GUIDE LINTER Monday, September 16, 13
  66. { STANDARDIZE: • Use (a subset of) jQuery • Prefer

    ES5 builtins (vs. Lodash, Underscore, & friends) Monday, September 16, 13
  67. { STANDARDIZE: Use Node-style callbacks for async things that happen

    once. Monday, September 16, 13
  68. { Segments.get(id, function(err, segment) { if(err) { return ready(err) }

    }) STANDARDIZE: A typical API. Monday, September 16, 13
  69. { Segments.get(id, onsegment) function onsegment(err, segment) { if(err) { return

    ready(err) } } STANDARDIZE: Most callbacks should be hoisted, named functions. Monday, September 16, 13
  70. { function get_segment_body(id, ready) { return Segments.get(id, onsegment) function onsegment(err,

    segment) { if(err) { return ready(err) } return ready(null, segment.criteria) } } STANDARDIZE: You should always delegate the error to your caller’s callback! Monday, September 16, 13
  71. { STANDARDIZE: Use Node-style event emitters for things that repeatedly

    happen. Monday, September 16, 13
  72. { var EE = require('./utils/eventemitter') var ee = new EE

    ee.on('eventname', function(item) { // fired on each “eventname” }) ee.emit('eventname', <anything>) STANDARDIZE: Monday, September 16, 13
  73. { var EE = require('./utils/eventemitter') var ee = new EE

    ee.once('eventname', function(item) { // fired on only // the *first* “eventname” }) ee.emit('eventname', <anything>) STANDARDIZE: Monday, September 16, 13
  74. { var EE = require('./utils/eventemitter') var ee = new EE

    ee.on('eventname', function(a, b) { // multiple args! }) ee.emit('eventname', 1, 2) STANDARDIZE: Monday, September 16, 13
  75. { STANDARDIZE: Use Node-style streams for series of information over

    time. Monday, September 16, 13
  76. { var through = require('through') , stream = through(write) $('[name=loudify]').stream('keyup')

    .pipe(stream) .pipe($('#output')) function write(input) { stream.queue(input.toUpperCase()) } STANDARDIZE: Monday, September 16, 13
  77. { // Streams take a *type* of data // turn

    it into a new *type* // -- a function applied over time. var template = require('path/to/html') // for instance, templates take // Object -> String template.stream('context') STANDARDIZE: Monday, September 16, 13
  78. { // ObjectStates are streams that // listen to EE’s

    (or other streams) // and produce a stream of distinct // application states var os = new ObjectState os.listen( $('[name=input]').stream() , 'data' // event to listen for , ['attr'] // attribute name to set ) os.pipe(template.stream('context')) STANDARDIZE: Monday, September 16, 13
  79. { // jQuery elements are readable // and writable. //

    when written to, their innerHTML is // replaced. os .pipe(template.stream('context')) .pipe($('#target')) STANDARDIZE: Monday, September 16, 13
  80. { // example: autocomplete input_el.stream() .pipe(template('/path/{{ qs(ctxt) }}')) .pipe(debounce()) .pipe(xhr.get())

    // <- backpressure .pipe(template('<html>')) .pipe(output_el) STANDARDIZE: Monday, September 16, 13
  81. { // a pattern: function transform() { var stream =

    through(write) return stream function write(data) { // EXAMPLE: randomly delay // the outgoing data! setTimeout(function() { stream.queue(data) }, Math.random() * 100 + 100) } } STANDARDIZE: Monday, September 16, 13
  82. { // using the aforementioned pattern, it’s // easy to

    write a stream in-situ, then // move it out to another module when // the time comes. STANDARDIZE: Monday, September 16, 13
  83. { STANDARDIZE: Example: UI Patterns Monday, September 16, 13

  84. { STANDARDIZE: BUTTONS, POPOVERS & PICKERS Monday, September 16, 13

  85. { STANDARDIZE: •All of these examples use “Popover”’s •Most of

    these examples are “Picker”’s Monday, September 16, 13
  86. { STANDARDIZE: •Pickers are reused in different ways by different

    products. •But they’re all implemented as streams of data over time. Monday, September 16, 13
  87. { STANDARDIZE: •Popovers show and hide a given content area

    and are “attached” to an element in the DOM. •Over time, the user can “show” or “hide” them. •These are exposed as “events”. Monday, September 16, 13
  88. { STANDARDIZE: •Some buttons have direct, one-time effects. •“Delete a

    Segment” is a good example. •It happens once and either succeeds or fails •“Delete XYZ” is implemented as a callback-accepting function. Monday, September 16, 13
  89. {We had to Modularize, Organize & Standardize our JS. Monday,

    September 16, 13
  90. {We had to Modularize, Organize & Standardize our JS. Oh,

    and TEST IT. Monday, September 16, 13
  91. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: Drive.JS was my first(ish) project after landing at UA. Monday, September 16, 13
  92. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: Drive.JS was patterned after Mocha’s TDD UI. Monday, September 16, 13
  93. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: It is both a test driver and a test runner. Monday, September 16, 13
  94. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: It runs in browser and in a headless JSDOM mode for quick local tests. Monday, September 16, 13
  95. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: This is just Require.JS’s require. Monday, September 16, 13
  96. { suite('suite name', function(require) { test('test name', function() { assert.equal(lhs,

    rhs) }) }) TESTS: This is just the CommonJS assert module. Monday, September 16, 13
  97. { suite('suite name', function(require) { test('test name', function(done) { setTimeout(function()

    { done() }, 100 * Math.random()) }) }) TESTS: Async tests take a “done” function, and have to call it when they’re finished. Monday, September 16, 13
  98. { suite('suite name', function(require) { suite.redirects('/hardcoded', '/test') }) TESTS: Some

    (bad) programs hard-code XHR URL paths. Drive lets you redirect all hardcoded paths to a different URL. Monday, September 16, 13
  99. { suite('suite name', function(require) { // yada yada }) html('../path/to/document.html')

    TESTS: Point to a different HTML document to serve as the base “HTML” for this test suite. Monday, September 16, 13
  100. { endpoints({ '/test': test_url }) function test_url(req, resp) { resp.end('hey

    buddy') } suite('suite name', function(require) { var request = require('./modules/request') test('suite name', function(done) { request.get('/test', function(err, data) { if(err) return done(err) assert.equal(data, 'hey buddy') }) }) }) TESTS: Stub out the server side of XHR interactions. Monday, September 16, 13
  101. { TESTS: We avoid testing direct “entry point” modules directly.

    (Those are a job for Selenium!) All other modules should be tested, though. Monday, September 16, 13
  102. { FAQ Monday, September 16, 13

  103. { FAQ: So why don’t we use an existing JS

    framework? Monday, September 16, 13
  104. { FAQ: •Ember •Backbone •Spine •BatmanJS •Knockout •AngularJS •JavaScriptMVC •Polymer

    •... Monday, September 16, 13
  105. { FAQ: We are not precluded from using these frameworks.

    Our system only dictates that we use CommonJS modules. Monday, September 16, 13
  106. { FAQ: That said, it’s my opinion that these frameworks

    don’t really fit our problem space. Monday, September 16, 13
  107. { FAQ: Are we NIH-ing things? Monday, September 16, 13

  108. { FAQ: Nope. We use quite a few popular frameworks

    and libraries. Monday, September 16, 13
  109. { FAQ: jQuery, D3, Highcharts, Flot, Polymaps, Require.JS, Browserify, Through,

    Duplexer, ES5-shim, AnalyticsJS, Esprima, Falafel, ... Monday, September 16, 13
  110. { FAQ: ...With the potential to use many more as

    we move to browserify and NPM. Monday, September 16, 13
  111. { FAQ: Why npm? Why not bower? Monday, September 16,

    13
  112. { FAQ: Monday, September 16, 13

  113. { FAQ: npm: 41k packages bower: 4245 repos component: 1263

    packages ... (not to mention volo, ender, and friends.) Monday, September 16, 13