Upgrade to Pro — share decks privately, control downloads, hide ads and more …

BigPipe

Arnout Kazemier
May 06, 2014
1.1k

 BigPipe

This talk introduced a brand new framework for Node.js which is based on the Blog post that facebook did back in 2010.

BigPipe allows you to slice up your side in re-usable components called pagelets which are flushed to the page fully async. These components can be released in to npm and re-mixed. Creating a full front to back npm experience.

Checkout the documentation on http://bigpipe.io

Arnout Kazemier

May 06, 2014
Tweet

Transcript

  1. !
    !
    !
    npm install --save bigpipe
    http://bigpipe.io

    View Slide

  2. 3rdEden 3rd-Eden arnout.kazemier

    View Slide

  3. what is bigpipe
    And why should you care

    View Slide

  4. facebook
    Introduced it on June 4, 2010
    on.fb.me/1kAHLer

    View Slide

  5. ◅ ▻ #
    tl;dr

    View Slide

  6. }
    Swaagie
    Martijn Swaagman
    3rd-Eden
    Arnout Kazemier

    View Slide

  7. ◅ ▻ #
    header
    item list
    side
    bar

    View Slide

  8. ◅ ▻ #
    pagelet

    View Slide

  9. ◅ ▻ #
    page

    View Slide








  10. !

    !


    !

    !



    !

    !

    View Slide



  11. <br/>pipe.arrive("login", {<br/>“css":["/b04d448dd7b67c3bdb2a75ff79198c840df88954.css"],<br/>“js":[],<br/>“processed”:1,<br/>“id":"0FWH-EFC8MTOQ-OLXRO9HG5H2-992RO1OR",<br/>“rpc":[],<br/>“remove":false,<br/>“authorized":true,<br/>“streaming":false,<br/>“data":{}<br/>});<br/>

    View Slide



  12. <br/>pipe.arrive("login", {<br/>“css":["/b04d448dd7b67c3bdb2a75ff79198c840df88954.css"],<br/>“js":[],<br/>“processed”:1,<br/>“id":"0FWH-EFC8MTOQ-OLXRO9HG5H2-992RO1OR",<br/>“rpc":[],<br/>“remove":false,<br/>“authorized":true,<br/>“streaming":false,<br/>“data":{}<br/>});<br/>

    View Slide

  13. why
    What do we gain by adopting this pattern

    View Slide

  14. ◅ ▻ #
    front-end/client & render performance

    JS



    %

    %



    %

    JS

    JS

    View Slide

  15. ◅ ▻ #
    & & & &
    & & & &
    & & & &
    optimal usage/performance

    View Slide

  16. improved perceived performance & time to interact
    generation latency rendering
    time

    View Slide

  17. perceived performance & time to interact
    generation latency rendering
    time

    View Slide

  18. developer satisfaction


    { }

    View Slide

  19. pagelet
    A small re-usable fragment which contains HTML, CSS and
    JavaScript

    View Slide

  20. ◅ ▻ #
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    //!
    // Alternatively, only depend on Pagelet!
    //!
    Pagelet = require(“pagelet”);

    View Slide

  21. ◅ ▻ #
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    //!
    // Code here!
    //!
    }).on(module);

    View Slide

  22. ◅ ▻ #
    var asyncthings = require(“./asyncthings”)!
    , Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    get: function get(done) {!
    asyncthings(function fetched(err, data) {!
    done(err, { packages: data });!
    });!
    }!
    }).on(module);

    View Slide

  23. ◅ ▻ #
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    things: require(“./asyncthings”),!
    get: function get(done) {!
    this.things(function fetched(err, data) {!
    done(err, { packages: data });!
    });!
    },!
    !
    post: function get(body, files, next) {!
    this.things(body, function process(err) {!
    next(err);!
    });!
    }!
    }).on(module);

    View Slide

  24. views
    Multiple built-in template
    engines supported
    ◅ ▻
    $ npm install --save ejs!
    !
    ..!
    !
    $ vim index.js!
    !
    ————————————————————————————————————————————
    !
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    view: “sidebar.ejs“!
    }).on(module);

    View Slide

  25. views
    Custom error template when
    this particular pagelet fails.
    Can be a different language.
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    view: “sidebar.ejs“,!
    error: “unavailable.jade“!
    }).on(module);

    View Slide

  26. auth
    Pagelet only shown when
    authorized
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    authorize: function (req, next) {!
    next(authenticate(req) === true);!
    }!
    }).on(module);

    View Slide

  27. auth
    Don’t remove the placeholder
    from the DOM
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    authorize: function (req, next) {!
    next(authenticate(req) === true);!
    },!
    remove: false!
    }).on(module);

    View Slide

  28. rpc
    Full real-time remote
    procedure calls
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet!
    , asyncthings = require(“./asyncthings”);!
    !
    Pagelet.extend({!
    RPC: [“search“],!
    search: function search(reply, arg) {!
    asyncthings(function (err, data) {!
    reply(err, data);!
    });!
    }!
    }).on(module);

    View Slide

  29. styling
    Name-spacing, pre-
    processing all happening
    transparently
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    css: “my-file.css“!
    }).on(module);

    View Slide

  30. styling
    Name-spacing, pre-
    processing all happening
    transparently
    ◅ ▻
    ul!
    &.foo!
    list-style: none!
    padding: 0!
    !
    ————————————————————————————————————————————
    !
    [data-pagelet="sidebar"] ul.foo {!
    list-style: none;!
    padding: 0;!
    }

    View Slide

  31. distribute
    Publish using npm
    ◅ ▻
    'use strict';!
    !
    var Pagelet = require(“bigpipe”).Pagelet;!
    !
    Pagelet.extend({!
    view: “view.jade”,!
    css: “my-file.sass“,!
    get: function get(done) {!
    setTimeout(function () {!
    done(undefined, { packages: 1 });!
    }, 10);!
    }!
    }).on(module);!
    !
    ————————————————————————————————————————————
    !
    $ npm publish packages-pagelet!

    View Slide

  32. distribute
    Share using npm
    ◅ ▻
    $ npm install packages-pagelet!
    !
    ————————————————————————————————————————————
    !
    'use strict';!
    !
    var Page = require(“bigpipe”).Page;!
    !
    Page.extend({!
    path: “/packages/:name“,!
    pagelets: {!
    “packages“: require(“packages-pagelet“)!
    }!
    }).on(module);

    View Slide

  33. distribute
    And remix existing pagelets
    ◅ ▻
    $ npm install packages-pagelet!
    !
    ————————————————————————————————————————————
    !
    'use strict’;!
    !
    var path = require(“path”)!
    , Pagelet = require(‘packages-pagelet');!
    !
    //!
    // Manually export the re-mixed instance!
    // which has the new custom css.!
    //!
    module.exports = Pagelet.extend({!
    css: path.join(__dirname, ‘custom.styl’)!
    });

    View Slide

  34. much
    much
    more
    ◅ ▻
    Pagelet.extend({!
    streaming: true,!
    engine: ‘hogan.js’,!
    query: [‘data.bar.sync.0.foo’],!
    fragment: fs.readFileSync(‘fragment’),!
    initialize: function initialize() {!
    this.debug(‘i broke something’);!
    },!
    dependencies: []!
    }).on(module);

    View Slide

  35. client
    Minimal, transparent framework

    View Slide

  36. ◅ ▻ #
    'use strict';!
    !
    pipe.once(“::render”, function (pagelet) {!
    //!
    // You can access the initialised pagelet here!
    //!
    });!

    View Slide

  37. #
    'use strict’;!
    !
    pagelet.render(‘hi’);
    render
    Refresh the HTML of every
    paglet

    View Slide

  38. #
    'use strict’;!
    !
    pagelet.submit(document.forms[0]);
    submit
    Submit a form over the real-
    time connection

    View Slide

  39. #
    'use strict';!
    !
    pagelet.search(‘foo’, function (err, row) {!
    console.log(err, row);!
    });
    rpc
    Simply call the same method
    name on the client

    View Slide

  40. sandboxing
    docker like containers for the browser

    View Slide

  41. asset loading
    async css & js

    View Slide

  42. page
    The entry point

    View Slide

  43. ◅ ▻ #
    'use strict';!
    !
    var Page = require(“bigpipe”).Page;

    View Slide

  44. ◅ ▻ #
    'use strict';!
    !
    var Page = require(“bigpipe”).Page;!
    !
    Page.extend({!
    pagelets: {!
    “pagelet-1”: “path to pagelet”,!
    “pagelet-2”: require(“custom-pagelet”)!
    }!
    }).on(module);

    View Slide

  45. ◅ ▻ #
    'use strict';!
    !
    var Page = require(“bigpipe”).Page;!
    !
    Page.extend({!
    pagelets: ”/directory/with/pagelets”!
    }).on(module);

    View Slide

  46. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/url/:param/:name”!
    }).on(module);
    path
    smart URL routing

    View Slide

  47. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: /^\/foo\/bar/i!
    }).on(module);
    path
    regexp based routing

    View Slide

  48. #
    path
    xRegExp name based routing
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: ‘/^\\/(?[\\d\\.]+)\\/foo/'!
    }).on(module);

    View Slide

  49. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”!
    }).on(module);
    view
    The first chunk and
    placeholders

    View Slide

  50. #
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !
    !

    view
    Manually include the
    {bootstrap} code

    View Slide

  51. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”,!
    authorize: function auth(req, next) {!
    next(false);!
    }!
    }).on(module);
    auth
    Authorize access to pages.

    View Slide

  52. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”,!
    mode: “async”,!
    !
    pagelets: {!
    “name”: “path to pagelet”,!
    “another”: require(“custom-pagelet”)!
    }!
    }).on(module);
    mode
    Different pagelet render
    modes.

    View Slide

  53. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”,!
    mode: “async”,!
    !
    pagelets: {!
    “name”: “path to pagelet”,!
    “another”: require(“custom-pagelet”)!
    }!
    }).on(module);
    mode
    async: fully async, unordered
    rendering

    View Slide

  54. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”,!
    mode: “pipeline”,!
    !
    pagelets: {!
    “name”: “path to pagelet”,!
    “another”: require(“custom-pagelet”)!
    }!
    }).on(module);
    mode
    pipeline: render in the
    specified order.

    View Slide

  55. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: “/“,!
    view: “index.ejs”,!
    mode: “sync”,!
    !
    pagelets: {!
    “name”: “path to pagelet”,!
    “another”: require(“custom-pagelet”)!
    }!
    }).on(module);
    mode
    sync: single flush, no fancy
    pancy rendering

    View Slide

  56. #
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    statusCode: 301,!
    method: ‘GET, POST, HEAD’,!
    contentType: ‘custom/html’,!
    data: { template: ‘data’ },!
    parsers: {},!
    charset: ‘UTF-7’,!
    dependencies: [],!
    initialize: function () {}!
    }).on(module);
    much
    much
    more

    View Slide

  57. putting it all together
    Creating your first BigPipe instance

    View Slide

  58. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’!
    });

    View Slide

  59. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: [Page, Page, Page, Page, Page]!
    dist: __dirname +’/dist’!
    });

    View Slide

  60. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’!
    });!
    !
    bigpipe.define([Page, Page, Page]);!
    bigpipe.define(__dirname +’/pages’);

    View Slide

  61. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(443, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’,!
    !
    key: fs.readFileSync(‘ssl.key’),!
    cert: fs.readFileSync(‘cert.key’)!
    });

    View Slide

  62. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(443, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’,!
    !
    key: fs.readFileSync(‘ssl.key’),!
    cert: fs.readFileSync(‘cert.key’),!
    redirect: 80!
    });

    View Slide

  63. ◅ ▻ #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(443, {!
    key: fs.readFileSync(‘ssl.key’),!
    cert: fs.readFileSync(‘cert.key’),!
    spdy: { /* SPDY configuration */ }!
    });

    View Slide

  64. #
    var BigPipe = require(“bigpipe”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’,!
    transformer: ‘engine.io’,!
    parser: ‘JSON’!
    });
    primus
    The transforming real-time
    engine

    View Slide

  65. primus
    Why is so damn _awesome_
    - seamless switching


    - additional fixes
    - plugin support
    - custom encoders
    - reconnect that works
    - offline detection
    - build with ♥
    engine.io, sockjs, socket.io, websockets
    and browserchannel

    View Slide

  66. primus
    started out as a wrapper
    around engine.io

    View Slide

  67. primus
    eventually got refactored to
    support more frameworks

    View Slide

  68. primus
    and finally it got named as
    we know it

    View Slide

  69. compiler
    optimising dependencies

    View Slide

  70. #
    status page
    custom and default error
    pages
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: ‘/404’,!
    statusCode: 404,!
    view: ‘404.ejs’!
    }).on(module);!
    !
    ————————————————————————————————————————————!
    'use strict’;!
    !
    var Page = require(‘bigpipe’).Page;!
    !
    Page.extend({!
    path: ‘/500’,!
    statusCode: 500,!
    view: ‘500.ejs’!
    }).on(module);

    View Slide

  71. #
    plugins
    a great way to extend BigPipe
    'use strict’;!
    !
    bigpipe.use('ack', {!
    // Only ran on the server.!
    server: function (bigpipe, options) {},!
    !
    // Runs on the client, it's automatically !
    // bundled.!
    client: function (bigpipe, options) {},!
    !
    // Optional library that needs to be!
    // bundled on the client!
    library: ‘',!
    !
    // Optional plugin specific options, will!
    // be merged with Bigpipe.options!
    options: {}!
    });

    View Slide

  72. #
    middleware
    for the sake of compatibility
    var BigPipe = require(“bigpipe”)!
    , serve = require(“static-serve”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’!
    });!
    !
    bigpipe.before(‘static’, serve());

    View Slide

  73. #
    middleware
    can be proxied to Primus
    var BigPipe = require(“bigpipe”)!
    , serve = require(“static-serve”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’!
    });!
    !
    bigpipe.before(‘static’, serve(), {!
    primus: true!
    });

    View Slide

  74. #
    middleware
    can be enabled/disabled at will
    var BigPipe = require(“bigpipe”)!
    , serve = require(“static-serve”);!
    !
    var bigpipe = BigPipe.createServer(8080, {!
    pages: __dirname +’/pages’,!
    dist: __dirname +’/dist’!
    });!
    !
    bigpipe.use(‘static’, serve());!
    bigpipe.disable(‘static’);!
    bigpipe.enable(‘static’);!
    bigpipe.remove(‘static’);!
    bigpipe.before(‘static’);

    View Slide

  75. live demo
    *crosses fingers*

    !

    View Slide

  76. browsenpm.org
    Open Source example of a BigPipe powered site

    View Slide

  77. public package page

    View Slide

  78. browsenpm.org
    github.com/nodejitsu/browsenpm.org
    !
    github.com/nodejitsu/packages-pagelet
    github.com/nodejitsu/npm-dependencies-pagelet
    github.com/nodejitsu/npm-documentation-pagelet
    github.com/nodejitsu/registry-status-pagelet
    github.com/nodejitsu/npm-search-pagelet

    View Slide

  79. future goals
    Quicklining or “turbo links”, pagelet & page cache, recursive/
    nested pagelets, streaming/diff-able assets, more tests and
    benchmarking.

    View Slide

  80. That’s all I got!
    Q/A: Tweet me @3rdEden or create github issues ;-)
    3rdEden
    3rd-Eden
    arnout.kazemier

    View Slide

  81. bigpipe.io
    !
    pagelet: github.com/bigpipe/paglet
    bigpipe: github.com/bigpipe/bigpipe
    pipe.js: github.com/bigpipe/pipe.js
    !
    primus: github.com/primus/primus
    browsenpm: browsenpm.org
    3rdEden
    3rd-Eden
    arnout.kazemier

    View Slide