Deploying client-side apps, 1000 (or so) at a time

Cascadia JS Server Day, July 2015

Transcript

1. Deploying client-side apps,
   
   1,000 (or so) at a time
   Rebecca Murphey / CascadiaJS 2015
   

   View Slide

2. @rmurphey ~ rmurphey.com
   

   View Slide

3. this is
   a talk about how we’re rethinking the systems we 

   use to build and deploy client-side web applications 

   at Bazaarvoice
   this is not
   a talk about how we should all start building 

   and deploying client-side web applications
   

   View Slide

4. the scene
   ratings & reviews for brands & retailers
   SPA in OPP*
   single code base, multiple customers
   one build per site/locale combo
   lots of sites, lots of locales
   * Credit to Mike Pennisi for this way-cooler acronym for 3PJS.
   

   View Slide

5. what’s a build?
   verify API
   fetch config
   generate config
   make core app js
   make scout js
   make css
   embed font in css
   download images
   publish to s3
   verify display
   publish scout file
   purge akamai
   probably some 

   other stuff i forgot
   

   View Slide

6. return util.format(
   'grunt %s %s', args.join(':'), options.join(' ')
   );
   

   View Slide

7. how does it even
   no clear contract between steps
   primitive, blocking retries
   builds are lost for most failures
   no demand-based scaling of resources
   limited observability overall
   

   View Slide

8. what’s a build?
   a set of steps
   some serial, some parallel
   triggered by some initial input
   producing some final output
   always with the possibility of failure
   

   View Slide

9. what’s a step?
   a unit of work
   likely asynchronous
   possibly requiring input
   possibly depending on a service
   ideally producing output
   

   View Slide

10. View Slide

11. {
    steps : [
    {
    name : 'step1',
    activityType : 'readMessageBody',
    input : 'message:workflow'
    },
    {
    name : 'step2',
    activityType : 'randomNumber'
    },
    {
    name : 'step3',
    activityType : 'email',
    input : 'step1:body,step2:number'
    }
    ]
    }
    

    View Slide

12. var Q = require('q');
    var SES = new AWS.SES();
    module.exports = function (input) {
    return Q.ninvoke(SES, 'sendEmail', {
    // ...
    Message : {
    Subject : {
    Data : 'A message from Pontiac'
    },
    Body : {
    Text : {
    Data : input.number + '\n' + input.body
    }
    }
    }
    });
    };
    

    View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. orchestration
    the automated arrangement, coordination, 

    and management of complex computer
    systems, middleware, and services
    

    View Slide

18. View Slide

19. “The one thing about Simple Workflow is it
    can be really hard to reason about,
    because things can happen out of order.”
    — (approximately) Joe Pollard
    

    View Slide

20. “The one thing about Simple Workflow is it
    can be really hard to reason about,
    because things can happen out of order.”
    — (approximately) Joe Pollard
    

    View Slide

21. pontiac
    a tool for describing and performing a workflow,
    using SWF, where a workflow consists of
    1. a set of steps,
    2. information about the input 

    required for each step,
    3. information about the ideal duration and

    concurrency of those steps, and
    4. the strategy for retrying failed steps
    

    View Slide

22. chopper
    receive a build message
    fetch locales
    create new messages per locale
    delete original message
    

    View Slide

23. View Slide

24. module.exports = {
    name : 'chopper',
    steps : [
    {
    name : 'getDeploymentLocales',
    activityType : 'chopper!getDeploymentLocales',
    input : 'message:workflow',
    retries : 1,
    timeout : 10
    },
    {
    name : 'createChoppedMessages',
    activityType : 'chopper!createChoppedMessages',
    input : 'message:workflow,locales:getDeploymentLocales'
    },
    {
    name : 'deleteMessage',
    activityType : 'shared!deleteMessage',
    input : 'message:workflow'
    }
    ]
    };
    

    View Slide

25. getDeploymentLocales
    module.exports = function (input, log) {
    var message = parseContents(input.message);
    var deploymentUrl = message.deploymentUrl;
    var deploymentRevision = message.deploymentRevision;
    var request = getDeploymentDoc({
    v : deploymentRevision
    }, deploymentUrl);
    var promise = request(function (response) {
    return response.locales;
    }, function (err) {
    log.error(err, 'BHive request failed');
    throw err;
    });
    return promise;
    };
    

    View Slide

26. size of step output is limited and must be a
    string; so, we have to store large or binary data
    no guarantee that steps will run on the
    same machine; can’t rely on filesystem
    retries, concurrency are first class
    clear contract between steps
    framework abstracts known complexities
    adding a step is exactly as hard as
    writing the step itself, and no harder
    

    View Slide

27. reality check
    none of this is in production yet;
    rewriting a mission-critical build system
    that isn’t visibly broken is a hard sell
    planning to pilot this instead with
    teams that have more critical
    deployment issues
    good news: lots of common steps
    across projects, and sharing will be
    easy now
    

    View Slide

28. lessons / reminders
    grunt is tools are easy to abuse
    identify and abstract common complexity
    write all code as testable modules
    there is great power in convention
    smart people have probably already solved 

    [at least part of] your problem
    

    View Slide

29. thank you
    @rmurphey
    @ttl_podcast
    rmurphey.com
    ttlpodcast.com
    

    View Slide