Deploying client-side apps, 1000 (or so) at a time

Deploying client-side apps, 1000 (or so) at a time

Cascadia JS Server Day, July 2015


Rebecca Murphey

July 10, 2015


  1. Deploying client-side apps, 1,000 (or so) at a time Rebecca

    Murphey / CascadiaJS 2015
  2. @rmurphey ~

  3. this is a talk about how we’re rethinking the systems

 use to build and deploy client-side web applications 
 at Bazaarvoice this is not a talk about how we should all start building 
 and deploying client-side web applications
  4. the scene ratings & reviews for brands & retailers SPA

    in OPP* single code base, multiple customers one build per site/locale combo lots of sites, lots of locales * Credit to Mike Pennisi for this way-cooler acronym for 3PJS.
  5. what’s a build? verify API fetch config generate config make

    core app js make scout js make css embed font in css download images publish to s3 verify display publish scout file purge akamai probably some 
 other stuff i forgot
  6. return util.format( 'grunt %s %s', args.join(':'), options.join(' ') );

  7. how does it even no clear contract between steps primitive,

    blocking retries builds are lost for most failures no demand-based scaling of resources limited observability overall
  8. what’s a build? a set of steps some serial, some

    parallel triggered by some initial input producing some final output always with the possibility of failure
  9. what’s a step? a unit of work likely asynchronous possibly

    requiring input possibly depending on a service ideally producing output
  10. None
  11. { steps : [ { name : 'step1', activityType :

    'readMessageBody', input : 'message:workflow' }, { name : 'step2', activityType : 'randomNumber' }, { name : 'step3', activityType : 'email', input : 'step1:body,step2:number' } ] }
  12. var Q = require('q'); var SES = new AWS.SES(); module.exports

    = function (input) { return Q.ninvoke(SES, 'sendEmail', { // ... Message : { Subject : { Data : 'A message from Pontiac' }, Body : { Text : { Data : input.number + '\n' + input.body } } } }); };
  13. None
  14. None
  15. None
  16. None
  17. orchestration the automated arrangement, coordination, 
 and management of complex

    computer systems, middleware, and services
  18. None
  19. “The one thing about Simple Workflow is it can be

    really hard to reason about, because things can happen out of order.” — (approximately) Joe Pollard
  20. “The one thing about Simple Workflow is it can be

    really hard to reason about, because things can happen out of order.” — (approximately) Joe Pollard
  21. pontiac a tool for describing and performing a workflow, using

    SWF, where a workflow consists of 1. a set of steps, 2. information about the input 
 required for each step, 3. information about the ideal duration and
 concurrency of those steps, and 4. the strategy for retrying failed steps
  22. chopper receive a build message fetch locales create new messages

    per locale delete original message
  23. None
  24. module.exports = { name : 'chopper', steps : [ {

    name : 'getDeploymentLocales', activityType : 'chopper!getDeploymentLocales', input : 'message:workflow', retries : 1, timeout : 10 }, { name : 'createChoppedMessages', activityType : 'chopper!createChoppedMessages', input : 'message:workflow,locales:getDeploymentLocales' }, { name : 'deleteMessage', activityType : 'shared!deleteMessage', input : 'message:workflow' } ] };
  25. getDeploymentLocales module.exports = function (input, log) { var message =

    parseContents(input.message); var deploymentUrl = message.deploymentUrl; var deploymentRevision = message.deploymentRevision; var request = getDeploymentDoc({ v : deploymentRevision }, deploymentUrl); var promise = request(function (response) { return response.locales; }, function (err) { log.error(err, 'BHive request failed'); throw err; }); return promise; };
  26. size of step output is limited and must be a

    string; so, we have to store large or binary data no guarantee that steps will run on the same machine; can’t rely on filesystem retries, concurrency are first class clear contract between steps framework abstracts known complexities adding a step is exactly as hard as writing the step itself, and no harder
  27. reality check none of this is in production yet; rewriting

    a mission-critical build system that isn’t visibly broken is a hard sell planning to pilot this instead with teams that have more critical deployment issues good news: lots of common steps across projects, and sharing will be easy now
  28. lessons / reminders grunt is tools are easy to abuse

    identify and abstract common complexity write all code as testable modules there is great power in convention smart people have probably already solved 
 [at least part of] your problem
  29. thank you @rmurphey @ttl_podcast