Deploying client-side apps, 1000 (or so) at a time

Cascadia JS Server Day, July 2015

Transcript

1. Deploying client-side apps, 1,000 (or so) at a time Rebecca Murphey / CascadiaJS 2015
2. @rmurphey ~ rmurphey.com
3. this is a talk about how we’re rethinking the systems we 
 use to build and deploy client-side web applications 
 at Bazaarvoice this is not a talk about how we should all start building 
 and deploying client-side web applications
4. the scene ratings & reviews for brands & retailers SPA in OPP* single code base, multiple customers one build per site/locale combo lots of sites, lots of locales * Credit to Mike Pennisi for this way-cooler acronym for 3PJS.
5. what’s a build? verify API fetch config generate config make core app js make scout js make css embed font in css download images publish to s3 verify display publish scout file purge akamai probably some 
 other stuff i forgot
6. return util.format( 'grunt %s %s', args.join(':'), options.join(' ') );
7. how does it even no clear contract between steps primitive, blocking retries builds are lost for most failures no demand-based scaling of resources limited observability overall
8. what’s a build? a set of steps some serial, some parallel triggered by some initial input producing some final output always with the possibility of failure
9. what’s a step? a unit of work likely asynchronous possibly requiring input possibly depending on a service ideally producing output
11. { steps : [ { name : 'step1', activityType : 'readMessageBody', input : 'message:workflow' }, { name : 'step2', activityType : 'randomNumber' }, { name : 'step3', activityType : 'email', input : 'step1:body,step2:number' } ] }
12. var Q = require('q'); var SES = new AWS.SES(); module.exports = function (input) { return Q.ninvoke(SES, 'sendEmail', { // ... Message : { Subject : { Data : 'A message from Pontiac' }, Body : { Text : { Data : input.number + '\n' + input.body } } } }); };
17. orchestration the automated arrangement, coordination, 
 and management of complex computer systems, middleware, and services
19. “The one thing about Simple Workflow is it can be really hard to reason about, because things can happen out of order.” — (approximately) Joe Pollard
20. “The one thing about Simple Workflow is it can be really hard to reason about, because things can happen out of order.” — (approximately) Joe Pollard
21. pontiac a tool for describing and performing a workflow, using SWF, where a workflow consists of 1. a set of steps, 2. information about the input 
 required for each step, 3. information about the ideal duration and
 concurrency of those steps, and 4. the strategy for retrying failed steps
22. chopper receive a build message fetch locales create new messages per locale delete original message
24. module.exports = { name : 'chopper', steps : [ { name : 'getDeploymentLocales', activityType : 'chopper!getDeploymentLocales', input : 'message:workflow', retries : 1, timeout : 10 }, { name : 'createChoppedMessages', activityType : 'chopper!createChoppedMessages', input : 'message:workflow,locales:getDeploymentLocales' }, { name : 'deleteMessage', activityType : 'shared!deleteMessage', input : 'message:workflow' } ] };
25. getDeploymentLocales module.exports = function (input, log) { var message = parseContents(input.message); var deploymentUrl = message.deploymentUrl; var deploymentRevision = message.deploymentRevision; var request = getDeploymentDoc({ v : deploymentRevision }, deploymentUrl); var promise = request(function (response) { return response.locales; }, function (err) { log.error(err, 'BHive request failed'); throw err; }); return promise; };
26. size of step output is limited and must be a string; so, we have to store large or binary data no guarantee that steps will run on the same machine; can’t rely on filesystem retries, concurrency are first class clear contract between steps framework abstracts known complexities adding a step is exactly as hard as writing the step itself, and no harder
27. reality check none of this is in production yet; rewriting a mission-critical build system that isn’t visibly broken is a hard sell planning to pilot this instead with teams that have more critical deployment issues good news: lots of common steps across projects, and sharing will be easy now
28. lessons / reminders grunt is tools are easy to abuse identify and abstract common complexity write all code as testable modules there is great power in convention smart people have probably already solved 
 [at least part of] your problem
29. thank you @rmurphey @ttl_podcast rmurphey.com ttlpodcast.com