Building_a_Custom_CMS_on_Node.js.pdf

Transcript

1. Building a Custom CMS on Node.js Recept.nu Technical Case Study - 2015 Peter Marklund, Tech Lead, TV4
2. Recept.nu History • Polopoly • EPiServer • In-house custom development
3. Our Mission • Build a new in-house CMS • Migrate all data • Keep the website as is (1:1 migration) • Time frame: september 2014 - february 2015 • Use the right tools for the job
4. The Dev Team • Peter Marklund (TV4, Tech Lead) • Leopold Roos (Netlight, Backend) • Eduardo Castaneda (Valtech, Frontend) • Anders Windelhed (Valtech, Backend) • Mikael Engver (Itera, EPiServer)
5. Recept.nu • 20,000+ recipes and articles • Celebrity chefs: Tommy Myllimäki, Leila Lindholm, Paulo Roberto • TV4 food programs: Sveriges Mästerkock, Mitt Kök, Nyhetsmorgon, Halv Åtta hos mig • Simple, responsive, and static recipe website
6. Data Model • Recipe • Chef/Source/Article (ContentItem) • Widget • Image • Category • Ingredient • URL (content lookup)
7. Architecture Three Node.js applications: • CMS (admin UI, talks to API) • REST API • Website (talks to API)
8. Data Hierarchy URL -> Page -> Section -> Widget -> Recipe -> Chef -> Source -> Image -> Category -> Ingredient
9. What is Node.js? • Released 2009 • Google V8 JavaScript engine • Module and package system (npm) • Async event loop for non-blocking IO • Network libraries (HTTP, streaming, sockets) • Used by: PayPal, LinkedIn, Microsoft etc.
10. Angular.js Take Aways • Much easier to get started with than Ember • Productive when building a CRUD app • Nobody in our team disliked it or had major problems working with it • It’s a big framework and the learning curve gets steeper down the road • All is being re-written in Angular 2.0
11. Atom Editor • autocomplete-plus • linter-jshint • linter-jscs • travis-ci-status
12. The Build (gulp test) • Code linting (“use strict”, jshint, jscs) • Check exact package versions and auto update • Unit tests and API tests • Migrate data from the old CMS • Angular.js Protractor tests (in the CMS) • Integration tests run on build machine
13. Deployment Heroku (AWS) with addons: • Algolia (search), Found (elasticsearch) • Compose (mongodb) • Newrelic (monitoring) • Sentry (error notification) • Logentries (logging) • loader.io (load testing) • Keen IO (analytics) • Stormpath (login)
14. API Test Example • jsonapitest Parse Example on Github
15. Caching and CDN • Akamai for all www content • Redis cache for all API requests from www
16. Travis CI • Builds and deploys the staging branch • Builds and deploys the production branch
17. Callback Hell Three async patterns: • Nested callbacks (traditional approach) • The async library • Promises (Bluebird)
18. Problems with Callbacks • Cannot return a value • Cannot raise an exception • Limited call stack (you’ll miss the stack trace when debugging) • No guarantees (can invoke callback never or too many times) • The invoker of the function may not pass a callback (use it in a synchronous fashion)
19. Callback Example function searchRecipes(query, callback) { assert(query); searchEngine.search({q: query, type: ‘recipe’}, function(err, result) { if (err) callback(err); if (result) { callback(null, result.hits); } }); }
20. JavaScript (ES5) is Messy • typeof • null, undefined • equality (==, ===) • true/false (+0, -0, NaN, “”) • variable scope • this • function arity
21. Selected Gotchas • The underscore each method breaks if you iterate an object with a length property (workaround is to iterate Object.keys(obj)) • Circular Node.js module (file) dependencies • assert.equal is non-strict
22. Utility Libraries to the Rescue • lodash - utility belt • ramda - functional programming
23. Node.js/JavaScript Strengths • Functions are first class • Simple/efficient modules • Object literals • JSON • JavaScript is everywhere
24. Unifying Frontend and Backend • Backend developers really learn JavaScript • Frontend developers can contribute on the backend • There is less context switching • Sometimes you can get confused about whether you are on the client or server though
25. The Future • IO.js (Node.js fork) • ES6 (next version of JavaScript) • ES6 generators and coroutines (better async patterns) • Google V8 Strong/Sane Mode, SoundScript • TypeScript and Angular 2 • React.js
26. ES6 • let (block scope) • Map, Set (collections with iterators) • generators • => (lambda) • destructuring • string interpolation • classes • modules • promises
27. Lesson: Node.js Good Practices • NODE_PATH=. • ‘use strict’; • jshint and jscs • modularize to small files • find a good async style
28. Lesson: First Rule of Engineering “Never build a CMS” • Versioning • Scheduled publishing • Widgets and sections • Menus
29. Lesson: Frameworks are Useful • Ruby on Rails is still very competitive for admin UI:s. It’s fun to reimplement Rails but it tends to get time consuming and/or messy • Model callbacks and validations • Layered architecture: DAO - Model - API - Controller/Router
30. Lesson: No, SQL! • MongoDB is not great at associations, structure, integrity, and data types • Our CMS happens to need all of the above • Most web developers know SQL • The mix of SQL and document store (JSON) in PostgreSQL is attractive
31. Lesson: Abandoning the Monolith can get Expensive and Repetitive • Code duplication • Source control (Git) duplication • Deployment duplication • Duplicated logging and debugging
32. Lesson: API Tests Help Refactorings • We use a REST API test framework called jsonapitest for API contractual tests • Our build makes 200+ HTTP requests against the API (takes only a few seconds) • Fairly easy to write • Good coverage
33. Lesson: You Need Integration Tests If you break your app up into services then you need to invest in integration tests that set up and test the entire system
34. Lesson: Performance and Scalability are Overrated • You usually don’t need to handle thousands of simultaneous connections. If you do you may well get away with scaling processes and hardware horizontally • With network latency it’s not always significant if your API call responds in 5 ms or 50 ms
35. Lesson: Success Factors • Well designed code base that is easy to maintain (regardless of language) • Frameworks and tools that make developers productive and happy • The skill level of every single developer on the team • Work close to stakeholders (remove any middleman) • Get feedback early • Migrate data early
36. Resources * Redemption from Callback Hell * Fibers, Event Loop and Meteor * Callbacks vs Coroutines * Goodbye MongoDB, Hello PostgreSQL * ES6: the future is now * Axel Rauschmayer on ES6 * Martin Fowler on Microservices * Microservices - Not a Free Lunch! * Experiments with Strengthening JavaScript