Building_a_Custom_CMS_on_Node.js.pdf

Slide 1

Slide 1 text

Building a Custom CMS on Node.js Recept.nu Technical Case Study - 2015 Peter Marklund, Tech Lead, TV4

Slide 2

Slide 2 text

Recept.nu History ● Polopoly ● EPiServer ● In-house custom development

Slide 3

Slide 3 text

Our Mission ● Build a new in-house CMS ● Migrate all data ● Keep the website as is (1:1 migration) ● Time frame: september 2014 - february 2015 ● Use the right tools for the job

Slide 4

Slide 4 text

The Dev Team ● Peter Marklund (TV4, Tech Lead) ● Leopold Roos (Netlight, Backend) ● Eduardo Castaneda (Valtech, Frontend) ● Anders Windelhed (Valtech, Backend) ● Mikael Engver (Itera, EPiServer)

Slide 5

Slide 5 text

Recept.nu ● 20,000+ recipes and articles ● Celebrity chefs: Tommy Myllimäki, Leila Lindholm, Paulo Roberto ● TV4 food programs: Sveriges Mästerkock, Mitt Kök, Nyhetsmorgon, Halv Åtta hos mig ● Simple, responsive, and static recipe website

Slide 6

Slide 6 text

Data Model ● Recipe ● Chef/Source/Article (ContentItem) ● Widget ● Image ● Category ● Ingredient ● URL (content lookup)

Slide 7

Slide 7 text

Architecture Three Node.js applications: ● CMS (admin UI, talks to API) ● REST API ● Website (talks to API)

Slide 8

Slide 8 text

Data Hierarchy URL -> Page -> Section -> Widget -> Recipe -> Chef -> Source -> Image -> Category -> Ingredient

Slide 9

Slide 9 text

What is Node.js? ● Released 2009 ● Google V8 JavaScript engine ● Module and package system (npm) ● Async event loop for non-blocking IO ● Network libraries (HTTP, streaming, sockets) ● Used by: PayPal, LinkedIn, Microsoft etc.

Slide 10

Slide 10 text

Angular.js Take Aways ● Much easier to get started with than Ember ● Productive when building a CRUD app ● Nobody in our team disliked it or had major problems working with it ● It’s a big framework and the learning curve gets steeper down the road ● All is being re-written in Angular 2.0

Slide 11

Slide 11 text

Atom Editor ● autocomplete-plus ● linter-jshint ● linter-jscs ● travis-ci-status

Slide 12

Slide 12 text

The Build (gulp test) ● Code linting (“use strict”, jshint, jscs) ● Check exact package versions and auto update ● Unit tests and API tests ● Migrate data from the old CMS ● Angular.js Protractor tests (in the CMS) ● Integration tests run on build machine

Slide 13

Slide 13 text

Deployment Heroku (AWS) with addons: ● Algolia (search), Found (elasticsearch) ● Compose (mongodb) ● Newrelic (monitoring) ● Sentry (error notification) ● Logentries (logging) ● loader.io (load testing) ● Keen IO (analytics) ● Stormpath (login)

Slide 14

Slide 14 text

API Test Example ● jsonapitest Parse Example on Github

Slide 15

Slide 15 text

Caching and CDN ● Akamai for all www content ● Redis cache for all API requests from www

Slide 16

Slide 16 text

Travis CI ● Builds and deploys the staging branch ● Builds and deploys the production branch

Slide 17

Slide 17 text

Callback Hell Three async patterns: ● Nested callbacks (traditional approach) ● The async library ● Promises (Bluebird)

Slide 18

Slide 18 text

Problems with Callbacks ● Cannot return a value ● Cannot raise an exception ● Limited call stack (you’ll miss the stack trace when debugging) ● No guarantees (can invoke callback never or too many times) ● The invoker of the function may not pass a callback (use it in a synchronous fashion)

Slide 19

Slide 19 text

Callback Example function searchRecipes(query, callback) { assert(query); searchEngine.search({q: query, type: ‘recipe’}, function(err, result) { if (err) callback(err); if (result) { callback(null, result.hits); } }); }

Slide 20

Slide 20 text

JavaScript (ES5) is Messy ● typeof ● null, undefined ● equality (==, ===) ● true/false (+0, -0, NaN, “”) ● variable scope ● this ● function arity

Slide 21

Slide 21 text

Selected Gotchas ● The underscore each method breaks if you iterate an object with a length property (workaround is to iterate Object.keys(obj)) ● Circular Node.js module (file) dependencies ● assert.equal is non-strict

Slide 22

Slide 22 text

Utility Libraries to the Rescue ● lodash - utility belt ● ramda - functional programming

Slide 23

Slide 23 text

Node.js/JavaScript Strengths ● Functions are first class ● Simple/efficient modules ● Object literals ● JSON ● JavaScript is everywhere

Slide 24

Slide 24 text

Unifying Frontend and Backend ● Backend developers really learn JavaScript ● Frontend developers can contribute on the backend ● There is less context switching ● Sometimes you can get confused about whether you are on the client or server though

Slide 25

Slide 25 text

The Future ● IO.js (Node.js fork) ● ES6 (next version of JavaScript) ● ES6 generators and coroutines (better async patterns) ● Google V8 Strong/Sane Mode, SoundScript ● TypeScript and Angular 2 ● React.js

Slide 26

Slide 26 text

ES6 ● let (block scope) ● Map, Set (collections with iterators) ● generators ● => (lambda) ● destructuring ● string interpolation ● classes ● modules ● promises

Slide 27

Slide 27 text

Lesson: Node.js Good Practices ● NODE_PATH=. ● ‘use strict’; ● jshint and jscs ● modularize to small files ● find a good async style

Slide 28

Slide 28 text

Lesson: First Rule of Engineering “Never build a CMS” ● Versioning ● Scheduled publishing ● Widgets and sections ● Menus

Slide 29

Slide 29 text

Lesson: Frameworks are Useful ● Ruby on Rails is still very competitive for admin UI:s. It’s fun to reimplement Rails but it tends to get time consuming and/or messy ● Model callbacks and validations ● Layered architecture: DAO - Model - API - Controller/Router

Slide 30

Slide 30 text

Lesson: No, SQL! ● MongoDB is not great at associations, structure, integrity, and data types ● Our CMS happens to need all of the above ● Most web developers know SQL ● The mix of SQL and document store (JSON) in PostgreSQL is attractive

Slide 31

Slide 31 text

Lesson: Abandoning the Monolith can get Expensive and Repetitive ● Code duplication ● Source control (Git) duplication ● Deployment duplication ● Duplicated logging and debugging

Slide 32

Slide 32 text

Lesson: API Tests Help Refactorings ● We use a REST API test framework called jsonapitest for API contractual tests ● Our build makes 200+ HTTP requests against the API (takes only a few seconds) ● Fairly easy to write ● Good coverage

Slide 33

Slide 33 text

Lesson: You Need Integration Tests If you break your app up into services then you need to invest in integration tests that set up and test the entire system

Slide 34

Slide 34 text

Lesson: Performance and Scalability are Overrated ● You usually don’t need to handle thousands of simultaneous connections. If you do you may well get away with scaling processes and hardware horizontally ● With network latency it’s not always significant if your API call responds in 5 ms or 50 ms

Slide 35

Slide 35 text

Lesson: Success Factors ● Well designed code base that is easy to maintain (regardless of language) ● Frameworks and tools that make developers productive and happy ● The skill level of every single developer on the team ● Work close to stakeholders (remove any middleman) ● Get feedback early ● Migrate data early

Slide 36

Slide 36 text

Resources * Redemption from Callback Hell * Fibers, Event Loop and Meteor * Callbacks vs Coroutines * Goodbye MongoDB, Hello PostgreSQL * ES6: the future is now * Axel Rauschmayer on ES6 * Martin Fowler on Microservices * Microservices - Not a Free Lunch! * Experiments with Strengthening JavaScript