Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Big app design for Node.js

Big app design for Node.js

Designing the code of a large application in production and scaling up to thousands of users while doing it all in Node.js is a challenge that no developer should face alone. I will discuss how we have developed a highly decoupled, plugin-based architecture and a decentralized infrastructure for Cloud9 IDE, along with the technologies we've developed and the difficulties we faced in order to build the largest Node.js application that exists in production.

B083b8207ccd0744a5abb18c8e75d24d?s=128

Sergi Mansilla

November 10, 2012
Tweet

More Decks by Sergi Mansilla

Other Decks in Technology

Transcript

  1. BIG application design Saturday, November 10, 12

  2. @sergimansilla Saturday, November 10, 12

  3. http://c9.io Saturday, November 10, 12

  4. Smart code completion Saturday, November 10, 12

  5. Static analysis Saturday, November 10, 12

  6. Free linux VM Saturday, November 10, 12

  7. Bring your own machine Saturday, November 10, 12

  8. Real terminal Saturday, November 10, 12

  9. Hundreds of thousands of lines of JavaScript Saturday, November 10,

    12
  10. dynamic, weakly-typed language Saturday, November 10, 12

  11. one single thread Saturday, November 10, 12

  12. Remote VM Cloud9 datacenter IDE server Remote VM Remote VM

    Openshift Saturday, November 10, 12
  13. Saturday, November 10, 12

  14. Saturday, November 10, 12

  15. Modularization Saturday, November 10, 12

  16. Black box coding Saturday, November 10, 12

  17. Saturday, November 10, 12

  18. Good at abstracting away require Saturday, November 10, 12

  19. Not that good at app-level abstraction require Saturday, November 10,

    12
  20. duplicated modules maps to folder names Relies on FS require

    hard to configure module Saturday, November 10, 12
  21. Dependency error handling coding time compile time run time KA-BOOM!

    - Server crash - Unhappy customers - Developer gets fired Saturday, November 10, 12
  22. How to fix it? Static dependency list Named services Easy

    configuration Resolve at startup No FS required Pass an object Saturday, November 10, 12
  23. Architect github.com/c9/architect Saturday, November 10, 12

  24. Everything is a plugin Plugins can consume plugins An application

    is just a set of plugins Architect Saturday, November 10, 12
  25. Dependency model require Architect runtime compile time Saturday, November 10,

    12
  26. Dependency error handling Architect coding time compile time run time

    - Happy customers - Developer keeps job - Fails before release Saturday, November 10, 12
  27. github.com/sergi/nodejsconfit2012 Saturday, November 10, 12

  28. Saturday, November 10, 12

  29. Declare entity ‘cook’ with behavior Use ‘cook’ to do some

    cooking Saturday, November 10, 12
  30. Cook Cooking action Dependency model Saturday, November 10, 12

  31. package.json Builds dependency tree executing code without Saturday, November 10,

    12
  32. { { cook/package.json do-­‐cooking/package.json Saturday, November 10, 12

  33. Next Extract the code Wrap in Architect plugin code Make

    two plugins Saturday, November 10, 12
  34. Function signature Call when done Saturday, November 10, 12

  35. Architect plugin code Module.exports Options - we’ll talk about it

    later Imports - everything you ‘consume’ Register - call when done Saturday, November 10, 12
  36. Saturday, November 10, 12

  37. Saturday, November 10, 12

  38. Easy to test Mock dependencies Saturday, November 10, 12

  39. ‘cook’ should be called 8 times Saturday, November 10, 12

  40. No black magic Specify dependency model Feed architect a config

    file Call ‘createApp’ Saturday, November 10, 12
  41. Saturday, November 10, 12

  42. Saturday, November 10, 12

  43. Configuration Per-plugin options No global options object Specify in config

    file Saturday, November 10, 12
  44. Saturday, November 10, 12

  45. Saturday, November 10, 12

  46. Options Passed in at startup Options can be dependencies Fail

    if options aren’t present Saturday, November 10, 12
  47. Saturday, November 10, 12

  48. Saturday, November 10, 12

  49. Your app as chunks of functionality rather than sets of

    classes Saturday, November 10, 12
  50. chunks of functionality Implicit type constraints Keep implementation private Swap

    features instead of interfaces Saturday, November 10, 12
  51. How does Cloud9 use it? Open source version Local version

    (OS + sync) Hosted version Normal FTP SSH Saturday, November 10, 12
  52. Saturday, November 10, 12

  53. Swap feature per implementation On OSS: talk local filesystem On

    FTP: talk FTP library On SSH: talk via a SSH bridge Saturday, November 10, 12
  54. Architect is a DI framework Saturday, November 10, 12

  55. But a quite special one Saturday, November 10, 12

  56. IDE instance (FTP) IDE instance (SSH) IDE instance (Normal) IDE

    instance (Normal) Single node.js process Other code (dashboard etc.) Saturday, November 10, 12
  57. Architect can do Multiple instances of same plugin Run independently

    But in the same process Saturday, November 10, 12
  58. Centralized event bus Loose coupling between plugins No hard dependencies!

    Can also do inter-context communication Saturday, November 10, 12
  59. Event bus Plugin Other plugin Emit event React on event

    Saturday, November 10, 12
  60. Saturday, November 10, 12

  61. Saturday, November 10, 12

  62. And now scale Swap it with i.e. Redis PubSub Plugins

    will never notice Modular awesomeness! Saturday, November 10, 12
  63. Modularize in feature blocks Don’t over engineer Don’t create too

    small blocks They aren’t interfaces! Saturday, November 10, 12
  64. Avoid context switching Less code! Less errors! Less boilerplate! Saturday,

    November 10, 12
  65. Loose coupling Use (don’t abuse) an event bus Smaller dependency

    graph Saturday, November 10, 12
  66. github.com/c9/architect Saturday, November 10, 12

  67. Happy coding! Saturday, November 10, 12

  68. @sergimansilla github.com/sergi http://c9.io Saturday, November 10, 12