Big app design for Node.js

Designing the code of a large application in production and scaling up to thousands of users while doing it all in Node.js is a challenge that no developer should face alone. I will discuss how we have developed a highly decoupled, plugin-based architecture and a decentralized infrastructure for Cloud9 IDE, along with the technologies we've developed and the difficulties we faced in order to build the largest Node.js application that exists in production.

Transcript

1. BIG application design Saturday, November 10, 12
2. @sergimansilla Saturday, November 10, 12
3. http://c9.io Saturday, November 10, 12
4. Smart code completion Saturday, November 10, 12
5. Static analysis Saturday, November 10, 12
6. Free linux VM Saturday, November 10, 12
7. Bring your own machine Saturday, November 10, 12
8. Real terminal Saturday, November 10, 12
9. Hundreds of thousands of lines of JavaScript Saturday, November 10, 12
10. dynamic, weakly-typed language Saturday, November 10, 12
11. one single thread Saturday, November 10, 12
12. Remote VM Cloud9 datacenter IDE server Remote VM Remote VM Openshift Saturday, November 10, 12
13. Saturday, November 10, 12
14. Saturday, November 10, 12
15. Modularization Saturday, November 10, 12
16. Black box coding Saturday, November 10, 12
17. Saturday, November 10, 12
18. Good at abstracting away require Saturday, November 10, 12
19. Not that good at app-level abstraction require Saturday, November 10, 12
20. duplicated modules maps to folder names Relies on FS require hard to configure module Saturday, November 10, 12
21. Dependency error handling coding time compile time run time KA-BOOM! - Server crash - Unhappy customers - Developer gets fired Saturday, November 10, 12
22. How to fix it? Static dependency list Named services Easy configuration Resolve at startup No FS required Pass an object Saturday, November 10, 12
23. Architect github.com/c9/architect Saturday, November 10, 12
24. Everything is a plugin Plugins can consume plugins An application is just a set of plugins Architect Saturday, November 10, 12
25. Dependency model require Architect runtime compile time Saturday, November 10, 12
26. Dependency error handling Architect coding time compile time run time - Happy customers - Developer keeps job - Fails before release Saturday, November 10, 12
27. github.com/sergi/nodejsconfit2012 Saturday, November 10, 12
28. Saturday, November 10, 12
29. Declare entity ‘cook’ with behavior Use ‘cook’ to do some cooking Saturday, November 10, 12
30. Cook Cooking action Dependency model Saturday, November 10, 12
31. package.json Builds dependency tree executing code without Saturday, November 10, 12
32. { { cook/package.json do-­‐cooking/package.json Saturday, November 10, 12
33. Next Extract the code Wrap in Architect plugin code Make two plugins Saturday, November 10, 12
34. Function signature Call when done Saturday, November 10, 12
35. Architect plugin code Module.exports Options - we’ll talk about it later Imports - everything you ‘consume’ Register - call when done Saturday, November 10, 12
36. Saturday, November 10, 12
37. Saturday, November 10, 12
38. Easy to test Mock dependencies Saturday, November 10, 12
39. ‘cook’ should be called 8 times Saturday, November 10, 12
40. No black magic Specify dependency model Feed architect a config file Call ‘createApp’ Saturday, November 10, 12
41. Saturday, November 10, 12
42. Saturday, November 10, 12
43. Configuration Per-plugin options No global options object Specify in config file Saturday, November 10, 12
44. Saturday, November 10, 12
45. Saturday, November 10, 12
46. Options Passed in at startup Options can be dependencies Fail if options aren’t present Saturday, November 10, 12
47. Saturday, November 10, 12
48. Saturday, November 10, 12
49. Your app as chunks of functionality rather than sets of classes Saturday, November 10, 12
50. chunks of functionality Implicit type constraints Keep implementation private Swap features instead of interfaces Saturday, November 10, 12
51. How does Cloud9 use it? Open source version Local version (OS + sync) Hosted version Normal FTP SSH Saturday, November 10, 12
52. Saturday, November 10, 12
53. Swap feature per implementation On OSS: talk local filesystem On FTP: talk FTP library On SSH: talk via a SSH bridge Saturday, November 10, 12
54. Architect is a DI framework Saturday, November 10, 12
55. But a quite special one Saturday, November 10, 12
56. IDE instance (FTP) IDE instance (SSH) IDE instance (Normal) IDE instance (Normal) Single node.js process Other code (dashboard etc.) Saturday, November 10, 12
57. Architect can do Multiple instances of same plugin Run independently But in the same process Saturday, November 10, 12
58. Centralized event bus Loose coupling between plugins No hard dependencies! Can also do inter-context communication Saturday, November 10, 12
59. Event bus Plugin Other plugin Emit event React on event Saturday, November 10, 12
60. Saturday, November 10, 12
61. Saturday, November 10, 12
62. And now scale Swap it with i.e. Redis PubSub Plugins will never notice Modular awesomeness! Saturday, November 10, 12
63. Modularize in feature blocks Don’t over engineer Don’t create too small blocks They aren’t interfaces! Saturday, November 10, 12
64. Avoid context switching Less code! Less errors! Less boilerplate! Saturday, November 10, 12
65. Loose coupling Use (don’t abuse) an event bus Smaller dependency graph Saturday, November 10, 12
66. github.com/c9/architect Saturday, November 10, 12
67. Happy coding! Saturday, November 10, 12
68. @sergimansilla github.com/sergi http://c9.io Saturday, November 10, 12