How to Build 12-Factor Application in NodeJS Using Docker

Nowadays, most applications are deployed on the cloud. It's important to build our application in cloud-native way so that it can gain the full potential of the cloud. The best practice to write cloud-native app is by following the 12-factor rule set, something published by Heroku in 2011 but still relevant today. In this talk, it will be discussed how to build a NodeJS application that follows 12-factor using Docker.

Transcript

1. How to Build 12-Factor Application in NodeJS using Docker Aditya Satrya Head of IT Development Jabar Digital Service JSDay Indonesia 2019
2. Outline 2 ▷ Cloud-native ▷ Docker ▷ Twelve-factor app ▷ How to write NodeJS app that follows 12-factor
3. Cloud-Native 3
4. 4 Cloud-native is a set of technologies and techniques….
5. 5 ...that empowers organization to...
6. 6 ...build and run scalable applications… Keyword: scalable
7. 7 ...in modern, dynamic environments such as public, private, and hybrid clouds Keyword: cloud
8. Cloud-Native application Able to get the maximum benefit from cloud computing: ▷ Portability ▷ Scalable ▷ Resilient ▷ Minimize time & cost with automation ▷ Continuous deployment 8
9. 9 Cloud-Native Trail-map 1. Containerization 2. CI/CD 3. Orchestration --below this are optional-- 4. Observability 5. Service Discovery 6. Networking & Policy 7. Distributed database & storage 8. Streaming & messaging 9. Container registry 10. Software distribution
10. Containerization (with Docker) 10
11. Containerization 11 ▷ You can’t do cloud-native without containerization ▷ Docker is the most popular
12. Containerization 12 ▷ a standard way to package your application's code, configurations, and dependencies into a single object. https://aws.amazon.com/containers/
13. Container vs VM 13 https://www.quora.com/What-is-Docker-Please-explain-it-in-simple-terms
14. Benefit 14 ▷ Run anywhere ▷ Scale quickly ▷ Better resource utilization
15. Container is an instance of image 15 https://www.freecodecamp.org/news/ a-practical-guide-to-containers-dfa66 d37ac30/
16. 16 https://www.freecodecamp.org/news/a-practical-guide-to-containers-dfa66d37ac30/
17. 12-Factor App 17
18. What is 12-Factor app? ▷ Best practices to build app optimized for the cloud (cloud-native) 18
19. 19 ▷ Drafted by developers at Heroku (2011) ▷ http://12factor.net
20. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 20 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
21. 21 How can we achieve 12-factor app using Docker?
22. 22 Example in NodeJS
23. NodeJS app that follow 12-factor 23
24. File:index.js var app = express(); ... app.get('/', function (req, res) { connection.query('SELECT * FROM names LIMIT 1' , function (err, rows, fields) { res.send('Hello ' + rows[0].first_name + ' ' + rows[0].last_name); }) }); app.listen(8000, function () { console.log('Server is listening on port 8000' ); }) 24
25. Run in browser 25
26. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 26 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
27. I. One Codebase 27 ▷ only one codebase per app ▷ many deploys of the app
28. I. One Codebase 28 ▷ Violation: ◦ Multiple app sharing one codebase ▷ Solution: ◦ Treat as dependency
29. I. One Codebase 29 ▷ Violation: ◦ Different code for different version of an app ▷ Solution: ◦ Use git branch/tag for different version
30. I. One Codebase 30 ▷ Benefit: ◦ Easy for automation (CI/CD) ◦ Encourage smaller codebase (build & run faster, improve maintainability)
31. ▷ Trunk-Based Use branching strategy 31
32. Use branching strategy 32 ▷ GitFlow
33. One Codebase, Many Deploys $ docker build -t asatrya/nodejs-12factor-app:1.0 . $ docker build -t asatrya/nodejs-12factor-app:2.0 . $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE asatrya/nodejs-12factor-app 1.0 ca4305291fb1 4 minutes ago 907MB asatrya/nodejs-12factor-app 2.0 eec4a2c8a951 2 minutes ago 907MB 33
34. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 34 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
35. II. Explicitly declare and isolate dependencies 35 ▷ Never relies on implicit existence of system-wide packages ▷ Isolation
36. II. Explicitly declare and isolate dependencies 36 ▷ Benefit: ◦ Improve predictability ◦ Simplifies setup
37. Directory Structure $ ls Dockerfile index.js package.json 37
38. File:package.json { "name": "nodejs_12factor_app", "version": "1.0.0", ... "dependencies": { "express": "^4.17.1", "mysql": "^2.17.1" } } 38
39. File:Dockerfile FROM node:10 # Create app directory RUN mkdir /app WORKDIR /app # Install app dependencies COPY package.json . RUN npm install COPY . . # Copy app source EXPOSE 8080 # Expose listened port CMD [ "node", "index.js" ] 39
40. Build Docker Image $ docker build -t asatrya/nodejs-12factor-app:1.0 . $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE asatrya/nodejs-12factor-app 1.0 ca4305291fb1 4 minutes ago 907MB 40
41. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 41 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
42. III. Store config in the environment 42 ▷ Config could be: ◦ Resource handles to backing services ◦ Credentials to external services ▷ Config varies across deploys, code does not
43. III. Store config in the environment 43 ▷ Strict separation of config from code ▷ Litmus test: ◦ Ready to be made open source anytime?
44. III. Store config in the environment 44 ▷ Violation: ◦ Store config as constants in the code ▷ Solution: ◦ Store config as environment variables
45. File:index.js ... var mysql = require('mysql'); var connection = mysql.createConnection({ host: process.env.MYSQL_HOST, user: process.env.MYSQL_USER, password: process.env.MYSQL_PASSWORD, database: process.env.MYSQL_DATABASE, }); ... 45
46. Run Container $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ asatrya/nodejs-12factor-app:1.0 46
47. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 47 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
48. ▷ Assume local resource are temporal and disposable ▷ Share-nothing architecture VI. Execute the app as stateless processes 48
49. ▷ Violation: ◦ Store persistent data/files in local filesystem ▷ Solution: ◦ Store in S3 VI. Execute the app as stateless processes 49
50. ▷ Benefit: ◦ Easy to scale out ◦ Resilient VI. Execute the app as stateless processes 50
51. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 51 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
52. IV. Treat backing services as attached resources 52
53. IV. Treat backing services as attached resources 53 ▷ Able to attach/detach backing services without any code change
54. IV. Treat backing services as attached resources 54 ▷ Benefit: ◦ Flexibility ◦ Resilience ◦ Loose coupling
55. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 55 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
56. V. Strictly separate build and run stages 56
57. V. Strictly separate build and run stages 57 ▷ Violation: ◦ Include config values in build artefact ◦ Alter code when running app
58. V. Strictly separate build and run stages 58 ▷ Benefit: ◦ Easy for rollback ◦ Promotes the separation of concerns
59. Separate Build,Release & Run # build $ docker build -t asatrya/nodejs-12factor-app:1.0 . # release $ docker push asatrya/nodejs-12factor-app:1.0 # run $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ asatrya/nodejs-12factor-app:1.0 59
60. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 60 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
61. X. Dev/prod parity 61 Traditional app 12-Factor App Code push to deploy Days Minutes/Hours Code authors vs code deployers Different people Same people Dev vs production environments Different Similar as possible
62. X. Dev/prod parity 62 ▷ Benefit: ◦ Faster delivery ◦ More predictability
63. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 63 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
64. ▷ your application might run as ◦ http://localhost:12001 on local ◦ http://192.168.1.10:2000 on staging ◦ http://app.company.com on production ▷ without having to change any code. VII. Export services via port binding 64
65. ▷ Benefit: ◦ Your apps can be a backing services for another app ◦ Flexibility VII. Export services via port binding 65
66. File:index.js var app = express(); ... app.get('/', function (req, res) { connection.query('SELECT * FROM names LIMIT 1' , function (err, rows, fields) { res.send('Hello ' + rows[0].first_name + ' ' + rows[0].last_name); }) }); app.listen(8000, function () { console.log('Server is listening on port 8000' ); }) 66
67. Run Container $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ -p 8001:8000 asatrya/nodejs-12factor-app:1.0 67
68. Run in browser 68
69. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 69 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
70. VI. Concurrency 70
71. VI. Concurrency 71 ▷ Promotes separation of concerns (microservices) ▷ Scale out instead of scale up
72. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 72 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
73. IX. Disposability 73 ▷ Minimize startup time ▷ Shutdown gracefully
74. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 74 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
75. XI. Treat logs as event streams 75 ▷ Write logs to stdout ▷ External system to collect logs into a centralized location
76. Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Config Store config in the environment VI. Processes Execute the app as one or more stateless processes 76 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate
77. XII. Run admin tasks as one-off processes 77 ▷ Examples of administrative processes: ◦ Database migrations ◦ Interactive programming consoles (REPLs) ◦ Timed scripts (ex: batch job) ◦ Executes custom code only once
78. XII. Run admin tasks as one-off processes 78 ▷ Traditional
79. XII. Run admin tasks as one-off processes 79 ▷ Cloud-native