How to Build 12-Factor Application in NodeJS Using Docker

Slide 1

Slide 1 text

How to Build 12-Factor Application in NodeJS using Docker Aditya Satrya Head of IT Development Jabar Digital Service JSDay Indonesia 2019

Slide 2

Slide 2 text

Outline 2 ▷ Cloud-native ▷ Docker ▷ Twelve-factor app ▷ How to write NodeJS app that follows 12-factor

Slide 3

Slide 3 text

Cloud-Native 3

Slide 4

Slide 4 text

4 Cloud-native is a set of technologies and techniques….

Slide 5

Slide 5 text

5 ...that empowers organization to...

Slide 6

Slide 6 text

6 ...build and run scalable applications… Keyword: scalable

Slide 7

Slide 7 text

7 ...in modern, dynamic environments such as public, private, and hybrid clouds Keyword: cloud

Slide 8

Slide 8 text

Cloud-Native application Able to get the maximum beneﬁt from cloud computing: ▷ Portability ▷ Scalable ▷ Resilient ▷ Minimize time & cost with automation ▷ Continuous deployment 8

Slide 9

Slide 9 text

9 Cloud-Native Trail-map 1. Containerization 2. CI/CD 3. Orchestration --below this are optional-- 4. Observability 5. Service Discovery 6. Networking & Policy 7. Distributed database & storage 8. Streaming & messaging 9. Container registry 10. Software distribution

Slide 10

Slide 10 text

Containerization (with Docker) 10

Slide 11

Slide 11 text

Containerization 11 ▷ You can’t do cloud-native without containerization ▷ Docker is the most popular

Slide 12

Slide 12 text

Containerization 12 ▷ a standard way to package your application's code, conﬁgurations, and dependencies into a single object. https://aws.amazon.com/containers/

Slide 13

Slide 13 text

Container vs VM 13 https://www.quora.com/What-is-Docker-Please-explain-it-in-simple-terms

Slide 14

Slide 14 text

Beneﬁt 14 ▷ Run anywhere ▷ Scale quickly ▷ Better resource utilization

Slide 15

Slide 15 text

Container is an instance of image 15 https://www.freecodecamp.org/news/ a-practical-guide-to-containers-dfa66 d37ac30/

Slide 16

Slide 16 text

16 https://www.freecodecamp.org/news/a-practical-guide-to-containers-dfa66d37ac30/

Slide 17

Slide 17 text

12-Factor App 17

Slide 18

Slide 18 text

What is 12-Factor app? ▷ Best practices to build app optimized for the cloud (cloud-native) 18

Slide 19

Slide 19 text

19 ▷ Drafted by developers at Heroku (2011) ▷ http://12factor.net

Slide 20

Slide 20 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 20 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 21

Slide 21 text

21 How can we achieve 12-factor app using Docker?

Slide 22

Slide 22 text

22 Example in NodeJS

Slide 23

Slide 23 text

NodeJS app that follow 12-factor 23

Slide 24

Slide 24 text

File:index.js var app = express(); ... app.get('/', function (req, res) { connection.query('SELECT * FROM names LIMIT 1' , function (err, rows, fields) { res.send('Hello ' + rows[0].first_name + ' ' + rows[0].last_name); }) }); app.listen(8000, function () { console.log('Server is listening on port 8000' ); }) 24

Slide 25

Slide 25 text

Run in browser 25

Slide 26

Slide 26 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 26 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 27

Slide 27 text

I. One Codebase 27 ▷ only one codebase per app ▷ many deploys of the app

Slide 28

Slide 28 text

I. One Codebase 28 ▷ Violation: ○ Multiple app sharing one codebase ▷ Solution: ○ Treat as dependency

Slide 29

Slide 29 text

I. One Codebase 29 ▷ Violation: ○ Different code for different version of an app ▷ Solution: ○ Use git branch/tag for different version

Slide 30

Slide 30 text

I. One Codebase 30 ▷ Beneﬁt: ○ Easy for automation (CI/CD) ○ Encourage smaller codebase (build & run faster, improve maintainability)

Slide 31

Slide 31 text

▷ Trunk-Based Use branching strategy 31

Slide 32

Slide 32 text

Use branching strategy 32 ▷ GitFlow

Slide 33

Slide 33 text

One Codebase, Many Deploys $ docker build -t asatrya/nodejs-12factor-app:1.0 . $ docker build -t asatrya/nodejs-12factor-app:2.0 . $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE asatrya/nodejs-12factor-app 1.0 ca4305291fb1 4 minutes ago 907MB asatrya/nodejs-12factor-app 2.0 eec4a2c8a951 2 minutes ago 907MB 33

Slide 34

Slide 34 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 34 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 35

Slide 35 text

II. Explicitly declare and isolate dependencies 35 ▷ Never relies on implicit existence of system-wide packages ▷ Isolation

Slide 36

Slide 36 text

II. Explicitly declare and isolate dependencies 36 ▷ Beneﬁt: ○ Improve predictability ○ Simpliﬁes setup

Slide 37

Slide 37 text

Directory Structure $ ls Dockerfile index.js package.json 37

Slide 38

Slide 38 text

File:package.json { "name": "nodejs_12factor_app", "version": "1.0.0", ... "dependencies": { "express": "^4.17.1", "mysql": "^2.17.1" } } 38

Slide 39

Slide 39 text

File:Dockerfile FROM node:10 # Create app directory RUN mkdir /app WORKDIR /app # Install app dependencies COPY package.json . RUN npm install COPY . . # Copy app source EXPOSE 8080 # Expose listened port CMD [ "node", "index.js" ] 39

Slide 40

Slide 40 text

Build Docker Image $ docker build -t asatrya/nodejs-12factor-app:1.0 . $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE asatrya/nodejs-12factor-app 1.0 ca4305291fb1 4 minutes ago 907MB 40

Slide 41

Slide 41 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 41 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 42

Slide 42 text

III. Store config in the environment 42 ▷ Config could be: ○ Resource handles to backing services ○ Credentials to external services ▷ Config varies across deploys, code does not

Slide 43

Slide 43 text

III. Store conﬁg in the environment 43 ▷ Strict separation of conﬁg from code ▷ Litmus test: ○ Ready to be made open source anytime?

Slide 44

Slide 44 text

III. Store config in the environment 44 ▷ Violation: ○ Store config as constants in the code ▷ Solution: ○ Store config as environment variables

Slide 45

Slide 45 text

File:index.js ... var mysql = require('mysql'); var connection = mysql.createConnection({ host: process.env.MYSQL_HOST, user: process.env.MYSQL_USER, password: process.env.MYSQL_PASSWORD, database: process.env.MYSQL_DATABASE, }); ... 45

Slide 46

Slide 46 text

Run Container $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ asatrya/nodejs-12factor-app:1.0 46

Slide 47

Slide 47 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 47 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 48

Slide 48 text

▷ Assume local resource are temporal and disposable ▷ Share-nothing architecture VI. Execute the app as stateless processes 48

Slide 49

Slide 49 text

▷ Violation: ○ Store persistent data/ﬁles in local ﬁlesystem ▷ Solution: ○ Store in S3 VI. Execute the app as stateless processes 49

Slide 50

Slide 50 text

▷ Beneﬁt: ○ Easy to scale out ○ Resilient VI. Execute the app as stateless processes 50

Slide 51

Slide 51 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 51 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 52

Slide 52 text

IV. Treat backing services as attached resources 52

Slide 53

Slide 53 text

IV. Treat backing services as attached resources 53 ▷ Able to attach/detach backing services without any code change

Slide 54

Slide 54 text

IV. Treat backing services as attached resources 54 ▷ Beneﬁt: ○ Flexibility ○ Resilience ○ Loose coupling

Slide 55

Slide 55 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 55 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 56

Slide 56 text

V. Strictly separate build and run stages 56

Slide 57

Slide 57 text

V. Strictly separate build and run stages 57 ▷ Violation: ○ Include conﬁg values in build artefact ○ Alter code when running app

Slide 58

Slide 58 text

V. Strictly separate build and run stages 58 ▷ Beneﬁt: ○ Easy for rollback ○ Promotes the separation of concerns

Slide 59

Slide 59 text

Separate Build,Release & Run # build $ docker build -t asatrya/nodejs-12factor-app:1.0 . # release $ docker push asatrya/nodejs-12factor-app:1.0 # run $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ asatrya/nodejs-12factor-app:1.0 59

Slide 60

Slide 60 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 60 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 61

Slide 61 text

X. Dev/prod parity 61 Traditional app 12-Factor App Code push to deploy Days Minutes/Hours Code authors vs code deployers Different people Same people Dev vs production environments Different Similar as possible

Slide 62

Slide 62 text

X. Dev/prod parity 62 ▷ Beneﬁt: ○ Faster delivery ○ More predictability

Slide 63

Slide 63 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 63 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 64

Slide 64 text

▷ your application might run as ○ http://localhost:12001 on local ○ http://192.168.1.10:2000 on staging ○ http://app.company.com on production ▷ without having to change any code. VII. Export services via port binding 64

Slide 65

Slide 65 text

▷ Beneﬁt: ○ Your apps can be a backing services for another app ○ Flexibility VII. Export services via port binding 65

Slide 66

Slide 66 text

Slide 67

Slide 67 text

Run Container $ docker run --rm \ -e MYSQL_HOST=db \ -e MYSQL_USER=dbuser \ -e MYSQL_PASSWORD=dbpassword \ -e MYSQL_DATABASE=dbname \ -p 8001:8000 asatrya/nodejs-12factor-app:1.0 67

Slide 68

Slide 68 text

Run in browser 68

Slide 69

Slide 69 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 69 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 70

Slide 70 text

VI. Concurrency 70

Slide 71

Slide 71 text

VI. Concurrency 71 ▷ Promotes separation of concerns (microservices) ▷ Scale out instead of scale up

Slide 72

Slide 72 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 72 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 73

Slide 73 text

IX. Disposability 73 ▷ Minimize startup time ▷ Shutdown gracefully

Slide 74

Slide 74 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 74 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 75

Slide 75 text

XI. Treat logs as event streams 75 ▷ Write logs to stdout ▷ External system to collect logs into a centralized location

Slide 76

Slide 76 text

Code I. One Codebase One codebase tracked in revision control, many deploys II. Dependencies Explicitly declare and isolate dependencies III. Conﬁg Store conﬁg in the environment VI. Processes Execute the app as one or more stateless processes 76 IV. Backing services Treat backing services as attached resources V. Build, release, run Strictly separate build and run stages X. Dev/prod parity Keep development, staging, and production as similar as possible VII. Port binding Export services via port binding VIII. Concurrency Scale out via the process model IX. Disposability Maximize robustness with fast startup and graceful shutdown XI. Logs Treat logs as event streams XII. Admin processes Run admin tasks as one-off processes Deploy Operate

Slide 77

Slide 77 text

XII. Run admin tasks as one-oﬀ processes 77 ▷ Examples of administrative processes: ○ Database migrations ○ Interactive programming consoles (REPLs) ○ Timed scripts (ex: batch job) ○ Executes custom code only once