Slide 1

Slide 1 text

TheReddest [email protected] PLANNING FOR THE HORIZONTAL SCALING NODE.JS APPLICATIONS Brandon Cannaday Thursday, April 4, 13

Slide 2

Slide 2 text

ME HOSTING, DATA, STATS FOR NODE.JS modulus.io Thursday, April 4, 13

Slide 3

Slide 3 text

WHEN TO SCALE? 1. RESPONSE TIMES 2. CPU USAGE 3. CONCURRENT CONNECTIONS Thursday, April 4, 13

Slide 4

Slide 4 text

NODEFLY.COM Thursday, April 4, 13

Slide 5

Slide 5 text

STARTING POINT mydomain.com SERVER > NODE Thursday, April 4, 13

Slide 6

Slide 6 text

NODE TWEAKS http.globalAgent.maxSockets = Number.MAX_VALUE; CONCURRENT OUTGOING CONNECTION LIMIT Thursday, April 4, 13

Slide 7

Slide 7 text

HURTLE: THE SERVER LINUX CONFIGURATION 1. FILE-MAX 2. SOMAXCONN 3. ULIMIT Thursday, April 4, 13

Slide 8

Slide 8 text

FILE-MAX SYSTEM FILE DESCRIPTOR LIMIT 1. Run sysctl -w fs.file-max=65535 2. Run sysctl -p Thursday, April 4, 13

Slide 9

Slide 9 text

SOMAXCONN SOCKET LISTEN QUEUE LENGTH 1. Run sysctl -w net.core.somaxconn=65535 2. Run sysctl -p Thursday, April 4, 13

Slide 10

Slide 10 text

ULIMIT PER PROCESS FILE DESCRIPTOR LIMIT 1. Edit /etc/security/limits.conf 2. Add the following: * soft nofile 65535 * hard nofile 65535 root soft nofile 65535 root hard nofile 65535 Thursday, April 4, 13

Slide 11

Slide 11 text

RUNNING SMOOTH mydomain.com SERVER > NODE Thursday, April 4, 13

Slide 12

Slide 12 text

HURTLE: THE CPU BUY A BIGGER BOX SERVER > NODE SERVER > NODE 1 CORE 4 CORES Thursday, April 4, 13

Slide 13

Slide 13 text

MULTICORE NODE 1 2 3 4 100% CORE USAGE Thursday, April 4, 13

Slide 14

Slide 14 text

CLUSTER MODULE SERVER > NODE > NODE > NODE > NODE mydomain.com Thursday, April 4, 13

Slide 15

Slide 15 text

CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } The Cluster Module Thursday, April 4, 13

Slide 16

Slide 16 text

CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Fork Children Thursday, April 4, 13

Slide 17

Slide 17 text

CLUSTER EXAMPLE var cluster = require('cluster'); var http = require('http'); var numCPUs = require('os').cpus().length; if(cluster.isMaster) { for(var i = 0; i < numCPUs; i++) { cluster.fork(); } } else { http.createServer(function(req, res) { res.writeHead(200); res.end('Hello World.'); }).listen(80); } Handle Requests Thursday, April 4, 13

Slide 18

Slide 18 text

CLUSTER LISTEN WORKER listen(...) MASTER Handle Thursday, April 4, 13

Slide 19

Slide 19 text

ROLLING UPDATES 1. UPDATE SCRIPT 2. WORKER -> STOP LISTENING 3. KILL WORKER 4. CALL FORK() AGAIN Thursday, April 4, 13

Slide 20

Slide 20 text

CLUSTER MODULE SERVER > NODE > NODE > NODE > NODE mydomain.com Thursday, April 4, 13

Slide 21

Slide 21 text

HURTLE: SHARED STATE SERVER > NODE > NODE > NODE > NODE NO SHARED STATE Thursday, April 4, 13

Slide 22

Slide 22 text

INSTALL REDIS SERVER > NODE > NODE > NODE > NODE REDIS Thursday, April 4, 13

Slide 23

Slide 23 text

EXAMPLE 1: SESSION var express = require('express'), app = express(); app.use(express.cookieParser()); app.use(express.session({ secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); MEMORY STORE Thursday, April 4, 13

Slide 24

Slide 24 text

EXAMPLE 1: SESSION var express = require('express'), RedisStore = require('connect-redis')(express), app = express(); app.use(express.cookieParser()); app.use(express.session({ store: new RedisStore({ host: 'localhost', port: 6379 }), secret: 'My Cookie Signing Secret' })); app.get('/', function(req, res) { req.session.somekey = 'some value'; }); REDIS STORE Thursday, April 4, 13

Slide 25

Slide 25 text

EXAMPLE 2: SOCKET.IO var RedisStore = require('socket.io/lib/stores/redis') , redis = require('socket.io/node_modules/redis') , pub = redis.createClient() , sub = redis.createClient() , client = redis.createClient(); io.set('store', new RedisStore({ redisPub : pub , redisSub : sub , redisClient : client })); https://github.com/LearnBoost/Socket.IO/wiki/Configuring-Socket.IO Thursday, April 4, 13

Slide 26

Slide 26 text

RUNNING SMOOTH SERVER > NODE > NODE > NODE > NODE REDIS mydomain.com Thursday, April 4, 13

Slide 27

Slide 27 text

LAST HURTLE: HORIZONTAL APP SERVER A > NODE > NODE > NODE > NODE REDIS APP SERVER B > NODE > NODE > NODE > NODE REDIS Thursday, April 4, 13

Slide 28

Slide 28 text

SEPARATE REDIS APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER Thursday, April 4, 13

Slide 29

Slide 29 text

LOAD BALANCING APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13

Slide 30

Slide 30 text

LOAD BALANCING 1. MANAGED 2. INSTALL ONE 3. WRITE YOUR OWN Thursday, April 4, 13

Slide 31

Slide 31 text

NGINX http { upstream mydomain_com { server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } LOAD BALANCER SERVER Thursday, April 4, 13

Slide 32

Slide 32 text

WRITE ONE https://github.com/substack/bouncy Thursday, April 4, 13

Slide 33

Slide 33 text

BOUNCY bouncy module var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13

Slide 34

Slide 34 text

BOUNCY Server collection var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13

Slide 35

Slide 35 text

BOUNCY Create server var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13

Slide 36

Slide 36 text

BOUNCY Bounce request var bouncy = require('bouncy'); var hosts = [ 'host1.mydomain.com', 'host2.mydomain.com' ]; var count = 0; var server = bouncy(function(req, res, bounce) { count++; var host = hosts[count % hosts.length]; bounce(host, 80); }); server.listen(80); Thursday, April 4, 13

Slide 37

Slide 37 text

AFFINITY SESSION AFFINITY STICKY SESSIONS SEND THE SAME PERSON BACK TO THE SAME SERVER Thursday, April 4, 13

Slide 38

Slide 38 text

NGINX AFFINITY http { upstream mydomain_com { sticky; server host1.mydomain.com:80; server host2.mydomain.com:80; } server { listen 80; server_name www.mydomain.com; location / { proxy_pass http://mydomain_com; } } } Thursday, April 4, 13

Slide 39

Slide 39 text

CUSTOM AFFINITY req.headers['x-forwarded-for'] req.connection.remoteAddress Thursday, April 4, 13

Slide 40

Slide 40 text

RUNNING SMOOTH APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LOAD BALANCER SERVER mydomain.com Thursday, April 4, 13

Slide 41

Slide 41 text

ROLLING UPDATES 1. REMOVE APP SERVER FROM LOAD BALANCER 2. UPGRADE APP SERVER 3. ADD BACK 4. REPEAT Thursday, April 4, 13

Slide 42

Slide 42 text

SSL TERMINATE EARLY Thursday, April 4, 13

Slide 43

Slide 43 text

SSL APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL SSL TERMINATOR Thursday, April 4, 13

Slide 44

Slide 44 text

SSL LB SERVER SSL mydomain.com 80 443 Thursday, April 4, 13

Slide 45

Slide 45 text

STUD https://github.com/bumptech/stud frontend = [*]:443 backend = [127.0.0.1]:80 ssl = on pem-file = "myCert.pem" EXAMPLE CONFIG FILE Thursday, April 4, 13

Slide 46

Slide 46 text

RUNNING SMOOTH W/SSL APP SERVER A > NODE > NODE > NODE > NODE APP SERVER B > NODE > NODE > NODE > NODE REDIS SERVER LB SERVER SSL mydomain.com Thursday, April 4, 13

Slide 47

Slide 47 text

HUGE REDIS SERVER LB SERVER SSL LB SERVER SSL Thursday, April 4, 13

Slide 48

Slide 48 text

DNS ROUND-ROBIN DNS MULTIPLE RECORDS, ONE DOMAIN Thursday, April 4, 13

Slide 49

Slide 49 text

ROUND-ROBIN DNS CLIENT 1 1. xxx.xxx.xxx.x 2. xxx.xxx.xxx.y CLIENT 2 1. xxx.xxx.xxx.y 2. xxx.xxx.xxx.x Thursday, April 4, 13

Slide 50

Slide 50 text

RUNNING SMOOTH REDIS SERVER LB SERVER SSL LB SERVER SSL Thursday, April 4, 13

Slide 51

Slide 51 text

BIG ENOUGH SERVER > NODE Thursday, April 4, 13

Slide 52

Slide 52 text

BIG ENOUGH SERVER > NODE Thursday, April 4, 13