Scalability Issues: Cure now, prevent later

Slides for my Future Of Web Apps talk about scalability in Prague.

2. Hi   my  name   is  Thijs
3. THIJS
5. TAZE
6. Excited   to  be   here!
7. I’m   an  evangelist  at
8. I’m   a  board  member   at
11. It  works  on  my  computer
13. It’s  the  internet’s  fault
14. Developer  !=  web  developer
15. HTTP?  HUH?
19. What  do  you  do  when  things  go  wrong?
20. Do  a  complete  rewrite  !
21. Absolutely   nothing!
22. Throw  servers   at  the  problem
23. Bigger  box
24. More  boxes
25. Reduce   impact
26. What  kind  of  infrastructure  do  you  need?
27. One  big   box?
28. Scale  across   100  nodes?
29. That  big  F5   loadbalancer?
30. Only  the  DB   gets   hammered
31. Lots  of  staCc   traffic
32. Shared  hosCng   will  do  ...   except  during  the   month  of  the  event
33. SaaS PaaS IaaS
34. DB  &   webserver? Or  specialized  tools?
35. m ulC-­‐server  setup About  a
36. m ulC-­‐server  setup About  a Loadbalancing
37. m ulC-­‐server  setup About  a What  about  uploaded  files?
38. m ulC-­‐server  setup About  a What  about  sessions?
39. m ulC-­‐server  setup About  a Database  replicaCon
40. Impact
41. rt_sigaction(SIGQUIT,  {0x7f11489295f0,  [QUIT],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [QUIT],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [TERM],  NULL,  8)  =  0 rt_sigaction(SIGTERM,  {0x7f11489295f0,  [TERM],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [TERM],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [XCPU],  NULL,  8)  =  0 rt_sigaction(SIGXCPU,  {0x7f11489295f0,  [XCPU],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [XCPU],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [XFSZ],  NULL,  8)  =  0 rt_sigaction(SIGXFSZ,  {0x7f11489295f0,  [XFSZ],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [XFSZ],  NULL,  8)  =  0 open("/proc/meminfo",  O_RDONLY)                  =  3 fstat(3,  {st_mode=S_IFREG|0444,  st_size=0,  ...})  =  0 mmap(NULL,  4096,  PROT_READ|PROT_WRITE,  MAP_PRIVATE|MAP_ANONYMOUS,  -­‐1,  0)  =  0x7f115022 read(3,  "MemTotal:                2097152  kB\nMemFr"...,  1024)  =  560 close(3)                                                                =  0 munmap(0x7f1150225000,  4096)                        =  0 getrlimit(RLIMIT_NOFILE,  {rlim_cur=1024,  rlim_max=1024})  =  0 brk(0x1ae1000)                                                    =  0x1ae1000 brk(0x1b02000)                                                    =  0x1b02000 socket(PF_FILE,  SOCK_STREAM,  0)                  =  3 fcntl(3,  F_SETFL,  O_RDWR|O_NONBLOCK)        =  0 connect(3,  {sa_family=AF_FILE,  path="/var/run/nscd/socket"...},  110)  =  -­‐1  ECONNREFUSE close(3)                                                                =  0 socket(PF_FILE,  SOCK_STREAM,  0)                  =  3 fcntl(3,  F_SETFL,  O_RDWR|O_NONBLOCK)        =  0 connect(3,  {sa_family=AF_FILE,  path="/var/run/nscd/socket"...},  110)  =  -­‐1  ECONNREFUSE close(3)                                                                =  0 open("/etc/services",  O_RDONLY|O_CLOEXEC)  =  3 fstat(3,  {st_mode=S_IFREG|0644,  st_size=18480,  ...})  =  0 System calls
42. rt_sigaction(SIGQUIT,  {0x7f11489295f0,  [QUIT],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [QUIT],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [TERM],  NULL,  8)  =  0 rt_sigaction(SIGTERM,  {0x7f11489295f0,  [TERM],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [TERM],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [XCPU],  NULL,  8)  =  0 rt_sigaction(SIGXCPU,  {0x7f11489295f0,  [XCPU],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [XCPU],  NULL,  8)  =  0 rt_sigprocmask(SIG_BLOCK,  [XFSZ],  NULL,  8)  =  0 rt_sigaction(SIGXFSZ,  {0x7f11489295f0,  [XFSZ],  SA_RESTORER|SA_INTERRUPT,  0x7f114d583f rt_sigprocmask(SIG_UNBLOCK,  [XFSZ],  NULL,  8)  =  0 open("/proc/meminfo",  O_RDONLY)                  =  3 fstat(3,  {st_mode=S_IFREG|0444,  st_size=0,  ...})  =  0 mmap(NULL,  4096,  PROT_READ|PROT_WRITE,  MAP_PRIVATE|MAP_ANONYMOUS,  -­‐1,  0)  =  0x7f115022 read(3,  "MemTotal:                2097152  kB\nMemFr"...,  1024)  =  560 close(3)                                                                =  0 munmap(0x7f1150225000,  4096)                        =  0 getrlimit(RLIMIT_NOFILE,  {rlim_cur=1024,  rlim_max=1024})  =  0 brk(0x1ae1000)                                                    =  0x1ae1000 brk(0x1b02000)                                                    =  0x1b02000 socket(PF_FILE,  SOCK_STREAM,  0)                  =  3 fcntl(3,  F_SETFL,  O_RDWR|O_NONBLOCK)        =  0 connect(3,  {sa_family=AF_FILE,  path="/var/run/nscd/socket"...},  110)  =  -­‐1  ECONNREFUSE close(3)                                                                =  0 socket(PF_FILE,  SOCK_STREAM,  0)                  =  3 fcntl(3,  F_SETFL,  O_RDWR|O_NONBLOCK)        =  0 connect(3,  {sa_family=AF_FILE,  path="/var/run/nscd/socket"...},  110)  =  -­‐1  ECONNREFUSE close(3)                                                                =  0 open("/etc/services",  O_RDONLY|O_CLOEXEC)  =  3 fstat(3,  {st_mode=S_IFREG|0644,  st_size=18480,  ...})  =  0
43. CPU RAM IO
44. Modular   architecture
45. Connection   handling strategy
46. Path   lookups
47. Caching
48. Is  caching   the   answer?
49. Frontend Backend Data   cache Opcode   cache Page   cache APC Memcached SQLite File Redis +more +more
50. What? When  not? How  long? CompaCbility Limited  size
51. APC
52. Do  you  have  a   data  access   strategy?
53. Distributed   cache
55. memcached.sess_prefix = "memc.sess.key." session.save_path="127.0.0.1:11211" session.save_handler = memcached Also   for  session   clustering
57. Reverse   (caching)   proxies
58. Protects  your   servers
59. Put  ‘em  before   your   webserver
60. They  honor   your  caching   headers
63. Content Delivery Network
66. I like Varnish
67. Exceptional   performance
68. Open  source   software
69. Varnish   Configuration   Language
70. Supports   ESI
71. Can  handle   purging  &   banning
72. backend default { .host = "127.0.0.1"; .port = "8080"; } acl purge { "localhost"; "127.0.0.1"; "some.host.name.com"; "1.2.3.4"; }
73. sub vcl_recv { if (req.request == "PURGE") { if (!client.ip ~ purge) { error 405 "Not allowed: " + client.ip; } return(lookup); } if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { return (pipe); }
74. if (req.request != "GET" && req.request != "HEAD") { return (pass); } if (req.http.Accept-Encoding) { if (req.url ~ "\.(gif|jpg|jpeg|swf|flv|mp3|mp4| pdf|ico|png|gz|tgz|bz2)(\?.*|)$") { remove req.http.Accept-Encoding; } else if (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate"){ set req.http.Accept-Encoding = "deflate"; } else { remove req.http.Accept-Encoding; } }
75. if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3| mp4|pdf|ico|png)(\?.*|)$") { unset req.http.cookie; set req.url = regsub(req.url, "\?.*$", ""); return (lookup); } if (req.url ~ "\?(utm_(campaign|medium|source|term)| adParams|client|cx|eid|fbid|feed|ref(id|src)?|v(er| iew))=") { set req.url = regsub(req.url, "\?.*$", ""); } if (req.http.cookie ~ "^ *$") { unset req.http.cookie; }
76. sub vcl_fetch { if (req.url ~ "wp-(login|admin)" || req.url ~ "preview=true" || req.url ~ "xmlrpc.php") { return (hit_for_pass); } if ( (!(req.url ~ "(wp-(login|admin)|login)"))) { unset beresp.http.set-cookie; set beresp.ttl = 1h; } if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3| mp4|pdf|ico|png)(\?.*|)$") { set beresp.ttl = 365d; } }
77. sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } } sub vcl_hit { if (req.request == "PURGE") { purge; set obj.ttl = 0s; error 200 "OK"; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "OK"; } }
78. Webserver     performance
79. Apache  vs  Nginx
80. Prefork  or  worker  mode Async,  event-­‐driven  workers Apache Nginx
81. Evented  non-­‐blocking  I/O We’ll  call   you  back!
82. Blocking var  results  =  db.query("SELECT  *  FROM  tablename"); //use  results   Non-­‐blocking db.query("SELECT  *  FROM  tablename"  ,  function  (results)  {   //use  results   });   //do  something  else  
83. Example var  http  =  require('http'); http.createServer(function  (req,  res)  {    res.writeHead(200,  {'Content-­‐Type':  'text/plain'});    res.end('Hello  World\n'); }).listen(1337,  '127.0.0.1'); console.log('Server  running  at  http://127.0.0.1:1337/');
84. We  can  do  it  in  PHP  too
85. Example <?php require 'vendor/autoload.php'; $app = function ($request, $response) { $response->writeHead(200, array('Content- Type' => 'text/plain')); $response->end("Hello World\n"); }; $loop = React\EventLoop\Factory::create(); $socket = new React\Socket\Server($loop); $http = new React\Http\Server($socket, $loop); $http->on('request', $app); echo "Server running at http://127.0.0.1:1337\n"; $socket->listen(1337); $loop->run();
86. Real  /me  data Ajax Websockets
87. Websockets var conn = new WebSocket('ws:// some.host'); conn.onmessage = function(e) { console.log(e.data); }; conn.send('Hello World!'); Client
88. Websockets Server Ratchet:   websocket   framework   on  top  of   React
89. <?php namespace MyApp; use Ratchet\MessageComponentInterface; use Ratchet\ConnectionInterface; class Chat implements MessageComponentInterface { protected $clients; public function __construct() { $this->clients = new \SplObjectStorage; } public function onOpen(ConnectionInterface $conn) { $this->clients->attach($conn); } public function onMessage(ConnectionInterface $from, $msg) { foreach ($this->clients as $client) { if ($from !== $client) { $client->send('Response: '.$msg); } } } public function onClose(ConnectionInterface $conn) { $this->clients->detach($conn); } public function onError(ConnectionInterface $conn, \Exception $e) { echo "An error has occurred: {$e->getMessage()}\n"; $conn->close(); } }
90. <?php require __DIR__.'/vendor/autoload.php'; require 'chat.php'; use Ratchet\WebSocket\WsServer; use Ratchet\Server\IoServer; use MyApp\Chat; $ws = new WsServer(new Chat()); $ws->disableVersion(0); $server = IoServer::factory($ws,80,'1.2.3.4'); $server->run();
91. FPM hJp://www.php.net/manual/en/install.fpm.configuraOon.php
92. [www] listen = 9000 user = www-data group = www-data pm = dynamic pm.max_children = 10 pm.start_servers = 4 pm.max_spare_servers = 6 pm.max_requests = 500 php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com php_flag[display_errors] = off php_admin_value[error_log] = /var/log/fpm-php.www.log php_admin_flag[log_errors] = on php_admin_value[memory_limit] = 32M
93. upstream fpm { server php1.server.com:9000; server php2.server.com:9000; } upstream memcached { server memcached1.server.com:11211; server memcached2.server.com:11211; } server { root /var/www; index index.php index.html index.htm; server_name nginx.server.com; location / { try_files $uri $uri/ /index.php; } location @php { fastcgi_pass fpm; fastcgi_index index.php; include fastcgi_params; }
94. location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }
95. Offloading Resources
96. fastcgi_finish_request();
97. Disconnects  from  webserver ConCnues  async  processing
100. # Reverse Worker Code $worker= new GearmanWorker(); $worker->addServer(); $worker->addFunction("reverse", "my_reverse_function"); while ($worker->work()); function my_reverse_function($job) { return strrev($job->workload()); } # Reverse Client Code $client= new GearmanClient(); $client->addServer(); print $client->do("reverse", "Hello World!");
101. # Reverse Client Code $client= new GearmanClient(); $client->addServer(); print $client->do("reverse", "Hello World!");
102. Don’t  be  clueless
103. Apply  the  tricks
104. And  don’t  forget  to  refactor  aPerwards
105. And  then  relax  ...  and  be  happy