Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scalability Issues ... cure first, prevent later
Search
Thijs Feryn
May 31, 2012
Technology
1
450
Scalability Issues ... cure first, prevent later
Slides for my PHPBenelux UG session in Houthalen on May 30th
Thijs Feryn
May 31, 2012
Tweet
Share
More Decks by Thijs Feryn
See All by Thijs Feryn
Caching the uncacheable with Varnish - PHP London 2020
thijsferyn
0
380
Accelerating OTT video platforms with Varnish - London Video Tech meetup 2020
thijsferyn
0
330
't Oncachebare cachen
thijsferyn
0
290
Caching the uncacheable with Varnish - PHP UG FFM 19
thijsferyn
1
610
Developing cacheable PHP applications - PHP Barcelona 2019
thijsferyn
0
570
Caching the uncacheable with Varnish - FullstackEU 2019
thijsferyn
0
430
Varnish beyond basic web acceleration - Symfony Live Berlin 2019
thijsferyn
0
350
Developing cacheable PHP applications
thijsferyn
0
360
Varnish beyond basic web acceleration - DAHO.AM 2019
thijsferyn
0
350
Other Decks in Technology
See All in Technology
Observability を実現するためにアセットを活用しよう(AWS 秋の Observability 祭り ~明日使えるアセット祭り~ )
tsujiba
0
120
リンクアンドモチベーション ソフトウェアエンジニア向け紹介資料 / Introduction to Link and Motivation for Software Engineers
lmi
4
300k
Microsoft Intune アプリのトラブルシューティング
sophiakunii
1
210
GraphRAGを用いたLLMによるパーソナライズド推薦の生成
naveed92
0
150
VPC間の接続方法を整理してみた #自治体クラウド勉強会
non97
1
1k
Platform Engineering ことはじめ
oracle4engineer
PRO
8
690
形式手法の 10 メートル手前 #kernelvm / Kernel VM Study Hokuriku Part 7
ytaka23
4
440
IaC運用を楽にするためにCDK Pipelinesを導入したけど、思い通りにいかなかった話
smt7174
1
130
Team Dynamicsを目指すウイングアーク1stのQAチーム
sadonosake
1
120
SREの前に
nwiizo
9
1.4k
フルカイテン株式会社 採用資料
fullkaiten
0
39k
Postmanの日本市場におけるDevRel (的) 活動 / Postman's DevRelish activities in Japan
yokawasa
1
110
Featured
See All Featured
No one is an island. Learnings from fostering a developers community.
thoeni
19
3k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
32
1.8k
How STYLIGHT went responsive
nonsquared
95
5.2k
Making the Leap to Tech Lead
cromwellryan
133
8.9k
Writing Fast Ruby
sferik
626
61k
Imperfection Machines: The Place of Print at Facebook
scottboms
264
13k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5k
VelocityConf: Rendering Performance Case Studies
addyosmani
325
24k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
Teambox: Starting and Learning
jrom
132
8.7k
Ruby is Unlike a Banana
tanoku
96
11k
Designing for humans not robots
tammielis
249
25k
Transcript
None
Evangelist
None
None
None
@ThijsFeryn
Please3 rate3my3talk3on3 Joind.in https://joind.in/6577
None
It3works3on3my3computer
None
It’s3the3internet’s3fault
Suits & hipsters
“Isn't'hos*ng' supposed'to'be'a' bulk'product?”
“The'Cloud'will' make'sure'my'app' scales”
“Large'campaigns' on'shared'hos*ng,' why'is'that'a' problem?”
“What's'a' database'index?”
“We'll'solve'that' with'an'Ajax'call”
“We'll'probably'have' 10.000'visitors'on' the'next'campaign”
GeBng3the3numbers3right
Let’s3do3a3complete3rewrite3!
Have3you3got3the3Gme?
Or3the3money?
What3do3you3do?
Nothing
Throw3 servers3at3 the3 problem
Do3some3 tricks3!
Throw3servers3at3the3problem
Scale3verGcally
Fast Cheap vs Limited Expensive Long% term Short% term
Scale3horizontally
Efficient Unlimited vs ConsideraGons IniGal3effort Long% term Short% term
What3kind3of3infrastructure3do3you3need?
What3kind3of3infrastructure3do3you3need? One3big3 box?
What3kind3of3infrastructure3do3you3need? Scale3across3 1003nodes?
What3kind3of3infrastructure3do3you3need? That3big3F53 loadbalancer?
What3kind3of3infrastructure3do3you3need? Only3the3DB3 gets3 hammered
What3kind3of3infrastructure3do3you3need? Lots3of3staGc3 traffic
What3kind3of3infrastructure3do3you3need? Shared3hosGng3 will3do3...3 except3during3the3 month3of3the3event
About3a3mulGYserver3setup
Goal:3(almost)3no3code3changes
Cache
Why?
Is3caching3 the3 answer?
Frontend Backend Data% cache Opcode% cache Page% cache APC Memcached
SQLite File Redis +more +more
What? When3not? How3long? CompaGbility Limited3size
Tricks
RewriteEngine*On RewriteCond*%{REQUEST_FILENAME}*=s*[OR] RewriteCond*%{REQUEST_FILENAME}*=l*[OR] RewriteCond*%{REQUEST_FILENAME}*=d RewriteRule*^.*$*=*[NC,L] RewriteRule*^.*$*index.php*[NC,L] Use3mod_rewrite Hit3frontcontroller3if3resource3doesn’t3exist
Use3mod_rewrite Write3output3to3file3and3print3to3screen <?php require('db.php'); $data = db_get_content_from_uri( $_SERVER['REQUEST_URI']); $fp =
fopen(dirname(__FILE__). $_SERVER['REQUEST_URI'], 'w'); fwrite($fp, $data); fclose($fp); echo $data;
APC
Auto3prepend3&3auto3append AutomaGcally3executed3before3every3request [PHP] auto_prepend_file*“/path/to/prepend.php” auto_append_file*“/path/to/append.php”
Auto3prepend Read3from3cache3or3start3output3buffer <?php if($_SERVER['REQUEST_METHOD'] == 'GET'){ $key = md5('http://'.$_SERVER['HTTP_HOST']. $_SERVER['REQUEST_URI']);
if(($output = apc_fetch($key)) !== false){ echo $output; exit(); } ob_start(); }
Auto3append Close3output3buffer,3store3data3in3cache3&3 print3output <?php if($_SERVER['REQUEST_METHOD'] == 'GET'){ $output = ob_get_contents();
ob_end_clean(); $key = md5('http://'.$_SERVER['HTTP_HOST']. $_SERVER['REQUEST_URI']); apc_store($key,$output,20); echo $output; }
Data3cache Caching3layer3 before3on3top3 of3the3 database
<?php class Data { private $_mysqlDependency; public function __construct($mysqlDependency) {
$this->_mysqlDependency = $mysqlDependency; } public function fetchQueryFromDatabase($key) { return $this->_mysqlDependency- >fetchQueryEscaped("SELECT * FROM `myTable` WHERE `myKey` = '%s'",$key); } } No% cache
<?php class Data { private $_namespace = 'myNamespace_'; private $_ttl
= 3600; private $_mysqlDependency; public function __construct($mysqlDependency) { $this->_mysqlDependency = $mysqlDependency; } public function fetchQueryFromDatabase($key) { if(($value = apc_fetch(md5( $this->_namespace.$key))) === false){ $value = $this->_fetchDataFromDatabase($key); $this->_storeDataInCache($key,$value); } return $value; } ... Override% &%use%cache Read% from%DB%&% store%in%cache
... private function _storeDataInCache($key,$value) { apc_store(md5($this->_namespace.$key),$value, $this->_ttl); return $this; }
private function _fetchQueryFromDatabase($key) { return $this->_mysqlDependency- >fetchQueryEscaped("SELECT * FROM `myTable` WHERE `myKey` = '%s'",$key); } } Fetch% from%DB Store% in%cache
You3can3also3use3Memcached
You3can3also3use3Memcached memcached.sess_prefix = "memc.sess.key." session.save_path="127.0.0.1:11211" session.save_handler = memcached Also% for%session%
handling
Reverse3 (caching)3 proxies
Put3‘em3before3 your3 webserver
They3honor3 your3caching3 headers
None
backend default { .host = "127.0.0.1"; .port = "8080"; }
acl purge { "localhost"; "127.0.0.1"; "some.host.name.com"; "1.2.3.4"; }
sub vcl_recv { if (req.request == "PURGE") { if (!client.ip
~ purge) { error 405 "Not allowed: " + client.ip; } return(lookup); } if (req.request != "GET" && req.request != "HEAD" && req.request != "PUT" && req.request != "POST" && req.request != "TRACE" && req.request != "OPTIONS" && req.request != "DELETE") { return (pipe); }
if (req.request != "GET" && req.request != "HEAD") { return
(pass); } if (req.http.Accept-Encoding) { if (req.url ~ "\.(gif|jpg|jpeg|swf|flv|mp3|mp4| pdf|ico|png|gz|tgz|bz2)(\?.*|)$") { remove req.http.Accept-Encoding; } else if (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } else if (req.http.Accept-Encoding ~ "deflate"){ set req.http.Accept-Encoding = "deflate"; } else { remove req.http.Accept-Encoding; } }
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3| mp4|pdf|ico|png)(\?.*|)$") { unset req.http.cookie; set req.url
= regsub(req.url, "\?.*$", ""); return (lookup); } if (req.url ~ "\?(utm_(campaign|medium|source|term)| adParams|client|cx|eid|fbid|feed|ref(id|src)?|v(er| iew))=") { set req.url = regsub(req.url, "\?.*$", ""); } if (req.http.cookie ~ "^ *$") { unset req.http.cookie; }
sub vcl_fetch { if (req.url ~ "wp-(login|admin)" || req.url ~
"preview=true" || req.url ~ "xmlrpc.php") { return (hit_for_pass); } if ( (!(req.url ~ "(wp-(login|admin)|login)"))) { unset beresp.http.set-cookie; set beresp.ttl = 1h; } if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3| mp4|pdf|ico|png)(\?.*|)$") { set beresp.ttl = 365d; } }
sub vcl_deliver { if (obj.hits > 0) { set resp.http.X-Cache
= "HIT"; } else { set resp.http.X-Cache = "MISS"; } } sub vcl_hit { if (req.request == "PURGE") { purge; set obj.ttl = 0s; error 200 "OK"; } } sub vcl_miss { if (req.request == "PURGE") { purge; error 200 "OK"; } }
FPM hfp://www.php.net/manual/en/install.fpm.configuraGon.php
[www] listen = 9000 user = www-data group = www-data
pm = dynamic pm.max_children = 10 pm.start_servers = 4 pm.max_spare_servers = 6 pm.max_requests = 500 php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f
[email protected]
php_flag[display_errors] = off php_admin_value[error_log] = /var/log/fpm-php.www.log php_admin_flag[log_errors] = on php_admin_value[memory_limit] = 32M
None
upstream fpm { server php1.server.com:9000; server php2.server.com:9000; } upstream memcached
{ server memcached1.server.com:11211; server memcached2.server.com:11211; } server { root /var/www; index index.php index.html index.htm; server_name nginx.server.com; location / { try_files $uri $uri/ /index.php; } location @php { fastcgi_pass fpm; fastcgi_index index.php; include fastcgi_params; }
location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream
not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }
location ~ \.php$ { set $memcached_key $request_uri; memcached_pass memcached; memcached_next_upstream
not_found; default_type text/html; error_page 404 405 502 = @php; } location ~* \.(?:ico|css|js|gif|jpe?g|png)$ { expires max; add_header Pragma public; add_header Cache-Control "public, must- revalidate, proxy-revalidate"; } }
None
Content Delivery Network
+3 W33Total3 Cache
None
None
None
None
✓Enable3caching ✓Enable3cron.php ✓Configure3log3retenGon ✓Flat3catalog ✓Enable3compilaGon ✓Install3Varnish3Cache3plugin ✓Split3frontend3&3backend3servers ✓Perform3indexing3on3separate3 server
<global> <cache> <backend>apc</backend> <prefix>MAGE_</prefix> </cache> </global>
<global> <cache> <backend>memcached</backend> <memcached> <servers> <server> <host><![CDATA[127.0.0.1]]></host> <port><![CDATA[11211]]></port> <persistent><![CDATA[1]]></persistent> </server>
</servers> <compression><![CDATA[0]]></compression> <cache_dir><![CDATA[]]></cache_dir> <hashed_directory_level><![CDATA[]]></hashed_directory_level> <hashed_directory_umask><![CDATA[]]></hashed_directory_umask> <file_name_prefix><![CDATA[]]></file_name_prefix> </memcached> </cache> </global>
None
Boost Varnish Memcached
Other3tricks3...
Mess3with3the3hosts3file Test3load3&3change3docroot Play3with3Apache3&3MySQL3 seBngs Add3a3database3index Enable3MySQL3query3caching
Don’t3be3clueless
Ask3for3help
Apply3the3tricks
And3don’t3forget3to3refactor3alerwards
And3then3relax3...3and3be3happy
None