Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PHP Projects Beyond The LAMP Stack - NomadPHP 2015

Thijs Feryn
December 17, 2015

PHP Projects Beyond The LAMP Stack - NomadPHP 2015

Slides for my "PHP Projects Beyond The LAMP Stack" talk at the December 2015 EU edition of NomadPHP.

Key topics: PHP, HHVM, Go, NodeJS, Redis, ElasticSearch, RabbitMQ

Thijs Feryn

December 17, 2015
Tweet

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. ✓ ~81% of the web ✓ Easy to learn ✓

    Mature (PHP renaissance) ✓ Frameworks & CMS’s ✓ Lots of online resources ✓ THE COMMUNITY
  2. ➡ Still considered a scripting language (by some) ➡ Slow(-ish)

    ➡ Internal variable structure causes overhead ➡ Everyone can program in PHP, unfortunately everyone does ➡ Doesn’t scale that well 
 (without the tricks)
  3. ✓ Fast ✓ Just In Time compiler ✓ FastCGI support

    ✓ Drop-in replacement for PHP-FPM ✓ Hack language HVVM
  4. Link HHVM to Nginx server { listen 80 default_server; root

    /var/www/html; index index.php server_name my-site.dev; location / { try_files $uri $uri/ /index.php?$args; } location ~ \.php$ { include snippets/fastcgi-php.conf; fastcgi_pass 127.0.0.1:9000; } }
  5. HACK language <?hh class MyClass { const int MyConst =

    0; private string $x = ''; public function increment(int $x): int { $y = $x + 1; return $y; } } Type annotations
  6. HACK language <?hh function foo(): (function(string): string) { $x =

    'bar'; return $y ==> $x . $y; } function test(): void { $fn = foo(); echo $fn('baz'); // barbaz } Lambdas != closures
  7. HACK language <?hh namespace Hack\UserDocumentation\Async\Intro\Examples\Curl; async function curl_A(): Awaitable<string> {

    $x = await \HH\Asio\curl_exec("http://example.com/"); return $x; } async function curl_B(): Awaitable<string> { $y = await \HH\Asio\curl_exec("http://example.net/"); return $y; } async function async_curl(): Awaitable<void> { $start = microtime(true); list($a, $b) = await \HH\Asio\v(array(curl_A(), curl_B())); $end = microtime(true); echo "Total time taken: " . strval($end - $start) . " seconds" . PHP_EOL; } \HH\Asio\join(async_curl()); Async
  8. ✓ Reverse caching proxy ✓ Load balancer ✓ Web application

    firewall (if you wish) ✓ HTTP accelerator Varnish
  9. ✓ Varnish Configuration Language ✓ Edge Side Include support ✓

    Gzip compression/decompression ✓ Cache purging ✓ HTTP streaming ✓ Grace mode ✓ Configure backends ✓ Backend loadbalancing ✓ ACL protection ✓ VMODs in C Varnish
  10. On the request side ✓ Only GET & HEAD ✓

    No cookies ✓ No auth headers On the response side ✓ No “no-cache, no store” ✓ TTL > 0 ✓ No set-cookies When does Varnish cache?
  11. vcl 4.0; sub vcl_recv { if (req.method != "GET" &&

    req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (hash); } Incoming request
  12. sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else

    { hash_data(server.ip); } return (lookup); } sub vcl_purge { return (synth(200, "Purged")); } Compose hash key Evict cache keys
  13. sub vcl_hit { if (obj.ttl >= 0s) { // A

    pure unadultered hit, deliver it return (deliver); } if (obj.ttl + obj.grace > 0s) { // Object is in grace, deliver it // Automatically triggers a background fetch return (deliver); } // fetch & deliver once we get the result return (fetch); } sub vcl_miss { return (fetch); } sub vcl_deliver { return (deliver); } Deliver output to client Fetch data from backend Or fetch if it’s stale Deliver stored object
  14. sub vcl_backend_response { if (beresp.ttl <= 0s || beresp.http.Set-Cookie ||

    beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store| private") || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120s; set beresp.uncacheable = true; } return (deliver); } Response from the backend
  15. ✓Strip tracking cookies (Google Analytics, …) ✓Sanitize URL ✓URL whitelist/blacklist

    ✓PURGE ACLs ✓Edge Side Include rules ✓Alway cache static files ✓Extend hash keys ✓Override TTL ✓Define grace mode What to extend?
  16. <!DOCTYPE html> <html> <head> <title>The Demo</title> <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="/css/bootstrap.min.css"> <link rel="stylesheet" href="/css/bootstrap-theme.min.css"> <script src=“/js/jquery-2.1.4.min.js"></script> <script src="/js/bootstrap.min.js"></script> </head> <body> <nav class="navbar navbar-inverse navbar-fixed-top"> <esi:include src=“http://mysite.dev/nav" /> </nav> <div class="jumbotron"> <esi:include src="http://mysite.dev/jumbotron" /> </div> <div class="container"> {% block content %}{% endblock %} <hr> <footer> <esi:include src="http://mysite.dev/footer" /> </footer> </div> </body> </html> ESI tags
  17. <!DOCTYPE html> <html> <head> <title>The Demo</title> <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="/css/bootstrap.min.css"> <link rel="stylesheet" href="/css/bootstrap-theme.min.css"> <script src=“/js/jquery-2.1.4.min.js"></script> <script src="/js/bootstrap.min.js"></script> </head> <body> <nav class="navbar navbar-inverse navbar-fixed-top"> {{ render_esi(url('nav')) }} </nav> <div class="jumbotron"> {{ render_esi(url('jumbotron')) }} </div> <div class="container"> {% block content %}{% endblock %} <hr> <footer> {{ render_esi(url('footer')) }} </footer> </div> </body> </html> Twig template in Silex ESI or internal subrequest Uses HttpFragmen tServiceProv ider
  18. ✓Cache pages and static assets ✓Fastest way ✓Hit rate may

    vary ✓Chop your content up in pieces ✓Use ESI or AJAX ✓Gateway to your application Where does Varnish fit in?
  19. ✓Cache all images, js, css, wof, … ✓Cache product pages

    ✓Cache category pages ✓Cache parts of the layout (via ESI) ✓Cache CMS-ish pages Where does Varnish fit in?
  20. Data is stored for flexibility, not for performance SQL (joins)

    allow different compositions of the same data
  21. ✓ Key-value store ✓ Fast ✓ Lightweight ✓ Data stored

    in RAM ✓ ~Memcached ✓ Data types ✓ Data persistance ✓ Replication ✓ Clustering Redis
  22. ✓ Strings ✓ Hashes ✓ Lists ✓ Sets ✓ Sorted

    sets ✓ Geo ✓ … Redis data types
  23. Redis $ redis-cli 127.0.0.1:6379> hset customer_1234 id 1234 (integer) 1

    127.0.0.1:6379> hset customer_1234 items_in_cart 2 (integer) 1 127.0.0.1:6379> hmset customer_1234 firstname Thijs lastname Feryn OK 127.0.0.1:6379> hgetall customer_1234 1) "id" 2) "1234" 3) "items_in_cart" 4) "2" 5) "firstname" 6) "Thijs" 7) "lastname" 8) "Feryn" 127.0.0.1:6379>
  24. $ redis-cli 127.0.0.1:6379> lpush products_for_customer_1234 5 (integer) 1 127.0.0.1:6379> lpush

    products_for_customer_1234 345 (integer) 2 127.0.0.1:6379> lpush products_for_customer_1234 78 12 345 (integer) 5 127.0.0.1:6379> llen products_for_customer_1234 (integer) 5 127.0.0.1:6379> lindex products_for_customer_1234 1 "12" 127.0.0.1:6379> lindex products_for_customer_1234 2 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "5" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "12" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 (nil) 127.0.0.1:6379> Redis
  25. daemonize yes pidfile /var/run/redis.pid port 6379 databases 16 maxmemory 1gb

    maxmemory-policy volatile-lru save 900 1 save 300 10 save 60 10000 dbfilename dump.rdb dir ./ slaveof 10.10.10.20 6379 appendonly yes appendfilename "appendonly.aof" Redis server config
  26. ✓ Database/API cache ✓ PHP session storage ✓ Message queue

    (lists) ✓ NoSQL database ✓ Real-time data retrieval Where does Redis fit in?
  27. ✓ Stock quantities ✓ Variable pricing information ✓ Shopping cart

    ✓ User profile information Where does Redis fit in?
  28. ✓Full-text search engine ✓Analytics engine ✓NoSQL database ✓Lucene based ✓Built-in

    clustering, replication, sharding ✓RESTful interface ✓Schemaless ElasticSearch
  29. { "name" : "Hijacker", "cluster_name" : "elasticsearch", "version" : {

    "number" : "2.1.0", "build_hash" : "72cd1f1a3eee09505e036106146dc1949dc5dc87", "build_timestamp" : "2015-11-18T22:40:03Z", "build_snapshot" : false, "lucene_version" : "5.3.1" }, "tagline" : "You Know, for Search" } http://localhost: 9200
  30. POST /my-index {"acknowledged":true} POST/my-index/my-type { "key" : "value", "date" :

    "2015-05-10", "counter" : 1, "tags" : ["tag1","tag2","tag3"] } { "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1, "created": true } Confirmation
  31. GET/my-index/my-type/AU089olr9oI99a_rK9fi?pretty { "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1,

    "found": true, "_source": { "key": "value", "date": "2015-05-10", "counter": 1, "tags": [ "tag1", "tag2", "tag3" ] } } Retrieve document by id Document & meta data
  32. GET /my-index/_mapping?pretty { "my-index": { "mappings": { "my-type": { "properties":

    { "counter": { "type": "long" }, "date": { "type": "date", "format": "dateOptionalTime" }, "key": { "type": "string" }, "tags": { "type": "string" } } } } } } Schemaless? Not really … “Guesses” mapping on insert
  33. POST /products { "mappings": { "product" : { "_id" :

    { "path" : "entity_id" }, "properties" : { "entity_id" : {"type" : "integer"}, "name" : { "type" : "string", "index" : "not_analyzed", "fields" : { "raw" : { "type" : "string", "analyzer": "english" } } }, "description" : { "type" : "string", "index" : "not_analyzed", "fields" : { "raw" : { "type" : "string", "analyzer": "english" } } }, "price" : {"type" : "double"}, "sku" : {"type" : "string", "index" : "not_analyzed"}, "created_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"}, "updated_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"} , "category" : { "type" : "string", "index" : "not_analyzed" } } } } } Explicit mapping at index creation time
  34. POST /products/product/_search?pretty { "query": { "match": { "name.raw": "Linen Blazer"

    } } } POST /products/product/_search?pretty { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "name": "Linen Blazer" } } } } } Matches 2 products Matches 1 product
  35. POST /products/product/_search?pretty { "query": { "filtered": { "filter": { "bool":

    { "must": [ { "range": { "price": { "gte": 100, "lte": 400 } } } ], "must_not": [ { "term": { "name": "Convertible Dress" } } ], "should": [ { "term": { "category": "Women" } }, { "term": { "category": "New Arrivals" } } ] } } } } }
  36. POST /products/product/_search?pretty { "fields": ["category","price","name"], "query": { "match": { "name.raw":

    "blazer" } }, "aggs": { "avg_price": { "avg": { "field": "price" } }, "min_price" : { "min": { "field": "price" } }, "max_price" : { "max": { "field": "price" } }, "number_of_products_per_category" : { "terms": { "field": "category", "size": 10 } } } } Multi-group by & query
  37. "aggregations": { "min_price": { "value": 455 }, "number_of_products_per_category": { "doc_count_error_upper_bound":

    0, "sum_other_doc_count": 0, "buckets": [ { "key": "Blazers", "doc_count": 2 }, { "key": "Default Category", "doc_count": 2 }, { "key": "Men", "doc_count": 2 } ] }, "max_price": { "value": 490 }, "avg_price": { "value": 472.5 } } Aggregation output
  38. ✓ Full-text search engine with drill-down search ✓ NoSQL database

    ✓ Big data analytics tool using Kibana Where does ElasticSearch fit in?
  39. ✓ All product information ✓ All categories & attributes ✓

    Log archive ✓ Both NoSQL DB & search engine Where does ElasticSearch fit in?
  40. ✓ Uses PHP-CLI ✓ Runs continuously ✓ Process forking ✓

    Pthreads ✓ Run worker scripts in parallel ✓ Managed by supervisord Worker scripts
  41. ✓ Sync MySQL & Redis ✓ Resize images ✓ Async

    logging & metrics ✓ Update quantities & prices Worker scripts
  42. ✓ Pub/sub ✓ Speaks AMQP protocol ✓ Supported by Pivotal

    ✓ Channels/Exchanges/ Queues ✓ Built-in clustering ✓ Reliable messaging RabbitMQ
  43. <?php require_once __DIR__ . '/vendor/autoload.php'; use PhpAmqpLib\Connection\AMQPConnection; use PhpAmqpLib\Message\AMQPMessage; $connection

    = new AMQPConnection('127.0.0.1', 5672, 'guest', 'guest'); $channel = $connection->channel(); $channel->queue_declare('hello', false, false, false, false); $msg = new AMQPMessage('Hello World!'); $channel->basic_publish($msg, '', 'hello'); echo " [x] Sent 'Hello World!'\n"; $channel->close(); $connection->close(); Send to queue
  44. <?php require_once __DIR__ . '/vendor/autoload.php'; use PhpAmqpLib\Connection\AMQPConnection; $callback = function($msg)

    { echo " [x] Received ", $msg->body, "\n"; }; $connection = new AMQPConnection('127.0.0.1', 5672, 'guest', 'guest'); $channel = $connection->channel(); echo ' [*] Waiting for messages. To exit press CTRL+C', "\n"; $channel->basic_consume('hello', '', false, true, false, false, $callback); while(count($channel->callbacks)) { $channel->wait(); } $channel->close(); $connection->close(); Receive from queue
  45. ✓ Take load away from user process ✓ Free up

    resources on frontend servers ✓ Elaborate messaging strategies ✓ Async event-based actions Where do RabbitMQ/workers fit in?
  46. ✓ Synchronize stock and price changes between Redis & MySQL

    ✓ Synchronize product changes between ElasticSearch & MysQL ✓ Stock/price/sales notifications ✓ Async checkout for busy event sites Where do RabbitMQ/workers fit in?
  47. ✓ Javascript runtime ✓ Async ✓ Event-driven ✓ Non-blocking I/O

    ✓ Callbacks ✓ Lightweight ✓ NPM packages ✓ Backend-code in Javascript NodeJS
  48. const http = require('http'); const hostname = '127.0.0.1'; const port

    = 1337; http.createServer((req, res) => { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Hello World\n'); }).listen(port, hostname, () => { console.log(`Server running at http://${hostname}:${port}/`); });
  49. var express = require('express'); var app = express(); app.get('/', function

    (req, res) { res.send('Hello World!'); }); app.post('/', function (req, res) { res.send('Got a POST request'); }); app.put('/user', function (req, res) { res.send('Got a PUT request at /user'); }); app.delete('/user', function (req, res) { res.send('Got a DELETE request at /user'); }); var server = app.listen(3000, function () { var host = server.address().address; var port = server.address().port; console.log('Example app listening at http://%s:%s', host, port); });
  50. var elasticsearch = require('elasticsearch');
 var express = require('express');
 var bodyParser

    = require('body-parser')
 var redis = require("redis");
 var amqp = require('amqplib/callback_api');
 
 var elasticsearchClient = new elasticsearch.Client({
 host: 'localhost:9200',
 log: 'error'
 });
 
 var redisClient = redis.createClient();
 redisClient.on("error", function (err) {
 console.log("Error " + err);
 });
 
 var app = express();
 app.use(bodyParser.urlencoded({ extended: false }))
 app.use(bodyParser.json()) Initialize
  51. app.get('/products', function (req, res) {
 elasticsearchClient.search({
 index: 'thedemo',
 type: 'product',


    body: {
 query: {
 match_all: {}
 }
 }
 }).then(function (resp) {
 res.json(resp.hits.hits)
 }, function (err) {
 console.trace(err.message);
 });
 }); app.get('/products/:id([0-9]+)/stock', function (req, res) {
 redisClient.get(req.params.id+':stock', function(err, reply) {
 res.json(parseInt(reply));
 });
 }); Get products from ES Get stock from Redis
  52. app.put('/products/:id([0-9]+)/stock', function (req, res) {
 
 var stock = req.body.stock;


    var action = req.body.action;
 
 if(action == 'increment') {
 redisClient.incrby(req.params.id+':stock',stock, function(err, reply) {
 amqp.connect('amqp://localhost', function(err, conn) {
 conn.createChannel(function(err, ch) {
 ch.assertExchange('stock', 'direct', {durable: false});
 ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock: parseInt(reply)})));
 res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
 });
 
 });
 });
 } else {
 redisClient.decrby(req.params.id+':stock',stock, function(err, reply) {
 amqp.connect('amqp://localhost', function(err, conn) {
 conn.createChannel(function(err, ch) {
 ch.assertExchange('stock', 'direct', {durable: false});
 ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock: parseInt(reply)})));
 res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
 });
 
 });
 });
 }
 }); Update stock in Redis Send message to queue
  53. ➜ ~ curl -XPUT localhost:3000/products/1/ stock -d"stock=2&action=increment" "Stock for product

    1 is now 7” ➜ ~ php stock.php info [*] Waiting for logs. To exit press CTRL+C [x] info:{"id":"1","stock":7} API call PHP queue worker
  54. ✓ Compiled language for the web ✓ Invented by Google

    ✓ Strictly typed ✓ Feels like your average interpreted language ✓ Async features ✓ Built for systems programming ✓ REALLY fast ✓ Not object oriented Go(lang)
  55. package main import ( "fmt" "log" "github.com/streadway/amqp" ) func failOnError(err

    error, msg string) { if err != nil { log.Fatalf("%s: %s", msg, err) panic(fmt.Sprintf("%s: %s", msg, err)) } } Initialize
  56. func main() { conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/") failOnError(err, "Failed to

    connect to RabbitMQ") defer conn.Close() ch, err := conn.Channel() failOnError(err, "Failed to open a channel") defer ch.Close() err = ch.ExchangeDeclare( "stock", // name "direct", // type false, // durable false, // auto-deleted false, // internal false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare an exchange") q, err := ch.QueueDeclare( "", // name false, // durable false, // delete when usused true, // exclusive false, // no-wait Runs from main function
  57. conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/") failOnError(err, "Failed to connect to RabbitMQ")

    defer conn.Close() ch, err := conn.Channel() failOnError(err, "Failed to open a channel") defer ch.Close() err = ch.ExchangeDeclare( "stock", // name "direct", // type false, // durable false, // auto-deleted false, // internal false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare an exchange") Initialize connection Initialize exchange
  58. q, err := ch.QueueDeclare( "", // name false, // durable

    false, // delete when usused true, // exclusive false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare a queue") err = ch.QueueBind( q.Name, // queue name "info", // routing key "stock", // exchange false, nil) failOnError(err, "Failed to bind a queue") Declare queue Bind to queue
  59. msgs, err := ch.Consume( q.Name, // queue "", // consumer

    true, // auto-ack false, // exclusive false, // no-local false, // no-wait nil, // args ) failOnError(err, "Failed to register a consumer") forever := make(chan bool) go func() { for d := range msgs { log.Printf(" [x] %s", d.Body) } }() log.Printf(" [*] Waiting for messages. To exit press CTRL+C") <-forever } Consume messages Async processing
  60. ✓ Go get ✓ Go run worker.go ✓ Go install

    worker.go ✓ env GOOS=linux GOARCH=amd64 go build worker.go Useful Go commands
  61. ✓ Cache pages (Varnish) ✓ Assemble content via ESI or

    AJAX ✓ Static assets on Nginx or CDN ✓ Business logic in lightweight API calls (NodeJS, Go) ✓ Key-value stores for volatile & real-time data in Redis ✓ ElasticSearch as a NoSQL database ✓ RabbitMQ for async communication ✓ Worker processes read from message queue End game