Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PHP projects beyond the LAMP Stack - PHPCafé 2016

Thijs Feryn
February 10, 2016

PHP projects beyond the LAMP Stack - PHPCafé 2016

Slides for my PHP projects beyond the LAMP stack talk at PHPCafé 2016 in Breda.

Thijs Feryn

February 10, 2016
Tweet

More Decks by Thijs Feryn

Other Decks in Technology

Transcript

  1. ✓ ~81% of the web ✓ Easy to learn ✓

    Mature (PHP renaissance) ✓ Frameworks & CMS’s ✓ Lots of online resources ✓ THE COMMUNITY
  2. ➡ Still considered a scripting language (by some) ➡ Slow(-ish)

    ➡ Internal variable structure causes overhead ➡ Everyone can program in PHP, unfortunately everyone does ➡ Doesn’t scale that well 
 (without the tricks)
  3. ✓ Fast ✓ Just In Time compiler ✓ FastCGI support

    ✓ Drop-in replacement for PHP-FPM ✓ Hack language HHVM
  4. HACK language <?hh class MyClass { const int MyConst =

    0; private string $x = ''; public function increment(int $x): int { $y = $x + 1; return $y; } } Type annotations
  5. HACK language <?hh function foo(): (function(string): string) { $x =

    'bar'; return $y ==> $x . $y; } function test(): void { $fn = foo(); echo $fn('baz'); // barbaz } Lambdas != closures
  6. HACK language <?hh namespace Hack\UserDocumentation\Async\Intro\Examples\Curl; async function curl_A(): Awaitable<string> {

    $x = await \HH\Asio\curl_exec("http://example.com/"); return $x; } async function curl_B(): Awaitable<string> { $y = await \HH\Asio\curl_exec("http://example.net/"); return $y; } async function async_curl(): Awaitable<void> { $start = microtime(true); list($a, $b) = await \HH\Asio\v(array(curl_A(), curl_B())); $end = microtime(true); echo "Total time taken: " . strval($end - $start) . " seconds" . PHP_EOL; } \HH\Asio\join(async_curl()); Async
  7. ✓ Varnish Configuration Language ✓ Edge Side Include support ✓

    Gzip compression/decompression ✓ Cache purging ✓ HTTP streaming ✓ Grace mode ✓ Configure backends ✓ Backend loadbalancing ✓ ACL protection ✓ VMODs in C Varnish
  8. On the request side ✓ Only GET & HEAD ✓

    No cookies ✓ No auth headers On the response side ✓ No “no-cache, no store” ✓ TTL > 0 ✓ No set-cookies When does Varnish cache?
  9. vcl 4.0; sub vcl_recv { if (req.method != "GET" &&

    req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (hash); } Incoming request
  10. sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else

    { hash_data(server.ip); } return (lookup); } sub vcl_purge { return (synth(200, "Purged")); } Compose hash key Evict cache keys
  11. sub vcl_hit { if (obj.ttl >= 0s) { // A

    pure unadultered hit, deliver it return (deliver); } if (obj.ttl + obj.grace > 0s) { // Object is in grace, deliver it // Automatically triggers a background fetch return (deliver); } // fetch & deliver once we get the result return (fetch); } sub vcl_miss { return (fetch); } sub vcl_deliver { return (deliver); } Deliver output to client Fetch data from backend Or fetch if it’s stale Deliver stored object
  12. sub vcl_backend_response { if (beresp.ttl <= 0s || beresp.http.Set-Cookie ||

    beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store| private") || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120s; set beresp.uncacheable = true; } return (deliver); } Response from the backend
  13. ✓Strip tracking cookies (Google Analytics, …) ✓Sanitize URL ✓URL whitelist/blacklist

    ✓PURGE ACLs ✓Edge Side Include rules ✓Alway cache static files ✓Extend hash keys ✓Override TTL ✓Define grace mode What to extend?
  14. <!DOCTYPE html> <html> <head> <title>The Demo</title> <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="/css/bootstrap.min.css"> <link rel="stylesheet" href="/css/bootstrap-theme.min.css"> <script src=“/js/jquery-2.1.4.min.js"></script> <script src="/js/bootstrap.min.js"></script> </head> <body> <nav class="navbar navbar-inverse navbar-fixed-top"> <esi:include src=“http://mysite.dev/nav" /> </nav> <div class="jumbotron"> <esi:include src="http://mysite.dev/jumbotron" /> </div> <div class="container"> {% block content %}{% endblock %} <hr> <footer> <esi:include src="http://mysite.dev/footer" /> </footer> </div> </body> </html> ESI tags
  15. <!DOCTYPE html> <html> <head> <title>The Demo</title> <meta name="viewport" content="width=device-width, initial-scale=1.0">

    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="/css/bootstrap.min.css"> <link rel="stylesheet" href="/css/bootstrap-theme.min.css"> <script src=“/js/jquery-2.1.4.min.js"></script> <script src="/js/bootstrap.min.js"></script> </head> <body> <nav class="navbar navbar-inverse navbar-fixed-top"> {{ render_esi(url('nav')) }} </nav> <div class="jumbotron"> {{ render_esi(url('jumbotron')) }} </div> <div class="container"> {% block content %}{% endblock %} <hr> <footer> {{ render_esi(url('footer')) }} </footer> </div> </body> </html> Twig template in Silex ESI or internal subrequest Uses HttpFragmen tServiceProv ider
  16. ✓Cache pages and static assets ✓Fastest way ✓Hit rate may

    vary ✓Chop your content up in pieces ✓Use ESI or AJAX ✓Gateway to your application Where does Varnish fit in?
  17. ✓Cache all images, js, css, wof, … ✓Cache product pages

    ✓Cache category pages ✓Cache parts of the layout (via ESI) ✓Cache CMS-ish pages Where does Varnish fit in?
  18. Data is stored for flexibility, not for performance SQL (joins)

    allow different compositions of the same data
  19. ✓ Key-value store ✓ Fast ✓ Lightweight ✓ Data stored

    in RAM ✓ ~Memcached ✓ Data types ✓ Data persistance ✓ Replication ✓ Clustering Redis
  20. ✓ Strings ✓ Hashes ✓ Lists ✓ Sets ✓ Sorted

    sets ✓ Geo ✓ … Redis data types
  21. Redis $ redis-cli 127.0.0.1:6379> hset customer_1234 id 1234 (integer) 1

    127.0.0.1:6379> hset customer_1234 items_in_cart 2 (integer) 1 127.0.0.1:6379> hmset customer_1234 firstname Thijs lastname Feryn OK 127.0.0.1:6379> hgetall customer_1234 1) "id" 2) "1234" 3) "items_in_cart" 4) "2" 5) "firstname" 6) "Thijs" 7) "lastname" 8) "Feryn" 127.0.0.1:6379>
  22. $ redis-cli 127.0.0.1:6379> lpush products_for_customer_1234 5 (integer) 1 127.0.0.1:6379> lpush

    products_for_customer_1234 345 (integer) 2 127.0.0.1:6379> lpush products_for_customer_1234 78 12 345 (integer) 5 127.0.0.1:6379> llen products_for_customer_1234 (integer) 5 127.0.0.1:6379> lindex products_for_customer_1234 1 "12" 127.0.0.1:6379> lindex products_for_customer_1234 2 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "5" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 "78" 127.0.0.1:6379> rpop products_for_customer_1234 "12" 127.0.0.1:6379> rpop products_for_customer_1234 "345" 127.0.0.1:6379> rpop products_for_customer_1234 (nil) 127.0.0.1:6379> Redis
  23. ✓ Database/API cache ✓ PHP session storage ✓ Message queue

    (lists) ✓ NoSQL database ✓ Real-time data retrieval Where does Redis fit in?
  24. ✓ Stock quantities ✓ Variable pricing information ✓ Shopping cart

    ✓ User profile information Where does Redis fit in?
  25. ✓Full-text search engine ✓Analytics engine ✓NoSQL database ✓Lucene based ✓Built-in

    clustering, replication, sharding ✓RESTful interface ✓Schemaless ElasticSearch
  26. { "name" : "Hijacker", "cluster_name" : "elasticsearch", "version" : {

    "number" : "2.1.0", "build_hash" : "72cd1f1a3eee09505e036106146dc1949dc5dc87", "build_timestamp" : "2015-11-18T22:40:03Z", "build_snapshot" : false, "lucene_version" : "5.3.1" }, "tagline" : "You Know, for Search" } http://localhost: 9200
  27. POST /my-index {"acknowledged":true} POST/my-index/my-type { "key" : "value", "date" :

    "2015-05-10", "counter" : 1, "tags" : ["tag1","tag2","tag3"] } { "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1, "created": true } Confirmation
  28. GET/my-index/my-type/AU089olr9oI99a_rK9fi?pretty { "_index": "my-index", "_type": "my-type", "_id": "AU089olr9oI99a_rK9fi", "_version": 1,

    "found": true, "_source": { "key": "value", "date": "2015-05-10", "counter": 1, "tags": [ "tag1", "tag2", "tag3" ] } } Retrieve document by id Document & meta data
  29. GET /my-index/_mapping?pretty { "my-index": { "mappings": { "my-type": { "properties":

    { "counter": { "type": "long" }, "date": { "type": "date", "format": "dateOptionalTime" }, "key": { "type": "string" }, "tags": { "type": "string" } } } } } } Schemaless? Not really … “Guesses” mapping on insert
  30. POST /products { "mappings": { "product" : { "_id" :

    { "path" : "entity_id" }, "properties" : { "entity_id" : {"type" : "integer"}, "name" : { "type" : "string", "index" : "not_analyzed", "fields" : { "raw" : { "type" : "string", "analyzer": "english" } } }, "description" : { "type" : "string", "index" : "not_analyzed", "fields" : { "raw" : { "type" : "string", "analyzer": "english" } } }, "price" : {"type" : "double"}, "sku" : {"type" : "string", "index" : "not_analyzed"}, "created_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"}, "updated_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"} , "category" : { "type" : "string", "index" : "not_analyzed" } } } } } Explicit mapping at index creation time
  31. POST /products/product/_search?pretty { "query": { "match": { "name.raw": "Linen Blazer"

    } } } POST /products/product/_search?pretty { "query": { "filtered": { "query": { "match_all": {} }, "filter": { "term": { "name": "Linen Blazer" } } } } } Matches 2 products Matches 1 product
  32. POST /products/product/_search?pretty { "query": { "filtered": { "filter": { "bool":

    { "must": [ { "range": { "price": { "gte": 100, "lte": 400 } } } ], "must_not": [ { "term": { "name": "Convertible Dress" } } ], "should": [ { "term": { "category": "Women" } }, { "term": { "category": "New Arrivals" } } ] } } } } }
  33. POST /products/product/_search?pretty { "fields": ["category","price","name"], "query": { "match": { "name.raw":

    "blazer" } }, "aggs": { "avg_price": { "avg": { "field": "price" } }, "min_price" : { "min": { "field": "price" } }, "max_price" : { "max": { "field": "price" } }, "number_of_products_per_category" : { "terms": { "field": "category", "size": 10 } } } } Multi-group by & query
  34. "aggregations": { "min_price": { "value": 455 }, "number_of_products_per_category": { "doc_count_error_upper_bound":

    0, "sum_other_doc_count": 0, "buckets": [ { "key": "Blazers", "doc_count": 2 }, { "key": "Default Category", "doc_count": 2 }, { "key": "Men", "doc_count": 2 } ] }, "max_price": { "value": 490 }, "avg_price": { "value": 472.5 } } Aggregation output
  35. ✓ Full-text search engine with drill-down search ✓ NoSQL database

    ✓ Big data analytics tool using Kibana Where does ElasticSearch fit in?
  36. ✓ All product information ✓ All categories & attributes ✓

    Log archive ✓ Both NoSQL DB & search engine Where does ElasticSearch fit in?
  37. ✓ Uses PHP-CLI ✓ Runs continuously ✓ Process forking ✓

    Pthreads ✓ Run worker scripts in parallel ✓ Managed by supervisord Worker scripts
  38. ✓ Sync MySQL & Redis ✓ Resize images ✓ Async

    logging & metrics ✓ Update quantities & prices Worker scripts
  39. ✓ Pub/sub ✓ Speaks AMQP protocol ✓ Supported by Pivotal

    ✓ Channels/Exchanges/ Queues ✓ Built-in clustering ✓ Reliable messaging RabbitMQ
  40. <?php require_once __DIR__ . '/vendor/autoload.php'; use PhpAmqpLib\Connection\AMQPConnection; use PhpAmqpLib\Message\AMQPMessage; $connection

    = new AMQPConnection('127.0.0.1', 5672, 'guest', 'guest'); $channel = $connection->channel(); $channel->queue_declare('hello', false, false, false, false); $msg = new AMQPMessage('Hello World!'); $channel->basic_publish($msg, '', 'hello'); echo " [x] Sent 'Hello World!'\n"; $channel->close(); $connection->close(); Send to queue
  41. <?php require_once __DIR__ . '/vendor/autoload.php'; use PhpAmqpLib\Connection\AMQPConnection; $callback = function($msg)

    { echo " [x] Received ", $msg->body, "\n"; }; $connection = new AMQPConnection('127.0.0.1', 5672, 'guest', 'guest'); $channel = $connection->channel(); echo ' [*] Waiting for messages. To exit press CTRL+C', "\n"; $channel->basic_consume('hello', '', false, true, false, false, $callback); while(count($channel->callbacks)) { $channel->wait(); } $channel->close(); $connection->close(); Receive from queue
  42. ✓ Take load away from user process ✓ Free up

    resources on frontend servers ✓ Elaborate messaging strategies ✓ Async event-based actions Where do RabbitMQ/workers fit in?
  43. ✓ Synchronize stock and price changes between Redis & MySQL

    ✓ Synchronize product changes between ElasticSearch & MysQL ✓ Stock/price/sales notifications ✓ Async checkout for busy event sites Where do RabbitMQ/workers fit in?
  44. ✓ Javascript runtime ✓ Async ✓ Event-driven ✓ Non-blocking I/O

    ✓ Callbacks ✓ Lightweight ✓ NPM packages ✓ Backend-code in Javascript NodeJS
  45. const http = require('http'); const hostname = '127.0.0.1'; const port

    = 1337; http.createServer((req, res) => { res.writeHead(200, { 'Content-Type': 'text/plain' }); res.end('Hello World\n'); }).listen(port, hostname, () => { console.log(`Server running at http://${hostname}:${port}/`); });
  46. var express = require('express'); var app = express(); app.get('/', function

    (req, res) { res.send('Hello World!'); }); app.post('/', function (req, res) { res.send('Got a POST request'); }); app.put('/user', function (req, res) { res.send('Got a PUT request at /user'); }); app.delete('/user', function (req, res) { res.send('Got a DELETE request at /user'); }); var server = app.listen(3000, function () { var host = server.address().address; var port = server.address().port; console.log('Example app listening at http://%s:%s', host, port); });
  47. var elasticsearch = require('elasticsearch');
 var express = require('express');
 var bodyParser

    = require('body-parser')
 var redis = require("redis");
 var amqp = require('amqplib/callback_api');
 
 var elasticsearchClient = new elasticsearch.Client({
 host: 'localhost:9200',
 log: 'error'
 });
 
 var redisClient = redis.createClient();
 redisClient.on("error", function (err) {
 console.log("Error " + err);
 });
 
 var app = express();
 app.use(bodyParser.urlencoded({ extended: false }))
 app.use(bodyParser.json()) Initialize
  48. app.get('/products', function (req, res) {
 elasticsearchClient.search({
 index: 'thedemo',
 type: 'product',


    body: {
 query: {
 match_all: {}
 }
 }
 }).then(function (resp) {
 res.json(resp.hits.hits)
 }, function (err) {
 console.trace(err.message);
 });
 }); app.get('/products/:id([0-9]+)/stock', function (req, res) {
 redisClient.get(req.params.id+':stock', function(err, reply) {
 res.json(parseInt(reply));
 });
 }); Get products from ES Get stock from Redis
  49. app.put('/products/:id([0-9]+)/stock', function (req, res) {
 
 var stock = req.body.stock;


    var action = req.body.action;
 
 if(action == 'increment') {
 redisClient.incrby(req.params.id+':stock',stock, function(err, reply) {
 amqp.connect('amqp://localhost', function(err, conn) {
 conn.createChannel(function(err, ch) {
 ch.assertExchange('stock', 'direct', {durable: false});
 ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock: parseInt(reply)})));
 res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
 });
 
 });
 });
 } else {
 redisClient.decrby(req.params.id+':stock',stock, function(err, reply) {
 amqp.connect('amqp://localhost', function(err, conn) {
 conn.createChannel(function(err, ch) {
 ch.assertExchange('stock', 'direct', {durable: false});
 ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock: parseInt(reply)})));
 res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
 });
 
 });
 });
 }
 }); Update stock in Redis Send message to queue
  50. ✓ Compiled language for the web ✓ Invented by Google

    ✓ Strictly typed ✓ Feels like your average interpreted language ✓ Async features ✓ Built for systems programming ✓ REALLY fast ✓ Not object oriented Go(lang)
  51. package main import ( "fmt" "log" "github.com/streadway/amqp" ) func failOnError(err

    error, msg string) { if err != nil { log.Fatalf("%s: %s", msg, err) panic(fmt.Sprintf("%s: %s", msg, err)) } } Initialize
  52. func main() { conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/") failOnError(err, "Failed to

    connect to RabbitMQ") defer conn.Close() ch, err := conn.Channel() failOnError(err, "Failed to open a channel") defer ch.Close() err = ch.ExchangeDeclare( "stock", // name "direct", // type false, // durable false, // auto-deleted false, // internal false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare an exchange") q, err := ch.QueueDeclare( "", // name false, // durable false, // delete when usused true, // exclusive false, // no-wait Runs from main function
  53. conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/") failOnError(err, "Failed to connect to RabbitMQ")

    defer conn.Close() ch, err := conn.Channel() failOnError(err, "Failed to open a channel") defer ch.Close() err = ch.ExchangeDeclare( "stock", // name "direct", // type false, // durable false, // auto-deleted false, // internal false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare an exchange") Initialize connection Initialize exchange
  54. q, err := ch.QueueDeclare( "", // name false, // durable

    false, // delete when usused true, // exclusive false, // no-wait nil, // arguments ) failOnError(err, "Failed to declare a queue") err = ch.QueueBind( q.Name, // queue name "info", // routing key "stock", // exchange false, nil) failOnError(err, "Failed to bind a queue") Declare queue Bind to queue
  55. msgs, err := ch.Consume( q.Name, // queue "", // consumer

    true, // auto-ack false, // exclusive false, // no-local false, // no-wait nil, // args ) failOnError(err, "Failed to register a consumer") forever := make(chan bool) go func() { for d := range msgs { log.Printf(" [x] %s", d.Body) } }() log.Printf(" [*] Waiting for messages. To exit press CTRL+C") <-forever } Consume messages Async processing
  56. ✓ Cache pages (Varnish) ✓ Assemble content via ESI or

    AJAX ✓ Static assets on Nginx or CDN ✓ Business logic in lightweight API calls (NodeJS, Go) ✓ Key-value stores for volatile & real-time data in Redis ✓ ElasticSearch as a NoSQL database ✓ RabbitMQ for async communication ✓ Worker processes read from message queue End game