Slide 1
Slide 1 text
projects
Beyond the
LAMP stack
By Thijs Feryn
PHP
Slide 2
Slide 2 text
Hi, I’m Thijs
Slide 3
Slide 3 text
I’m
@ThijsFeryn
on Twitter
Slide 4
Slide 4 text
I’m an
Evangelist
At
Slide 5
Slide 5 text
I’m an
at
Evangelist
Slide 6
Slide 6 text
I’m a
at
board member
Slide 7
Slide 7 text
Jan 29th & 30th
Antwerpen, Belgium
http://conference.phpbenelux.eu/2016/
http://shop.phpbenelux.eu
Slide 8
Slide 8 text
https://joind.in/talk/view/16524
I need
feedback
Slide 9
Slide 9 text
LAMP
Stack?
Slide 10
Slide 10 text
Linux
Apache
MySQL
PHP
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
✓ ~81% of the web
✓ Easy to learn
✓ Mature (PHP renaissance)
✓ Frameworks & CMS’s
✓ Lots of online resources
✓ THE COMMUNITY
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
➡ Still considered a scripting
language (by some)
➡ Slow(-ish)
➡ Internal variable structure
causes overhead
➡ Everyone can program in PHP,
unfortunately everyone does
➡ Doesn’t scale that well
(without the tricks)
Slide 15
Slide 15 text
CPU memory I/O
PHP consumes lots of
Slide 16
Slide 16 text
It gets worse “at scale”
Slide 17
Slide 17 text
Faster PHP
Slide 18
Slide 18 text
No content
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
✓ Fast
✓ Just In Time compiler
✓ FastCGI support
✓ Drop-in replacement for
PHP-FPM
✓ Hack language
HVVM
Slide 22
Slide 22 text
Link HHVM to Nginx
server {
listen 80 default_server;
root /var/www/html;
index index.php
server_name my-site.dev;
location / {
try_files $uri $uri/ /index.php?$args;
}
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass 127.0.0.1:9000;
}
}
Slide 23
Slide 23 text
Drop-in
replacement for
PHP-FPM
Slide 24
Slide 24 text
HACK language
Slide 25
Slide 25 text
HACK language
$x . $y;
}
function test(): void {
$fn = foo();
echo $fn('baz'); // barbaz
}
Lambdas
!=
closures
Slide 26
Slide 26 text
HACK language
{
$x = await \HH\Asio\curl_exec("http://example.com/");
return $x;
}
async function curl_B(): Awaitable {
$y = await \HH\Asio\curl_exec("http://example.net/");
return $y;
}
async function async_curl(): Awaitable {
$start = microtime(true);
list($a, $b) = await \HH\Asio\v(array(curl_A(), curl_B()));
$end = microtime(true);
echo "Total time taken: " . strval($end - $start) . " seconds" .
PHP_EOL;
}
\HH\Asio\join(async_curl());
Async
Slide 27
Slide 27 text
What kind of PHP
code do we run?
Why does it slow
down?
Slide 28
Slide 28 text
Waiting for DB or API
Slide 29
Slide 29 text
HHVM won’t
really speed up
that process.
Slide 30
Slide 30 text
Different approach
Slide 31
Slide 31 text
Different
tools
Slide 32
Slide 32 text
Caching
Slide 33
Slide 33 text
Optimize database
Optimize runtime
A
void
A
void
Slide 34
Slide 34 text
Don’t
recompute
if the data
hasn’t
changed
Slide 35
Slide 35 text
Varnish
Slide 36
Slide 36 text
Reverse caching
Proxy
Slide 37
Slide 37 text
Normally
User Server
Slide 38
Slide 38 text
Forward proxy
User Proxy Server
Proxy in
the office
Slide 39
Slide 39 text
Reverse proxy
User Varnish Server
Proxy
in the data
center
Slide 40
Slide 40 text
It caches pages
It’s all
about
HTTP
Slide 41
Slide 41 text
✓ Reverse caching proxy
✓ Load balancer
✓ Web application firewall
(if you wish)
✓ HTTP accelerator
Varnish
Slide 42
Slide 42 text
✓ Varnish Configuration Language
✓ Edge Side Include support
✓ Gzip compression/decompression
✓ Cache purging
✓ HTTP streaming
✓ Grace mode
✓ Configure backends
✓ Backend loadbalancing
✓ ACL protection
✓ VMODs in C
Varnish
Slide 43
Slide 43 text
On the request side
✓ Only GET & HEAD
✓ No cookies
✓ No auth headers
On the response side
✓ No “no-cache, no store”
✓ TTL > 0
✓ No set-cookies
When does Varnish cache?
Slide 44
Slide 44 text
No content
Slide 45
Slide 45 text
Extend the
default behavior
in VCL
Slide 46
Slide 46 text
This is the default
behavior
Even
if you
don’t use
the VCL
Slide 47
Slide 47 text
vcl 4.0;
sub vcl_recv {
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
return (hash);
}
Incoming
request
Slide 48
Slide 48 text
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
return (lookup);
}
sub vcl_purge {
return (synth(200, "Purged"));
}
Compose
hash key
Evict
cache keys
Slide 49
Slide 49 text
sub vcl_hit {
if (obj.ttl >= 0s) {
// A pure unadultered hit, deliver it
return (deliver);
}
if (obj.ttl + obj.grace > 0s) {
// Object is in grace, deliver it
// Automatically triggers a background fetch
return (deliver);
}
// fetch & deliver once we get the result
return (fetch);
}
sub vcl_miss {
return (fetch);
}
sub vcl_deliver {
return (deliver);
}
Deliver
output to
client
Fetch
data from
backend
Or fetch if
it’s stale
Deliver
stored
object
Slide 50
Slide 50 text
sub vcl_backend_response {
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Surrogate-control ~ "no-store" ||
(!beresp.http.Surrogate-Control &&
beresp.http.Cache-Control ~ "no-cache|no-store|
private") ||
beresp.http.Vary == "*") {
/*
* Mark as "Hit-For-Pass" for the next 2 minutes
*/
set beresp.ttl = 120s;
set beresp.uncacheable = true;
}
return (deliver);
}
Response
from the
backend
Slide 51
Slide 51 text
What to extend?
Slide 52
Slide 52 text
✓Strip tracking cookies (Google
Analytics, …)
✓Sanitize URL
✓URL whitelist/blacklist
✓PURGE ACLs
✓Edge Side Include rules
✓Alway cache static files
✓Extend hash keys
✓Override TTL
✓Define grace mode
What to extend?
Slide 53
Slide 53 text
Caching in your architecture
Slide 54
Slide 54 text
Respect HTTP
Slide 55
Slide 55 text
Cache-control: public, max-
age=3600, s-maxage=7200
Cache-control: no-cache, no-
store
VS
Slide 56
Slide 56 text
Cookies?
Slide 57
Slide 57 text
The example
Slide 58
Slide 58 text
Don’t cache header
Slide 59
Slide 59 text
Cache products for 1 hour
Slide 60
Slide 60 text
Cache menus for 1 day
Slide 61
Slide 61 text
Edge Side
Includes
Slide 62
Slide 62 text
The Demo
{% block content %}{% endblock %}
ESI tags
Slide 63
Slide 63 text
The Demo
{{ render_esi(url('nav')) }}
{{ render_esi(url('jumbotron')) }}
{% block content %}{% endblock %}
{{ render_esi(url('footer')) }}
Twig
template in
Silex
ESI or
internal
subrequest
Uses
HttpFragmen
tServiceProv
ider
{{ render_esi(url('footer')) }}
Slide 64
Slide 64 text
Or just
use AJAX
Async
Graceful
degradition
Slide 65
Slide 65 text
Hit rate
Slide 66
Slide 66 text
Flexibility
Slide 67
Slide 67 text
Minimal VCL
Let the
application
handle it
Slide 68
Slide 68 text
Where does it fit in?
Slide 69
Slide 69 text
✓Cache pages and static assets
✓Fastest way
✓Hit rate may vary
✓Chop your content up in pieces
✓Use ESI or AJAX
✓Gateway to your application
Where does Varnish fit in?
Slide 70
Slide 70 text
No content
Slide 71
Slide 71 text
✓Cache all images, js, css, wof, …
✓Cache product pages
✓Cache category pages
✓Cache parts of the layout (via ESI)
✓Cache CMS-ish pages
Where does Varnish fit in?
Slide 72
Slide 72 text
You can also
host your
static files on
a separate set
of Nginx
servers
Slide 73
Slide 73 text
Content Delivery Network
Slide 74
Slide 74 text
CDNs are
nothing but a
bunch of
reverse
caching
proxies
Slide 75
Slide 75 text
Put the content
where your
user is
Slide 76
Slide 76 text
A
voids network saturation
Slide 77
Slide 77 text
No content
Slide 78
Slide 78 text
User specific content
Slide 79
Slide 79 text
Keeping track of state
Slide 80
Slide 80 text
Cookies
Slide 81
Slide 81 text
Stateful
caching
vs
high hit rate
Slide 82
Slide 82 text
Not all pages
can be
cached
Slide 83
Slide 83 text
Make the data retrieval faster
Slide 84
Slide 84 text
Typical MySQL issues
Slide 85
Slide 85 text
Data is stored for
flexibility, not for
performance
SQL (joins) allow
different
compositions of the
same data
Slide 86
Slide 86 text
Offload the database
Slide 87
Slide 87 text
No content
Slide 88
Slide 88 text
✓ Key-value store
✓ Fast
✓ Lightweight
✓ Data stored in RAM
✓ ~Memcached
✓ Data types
✓ Data persistance
✓ Replication
✓ Clustering
Redis
Slide 89
Slide 89 text
Redis
$ redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> set mykey somevalue
OK
127.0.0.1:6379> get mykey
"somevalue
Slide 90
Slide 90 text
✓ Strings
✓ Hashes
✓ Lists
✓ Sets
✓ Sorted sets
✓ Geo
✓ …
Redis data types
Slide 91
Slide 91 text
Redis
$ redis-cli
127.0.0.1:6379> hset customer_1234 id 1234
(integer) 1
127.0.0.1:6379> hset customer_1234 items_in_cart 2
(integer) 1
127.0.0.1:6379> hmset customer_1234 firstname Thijs lastname
Feryn
OK
127.0.0.1:6379> hgetall customer_1234
1) "id"
2) "1234"
3) "items_in_cart"
4) "2"
5) "firstname"
6) "Thijs"
7) "lastname"
8) "Feryn"
127.0.0.1:6379>
Slide 92
Slide 92 text
$ redis-cli
127.0.0.1:6379> lpush products_for_customer_1234 5
(integer) 1
127.0.0.1:6379> lpush products_for_customer_1234 345
(integer) 2
127.0.0.1:6379> lpush products_for_customer_1234 78 12 345
(integer) 5
127.0.0.1:6379> llen products_for_customer_1234
(integer) 5
127.0.0.1:6379> lindex products_for_customer_1234 1
"12"
127.0.0.1:6379> lindex products_for_customer_1234 2
"78"
127.0.0.1:6379> rpop products_for_customer_1234
"5"
127.0.0.1:6379> rpop products_for_customer_1234
"345"
127.0.0.1:6379> rpop products_for_customer_1234
"78"
127.0.0.1:6379> rpop products_for_customer_1234
"12"
127.0.0.1:6379> rpop products_for_customer_1234
"345"
127.0.0.1:6379> rpop products_for_customer_1234
(nil)
127.0.0.1:6379>
Redis
Slide 93
Slide 93 text
daemonize yes
pidfile /var/run/redis.pid
port 6379
databases 16
maxmemory 1gb
maxmemory-policy volatile-lru
save 900 1
save 300 10
save 60 10000
dbfilename dump.rdb
dir ./
slaveof 10.10.10.20 6379
appendonly yes
appendfilename "appendonly.aof"
Redis server config
Slide 94
Slide 94 text
Predis
https://github.com/nrk/predis
Slide 95
Slide 95 text
Where does it fit in?
Slide 96
Slide 96 text
✓ Database/API cache
✓ PHP session storage
✓ Message queue (lists)
✓ NoSQL database
✓ Real-time data retrieval
Where does Redis fit in?
Slide 97
Slide 97 text
No content
Slide 98
Slide 98 text
✓ Stock quantities
✓ Variable pricing
information
✓ Shopping cart
✓ User profile information
Where does Redis fit in?
Slide 99
Slide 99 text
Basically:
Real-time
&
volatile data
Slide 100
Slide 100 text
BUT:
MySQL can still
remain “source of
truth” database
Slide 101
Slide 101 text
No content
Slide 102
Slide 102 text
✓Full-text search engine
✓Analytics engine
✓NoSQL database
✓Lucene based
✓Built-in clustering, replication,
sharding
✓RESTful interface
✓Schemaless
ElasticSearch
Slide 103
Slide 103 text
{
"name" : "Hijacker",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.1.0",
"build_hash" :
"72cd1f1a3eee09505e036106146dc1949dc5dc87",
"build_timestamp" : "2015-11-18T22:40:03Z",
"build_snapshot" : false,
"lucene_version" : "5.3.1"
},
"tagline" : "You Know, for Search"
}
http://localhost:
9200
Slide 104
Slide 104 text
POST /my-index
{"acknowledged":true}
POST/my-index/my-type
{
"key" : "value",
"date" : "2015-05-10",
"counter" : 1,
"tags" : ["tag1","tag2","tag3"]
}
{
"_index": "my-index",
"_type": "my-type",
"_id": "AU089olr9oI99a_rK9fi",
"_version": 1,
"created": true
}
Confirmation
Slide 105
Slide 105 text
GET/my-index/my-type/AU089olr9oI99a_rK9fi?pretty
{
"_index": "my-index",
"_type": "my-type",
"_id": "AU089olr9oI99a_rK9fi",
"_version": 1,
"found": true,
"_source": {
"key": "value",
"date": "2015-05-10",
"counter": 1,
"tags": [
"tag1",
"tag2",
"tag3"
]
}
}
Retrieve
document by
id
Document &
meta data
Slide 106
Slide 106 text
GET /my-index/_mapping?pretty
{
"my-index": {
"mappings": {
"my-type": {
"properties": {
"counter": {
"type": "long"
},
"date": {
"type": "date",
"format": "dateOptionalTime"
},
"key": {
"type": "string"
},
"tags": {
"type": "string"
}
}
}
}
}
}
Schemaless?
Not really …
“Guesses”
mapping on
insert
Slide 107
Slide 107 text
POST /products
{
"mappings": {
"product" : {
"_id" : {
"path" : "entity_id"
},
"properties" : {
"entity_id" : {"type" : "integer"},
"name" : {
"type" : "string",
"index" : "not_analyzed",
"fields" : {
"raw" : {
"type" : "string",
"analyzer": "english"
}
}
},
"description" : {
"type" : "string",
"index" : "not_analyzed",
"fields" : {
"raw" : {
"type" : "string",
"analyzer": "english"
}
}
},
"price" : {"type" : "double"},
"sku" : {"type" : "string", "index" : "not_analyzed"},
"created_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"},
"updated_at" : {"type" : "date", "format" : "YYYY-MM-dd HH:mm:ss"} ,
"category" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
}
}
Explicit
mapping at
index creation
time
Slide 108
Slide 108 text
Analyzed
vs
non-analyzed
Full-text
vs
exact value
Filter
vs
Query
Slide 109
Slide 109 text
Search
Slide 110
Slide 110 text
POST /products/product/_search?pretty
{
"query": {
"match": {
"name.raw": "Linen Blazer"
}
}
}
POST /products/product/_search?pretty
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"term": {
"name": "Linen Blazer"
}
}
}
}
}
Matches
2 products
Matches 1
product
Slide 111
Slide 111 text
POST /products/product/_search?pretty
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"range": {
"price": {
"gte": 100,
"lte": 400
}
}
}
],
"must_not": [
{
"term": {
"name": "Convertible Dress"
}
}
],
"should": [
{
"term": {
"category": "Women"
}
},
{
"term": {
"category": "New Arrivals"
}
}
]
}
}
}
}
}
Slide 112
Slide 112 text
Aggregations
Slide 113
Slide 113 text
Group by on steroids
Slide 114
Slide 114 text
POST /products/product/_search?pretty
{
"fields": ["category","price","name"],
"query": {
"match": {
"name.raw": "blazer"
}
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
},
"min_price" : {
"min": {
"field": "price"
}
},
"max_price" : {
"max": {
"field": "price"
}
},
"number_of_products_per_category" : {
"terms": {
"field": "category",
"size": 10
}
}
}
}
Multi-group
by
&
query
Slide 115
Slide 115 text
"aggregations": {
"min_price": {
"value": 455
},
"number_of_products_per_category": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "Blazers",
"doc_count": 2
},
{
"key": "Default Category",
"doc_count": 2
},
{
"key": "Men",
"doc_count": 2
}
]
},
"max_price": {
"value": 490
},
"avg_price": {
"value": 472.5
}
}
Aggregation
output
Slide 116
Slide 116 text
Clustering
Slide 117
Slide 117 text
Single
node
2 node
cluster
3 node
cluster
Slide 118
Slide 118 text
Where does it fit in?
Slide 119
Slide 119 text
✓ Full-text search engine
with drill-down search
✓ NoSQL database
✓ Big data analytics tool
using Kibana
Where does ElasticSearch fit in?
Slide 120
Slide 120 text
No content
Slide 121
Slide 121 text
No content
Slide 122
Slide 122 text
✓ All product information
✓ All categories &
attributes
✓ Log archive
✓ Both NoSQL DB & search
engine
Where does ElasticSearch fit in?
Slide 123
Slide 123 text
CPU memory I/O
PHP consumes lots of
Slide 124
Slide 124 text
Depends on what
you do with it
Slide 125
Slide 125 text
✓ Image resizing
✓ Logging/reporting
✓ PDF generation
✓ Check-out on super busy
event sites
✓ …
Slide 126
Slide 126 text
Offload the webserver
Slide 127
Slide 127 text
Worker scripts
Slide 128
Slide 128 text
✓ Uses PHP-CLI
✓ Runs continuously
✓ Process forking
✓ Pthreads
✓ Run worker scripts in
parallel
✓ Managed by supervisord
Worker scripts
Slide 129
Slide 129 text
✓ Sync MySQL & Redis
✓ Resize images
✓ Async logging & metrics
✓ Update quantities &
prices
Worker scripts
Slide 130
Slide 130 text
How does your app
communicate with the
workers?
Slide 131
Slide 131 text
Message queues
Slide 132
Slide 132 text
No content
Slide 133
Slide 133 text
✓ Pub/sub
✓ Speaks AMQP protocol
✓ Supported by Pivotal
✓ Channels/Exchanges/
Queues
✓ Built-in clustering
✓ Reliable messaging
RabbitMQ
Slide 134
Slide 134 text
No content
Slide 135
Slide 135 text
Exchanges
Slide 136
Slide 136 text
https://github.com/
videlalvaro/php-amqplib
Slide 137
Slide 137 text
channel();
$channel->queue_declare('hello', false, false, false, false);
$msg = new AMQPMessage('Hello World!');
$channel->basic_publish($msg, '', 'hello');
echo " [x] Sent 'Hello World!'\n";
$channel->close();
$connection->close();
Send to
queue
Slide 138
Slide 138 text
body, "\n";
};
$connection = new AMQPConnection('127.0.0.1', 5672, 'guest', 'guest');
$channel = $connection->channel();
echo ' [*] Waiting for messages. To exit press CTRL+C', "\n";
$channel->basic_consume('hello', '', false, true, false, false, $callback);
while(count($channel->callbacks)) {
$channel->wait();
}
$channel->close();
$connection->close();
Receive
from
queue
Slide 139
Slide 139 text
Where does it fit in?
Slide 140
Slide 140 text
✓ Take load away from user
process
✓ Free up resources on
frontend servers
✓ Elaborate messaging
strategies
✓ Async event-based actions
Where do RabbitMQ/workers fit in?
Slide 141
Slide 141 text
No content
Slide 142
Slide 142 text
✓ Synchronize stock and price
changes between Redis & MySQL
✓ Synchronize product changes
between ElasticSearch & MysQL
✓ Stock/price/sales notifications
✓ Async checkout for busy event
sites
Where do RabbitMQ/workers fit in?
Slide 143
Slide 143 text
CPU memory I/O
PHP consumes lots of
Slide 144
Slide 144 text
But with most of
these tools we still
go through the
PHP runtime …
Slide 145
Slide 145 text
Other backend
tools to handle
stateful data
Slide 146
Slide 146 text
SOA
Microservices
AJAX
Websockets
Slide 147
Slide 147 text
No content
Slide 148
Slide 148 text
✓ Javascript runtime
✓ Async
✓ Event-driven
✓ Non-blocking I/O
✓ Callbacks
✓ Lightweight
✓ NPM packages
✓ Backend-code in Javascript
NodeJS
Slide 149
Slide 149 text
const http = require('http');
const hostname = '127.0.0.1';
const port = 1337;
http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello World\n');
}).listen(port, hostname, () => {
console.log(`Server running at http://${hostname}:${port}/`);
});
Slide 150
Slide 150 text
ExpressJS
framework for
Node
Slide 151
Slide 151 text
var express = require('express');
var app = express();
app.get('/', function (req, res) {
res.send('Hello World!');
});
app.post('/', function (req, res) {
res.send('Got a POST request');
});
app.put('/user', function (req, res) {
res.send('Got a PUT request at /user');
});
app.delete('/user', function (req, res) {
res.send('Got a DELETE request at /user');
});
var server = app.listen(3000, function () {
var host = server.address().address;
var port = server.address().port;
console.log('Example app listening at http://%s:%s', host, port);
});
Slide 152
Slide 152 text
Gateway to
ElasticSearch,
RabbitMQ & Redis
Slide 153
Slide 153 text
var elasticsearch = require('elasticsearch');
var express = require('express');
var bodyParser = require('body-parser')
var redis = require("redis");
var amqp = require('amqplib/callback_api');
var elasticsearchClient = new elasticsearch.Client({
host: 'localhost:9200',
log: 'error'
});
var redisClient = redis.createClient();
redisClient.on("error", function (err) {
console.log("Error " + err);
});
var app = express();
app.use(bodyParser.urlencoded({ extended: false }))
app.use(bodyParser.json())
Initialize
Slide 154
Slide 154 text
app.get('/products', function (req, res) {
elasticsearchClient.search({
index: 'thedemo',
type: 'product',
body: {
query: {
match_all: {}
}
}
}).then(function (resp) {
res.json(resp.hits.hits)
}, function (err) {
console.trace(err.message);
});
});
app.get('/products/:id([0-9]+)/stock', function (req, res) {
redisClient.get(req.params.id+':stock', function(err,
reply) {
res.json(parseInt(reply));
});
});
Get
products
from ES
Get
stock from
Redis
Slide 155
Slide 155 text
app.put('/products/:id([0-9]+)/stock', function (req, res) {
var stock = req.body.stock;
var action = req.body.action;
if(action == 'increment') {
redisClient.incrby(req.params.id+':stock',stock, function(err, reply) {
amqp.connect('amqp://localhost', function(err, conn) {
conn.createChannel(function(err, ch) {
ch.assertExchange('stock', 'direct', {durable: false});
ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock:
parseInt(reply)})));
res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
});
});
});
} else {
redisClient.decrby(req.params.id+':stock',stock, function(err, reply) {
amqp.connect('amqp://localhost', function(err, conn) {
conn.createChannel(function(err, ch) {
ch.assertExchange('stock', 'direct', {durable: false});
ch.publish('stock', 'info', new Buffer(JSON.stringify({id: req.params.id, stock:
parseInt(reply)})));
res.json('Stock for product '+req.params.id+' is now '+parseInt(reply));
});
});
});
}
});
Update
stock in
Redis
Send
message to
queue
Slide 156
Slide 156 text
➜ ~ node node.js
Example app listening at http://:::3000
Running
node.js
Slide 157
Slide 157 text
➜ ~ curl -XPUT localhost:3000/products/1/
stock -d"stock=2&action=increment"
"Stock for product 1 is now 7”
➜ ~ php stock.php info
[*] Waiting for logs. To exit press CTRL+C
[x] info:{"id":"1","stock":7}
API call
PHP
queue
worker
Slide 158
Slide 158 text
No content
Slide 159
Slide 159 text
✓ Compiled language for the web
✓ Invented by Google
✓ Strictly typed
✓ Feels like your average
interpreted language
✓ Async features
✓ Built for systems programming
✓ REALLY fast
✓ Not object oriented
Go(lang)
Slide 160
Slide 160 text
Where does it fit in?
Slide 161
Slide 161 text
Really fast workers
Really fast APIs
Could
replace PHP
workers
Could
replace
NodeJS
Slide 162
Slide 162 text
Microservices
Slide 163
Slide 163 text
package main
import (
"fmt"
"log"
"github.com/streadway/amqp"
)
func failOnError(err error, msg string) {
if err != nil {
log.Fatalf("%s: %s", msg, err)
panic(fmt.Sprintf("%s: %s", msg, err))
}
}
Initialize
Slide 164
Slide 164 text
func main() {
conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
failOnError(err, "Failed to connect to RabbitMQ")
defer conn.Close()
ch, err := conn.Channel()
failOnError(err, "Failed to open a channel")
defer ch.Close()
err = ch.ExchangeDeclare(
"stock", // name
"direct", // type
false, // durable
false, // auto-deleted
false, // internal
false, // no-wait
nil, // arguments
)
failOnError(err, "Failed to declare an exchange")
q, err := ch.QueueDeclare(
"", // name
false, // durable
false, // delete when usused
true, // exclusive
false, // no-wait
Runs from
main
function
Slide 165
Slide 165 text
conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
failOnError(err, "Failed to connect to RabbitMQ")
defer conn.Close()
ch, err := conn.Channel()
failOnError(err, "Failed to open a channel")
defer ch.Close()
err = ch.ExchangeDeclare(
"stock", // name
"direct", // type
false, // durable
false, // auto-deleted
false, // internal
false, // no-wait
nil, // arguments
)
failOnError(err, "Failed to declare an exchange")
Initialize
connection
Initialize
exchange
Slide 166
Slide 166 text
q, err := ch.QueueDeclare(
"", // name
false, // durable
false, // delete when usused
true, // exclusive
false, // no-wait
nil, // arguments
)
failOnError(err, "Failed to declare a queue")
err = ch.QueueBind(
q.Name, // queue name
"info", // routing key
"stock", // exchange
false,
nil)
failOnError(err, "Failed to bind a queue")
Declare
queue
Bind to
queue
Slide 167
Slide 167 text
msgs, err := ch.Consume(
q.Name, // queue
"", // consumer
true, // auto-ack
false, // exclusive
false, // no-local
false, // no-wait
nil, // args
)
failOnError(err, "Failed to register a consumer")
forever := make(chan bool)
go func() {
for d := range msgs {
log.Printf(" [x] %s", d.Body)
}
}()
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
<-forever
}
Consume
messages
Async
processing
Slide 168
Slide 168 text
✓ Go get
✓ Go run worker.go
✓ Go install worker.go
✓ env GOOS=linux GOARCH=amd64
go build worker.go
Useful Go commands
Slide 169
Slide 169 text
The end game
Slide 170
Slide 170 text
✓ Cache pages (Varnish)
✓ Assemble content via ESI or AJAX
✓ Static assets on Nginx or CDN
✓ Business logic in lightweight API calls
(NodeJS, Go)
✓ Key-value stores for volatile & real-time
data in Redis
✓ ElasticSearch as a NoSQL database
✓ RabbitMQ for async communication
✓ Worker processes read from message
queue
End game
Slide 171
Slide 171 text
Use the right tool
for the job
Slide 172
Slide 172 text
Use the tools you like
Slide 173
Slide 173 text
No content
Slide 174
Slide 174 text
https://joind.in/talk/view/16524
I need
feedback