Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elasticsearch - Turbinando sua aplicação PHP
Search
Breno Oliveira
March 21, 2016
Programming
1
150
Elasticsearch - Turbinando sua aplicação PHP
Utilizando Elasticsearch em suas aplicações PHP
Breno Oliveira
March 21, 2016
Tweet
Share
More Decks by Breno Oliveira
See All by Breno Oliveira
Spring e Eureka Service Discovery
brenooliveira
0
59
Como Fazer Minha Gem e o Rails Conversarem Bem
brenooliveira
0
45
Javascript Pitfalls
brenooliveira
0
87
Elasticsearch
brenooliveira
1
120
Falando sobre testes automatizados
brenooliveira
0
87
Other Decks in Programming
See All in Programming
テーブル定義書の構造化抽出して、生成AIでDWH分析を試してみた / devio2025tokyo
kasacchiful
0
220
Introduce Hono CLI
yusukebe
6
2.9k
チームの境界をブチ抜いていけ
tokai235
0
200
『毎日の移動』を支えるGoバックエンド内製開発
yutautsugi
2
260
実践Claude Code:20の失敗から学ぶAIペアプログラミング
takedatakashi
15
6.3k
Devoxx BE - Local Development in the AI Era
kdubois
0
130
XP, Testing and ninja testing ZOZ5
m_seki
3
770
「ちょっと古いから」って避けてた技術書、今だからこそ読もう
mottyzzz
11
6.9k
Developer Joy - The New Paradigm
hollycummins
1
280
大規模アプリのDIフレームワーク刷新戦略 ~過去最大規模の並行開発を止めずにアプリ全体に導入するまで~
mot_techtalk
1
470
kiroとCodexで最高のSpec駆動開発を!!数時間で web3ネイティブなミニゲームを作ってみたよ!
mashharuki
0
770
All About Angular's New Signal Forms
manfredsteyer
PRO
0
200
Featured
See All Featured
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
36
6.1k
Side Projects
sachag
455
43k
What's in a price? How to price your products and services
michaelherold
246
12k
Imperfection Machines: The Place of Print at Facebook
scottboms
269
13k
The Illustrated Children's Guide to Kubernetes
chrisshort
49
51k
Being A Developer After 40
akosma
91
590k
Navigating Team Friction
lara
190
15k
Typedesign – Prime Four
hannesfritz
42
2.8k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
48
9.7k
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
194
16k
Done Done
chrislema
185
16k
How to train your dragon (web standard)
notwaldorf
97
6.3k
Transcript
Breno Oliveira @brenoholiveira
[email protected]
B R E N O O
L I V E I R A Tech Leader at Moip ElasticSearch Turbinando sua aplicação PHP
None
+ 86k de contas vendendo
+ 100K de pedidos por dia
None
None
VENDAS AUMENTAM VENDEDOR QUER SABER
None
Ordenação FullText Search Paginação Agregação Filtros
REALTIME
E O SEU SERVER DEPOIS DE VÁRIOS F5 …
None
None
NOSSA VONTADE É DE …
SEU CLIENTA VAI…
PHP + MySQL PHP + postgresql PHP + nosql PHP
+ ????
CASO DE USO
+
502 200 304
+
Aplicação MOIP Base de Dados Master
Single Term SELECT * FROM pedidos p INNER JOIN cliente
ON p.cliente_id = c.id WHERE p.descricao LIKE "%superman%"
multi Term SELECT * FROM pedidos p INNER JOIN cliente
ON p.cliente_id = c.id WHERE p.descricao LIKE "%superman%" OR p.descricao LIKE "%batman%"
ORDER ARRGH!!!! SELECT * FROM pedidos p INNER JOIN cliente
ON p.cliente_id = c.id WHERE p.descricao LIKE "%superman%" OR p.descricao LIKE "%batman%" ORDER BY ??
PROBLEMAS • Difícil manter • Muitos JOINS • Lento
.Primeiro REFACTOR
MICRO SERVIÇO + BANCO denormalizado
APLICAÇÃO MOIP Base de Dados Master REPORTS API Base de
Dados denormalizada
MELHORIAS • Micro serviço • Concorrência no banco de dados
• No more JOINS • Full-Text Search • Melhorias de desempenho
PROBLEMAS • Periodos longos • Filtros • Lentidão
.SEgundo REFACTOR
MICRO SERVIÇO + Banco colunar
APLICAÇÃO MOIP Base de Dados Master REPORTS API
MELHORIAS • Consultas significantemente mais rápidas
PROBLEMAS • BULK INSERTs Too Slow
.terceiro REFACTOR
MICRO SERVIÇO + Elasticseach
APLICAÇÃO MOIP Base de Dados Master REPORTS API Elasticsearch
MELHORIAS • Bulk INSERT • Desempenho • Aggregations • Scalable
• Redundancy
.Atualmente
None
Last Black Friday
None
None
None
.não é banco da dados
.nem nosql I T ' ’ N O T K
E Y- VA L U E S T O R A G E
.Escalável
.Alta resiliencia
.restful api
.orientado a documentos
.full-text search
.free schema
.Eventualmente consistente
.apache lucene
None
.Conceitos básicos Elasticsearch
MySQL Elasticsearch Database Index Table Type Column Field Schema Mapping
Partition Shard
curl -XPUT http://localhost:9200/tips/tip/1 -d '{"tip": "Using Elasticsearch in Production", "tags":
["protip","success"]} ' Endpoint Index Type Document ID Document { "_index": "tips", "_type": "tip", "_id": "1", "_version": 1, "created": true } Response
$ curl http://localhost:9200/tips/tip/1?pretty { "_index": "tips", "_type": "tip", "_id": "1",
"_version": 1, "found": true, "_source": { "tip": "Using Elasticsearch in Production", "tags": [ "protip", "success" ] } }
None
VAGRANT
$ git clone
[email protected]
:brenooliveira/elastic-php-demo.git $ cd elastic-php-demo $ vagrant up
—provision $ vagrant ssh vagrant@default:~/$
vagrant@default:$ curl http://localhost:9200 { "status" : 200, "name" : "Corruptor",
"cluster_name" : "elasticsearch", "version" : { "number" : "1.5.2", "build_hash" : "62ff9868b4c8a0c45860bebb259e21980778ab1c", "build_timestamp" : "2015-04-27T09:21:06Z", "build_snapshot" : false, "lucene_version" : "4.10.4" }, "tagline" : "You Know, for Search" }
{ "require": { "elasticsearch/elasticsearch": "~2.0@beta" } } composer.json $ curl
-s http://getcomposer.org/installer | php $ php composer.phar install --no-dev
<?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create() ->build(); $params = array();
$params['index'] = 'tips'; $params['type'] = 'tip'; $params['id'] = '1'; $result = $client->get($params) ?> <pre> <?= print_r($result); ?> </pre> exemplo1.php
Array ( [_index] => tips [_type] => tip [_id] =>
1 [_version] => 1 [found] => 1 [_source] => Array ( [tip] => Using Elasticsearch in Production [tags] => Array ( [0] => protip [1] => success ) ) )
exemplo2.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create() ->build(); $params =
[ 'index' => 'tips', 'type' => 'tip', 'id' => '2', 'body' => [ 'tip' => 'Using Marvel to get metrics from Elasticsearch', 'tags' => ['protip', 'metrics'] ] ]; // Documento indexado em tips/tip/2 $response = $client->index($params); ?> <pre> <?= print_r($response) ?> </pre>
exemplo3.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create() ->build(); for($i =
0; $i < 100; $i++) { $params['body'][] = [ 'index' => [ '_index' => 'jobs', '_type' => 'job', ] ]; $params['body'][] = [ 'title' => 'Job position '. $i, 'email' => 'email-'. $i . '@gmail.com' ]; } $responses = $client->bulk($params); ?>
exemplo4.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); $json = '{
"query" : { "match" : { "tags" : "protip" } } }'; $params = [ 'index'=> 'tips', 'type' => 'tip', 'body' => $json ]; $responses = $client->search($params); ?> <pre> <?= print_r($responses) ?> </pre>
exemplo5.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); $params = [
'index' => 'tips', 'type' => 'tip', 'id' => '3', 'body' => [ 'tip' => 'Wrong insert', 'tags' => ['missing', 'error', 'no_logs'] ] ]; $response_insert = $client->index($params); $params = [ 'index' => 'tips', 'type' => 'tip', 'id' => '3' ]; $responde_delete = $client->delete($params); ?>
exemplo6.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); $params = [
'index' => 'tips', 'type' => 'tip', 'id' => '4', 'body' => [ 'tip' => 'Loren Ipsum', 'tags' => ['lorem'] ] ]; $response_insert = $client->index($params); $params = [ 'index' => 'tips', 'type' => 'tip', 'id' => '4', 'body' => [ 'doc' => [ 'text' => 'lipsum generator' ] ] ]; $response_update = $client->update($params); $params = [ 'index' => 'tips', 'type' => 'tip', 'id' => '4' ]; $response_search = $client->get($params); ?>
exemplo7.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); $json = '{
"my-suggestion" : { "text" : "Elastiscearch", "term" : { "field" : "tip" } } }'; $params = [ 'index' => 'tips', 'body' => $json ]; $response = $client->suggest($params); ?>
exemplo8.php <?php require 'vendor/autoload.php'; $client = Elasticsearch\ClientBuilder::create()->build(); $json = '{
"query" : { "more_like_this" : { "like_text" : "In production", "min_term_freq" : 1, "min_doc_freq" : 1 } } }'; $params = [ 'index' => 'tips', 'type' => 'tip', 'body' => $json ]; $response = $client->search($params); ?>
.Vamos falar de arquitetura
.Como devo sincronizar mysql & Elasticsearch
E-commerce app Base de Dados Elasticsearch Cadastrar novo produto
E-commerce app Base de Dados Elasticsearch rabbitmq Publish Message php
worker Consume Message & Sync Cadastrar novo produto
.Lessons learned
Utilize mais mappings
Em produção use alias
None