Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
PHPで支える大規模・ラムダアーキテクチャ
Search
Sponsored
·
SiteGround - Reliable hosting with speed, security, and support you can count on.
→
yuuki takezawa
July 15, 2017
Technology
1.4k
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
PHPで支える大規模・ラムダアーキテクチャ
yuuki takezawa
July 15, 2017
Transcript
42 slides
PHPͰࢧ͑Δ େنϥϜμΞʔΩςΫνϟ takezawa yuuki <ytake> PHP conference kansai 2017
Lamda Architecture
ҰൠతͳwebαʔϏε • PHP • MySQL, PostgreSQL, Oracle, SQL Server •
Apache, Nginx etc
େ͖͘ͳ͖ͬͯͨwebαʔϏε • DatabaseʹՃ͑ɺElasitcsearch, SolrͳͲ • ύϑΥʔϚϯεվળͷͨΊʹશจݕࡧͳͲΛՃ͠ RDBMSͷ͍͠ͱ͜ΖΛิ͏ • ͍͔ͭ͘ͷindexੜύλʔϯ
Application Database Elasticsearch
ύλʔϯ1 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ElasticseachͳͲʹૠೖ͢Δύλʔϯ • ΞϓϦέʔγϣϯଆͰίϯτϩʔϧͰ͖Δ͕ɺ ΞϓϦέʔγϣϯͷίʔυ͕ංେԽ
Application Database Elasticsearch Batch
ύλʔϯ2 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ఆظ࣮ߦ͞ΕΔόονͰૠೖ͢Δύλʔϯ • webΞϓϦέʔγϣϯଆͰσʔλϕʔεʹૠೖͷΈ • batchͰͲ͜·Ͱ࡞͔ͨ֬͠ೝ͠ͳ͕Βɺ ະ࡞ͷͷͷΈ࡞͢Δ •
ͨͩ͠ϦΞϧλΠϜͰͳ͍
Application Database Elasticsearch Message Queue Consumer Producer
ύλʔϯ3 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ProducerΛհͯ͠Message Queueૠೖ • webΞϓϦέʔγϣϯଆͰσʔλϕʔεʹૠೖͷΈ • Consumer͕Ԡ͠ɺElasticsearchͷindexΛ࡞ •
Message͕ফࣦ͠ͳ͍ݶΓϦΞϧλΠϜʹ͍ۙ
ΑΓେ͖ͳΞϓϦέʔγϣϯ
ࣄۀʹΑΔେ͖ͳΞϓϦέʔγϣϯ • ϢʔβʔͷߦಈΛੳ͍ͨ͠ • ଟ͘ͷϢʔβʔʹར༻͞Ε͍ͯΔݕࡧจࣈΛαδΣετ ʹར༻͍ͨ͠ • Ϣʔβʔͷߦಈʹج͍ͮͨίϯςϯπΛද͍ࣔͤͨ͞ • ࢄͨ͠αʔϏεͷσʔλΛूͯ͠৽͍͠ίϯςϯ
πΛఏڙ͍ͨ͠ • BigData
Big Data + Fast Data
BigDataʹ͏ΞϓϦέʔγϣϯͷ՝ • ͦΕͧΕͷΞϓϦέʔγϣϯͰ࣮ߦ͍ͯͨ͠όονॲ ཧ͕ऴΘΒͳ͍ • Ϩίʔυ͍ظؒͰԯͱେʹͳΓɺ σʔλϕʔεͷindexΑΓI/O͕ݫ͍͠ • ϨϓϦέʔγϣϯԆ୲อ͕͍͠ •
ઍສϢʔβʔͷϦΞϧλΠϜͷੳΛ͢Δʹݫ͍͠ • ਖ਼نԽͨ͠σʔλઈରʹRDBMS • ࢄͨ͠σʔλϕʔεʹͲ͏ཱ͔ͪ͏͔
None
BigDataͷΞϓϩʔν • σʔλͦͷͷͷू • લ·Ͱʹूܭ͓͚ͯ͠ྑ͍σʔλΛ͋Β͔͡Ί༻ ҙ͢Δ • ϦΞϧλΠϜʹೖྗ͞ΕΔσʔλʹରͯ͠ͷ MessageॲཧͱɺࢄՄೳͳσʔλετϨʔδ •
લड़ͷdatabase, elasticsearchซ༻ͷऔΓΈΛɺ ΑΓେ͖ͳεέʔϧͰߏங͢Δ
ετϦʔϜॲཧ • େྔͷσʔλΛϦΞϧλΠϜͰॲཧ͢Δͷ͕ɺ ετϦʔϜσʔλॲཧͷత • ऴΘΓ͕ͳ͘ɺແݶʹͬͯ͘ΔͷͷΞϓϩʔν • ϝϞϦͰॲཧ͞Εɺͦͷޙഁغ͞ΕΔ • ࢹܥͷॲཧΑ͘ར༻͞Ε͍ͯΔͷ
• ηϯαʔΛར༻ͨ͠ΞϓϦέʔγϣϯͳͲ
ϥϜμΞʔΩςΫνϟ • όονɺαʔϏεɺεϐʔυͰߏ • όονɺେ͖ͳσʔλͷूܭɺେྔσʔλͷੳͳͲΛ୲ ͢Δ -> Hadoop(MapReduce), Spark •
αʔϏεόονͷू݁ՌΛఏڙ͢Δ Hive, HBase, ElephantDB, Splout SQL, pipelineDB… • εϐʔυϦΞϧλΠϜॲཧͷ݁ՌΛఏڙ͢Δ Spark, Storm, Kafka, Cassandra etc.. • αʔϏεͱεϐʔυͷ྆ํͷΛϚʔδͯ͠ฦ٫ қߴ͍ɾɾɾ -> KafkaͳͲʹूͤͨ͞Kappa Architecure
KappaΞʔΩςΫνϟ
KappaΞʔΩςΫνϟ
OSSͰߏங
PHPϝΠϯͰ࡞Δ͜ͱ͍͠ɾɾ
Apache Cassandra
Apache Cassandra • Ϩίʔυ͕େྔʹ૿͑Δ͜ͱ͕Θ͔͓ͬͯΓɺ ੳʹར༻͢Δ༧ఆͰ͋ͬͨͨΊɺ εέʔϧ͕༰қͱ͍͏Ͱ࠾༻ • PHP͔Βར༻Մೳ(ext-cassandra) • େྔσʔλͷॻ͖ࠐΈʹରԠ
• ؆୯ͳτϥϯβΫγϣϯαϙʔτ • σʔληϯλʔލ͗ͷΫϥελʔߏங • Availability ͱ Partition Tolerance • SQLΠϯλʔϑΣʔε • ୯Ұোͳ͠
Apache Cassandra Architecture
ؾΛ͚ͭΔ • RDBMSײ֮Ͱ͏·͘ར༻Ͱ͖·ͤΜ • ύʔςΟγϣϯΩʔͰ͏·͘ઃܭ͢Δ • ݅ʹΑΔΦʔμʔࢦఆͰ͖ͳ͍ • ϚςϦΞϥΠζυϏϡʔซ༻͢͠ •
ো࣌ͷϩάੳͨͩ͘͠ • ίϯύΫγϣϯͱઓ͏(࣮ࡍʹར༻͢Δ༰ྔ*2Ͱܭࢉ) • ݕࡧͰҾ͔͔ͬΔهࣄେମچόʔδϣϯͰɺ ݱߦͱશ͘ผ
ςʔϒϧઃܭ • Primary KeyࣝผΩʔͰ͋Γͳ͕Βɺ Ͳͷnodeʹ֨ೲ͢Δ͔Λܾఆ͢ΔύʔςΟγϣϯΩʔ • ҟͳΔnodeʹ͋Δͷͷݕࡧ͔ͳ͍ ඞཁͳέʔε͕ੜͨ͡߹ςʔϒϧઃܭΛݟ͢ • ߋ৽࣌ɺআ࣌ʹؚΊͳ͚ΕͳΒͳ͍
• ར༻ՄೳͳͷηΧϯμϦΠϯσοΫε·Ͱ • JOINLIKEଘࡏ͠ͳ͍ͨΊɺෳࡶͳͷSparkͰ
ςʔϒϧઃܭ CREATE TABLE timeline.user_timeline ( uuid uuid, user_id int, reference
map<text, text>, body text, is_read tinyint, published_at timestamp, PRIMARY KEY (user_id) );
Ϩίʔυͷॱ൪Λܾఆ͢Δ CREATE TABLE timeline.user_timeline ( uuid uuid, user_id int, reference
map<text, text>, body text, is_read tinyint, published_at timestamp, PRIMARY KEY (user_id) ) WITH CLUSTERING ORDER BY (published_at DESC);
MATERIALIZED VIEW CREATE MATERIALIZED VIEW timeline.desc_user_timeline AS SELECT uuid, user_id,
published_at, reference, body FROM timeline.user_timeline WHERE user_id IS NOT NULL AND published_at IS NOT NULL AND uuid IS NOT NULL PRIMARY KEY (user_id, published_at, uuid) WITH CLUSTERING ORDER BY (published_at DESC);
From PHP $cluster = Cassandra::cluster() ->withContactPoints('10.0.1.24', ‘localhost') ->withPort(9042) ->build(); $statement
= $session->prepare( "UPDATE users SET age = ? WHERE user_name = ?” ); $futures = array(); // execute all statements in background foreach ($data as $arguments) { $futures[] = $session->executeAsync( $statement, [ ‘arguments' => $arguments ]; }
PHP extension • Batchʹ࠷దԽ͞ΕͨI/F Batch Statement • ฒྻར༻Մೳ • Pagination͕༻ҙ͞Ε͍ͯΔ(Generatorར༻)
• Cassandraͷ΄ͱΜͲͷػೳ͕ར༻Ͱ͖ΔͷͰɺ Java͔Βར༻ͤͣͱे׆༻Ͱ͖Δ
Apache Kafka
Apache Kafka • Streamαϙʔτ(ϥϜμΞʔΩςΫνϟͰඞཁෆՄܽ) • ΫϥελϦϯά͕ࣗ༝ࣗࡏ • Zookeeperͱ࿈ܞͨ͠ࢄγεςϜ • োʹڧ͘ɺϝοηʔδͷ࠶औಘ͕Մೳ
• SparkͱStormͱ༰қʹ࿈ܞͰ͖Δ͜ͱ͔Β࠾༻ • ϝοηʔδૹ৴ޙͰࢦఆظؒอ࣋͠ɺ ଞͷΫΤϦΤϯδϯ͔Βϝοηʔδ༰औಘՄೳ • PHP͔Βར༻Մೳ(rdkafka)
Message QueueͰൃੜ͢Δ • Producer͔ΒBrokerૹ৴࣌ʹܽଛ͢Δ͜ͱ͕͋Δ • Broker͕ड৴Λࣦഊ͢Δέʔε • Brokerͷૹ৴͕ࣦഊ͢Δέʔε • ॏෳͯ͠ड৴ͯ͠͠·͏έʔε
0.11 • Exactly-once delivery and transactional messaging • ਖ਼֬ʹҰ͚ͩɺ࣮֬ʹಧ͚Δ •
ϝοηʔδૹ৴ͱड৴ʹτϥϯβΫγϣϯʂ • ΑΓڧݻʹ
Partition • ฒྻࢄॲཧ͕ઃܭ • topicΛPartitionͰׂ͠ɺProducer, Consumer͕ҙ ͷPartitionʹΞΫηε • ࡉԽͱޮԽ͕ࣗ༝ʹ
Partition
BigDataͷ࢝·ΓPHP͔Β
None
webΞϓϦέʔγϣϯ͔Β BigData·Ͱࢧ͑ΔPHP
·ͱΊ • ෳࡶԽ͢ΔΞϓϦέʔγϣϯɺ ՝ղܾେ͖͘ͳΔޣຯ • PHPͰϏδωεΛαϙʔτ͢Δཧπʔϧ • PHP͔Β࢝·ΔBigData + FastDataΞʔΩςΫνϟ
• PHPͰେ͖͘ߩݙ
More Decks by yuuki takezawa
See All by yuuki takezawa
なぜAI時代に 「イベント」を中心に考えるのか? / Why focus on "events" in the age of AI?
ytake
4
2.3k
PHPでアクターモデルを活用したSagaパターンの実践法 / php-saga-pattern-with-actor-model
ytake
0
3.2k
PHP ステートレス VS ステートフル 状態管理と並行性 / php-stateless-stateful
ytake
0
410
PHPでアクターモデルを理解・体験しよう / Understand and experience the actor model in PHP
ytake
2
1.1k
再考 アクターモデル/ reconsider actor model
ytake
0
1.7k
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
750
Phluxorでアクターモデルを 理解・体験しよう / toolkit-for-flexible-actor-models-in-php-phluxor
ytake
1
410
オブジェクトのおしゃべり大失敗 メッセージングアンチパターン集 / messaging anti-pattern collection
ytake
2
1.4k
DRE/SREのプラクティス融合によるクラウドネイティブなデータ基盤作り / dre_sre
ytake
0
1.1k
Other Decks in Technology
See All in Technology
変更差分を読むために環境を整備しよう
pipepasta
1
480
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
17
120k
Amazon Bedrock AgentCore の基礎とセキュリティ・ガバナンス
ren8k
0
210
AIネイティブ組織の作り方。1人のPOとAIから始める、自社での実験
ama_ch
2
1.2k
DevOps Agentにあらゆるセキュリティインシデントのトリアージを任せた件
jirtosterone
1
190
セキュリティチームは どのコンテナイメージの脆弱性対応を優先すべきか 〜CISA新指針とSCCで作る脆弱性トリアージ〜
swdtkuy
0
150
こんなに変わった! AWSネットワークの常識
shotashiratori
16
11k
人の努力に頼らないセキュリティをエンジニアリングする在り方 / An approach to engineering security that does not rely on human effort
carta_engineering
0
230
AI×アジャイルで、石橋を高速で叩きながら渡る~電力会社がゼロから築いた会員サイト改善プロセス~ / 20261112 Ayami Kanazawa & Shigeru Hirai
shift_evolve
PRO
1
110
HTTP通信に使おうとすると実はめんどくさいfile_get_contents
maguroalternative
0
230
認知の質も文化醸成も。数年取り組んで見えてきた技術広報のKPI設計 / Tech PR KPIs: Lessons on Awareness Quality and Culture After Several Years
bitkey
PRO
0
140
AWS DevOps Agentの構造から読み解く運用 AI エージェントの設計原則/aws-devops-agent-design-principles
tomoki10
0
140
Featured
See All Featured
From π to Pie charts
rasagy
1
450
Docker and Python
trallard
47
4.3k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
360
31k
GitHub's CSS Performance
jonrohan
1033
470k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
23k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
marketingsoph
0
470
Embracing the Ebb and Flow
colly
88
5.2k
A Modern Web Designer's Workflow
chriscoyier
699
190k
[RailsConf 2023] Rails as a piece of cake
palkan
59
7.2k
A better future with KSS
kneath
240
18k
How to Grow Your eCommerce with AI & Automation
katarinadahlin
PRO
2
350
How to Get Subject Matter Experts Bought In and Actively Contributing to SEO & PR Initiatives.
livdayseo
1
250