Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
PHPで支える大規模・ラムダアーキテクチャ
Search
yuuki takezawa
July 15, 2017
Technology
0
1.3k
PHPで支える大規模・ラムダアーキテクチャ
yuuki takezawa
July 15, 2017
Tweet
Share
More Decks by yuuki takezawa
See All by yuuki takezawa
PHP ステートレス VS ステートフル 状態管理と並行性 / php-stateless-stateful
ytake
0
110
PHPでアクターモデルを理解・体験しよう / Understand and experience the actor model in PHP
ytake
2
250
再考 アクターモデル/ reconsider actor model
ytake
0
1k
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
420
Phluxorでアクターモデルを 理解・体験しよう / toolkit-for-flexible-actor-models-in-php-phluxor
ytake
1
260
オブジェクトのおしゃべり大失敗 メッセージングアンチパターン集 / messaging anti-pattern collection
ytake
2
1.1k
DRE/SREのプラクティス融合によるクラウドネイティブなデータ基盤作り / dre_sre
ytake
0
770
技術的負債と向き合う取り組みでよかったもの / positive_efforts_to_tackle_technical_debt
ytake
10
3.8k
アプリケーションエンジニアから強いデータエンジニアへの歩き方 / How to transition and become a Data Engineer from an Application Engineer
ytake
1
550
Other Decks in Technology
See All in Technology
コンピュータビジョンの社会実装について考えていたらゲームを作っていた話
takmin
1
600
依存パッケージの更新はコツコツが勝つコツ! / phpcon_nagoya2025
blue_goheimochi
3
210
Aurora PostgreSQLがCloudWatch Logsに 出力するログの課金を削減してみる #jawsdays2025
non97
1
200
Autonomous Database Serverless 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
17
45k
AIエージェント元年@日本生成AIユーザ会
shukob
1
210
AIエージェント元年
shukob
0
160
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
150
生成AI “再”入門 2025年春@WIRED TUESDAY EDITOR'S LOUNGE
kajikent
0
100
Goで作って学ぶWebSocket
ryuichi1208
3
2.7k
クラウド食堂とは?
hiyanger
0
110
【内製開発Summit 2025】イオンスマートテクノロジーの内製化組織の作り方/In-house-development-summit-AST
aeonpeople
2
640
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
230
Featured
See All Featured
GitHub's CSS Performance
jonrohan
1030
460k
The Invisible Side of Design
smashingmag
299
50k
KATA
mclloyd
29
14k
Documentation Writing (for coders)
carmenintech
67
4.6k
A Tale of Four Properties
chriscoyier
158
23k
VelocityConf: Rendering Performance Case Studies
addyosmani
328
24k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
46
2.3k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3k
Testing 201, or: Great Expectations
jmmastey
42
7.2k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
Into the Great Unknown - MozCon
thekraken
35
1.6k
Designing for Performance
lara
604
68k
Transcript
PHPͰࢧ͑Δ େنϥϜμΞʔΩςΫνϟ takezawa yuuki <ytake> PHP conference kansai 2017
Lamda Architecture
ҰൠతͳwebαʔϏε • PHP • MySQL, PostgreSQL, Oracle, SQL Server •
Apache, Nginx etc
େ͖͘ͳ͖ͬͯͨwebαʔϏε • DatabaseʹՃ͑ɺElasitcsearch, SolrͳͲ • ύϑΥʔϚϯεվળͷͨΊʹશจݕࡧͳͲΛՃ͠ RDBMSͷ͍͠ͱ͜ΖΛิ͏ • ͍͔ͭ͘ͷindexੜύλʔϯ
Application Database Elasticsearch
ύλʔϯ1 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ElasticseachͳͲʹૠೖ͢Δύλʔϯ • ΞϓϦέʔγϣϯଆͰίϯτϩʔϧͰ͖Δ͕ɺ ΞϓϦέʔγϣϯͷίʔυ͕ංେԽ
Application Database Elasticsearch Batch
ύλʔϯ2 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ఆظ࣮ߦ͞ΕΔόονͰૠೖ͢Δύλʔϯ • webΞϓϦέʔγϣϯଆͰσʔλϕʔεʹૠೖͷΈ • batchͰͲ͜·Ͱ࡞͔ͨ֬͠ೝ͠ͳ͕Βɺ ະ࡞ͷͷͷΈ࡞͢Δ •
ͨͩ͠ϦΞϧλΠϜͰͳ͍
Application Database Elasticsearch Message Queue Consumer Producer
ύλʔϯ3 • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ ProducerΛհͯ͠Message Queueૠೖ • webΞϓϦέʔγϣϯଆͰσʔλϕʔεʹૠೖͷΈ • Consumer͕Ԡ͠ɺElasticsearchͷindexΛ࡞ •
Message͕ফࣦ͠ͳ͍ݶΓϦΞϧλΠϜʹ͍ۙ
ΑΓେ͖ͳΞϓϦέʔγϣϯ
ࣄۀʹΑΔେ͖ͳΞϓϦέʔγϣϯ • ϢʔβʔͷߦಈΛੳ͍ͨ͠ • ଟ͘ͷϢʔβʔʹར༻͞Ε͍ͯΔݕࡧจࣈΛαδΣετ ʹར༻͍ͨ͠ • Ϣʔβʔͷߦಈʹج͍ͮͨίϯςϯπΛද͍ࣔͤͨ͞ • ࢄͨ͠αʔϏεͷσʔλΛूͯ͠৽͍͠ίϯςϯ
πΛఏڙ͍ͨ͠ • BigData
Big Data + Fast Data
BigDataʹ͏ΞϓϦέʔγϣϯͷ՝ • ͦΕͧΕͷΞϓϦέʔγϣϯͰ࣮ߦ͍ͯͨ͠όονॲ ཧ͕ऴΘΒͳ͍ • Ϩίʔυ͍ظؒͰԯͱେʹͳΓɺ σʔλϕʔεͷindexΑΓI/O͕ݫ͍͠ • ϨϓϦέʔγϣϯԆ୲อ͕͍͠ •
ઍສϢʔβʔͷϦΞϧλΠϜͷੳΛ͢Δʹݫ͍͠ • ਖ਼نԽͨ͠σʔλઈରʹRDBMS • ࢄͨ͠σʔλϕʔεʹͲ͏ཱ͔ͪ͏͔
None
BigDataͷΞϓϩʔν • σʔλͦͷͷͷू • લ·Ͱʹूܭ͓͚ͯ͠ྑ͍σʔλΛ͋Β͔͡Ί༻ ҙ͢Δ • ϦΞϧλΠϜʹೖྗ͞ΕΔσʔλʹରͯ͠ͷ MessageॲཧͱɺࢄՄೳͳσʔλετϨʔδ •
લड़ͷdatabase, elasticsearchซ༻ͷऔΓΈΛɺ ΑΓେ͖ͳεέʔϧͰߏங͢Δ
ετϦʔϜॲཧ • େྔͷσʔλΛϦΞϧλΠϜͰॲཧ͢Δͷ͕ɺ ετϦʔϜσʔλॲཧͷత • ऴΘΓ͕ͳ͘ɺແݶʹͬͯ͘ΔͷͷΞϓϩʔν • ϝϞϦͰॲཧ͞Εɺͦͷޙഁغ͞ΕΔ • ࢹܥͷॲཧΑ͘ར༻͞Ε͍ͯΔͷ
• ηϯαʔΛར༻ͨ͠ΞϓϦέʔγϣϯͳͲ
ϥϜμΞʔΩςΫνϟ • όονɺαʔϏεɺεϐʔυͰߏ • όονɺେ͖ͳσʔλͷूܭɺେྔσʔλͷੳͳͲΛ୲ ͢Δ -> Hadoop(MapReduce), Spark •
αʔϏεόονͷू݁ՌΛఏڙ͢Δ Hive, HBase, ElephantDB, Splout SQL, pipelineDB… • εϐʔυϦΞϧλΠϜॲཧͷ݁ՌΛఏڙ͢Δ Spark, Storm, Kafka, Cassandra etc.. • αʔϏεͱεϐʔυͷ྆ํͷΛϚʔδͯ͠ฦ٫ қߴ͍ɾɾɾ -> KafkaͳͲʹूͤͨ͞Kappa Architecure
KappaΞʔΩςΫνϟ
KappaΞʔΩςΫνϟ
OSSͰߏங
PHPϝΠϯͰ࡞Δ͜ͱ͍͠ɾɾ
Apache Cassandra
Apache Cassandra • Ϩίʔυ͕େྔʹ૿͑Δ͜ͱ͕Θ͔͓ͬͯΓɺ ੳʹར༻͢Δ༧ఆͰ͋ͬͨͨΊɺ εέʔϧ͕༰қͱ͍͏Ͱ࠾༻ • PHP͔Βར༻Մೳ(ext-cassandra) • େྔσʔλͷॻ͖ࠐΈʹରԠ
• ؆୯ͳτϥϯβΫγϣϯαϙʔτ • σʔληϯλʔލ͗ͷΫϥελʔߏங • Availability ͱ Partition Tolerance • SQLΠϯλʔϑΣʔε • ୯Ұোͳ͠
Apache Cassandra Architecture
ؾΛ͚ͭΔ • RDBMSײ֮Ͱ͏·͘ར༻Ͱ͖·ͤΜ • ύʔςΟγϣϯΩʔͰ͏·͘ઃܭ͢Δ • ݅ʹΑΔΦʔμʔࢦఆͰ͖ͳ͍ • ϚςϦΞϥΠζυϏϡʔซ༻͢͠ •
ো࣌ͷϩάੳͨͩ͘͠ • ίϯύΫγϣϯͱઓ͏(࣮ࡍʹར༻͢Δ༰ྔ*2Ͱܭࢉ) • ݕࡧͰҾ͔͔ͬΔهࣄେମچόʔδϣϯͰɺ ݱߦͱશ͘ผ
ςʔϒϧઃܭ • Primary KeyࣝผΩʔͰ͋Γͳ͕Βɺ Ͳͷnodeʹ֨ೲ͢Δ͔Λܾఆ͢ΔύʔςΟγϣϯΩʔ • ҟͳΔnodeʹ͋Δͷͷݕࡧ͔ͳ͍ ඞཁͳέʔε͕ੜͨ͡߹ςʔϒϧઃܭΛݟ͢ • ߋ৽࣌ɺআ࣌ʹؚΊͳ͚ΕͳΒͳ͍
• ར༻ՄೳͳͷηΧϯμϦΠϯσοΫε·Ͱ • JOINLIKEଘࡏ͠ͳ͍ͨΊɺෳࡶͳͷSparkͰ
ςʔϒϧઃܭ CREATE TABLE timeline.user_timeline ( uuid uuid, user_id int, reference
map<text, text>, body text, is_read tinyint, published_at timestamp, PRIMARY KEY (user_id) );
Ϩίʔυͷॱ൪Λܾఆ͢Δ CREATE TABLE timeline.user_timeline ( uuid uuid, user_id int, reference
map<text, text>, body text, is_read tinyint, published_at timestamp, PRIMARY KEY (user_id) ) WITH CLUSTERING ORDER BY (published_at DESC);
MATERIALIZED VIEW CREATE MATERIALIZED VIEW timeline.desc_user_timeline AS SELECT uuid, user_id,
published_at, reference, body FROM timeline.user_timeline WHERE user_id IS NOT NULL AND published_at IS NOT NULL AND uuid IS NOT NULL PRIMARY KEY (user_id, published_at, uuid) WITH CLUSTERING ORDER BY (published_at DESC);
From PHP $cluster = Cassandra::cluster() ->withContactPoints('10.0.1.24', ‘localhost') ->withPort(9042) ->build(); $statement
= $session->prepare( "UPDATE users SET age = ? WHERE user_name = ?” ); $futures = array(); // execute all statements in background foreach ($data as $arguments) { $futures[] = $session->executeAsync( $statement, [ ‘arguments' => $arguments ]; }
PHP extension • Batchʹ࠷దԽ͞ΕͨI/F Batch Statement • ฒྻར༻Մೳ • Pagination͕༻ҙ͞Ε͍ͯΔ(Generatorར༻)
• Cassandraͷ΄ͱΜͲͷػೳ͕ར༻Ͱ͖ΔͷͰɺ Java͔Βར༻ͤͣͱे׆༻Ͱ͖Δ
Apache Kafka
Apache Kafka • Streamαϙʔτ(ϥϜμΞʔΩςΫνϟͰඞཁෆՄܽ) • ΫϥελϦϯά͕ࣗ༝ࣗࡏ • Zookeeperͱ࿈ܞͨ͠ࢄγεςϜ • োʹڧ͘ɺϝοηʔδͷ࠶औಘ͕Մೳ
• SparkͱStormͱ༰қʹ࿈ܞͰ͖Δ͜ͱ͔Β࠾༻ • ϝοηʔδૹ৴ޙͰࢦఆظؒอ࣋͠ɺ ଞͷΫΤϦΤϯδϯ͔Βϝοηʔδ༰औಘՄೳ • PHP͔Βར༻Մೳ(rdkafka)
Message QueueͰൃੜ͢Δ • Producer͔ΒBrokerૹ৴࣌ʹܽଛ͢Δ͜ͱ͕͋Δ • Broker͕ड৴Λࣦഊ͢Δέʔε • Brokerͷૹ৴͕ࣦഊ͢Δέʔε • ॏෳͯ͠ड৴ͯ͠͠·͏έʔε
0.11 • Exactly-once delivery and transactional messaging • ਖ਼֬ʹҰ͚ͩɺ࣮֬ʹಧ͚Δ •
ϝοηʔδૹ৴ͱड৴ʹτϥϯβΫγϣϯʂ • ΑΓڧݻʹ
Partition • ฒྻࢄॲཧ͕ઃܭ • topicΛPartitionͰׂ͠ɺProducer, Consumer͕ҙ ͷPartitionʹΞΫηε • ࡉԽͱޮԽ͕ࣗ༝ʹ
Partition
BigDataͷ࢝·ΓPHP͔Β
None
webΞϓϦέʔγϣϯ͔Β BigData·Ͱࢧ͑ΔPHP
·ͱΊ • ෳࡶԽ͢ΔΞϓϦέʔγϣϯɺ ՝ղܾେ͖͘ͳΔޣຯ • PHPͰϏδωεΛαϙʔτ͢Δཧπʔϧ • PHP͔Β࢝·ΔBigData + FastDataΞʔΩςΫνϟ
• PHPͰେ͖͘ߩݙ