Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PHPで支える大規模・ラムダアーキテクチャ

 PHPで支える大規模・ラムダアーキテクチャ

yuuki takezawa

July 15, 2017
Tweet

More Decks by yuuki takezawa

Other Decks in Technology

Transcript

  1. PHPͰࢧ͑Δ
    େن໛ϥϜμΞʔΩςΫνϟ
    takezawa yuuki
    PHP conference kansai 2017

    View Slide

  2. Lamda Architecture

    View Slide

  3. ҰൠతͳwebαʔϏε
    • PHP
    • MySQL, PostgreSQL, Oracle, SQL Server
    • Apache, Nginx etc

    View Slide

  4. େ͖͘ͳ͖ͬͯͨwebαʔϏε
    • DatabaseʹՃ͑ɺElasitcsearch, SolrͳͲ
    • ύϑΥʔϚϯεվળͷͨΊʹશจݕࡧͳͲΛ௥Ճ͠

    RDBMSͷ೉͍͠ͱ͜ΖΛิ͏
    • ͍͔ͭ͘ͷindexੜ੒ύλʔϯ

    View Slide

  5. Application
    Database Elasticsearch

    View Slide

  6. ύλʔϯ1
    • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ

    ElasticseachͳͲʹૠೖ͢Δύλʔϯ
    • ΞϓϦέʔγϣϯଆͰίϯτϩʔϧͰ͖Δ͕ɺ

    ΞϓϦέʔγϣϯͷίʔυ͕ංେԽ

    View Slide

  7. Application
    Database Elasticsearch
    Batch

    View Slide

  8. ύλʔϯ2
    • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ

    ఆظ࣮ߦ͞ΕΔόονͰૠೖ͢Δύλʔϯ
    • webΞϓϦέʔγϣϯଆͰ͸σʔλϕʔεʹૠೖͷΈ
    • batchͰͲ͜·Ͱ࡞੒͔ͨ֬͠ೝ͠ͳ͕Βɺ

    ະ࡞੒ͷ΋ͷͷΈ࡞੒͢Δ
    • ͨͩ͠ϦΞϧλΠϜͰ͸ͳ͍

    View Slide

  9. Application
    Database
    Elasticsearch
    Message Queue
    Consumer
    Producer

    View Slide

  10. ύλʔϯ3
    • webΞϓϦέʔγϣϯଆͰɺσʔλϕʔεʹૠೖޙɺ

    ProducerΛհͯ͠Message Queue΁ૠೖ
    • webΞϓϦέʔγϣϯଆͰ͸σʔλϕʔεʹૠೖͷΈ
    • Consumer͕൓Ԡ͠ɺElasticsearchͷindexΛ࡞੒
    • Message͕ফࣦ͠ͳ͍ݶΓϦΞϧλΠϜʹ͍ۙ

    View Slide

  11. ΑΓେ͖ͳΞϓϦέʔγϣϯ΁

    View Slide

  12. ࣄۀ੒௕ʹΑΔେ͖ͳΞϓϦέʔγϣϯ΁
    • ϢʔβʔͷߦಈΛ෼ੳ͍ͨ͠
    • ଟ͘ͷϢʔβʔʹར༻͞Ε͍ͯΔݕࡧจࣈΛαδΣετ
    ʹར༻͍ͨ͠
    • Ϣʔβʔͷߦಈʹج͍ͮͨίϯςϯπΛද͍ࣔͤͨ͞
    • ෼ࢄͨ͠αʔϏεͷσʔλΛू໿ͯ͠৽͍͠ίϯςϯ
    πΛఏڙ͍ͨ͠
    • BigData΁

    View Slide

  13. Big Data + Fast Data

    View Slide

  14. BigDataʹ൐͏ΞϓϦέʔγϣϯͷ՝୊
    • ͦΕͧΕͷΞϓϦέʔγϣϯͰ࣮ߦ͍ͯͨ͠όονॲ
    ཧ͕ऴΘΒͳ͍
    • Ϩίʔυ਺΋୹͍ظؒͰ਺ԯͱ๲େʹͳΓɺ

    σʔλϕʔεͷindexΑΓ΋I/O͕ݫ͍͠
    • ϨϓϦέʔγϣϯ஗Ԇ୲อ͕೉͍͠
    • ਺ઍສϢʔβʔͷϦΞϧλΠϜͷ෼ੳΛ͢Δʹ͸ݫ͍͠
    • ਖ਼نԽͨ͠σʔλ͸ઈରʹRDBMS
    • ෼ࢄͨ͠σʔλϕʔεʹͲ͏ཱͪ޲͔͏͔

    View Slide

  15. View Slide

  16. BigData΁ͷΞϓϩʔν
    • σʔλͦͷ΋ͷͷू໿
    • લ೔·Ͱʹूܭ͓͚ͯ͠͹ྑ͍σʔλΛ͋Β͔͡Ί༻
    ҙ͢Δ
    • ϦΞϧλΠϜʹೖྗ͞ΕΔσʔλʹରͯ͠ͷ

    Messageॲཧͱɺ෼ࢄՄೳͳσʔλετϨʔδ
    • લड़ͷdatabase, elasticsearchซ༻ͷऔΓ૊ΈΛɺ

    ΑΓେ͖ͳεέʔϧͰߏங͢Δ

    View Slide

  17. ετϦʔϜॲཧ
    • େྔͷσʔλΛϦΞϧλΠϜͰॲཧ͢Δͷ͕ɺ

    ετϦʔϜσʔλॲཧͷ໨త
    • ऴΘΓ͕ͳ͘ɺແݶʹ΍ͬͯ͘Δ΋ͷ΁ͷΞϓϩʔν
    • ϝϞϦ಺Ͱॲཧ͞Εɺͦͷޙഁغ͞ΕΔ
    • ؂ࢹܥͷॲཧΑ͘ར༻͞Ε͍ͯΔ΋ͷ
    • ηϯαʔΛར༻ͨ͠ΞϓϦέʔγϣϯͳͲ

    View Slide

  18. ϥϜμΞʔΩςΫνϟ
    • όον૚ɺαʔϏε૚ɺεϐʔυ૚Ͱߏ੒
    • όον૚͸ɺେ͖ͳσʔλͷूܭ΍ɺେྔσʔλͷ෼ੳͳͲΛ୲౰
    ͢Δ -> Hadoop(MapReduce), Spark
    • αʔϏε૚͸όον૚ͷू໿݁ՌΛఏڙ͢Δ

    Hive, HBase, ElephantDB, Splout SQL, pipelineDB…
    • εϐʔυ૚͸ϦΞϧλΠϜॲཧͷ݁ՌΛఏڙ͢Δ૚

    Spark, Storm, Kafka, Cassandra etc..
    • αʔϏε૚ͱεϐʔυ૚ͷ྆ํͷ஋ΛϚʔδͯ͠ฦ٫

    ೉қ౓͸ߴ͍ɾɾɾ

    -> KafkaͳͲʹू໿ͤͨ͞Kappa Architecure΁

    View Slide

  19. KappaΞʔΩςΫνϟ

    View Slide

  20. KappaΞʔΩςΫνϟ

    View Slide

  21. OSSͰߏங

    View Slide

  22. PHPϝΠϯͰ࡞Δ͜ͱ͸೉͍͠ɾɾ

    View Slide

  23. Apache Cassandra

    View Slide

  24. Apache Cassandra
    • Ϩίʔυ͕େྔʹ૿͑Δ͜ͱ͕Θ͔͓ͬͯΓɺ

    ෼ੳʹར༻͢Δ༧ఆͰ͋ͬͨͨΊɺ

    εέʔϧ͕༰қͱ͍͏఺Ͱ࠾༻
    • PHP͔Βར༻Մೳ(ext-cassandra)
    • େྔσʔλͷॻ͖ࠐΈʹରԠ
    • ؆୯ͳτϥϯβΫγϣϯαϙʔτ
    • σʔληϯλʔލ͗ͷΫϥελʔߏங
    • Availability ͱ Partition Tolerance
    • SQLΠϯλʔϑΣʔε
    • ୯Ұো֐఺ͳ͠

    View Slide

  25. Apache Cassandra Architecture

    View Slide

  26. ؾΛ͚ͭΔ఺
    • RDBMSײ֮Ͱ͸͏·͘ར༻Ͱ͖·ͤΜ
    • ύʔςΟγϣϯΩʔͰ͏·͘ઃܭ͢Δ
    • ৚݅ʹΑΔΦʔμʔ͸ࢦఆͰ͖ͳ͍
    • ϚςϦΞϥΠζυϏϡʔซ༻͢΂͠
    • ো֐࣌ͷϩά෼ੳ͸ͨͩ͘͠
    • ίϯύΫγϣϯͱઓ͏(࣮ࡍʹར༻͢Δ༰ྔ*2Ͱܭࢉ)
    • ݕࡧͰҾ͔͔ͬΔهࣄ͸େମچόʔδϣϯͰɺ

    ݱߦͱશ͘ผ෺

    View Slide

  27. ςʔϒϧઃܭ
    • Primary Key͸ࣝผΩʔͰ͋Γͳ͕Βɺ

    Ͳͷnodeʹ֨ೲ͢Δ͔Λܾఆ͢ΔύʔςΟγϣϯΩʔ
    • ҟͳΔnodeʹ͋Δ΋ͷͷݕࡧ͸޲͔ͳ͍

    ඞཁͳέʔε͕ੜͨ͡৔߹͸ςʔϒϧઃܭΛݟ௚͢
    • ߋ৽࣌ɺ࡟আ࣌ʹؚΊͳ͚Ε͹ͳΒͳ͍
    • ར༻Մೳͳ΋ͷ͸ηΧϯμϦΠϯσοΫε·Ͱ
    • JOIN΍LIKE͸ଘࡏ͠ͳ͍ͨΊɺෳࡶͳ΋ͷ͸SparkͰ

    View Slide

  28. ςʔϒϧઃܭ
    CREATE TABLE timeline.user_timeline (
    uuid uuid,
    user_id int,
    reference map,
    body text,
    is_read tinyint,
    published_at timestamp,
    PRIMARY KEY (user_id)
    );

    View Slide

  29. Ϩίʔυͷॱ൪Λܾఆ͢Δ
    CREATE TABLE timeline.user_timeline (
    uuid uuid,
    user_id int,
    reference map,
    body text,
    is_read tinyint,
    published_at timestamp,
    PRIMARY KEY (user_id)
    )
    WITH CLUSTERING ORDER BY (published_at DESC);

    View Slide

  30. MATERIALIZED VIEW
    CREATE MATERIALIZED VIEW timeline.desc_user_timeline
    AS SELECT uuid, user_id, published_at, reference, body
    FROM timeline.user_timeline
    WHERE user_id IS NOT NULL
    AND published_at IS NOT NULL
    AND uuid IS NOT NULL
    PRIMARY KEY (user_id, published_at, uuid)
    WITH CLUSTERING ORDER BY (published_at DESC);

    View Slide

  31. From PHP
    $cluster = Cassandra::cluster()
    ->withContactPoints('10.0.1.24', ‘localhost')
    ->withPort(9042)
    ->build();
    $statement = $session->prepare(
    "UPDATE users SET age = ? WHERE user_name = ?”
    );
    $futures = array();
    // execute all statements in background
    foreach ($data as $arguments) {
    $futures[] = $session->executeAsync(
    $statement,
    [
    ‘arguments' => $arguments
    ];
    }

    View Slide

  32. PHP extension
    • Batchʹ࠷దԽ͞ΕͨI/F Batch Statement
    • ฒྻར༻Մೳ
    • Pagination͕༻ҙ͞Ε͍ͯΔ(Generatorར༻)
    • Cassandraͷ΄ͱΜͲͷػೳ͕ར༻Ͱ͖ΔͷͰɺ

    Java͔Βར༻ͤͣͱ΋े෼׆༻Ͱ͖Δ

    View Slide

  33. Apache Kafka

    View Slide

  34. Apache Kafka
    • Streamαϙʔτ(ϥϜμΞʔΩςΫνϟͰ͸ඞཁෆՄܽ)
    • ΫϥελϦϯά͕ࣗ༝ࣗࡏ
    • Zookeeperͱ࿈ܞͨ͠෼ࢄγεςϜ
    • ো֐ʹڧ͘ɺϝοηʔδͷ࠶औಘ͕Մೳ
    • SparkͱStormͱ༰қʹ࿈ܞͰ͖Δ͜ͱ͔Β࠾༻
    • ϝοηʔδૹ৴ޙͰ΋ࢦఆظؒอ࣋͠ɺ

    ଞͷΫΤϦΤϯδϯ͔Βϝοηʔδ಺༰औಘՄೳ
    • PHP͔Βར༻Մೳ(rdkafka)

    View Slide

  35. Message QueueͰൃੜ͢Δ໰୊
    • Producer͔ΒBroker΁ૹ৴࣌ʹܽଛ͢Δ͜ͱ͕͋Δ
    • Broker͕ड৴Λࣦഊ͢Δέʔε
    • Brokerͷૹ৴͕ࣦഊ͢Δέʔε
    • ॏෳͯ͠ड৴ͯ͠͠·͏έʔε

    View Slide

  36. 0.11
    • Exactly-once delivery and transactional messaging
    • ਖ਼֬ʹҰ౓͚ͩɺ࣮֬ʹಧ͚Δ
    • ϝοηʔδૹ৴ͱड৴ʹτϥϯβΫγϣϯʂ
    • ΑΓڧݻʹ

    View Slide

  37. Partition
    • ฒྻ΍෼ࢄॲཧ͕ઃܭ
    • topicΛPartitionͰ෼ׂ͠ɺProducer, Consumer͕೚ҙ
    ͷPartitionʹΞΫηε
    • ࡉ෼Խͱޮ཰Խ͕ࣗ༝ʹ

    View Slide

  38. Partition

    View Slide

  39. BigDataͷ࢝·Γ͸PHP͔Β

    View Slide

  40. View Slide

  41. webΞϓϦέʔγϣϯ͔Β
    BigData·Ͱࢧ͑ΔPHP

    View Slide

  42. ·ͱΊ
    • ෳࡶԽ͢ΔΞϓϦέʔγϣϯɺ

    ՝୊ղܾ͸େ͖͘ͳΔ୉ޣຯ
    • PHPͰϏδωεΛαϙʔτ͢Δ؅ཧπʔϧ
    • PHP͔Β࢝·ΔBigData + FastDataΞʔΩςΫνϟ
    • PHPͰ΋େ͖͘ߩݙ

    View Slide