Google App Engine を利用した、新規 Web サービスの立ち上げ方

by Osamu TONOMORI

Slide 1

Slide 1 text

Google App Engine Λར༻ͨ͠  ৽ن Web αʔϏεͷ্ཱͪ͛ํ Osamu TONOMORI TECH PLAY CONFERENCE 2017  (2017/08/20)

Slide 2

Slide 2 text

Agenda 1. Who Am I ? 2. Overview of Google App Engine 3. In the case of Mercari Kauru 4. Summary

Slide 3

Slide 3 text

Who Am I ?

Slide 4

Slide 4 text

Who Am I ? • ओ৿ ཧ - Osamu TONOMORI  ʢTwitter & GitHub: @osamingoʣ • גࣜձࣾι΢κ΢ - ϝϧΧϦΧ΢ϧνʔϜ  ʢEx-Company: CyberAgent, Inc. ~ AbemaTV, Inc.ʣ • ιϑτ΢ΣΞΤϯδχΞ - Gopher  ʢServer-side Leader, Scrum Master, Mood Makerʣ

Slide 5

Slide 5 text

My GitHub - https://github.com/osamingo

Slide 6

Slide 6 text

Latest interview - http://mercan.mercari.com

Slide 7

Slide 7 text

Overview of Google App Engine

Slide 8

Slide 8 text

About Google App Engine (GAE) • Google Cloud Platform ͕ఏڙ͢ΔαʔϏεͷͻͱͭɻ • 2007೥ʹ Python, Java ͕࢖͑Δ PaaS ͱͯ͠ϦϦʔεɻ • GAE ʹ͸ɺStandard Environment, Flexible Environment ͷ2छྨ͕͋Δɻ • ར༻Ͱ͖ΔϦʔδϣϯ͕ݶΒΕ͍ͯΔ͕ɺ౦ژϦʔδϣϯʹ͸༗Γ㽂ɻ • ओͳڝ߹͸ɺHeroku, Engine Yard ͳͲɻ

Slide 9

Slide 9 text

Google App Engine - Standard Environment (SE) • 2007೥ͷϦϦʔε౰ॳ͔Β͋Δͷ͕ɺStandard Environment Ͱ͢ɻ • ࢖༻Ͱ͖Δݴޠ͕ɺݶΒΕ͍ͯΔɻ  Python 2.7.x, Java 7 or 8(β), PHP 5.5.x, Go 1.6 or 1.8 (β) • ϧʔϧ͸ଟ͍͕ɺͦͷ෼։ൃ΍ӡ༻ָ͕ʹͰ͖Δɻ • ઃఆΛ޻෉͢Ε͹ɺແྉͰ࢖͍ଓ͚ΒΕΔɻ

Slide 10

Slide 10 text

Google App Engine - Flexible Environment (FE) • Google Cloud Next 2017 ʹͯɺGA ʹͳͬͨʂ • ΞϓϦέʔγϣϯͷத਎͸ɺ nginx + docker ߏ੒ʹͳ͍ͬͯΔɻ  docker ͳͷͰɺՄೳੑ͸ແݶେͱ͍ͬͯ΋աݴͰ͸ແ͍ʂ • Google ͸ɺ Flexible Environment Λਪͯ͠ߦ͖͍ͨงғؾΛײ͡Δɻ  Google App Engine ͷ LP ͕ɺFE લఏͰͷઆ໌͕ଟ͍ɻ

Slide 11

Slide 11 text

Google App Engine - How to choose SE or FE? • ࠓͷͱ͜Ζ͸ɺStandard Environment Λબ୒͢Ε͹ؒҧ͍ͳ͍ɻ • Flexible Environment Λબ୒͢Δ৔߹ͷνΣοΫ߲໨  ✔ Python, Java, PHP, Go Ҏ֎ͷݴޠͰ։ൃ͍ͨ͠ɻ  ✔ ϑΝΠϧ࡞੒ͳͲɺҰ࣌తͳσΟεΫॻ͖ࠐΈ͕ඞཁʹͳΔɻ  ✔ NAT ήʔτ΢ΣΠͳͲɺ಺෦ωοτϫʔΫʹॴଐ͢Δඞཁ͕͋Δɻ  ✔ SSH ઀ଓΛߦ͍ɺσόοά͢Δඞཁ͕͋Δɻ

Slide 12

Slide 12 text

Google App Engine - Scaling • Automatic (SE, FE)  Application ͷϝτϦΫε৘ใʹΑͬͯ Instance ਺͕ࣗಈՄม͢Δɻ  ࣌ؒଳʹΑΔΞΫηεมಈ͕ଟ͍ɺAPI ͳͲʹ޲͍͍ͯΔɻ • Basic (SE)  ىಈ͢Δ࠷େ Instance ਺Λઃఆ͢Δ৔߹ʹઃఆ͢Δɻ  ઃఆͰ͖Δ Instance class ΋๛෋ͰɺBatch ͳͲʹ޲͍͍ͯΔɻ • Manual (SE, FE)  ىಈ͍ͯ͠Δ Instance Λݻఆ͢Δ৔߹ʹઃఆ͢Δɻ

Slide 13

Slide 13 text

Google App Engine - Database • Cloud Datastore  Google ۘ੡ NoSQL ͷ Full-Managed αʔϏεɻ  GAE ͷ৔߹͸ɺجຊతʹ Datastore Λ࢖͏͜ͱʹͳΔɻ • Cloud SQL  Full-Managed ͷ RDB αʔϏεɻ  ରԠ͍ͯ͠Δͷ͸ɺ MySQL 5.6 or 5.7, PostgreSQL 9.6 (β) ͷ 2 छྨ • Others  Cloud Bigtable, Cloud Spanner, etc.

Slide 14

Slide 14 text

Google App Engine - Caching System • Memcache  GAE ͔ΒʹݶΓɺ Managed ͷ Memcache ͕ར༻Ͱ͖Δɻ  ΍ͨΒૣ͘ 5ms ͱ͔ͰɺΩϟογϡऔಘ͕Ͱ͖Δɻ  ͔͠͠ɺ՝ۚͯͨ͠ͱͯ͠΋ SLA ͸อূ͞Εͳ͍ͷͰ஫ҙ

Slide 15

Slide 15 text

Google App Engine - Object Storage • Blobstore  Cloud Storage ͕ϦϦʔε͞ΕΔ·Ͱ͸ɺBlobstore ͕ओྲྀͩͬͨɻ  URL ൃߦ͢Δඞཁ͕͋Δ͕ɺେ͖͍αΠζʢ2GBʣͷϑΝΠϧΛѻ͑Δɻ • Cloud Storage  ݱࡏͷओྲྀ͸ɺCloud Storage Ͱར༻Λਪ঑͞Ε͍ͯΔɻ  Blobstore ͷػೳ΋ར༻Ͱ͖ΔΑ͏ʹޓ׵ੑ͕อͨΕ͍ͯΔɻ

Slide 16

Slide 16 text

Google App Engine - Message Queueing • Taskqueue  ֤ Service (Module) ؒͷඇಉظॲཧʹΑ͘༻͍Δɻ  GAE ͔ΒͷΈ & ႈ౳ੑ୲อΛ͠ͳ͍͍͚ͳ͍͕ɺ࢖͍উख͕ͱͯ΋ྑ͍ɻ • Cloud Pub/Sub  GAE ಺ʹ͸ดͣ͡ɺGAE ͱଞαʔϏεΛܨ͙͜ͱ͕Ͱ͖Δɻ  Cloud Storage ͷ Upload Notiﬁcation ͳͲɻ

Slide 17

Slide 17 text

Google App Engine - Job Scheduler • Taskqueue - cron job  Taskqueue ͷ 1 ػೳͱͯ͠ɺఏڙ͞Ε͍ͯΔɻ  ઃఆͨ͠ Endpoint ʹରͯ͠ Kick ͢ΔɻϦτϥΠػೳ΋༗Γɻ  ਓؒʹ༏͍͠ઃఆه๏Ͱخ͍͠

Slide 18

Slide 18 text

Google App Engine - Search • Search API  GAE ͔ΒݶΓར༻Ͱ͖Δɺݕࡧػೳɻ  ܗଶૉղੳ΍ɺϑΝηοτݕࡧ΋͋ΔͷͰߜࠐΈػೳ΋ڧ͍ɻ  ͨͩ͠ɺN-gram ΍ɺSynonym ʹ͸ରԠ͍ͯ͠ͳ͍ɻ

Slide 19

Slide 19 text

Google App Engine - Logging • Stackdriver Logging  ΞΫηεϩά͸ɺࣗಈతʹ Stackdriver Logging ʹू໿͞ΕΔɻ  ΞϓϦέʔγϣϯ಺Ͱग़ྗ͢Δϩά΋ɺಉ༷ʹू໿͞ΕΔɻ  ಛఆͷ৚݅ͷϩάΛ BigQuery ౳ʹྲྀ͢͜ͱ΋ग़དྷΔɻ

Slide 20

Slide 20 text

Google App Engine - Monitoring & Alert • Stackdriver Monitoring  Stackdriver Logging ͱಉ༷ʹࣗಈతʹϝτϦΫε͕ऩू͞ΕΔɻ  GAE ͚ͩͰͳ͘ɺGCP ্ʹ͋ΔαʔϏε΋ࣗಈతʹऩू͞ΕΔɻ  ऩू͞ΕͨϝτϦΫεΛݩʹ Alert ͷઃఆ΋Ͱ͖Δɻ

Slide 21

Slide 21 text

Google App Engine - Analytics • BigQuery  ΈΜͳ͍͖ͩ͢ͳ BigQuery େઌੜɻ  ৄࡉ͸ɺࣳඌ͞Μ͕࿩ͯ͘͠ΕΔ͸ͣɻ

Slide 22

Slide 22 text

Google App Engine - Others • Cloud Endpoint  ΞΫηε؅ཧ΍ɺϞχλϦϯά͕Ͱ͖Δ API ήʔτ΢ΣΠɻ  Open API ࢓༷Λ༻͍ͯɺΫϥΠΞϯτͱαʔόؒͰ࢓༷Λڞ༗Ͱ͖Δɻ  gRPC (β) ൛΋͋Δ͕ɺݱࡏ GAE ʹ͸ະରԠ

Slide 23

Slide 23 text

Google App Engine - Use cases • Snapchat (Snap, Inc.)  Google Cloud Platform Live: Interview with SnapChat's Bobby Murphy  https://youtu.be/17PtS1Qx8kU • Super Mario Run (Nintendo Co., Ltd. & DeNA, Inc.)  Google Cloud Platform ಋೖࣄྫ - ೚ఱಊגࣜձ༷ࣾ  https://youtu.be/-29lW2EWock

Slide 24

Slide 24 text

In the case of Mercari Kauru

Slide 25

Slide 25 text

What is Mercari Kauru ?

Slide 26

Slide 26 text

In 3 months since launch About 1,000,000 items! About 500,000 products!

Slide 27

Slide 27 text

Team Members at Launch

Slide 28

Slide 28 text

Team Members - Server-side • ࠷ॳͷ 3 ϲ݄͙Β͍͸ɺࣗ෼ͻͱΓ͚ͩɻ  GAE ͩͱɺΠϯϑϥߏங͍Βͳ͍ͷͰଈ೔͔Βίʔυॻ͚Δɻ • ݱࡏ͸ɺ4 ਓʢ1ਓ͸ผνʔϜ݉೚ʣମ੍Ͱ։ൃ͍ͯ͠Δɻ  શһ Server-side ΤϯδχΞͰɺInfra ઐ೚͸ډͳ͍ɻ

Slide 29

Slide 29 text

About Technology

Slide 30

Slide 30 text

Architecture

Slide 31

Slide 31 text

Service • ؆୯ʹ Microservices Λ࣮ݱ͢Δػೳ͕ Service ͱ͍͏֓೦Ͱ͢ɻ • Service ຖʹ Endpoints Λ࣋ͪɺScaling ͷઃఆ΋ग़དྷ·͢ɻ

Slide 32

Slide 32 text

API Service • ٸͳߴෛՙʹ΋ରԠͰ͖ΔΑ͏ʹ Automatic Scaling Λ࠾༻ɻ • e.g. Mercari ͔Βͷ༠ಋͰҰؾʹෛՙ͕2ഒҎ্ʹͳΔ͜ͱ͕͋Δɻ

Slide 33

Slide 33 text

Webhook Service • ֎෦αʔϏε͔Βͷ Event Λड͚෇͚Δɻ • Mercari, Tracking ͳͲ • Webhook Ͱ͸ɺ۩ମతͳॲཧ͸͠ͳ͍ɻ • Taskqueue ʹྲྀ͔͢ɺLogging Λߦ͏͚ͩɻ • ڧ͍ Instance ͸ཁΒͳ͍͠ɺScale out ͚ͩͯ͘͠ΕΕ͹ྑ͍ɻ

Slide 34

Slide 34 text

Management Service (Event & Batch) • ඇಉظͰߦ͏ॲཧΛશͯ੥͚ෛ͏αʔϏε • Basic Scaling Ͱ Instance Class ΛڧΊʹઃఆͯ͠ॲཧ͠·͘Δɻ • ࣌ؒͷ͔͔Δ Batch ॲཧ΋ߦ͏ɻ  ʢਖ਼௚ɺSubscriber, Batch ʹ෼ׂ͢Ε͹ྑ͔ͬͨͱޙչ͍ͯ͠Δɻʣ

Slide 35

Slide 35 text

Other Services • Admin  ؅ཧπʔϧͳͷͰɺInstance ΛཱͯΔඞཁແ͠ɻ • Admin API  ؅ཧπʔϧ޲͚ͷ API ͳͷͰɺ0~1 instance ͰOKɻ

Slide 36

Slide 36 text

Database • Mercari Kauru Ͱ͸ɺ  ɹCloud Datastore  ɹCloud SQLʢMySQL 5.7ʣ  ͷͲͪΒ΋ར༻͍ͯ͠·͢ɻ • DDD Λ࠾༻͍ͯͯ͠ɺ  Domain Repository ͰӅṭ͍ͯ͠Δɻ

Slide 37

Slide 37 text

Database - Cloud Datastore • Datastore Ͱ͸ɺMercari Kauru ݻ༗ͷσʔλΛอ͍࣋ͯ͠·͢ɻ • ಛʹແݶʹ஝ੵ͞Ε͍ͯ͘ܥͷσʔλ͸ɺNoSQL ͷಘҙ෼໺ɻ • Like, Stock, Notiﬁcation, PushNotiﬁcationToken, etc.

Slide 38

Slide 38 text

Database - Cloud SQL • MySQL ʹ͸ɺ੡඼ͷϚελσʔλΛอ͍࣋ͯ͠·͢ɻ • ݩͷ Schema ͕ɺਖ਼نԽ͞Ε͍ͯͯ NoSQL ʹ֨ೲ͢Δͷ͸೉͍͠ɻ • Ϛελσʔλ͸ɺ੔ܗ͞Ε Cache Խ͞ΕΔͷͰɺগΞΫηεɻ  ʢGAE ͔Β͸ɺ12 connections/instance ͱ੍ݶ͞Ε͍ͯΔɻʣ • ͜͜Λؒҧ͑ΔͱΤά͍ͷͰɺอશ͕ୈҰͷબ୒Λͨ͠ɻ

Slide 39

Slide 39 text

Version • Service ͸ɺෳ਺ͷ Version Λ࣋ͭ͜ͱ͕Ͱ͖Δɻ  1SPKFDU 4FSWJDF 7FSTJPO .FSDBSJ,BVSV "1* W W #BUDI W W

Slide 40

Slide 40 text

Version - Production • Production Ͱ͸ɺCanary release Λͨ͠ΓɺRollback ΋༻ҙʹՄೳɻ • 10% ͚ͩ։์ͨ͠Γͯ͠ɺ༷ࢠΛݟͨΓͰ͖Δɻ

Slide 41

Slide 41 text

Version - Development • Development Ͱ͸ɺbranch ຖʹ؀ڥΛ؆୯ʹ༻ҙ͍ͯ͠Δɻ • ֤ Version ʹ͸ݻ༗ͷαϒυϝΠϯ͕ൃߦ͞ΕΔɻ • ฒྻͰػೳ։ൃ΍ɺQA Λߦ͑ΔͷͰͱͯ΋ศརɻ

Slide 42

Slide 42 text

Logging & Analytics • Application ͔Βॻ͖ग़͢ϩά͸ JSON Format Ͱॻ͖ग़͍ͯ͠Δɻ • Stackdriver Logging ʹू໿͞ΕͨޙɺBigQuery ʹྲྀ͠ࠐΜͰ͍Δɻ  BigQuery ʹ͸ɺJSON ؔ਺͕͋ΔͷͰѻָ͍͕ʹͳΔɻ • Firebase Analytics ͔Β΋ BigQuery ʹ Event ϩάΛྲྀ͠ࠐΜͰ͍Δɻ  Server, Client Λॎஅͯ͠෼ੳΛߦ͑Δ (๑•̀ŷ•́)و✧

Slide 43

Slide 43 text

Release & Deploy • શͯ ChatOps Ͱߦ͍ͬͯΔɻ  @kaurungo [release, migrate, trafﬁc, etc.] • GAE ʹ͸ Admin API ͱ͍͏ɺ༷ʑͳૢ࡞͕Ͱ͖Δ API ͕͋Δɻ  ops-bot ͱ͍͏ Service Λ੾ͬͯɺ GAE ্Ͱӡ༻͍ͯ͠Δɻ

Slide 44

Slide 44 text

Summary

Slide 45

Slide 45 text

Summary • খ࢝͘͞ΊΔͳΒ Google App Engine ͸ɺྑ͍બ୒ࢶͷͻͱͭɻ  ಛʹ։ൃΑΓ΋ɺӡ༻ΛΞάϨογϒʹߦ͑Δɻ • େ͖͘ҭͯΔͷ΋ྑ͠ɺGCP ͷԸܙΛ਺ଟ͘ड͚Δ͜ͱ͕Ͱ͖Δɻ  ͱͯͭ΋ͳ͘ώοτͨ͠Βɺ࠷খݶͷίετͰ GCE, GKE ʹҠߦͰ͖Δɻ • φϨοδ΋஝ੵ͞Ε͍ͯΔ͠ɺϝϯςφϯε΋׆ൃʹߦΘΕ͍ͯΔɻ  ಛʹ೔ຊͰ͸ɺGCPUG ίϛϡχςΟͷԸܙ͕ड͚ΒΕΔɻ

Slide 46

Slide 46 text

Thanks for your attention!