Cloud connect the world as a Glue - AWS Dev Day 2017

Transcript

1. Cloud connect the world as a Glue
   AWS Dev Day 2017 Track 2
   Masahiro Nagano @kazeburo
   

   View Slide

2. Me
   • Masahiro Nagano / ௕໺խ޿
   • @kazeburo
   • Mercari, Inc

   Principal Engineer

   Site Reliability Engineering (SRE) Team
   • BASE, Inc Technical Advisor
   

   View Slide

3. SRE Team ͷ঺հ
   

   View Slide

4. SRE
   • Site Reliability Engineering ͷུ
   • Google ͷӡ༻νʔϜΛ཰͍Δ Ben Treynor ͕ఏএ
   • Google ͷ༷ʑͳϓϩμΫτɾαʔϏεΛԣஅͯ͠ɺιϑτ΢ΣΞΤ
   ϯδχΞϦϯάΑΓαΠτ/αʔϏεͷ৴པੑΛ޲্ͤ͞Δ Software
   Engineering/Teamͱͦͷ࣮ફ = Google SRE
   

   View Slide

5. Google SRE
   • ιϑτ΢ΣΞΤϯδχΞ(SWE)ͱͯ͠࠾༻
   • ӡ༻ͷۀ຿Λ50%ҎԼʹ཈͑Δ
   • 50%͸ΦϖϨʔγϣϯͷࣗಈԽɺιϑτ΢ΣΞͷ৴པੑ޲্ʹ͋ͯΔ
   • ΤϥʔόδΣοτͱ͍͏ߟ͑ํ
   • SREͱSWEͷSLAΛऔΓܾΊɺར֐ΛҰகͤ͞Δ
   http://landing.google.com/sre/book.html
   

   View Slide

6. View Slide

7. Mercari SRE
   • ͍ͭͰ΋շద͔ͭ҆શʹར༻Ͱ͖Δʮ৴པੑͷߴ͍ʯαʔϏεͷ࣮ݱ
   • ʮ৽نαʔϏεͷ։ൃҎ֎ͷΤϯδχΞϦϯά͸શ෦΍Δʯ
   • 2015/11 ʮΠϯϑϥνʔϜʯ͔ΒSRE΁
   • ʮΠϯϑϥʯΑΓ΋αʔϏεࢦ޲
   • ݱࡏϝϯόʔ͸ʮ6ਓʯઈࢍืूத
   

   View Slide

8. Mercari SRE ͷۀ຿ൣғ
   Operations Software Eng.
   ج൫ߏங
   OnCall (ো֐ରԠ)
   Automation
   εέʔϥϏϦςΟɾՄ༻ੑվળ
   ϛυϧ΢ΣΞߏங
   ΞϓϦέʔγϣϯͷઃܭϨϏϡʔ
   ϩάऩूɾ෼ੳج൫ͷߏஙɺӡ༻
   αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ
   ηΩϡϦςΟʗෆਖ਼ར༻ݕग़
   

   View Slide

9. Agenda
   • ϝϧΧϦͱ͸ / ੈք3ڌ఺Ͱͷ։ൃӡ༻ମ੍ʹ͍ͭͯ
   • ϝϧΧϦͷΞʔΩςΫνϟ / Ϋϥ΢υͷར༻
   • ڑ཭Λ௒͑ΔɺੈքΛܨ͙γεςϜ
   

   View Slide

10. Mercari
    • ࠃ಺࠷େڃͷϑϦϚΞϓϦ
    • 3෼Ͱ؆୯ʹग़඼
    • ҆৺҆શͳܾࡁ
    

    View Slide

11. Mercari KPI
    μ΢ϯϩʔυ਺
    GMV(૯औҾֹ)
    6500ສDL(JP+US)
    ݄ؒ100ԯԁҎ্
    ग़඼਺ 1೔100ສ඼Ҏ্
    

    View Slide

12. μ΢ϯϩʔυ਺ਪҠ(JP)
    JP μ΢ϯϩʔυ਺ 4000ສ (2016/11)
    

    View Slide

13. ೔ຊ࠷େͷϑϦϚΞϓϦ
    1,200
    1෼ؒͷग़඼਺(peek࣌ؒଳ)
    ඼Ҏ্
    

    View Slide

14. ग़඼͔Β͙͢ʹചΕΔ
    24
    ࣌ؒҎ಺
    ചΕͨ঎඼ͷ໿50%͕
    24࣌ؒҎ಺ʹऔҾ੒ཱ
    

    View Slide

15. Global Service
    JP
    2016/08
    US AppStore
    3Ґ
    US UK
    2017/03/15
    ϦϦʔε
    

    View Slide

16. Global Development Team
    Tokyo
    San Francisco
    London
    San Francisco/London ʹΦϑΟε
    ݱ஍࠾༻ɺग़޲ऀɺ௕ظग़ு߹Θͤͯ
    ΤϯδχΞ͕਺໊͔Β਺े໊
    

    View Slide

17. Global Development Team
    • Tokyo
    • ։ൃͷத৺ɻJPʹՃ͑ͯશͯͷregionͷ։ൃ
    • San Fransisco
    • αʔϏεͷϩʔΧϥΠζ
    • ݴޠ͚ͩͰ͸ͳ͘ɺจԽ΍श׳ͷϩʔΧϥΠζ
    • London
    • αʔϏε্ཱͪ͛ϑΣʔζ / ݱ஍ͷ๏ྩͳͲʹ߹ΘͤͨϩʔΧϥΠζ
    

    View Slide

18. Global Development ͷ೉͠͞
    10:00
    1:00
    18:00
    9࣌ؒ
    7࣌ؒ
    લ೔
    Tokyo
    San Francisco
    London
    3ڌ఺ἧͬͯإΛ߹ΘͤΔ͜ͱ͕͔ͳΓ೉͍͠
    

    View Slide

19. Global Development ͷਐΊํ(1)
    • Ϋϥ΢υΛ׆༻ͯ͠ίϛϡχέʔγϣϯΛਤΔ
    • ଠฏ༸ɾେ੢༸Λ·͍ͨͩPull ReqeuestϨϏϡʔ
    • Slack
    • Video Conference
    • ϦϞʔτϖΞϓϩ(εΫϦʔϯڞ༗)
    

    View Slide

20. Global Development ͷਐΊํ(2)
    • ཱࣗͨ͠νʔϜͱͯ͠՝୊ղܾ͢Δ
    • ϓϩδΣΫτϚωʔδϟͱɺΫϥΠΞϯτ͔ΒαʔόαΠυΤϯ
    δχΞ·ͰϑϧελοΫͷϓϩμΫτνʔϜΛݱ஍Ͱ݁੒
    • νʔϜؙ͝ͱग़ு
    • iOS/Android͸RegionʹΑͬͯfork͋Δ͍͸ɺbranchΛ෼͚ɺޓ͍
    ͷӨڹΛݮΒ͢
    

    View Slide

21. SREͷέʔε
    • 6ਓͷ͏ͪɺ1ਓ͕௕ظUSग़ுத
    • ݱ஍։ൃͷαʔϏεͷΦϖϨʔγϣϯͷ೺Ѳ
    • ि1ͰUSͱͷSync MTG
    • ே9࣌(PDT 17:00) ʹࣗ୐ʹͯ Video Conference
    • UKͱ͸Ҋ݅ϕʔεͰ༦ํʹMTG
    • OnCall ౰൪͸ே9͔࣌Βࣗ୐଴ػɻUS͔Βͷ࡞ۀґཔʹ͋ͨΔ
    

    View Slide

22. Mercari Architecture
    

    View Slide

23. Infrastructure
    ੴङDC
    ઐ༻αʔό
    JP
    Cloud
    US
    Cloud
    UK
    Hybrid & Multi Cloud
    

    View Slide

24. Infrastructure history (1)
    • 2013/07 JP ϦϦʔε
    • ͘͞ΒΠϯλʔωοτͷVPS 1୆ʹWeb΋DB΋͢΂ͯࡌͤͨ
    • ΠϯϑϥετϥΫνϟઐ೚ऀ͍ͳ͍தͰɺ਎ۙͳٕज़Λબ୒
    • ϦϦʔεޙ2ϲ݄Ͱ͘͞ΒΫϥ΢υɺઐ༻αʔόʹҠߦ͖ͯͨ͠
    

    View Slide

25. Infrastructure history (2)
    • 2014/09 US ϦϦʔε
    • AWS (Oregon) ʹͯαʔϏεߏங
    • JPϦϦʔε౰ॳʹൺ΂ͯΤϯδχΞ͕૿͑ɺAWSܦݧऀ΋ଟ͘ͳͬͨ
    • ͦΕͰ΋·ͩΠϯϑϥετϥΫνϟઐ೚ऀ͸গͳ͘ɺAWSͷϚωʔδυ
    αʔϏεΛଟ͘ར༻ͯ͠αʔϏεΛߏங
    • USࠃ಺ͷઐ༻αʔόར༻΋ݕ౼͕ͨ͠ɺUSͷεέʔϧ͸༧૝ͮ͠Β͘ɺ
    Ϋϥ΢υͷॊೈ͞Λ೔ຊΑΓ΋ॏཁࢹͨ͠
    

    View Slide

26. Infrastructure history (3)
    • (2015/02 kazeburo ೖࣾ)
    • 2015/11 SREνʔϜൃ଍
    • ͘͞ΒΠϯλʔωοτͱAWSͷϋΠϒϦουͳΠϯϑϥετϥΫνϟ
    ͷ্ͷΞʔΩςΫνϟΛਐԽͤ͞ɺ৴པੑͱεέʔϥϏϦςΟͷ޲্
    • 2017/03 UK ϦϦʔε
    • ৽͍ٕ͠ज़తνϟϨϯδͱͯ͠GCPΛબ୒
    

    View Slide

27. Architecture
    • ࡾ૚+αͳγϯϓϧͳΞʔΩςΫνϟ
    • Reverse Proxy = nginx

    Application = Apache+mod_php

    Database = MySQL

    Cache = memcached

    Search = Solr
    • ଟ͘Λ෺ཧαʔόʹͯߏ੒
    • εέʔϧΞοϓ΋εέʔϧΞ΢τ΋ߦ͏Diagonal Scaleࢦ޲
    • Databaseʹ͸ ioMemory ΍ NVMe Λ౥ࡌͨ͠αʔόΛ࠾༻
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    WS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    fic
    DNS-RR
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    cloud cloud
    JP
    

    View Slide

28. Architecture
    JP ͷΞʔΩςΫνϟΛجຊ౿ऻ
    EC2/GCE (αʔό) Λத৺ͨ͠ߏ੒
    ɾ
    USಠࣗͷαʔϏε΍

    খن໛ʙதن໛DBʹ͸ RDS
    UKͰ͸Cloud Load BalancerΛར༻
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    anagement
    onsole
    IAM Add-on Example:
    IAM Add-on
    man Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    DNS-RR
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    US
    RDS
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2 EC2
    EC2
    EC2
    EC2 EC2
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corpora
    data cen
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    GCE
    cloud load balancer
    GCE GCE
    GCE GCE GCE
    GCE GCE GCE
    GCE GCE
    GCE
    GCE
    GCE GCE
    UK
    

    View Slide

29. αʔόத৺ͷ Architecture
    • ϝϯςϯεϏϦςΟɾεέʔϥϏϦςΟઓུͷڞ௨Խ
    • গਓ਺Ͱͷӡ༻
    • Ansible Playbook ࠶ར༻
    • εέʔϧ͕ઌߦ͍ͯ͠ΔJPͰ࣮੷͋Δߏ੒
    • US Ͱͷ App Store ϥϯΩϯά3ҐͷτϥϑΟοΫ΋໰୊ͳ͘ӡ༻
    • EC2ͷIaaSͱͯ͠ͷύϑΥʔϚϯεɺ৴པੑ͸͔ͳΓ޲্͍ͯ͠Δ
    

    View Slide

30. Mercari Architecture ·ͱΊ
    • 3ͭͷRegionͰ࠾༻͢ΔΠϯϑϥετϥΫνϟ͕ҟͳΔ
    • JP/US/UK ͸αʔόΛத৺ͱͨ͠ArchitectureΛ࠾༻
    • AWSͰ΋Ϋϥ΢υΒ͍͠ઃܭ͸ͤͣɺن໛Ͱઌߦ͢ΔJPʹ߹Θͤ
    Δ͜ͱͰɺӡ༻ͷڞ௨ԽͱলྗԽ
    • ϝϧΧϦͰ͸Ϋϥ΢υΛੵۃతʹ࢖͍ͬͯͳ͍ʁ
    • JP/US/UK ڞ௨ͷΠϯϑϥετϥΫνϟͰར༻͍ͯ͠·͢
    

    View Slide

31. Mercari Global Infrastructure
    

    View Slide

32. Global Infrastructure
    • Mercari JP/US/UK ͷΠϯϑϥετϥΫνϟ͸ಠཱ͍ͯ͠Δ
    • σʔλΛαʔϏεΛߦ͏Ҭ಺ʹཹΊΔඞཁੑ
    • ͍͔ͭ͘ͷΫϥ΢υαʔϏεΛڞ௨ͯ͠ར༻
    • ւ֎ͰͷΞΫηεվળ
    • Ϋϥ΢υͷߴ͍εέʔϥϏϦςΟɾ৴པੑʹΑΓαʔϏεͷՄ༻
    ੑΛอͭ
    

    View Slide

33. Global Infrastructure
    JP US UK
    DNS: Amazon Route53
    CDN: Akamai, CloudFront
    Storage: Amazon S3
    Analysis: Google BigQuery
    Common Micro Services
    ܾࡁ/෺ྲྀ/Domestic Service ܾࡁ/෺ྲྀ/Domestic Service ܾࡁ/෺ྲྀ/Domestic Service
    ڞ௨ΞʔΩςΫνϟ
    αʔό͕த৺
    ֤Region
    Ϋϥ΢υ͕த৺
    ৴པੑͷߴ͍AWSͷ
    αʔϏε͕ڬΈࠐΉ
    

    View Slide

34. Amazon Route53
    • ߴ͍Մ༻ੑͱ৴པੑͷDNS
    • Roadworker Λར༻
    • github.com/codenize-tools/roadworker
    • Routefile ΛGithubͰ؅ཧ
    • Pull RequestͷmergeޙɺCIΛܦ༝ͯ͠
    ࣗಈ൓ө
    hosted_zone "mercari.jp." do
    rrset "api.mercari.jp.", "CNAME" do
    ttl 30
    resource_records(
    "endpoint-api.mercari.jp"
    )
    end
    end
    #Routefile
    PR
    Github Travis-CI Route53
    

    View Slide

35. Amazon Route53 + HealthCheck
    • DNS-RR ӡ༻࣌ͷ໰୊఺
    • αʔόো֐࣌ʹDNSͷॻ͖׵͑ʹ͕͔͔࣌ؒΔ
    • ϒϥ΢βͳͲͷҰ෦ΫϥΠΞϯτ͸DNS-RRͷ৔߹ɺҰ෦ͷαʔόʹ઀ଓ͕Ͱ͖
    ͳ͍৔߹ɺଞͷαʔό΁઀ଓ͠௚͢ͷͰো֐ʹΑΔӨڹ͸େ͖͘ͳΓʹ͍͘ɻ
    • ϚΠΫϩαʔϏεԽ͕ਐΉͱ༷ʑͳϒϥ΢βҎ֎ͷΫϥΠΞϯτ͕઀ଓ͢Δɻ
    ଟ͘͸DNS-RRͷো֐࣌ͷ࠶઀ଓ͸࣮૷͞Εͯͳ͍
    • Route53 ͷ Health CheckΛ࢖͍ղܾ(Λݕূத)
    

    View Slide

36. Route53 + Health Check with Roadworker
    [“153.x.y.150”, "153.x.y.151"].each do |ip|
    rrset "endpoint-ha.mercari.jp.", "A" do
    ttl 30
    weight 1
    set_identifier “endpoint-ha-“ + ip.gsub(/\./,'-')
    health_check "http://#{ip}/hc", :request_interval => 30, :failure_threshold => 3
    resource_records(
    "#{ip}"
    )
    end
    end
    Health CheckʹΑΓ DNS-RR Ͱ΋Մ༻ੑΛߴΊΒΕΔ
    #Routefile
    

    View Slide

37. (࿩͸ͦΕ·͕͢)
    ಺෦ DNS
    • શͯͷαʔόʹunboundΛಋೖ
    • ϩʔΧϧΩϟογϡʹΑΔύϑΥʔϚϯε޲্
    • resolv.confΑΓՄ༻ੑ্͕͕Δ
    • DNSαʔόͷunboundͰϦΫΤετΛৼΓ෼͚
    • *.local ͸BIND͕ݖҖαʔό
    • *.consul ͸consul DNS interface
    App App App
    App App App
    DNS DNS
    unbound unbound unbound
    unbound unbound unbound
    DNS
    unbound
    Consul/DNS
    BIND
    *.consul
    *.local
    

    View Slide

38. (࿩͸ͦΕ·͕͢)
    ಺෦DNSͰCNAME
    • ಺෦DNSͰϚωʔδυαʔϏεͷΤϯυϙΠϯτͷCNAMEΛઃఆ
    • ΞϓϦέʔγϣϯ͔Β઀ଓ͸CNAMEܦ༝
    • ϚωʔδυαʔϏε͔ΒϚωʔδυαʔϏε΁ͷҠߦɺϚωʔδυ
    αʔϏε͔ΒEC2΁ͷҠߦɺ·ͨͦͷٯͷҠߦ͕΍Γ΍͍͢
    db-cstool-master IN CNAME cstool-db.XXXXX.us-west-2.rds.amazonaws.com.
    

    View Slide

39. Amazon S3
    • ߴ͍Մ༻ੑͱ৴པੑͷετϨʔδ
    • ঎඼ը૾ɺϩάɺσʔλϕʔεͷόοΫΞοϓͳͲ͋ΒΏΔσʔλ
    Λ֨ೲ
    • IAMΛར༻ͨ͠ߴ౓ͳΞΫηε؅ཧͱૄ݁߹ͷ࣮ݱ
    • αϒγεςϜ͔ΒͷσʔλΠϯϙʔτɾΤΫεϙʔτ
    • ֎෦αʔϏεɾύʔτφʔͱͷσʔλड͚౉͠खஈ
    

    View Slide

40. ͋ΒΏΔσʔλͷετϨʔδ: ঎඼ը૾
    App App App
    App App App
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    ent IAM Add-on Example:
    IAM Add-on
    ligence
    HIT)
    Assignment/
    Task
    Requester
    Workers
    ঎඼ը૾σʔλ͸ಉظతʹॖখ/Ξοϓϩʔυ
    AWS SDK for PHPΛར༻ɻෳ਺ͷը૾Λฒߦͯ͠PUTͯ͠଎౓޲্
    ग़඼!
    ਺ඦສຕ/day
    

    View Slide

41. ͋ΒΏΔσʔλͷετϨʔδ: ϩά
    Log
    Log
    ΞΫηεϩάʗΤϥʔϩάͳͲ֤छϩά͸fluentܦ༝Ͱू໿ͯ͠S3ʹ֨ೲ
    aws-cli ·ͨ͸ fluent-plugin-s3
    batch + aws-cli
    > 1TB/day
    App App App
    App App App
    

    View Slide

42. ͋ΒΏΔσʔλͷετϨʔδ: όοΫΞοϓ
    MySQL
    Master
    MySQL
    BackupSlave
    MySQL͸ຖ೔xtrabackup(ि1Ͱmysqldump)
    backup༻slave͔ΒbackupΛऔಘɻaws-cliͰసૹ
    > 1.2TB(ѹॖࡁ)/day
    xtrabackup + aws-cli
    MySQL
    Master
    MySQL
    BackupSlave
    

    View Slide

43. Amazon S3 as a Hub
    MySQL
    SaaS / ࣾ಺
    Microservices
    + IAM
    ػցֶशʗෆਖ਼ݕ஌
    ෺ྲྀɾܾࡁ
    ৴པੑͷߴ͍S3ΛHubͱͯ͠ɺૄ݁߹Λ࣮ݱ
    SFTP Partner
    goofys
    App App App
    App App App
    nginx nginx nginx
    consulͰઃఆΛ഑෍
    σʔλͷimport/export
    ACL
    ML API ML API
    ֶशσʔλͷimport
    

    View Slide

44. ػցֶश΁ͷऔΓ૊Έ
    • αʔϏεͰར༻தɾݕূத
    • ݕࡧ݁ՌͷվળɻߦಈղੳʹΑΓɺ঎඼ͷݕࡧΠϯσοΫεʹΩʔ
    ϫʔυΛ௥Ճ͠ɺΑΓݟ͚ͭ΍͘͢
    • ग़඼࣌ͷՁ֨αδΣετ
    • ػցֶशΛͩΕͰ΋ࢼ͢͜ͱ͕Ͱ͖Δ؀ڥΛ
    • Amazon ML΋ݕ౼
    

    View Slide

45. ڑ཭Λ௒͑ͯੈքΛܨ͙
    

    View Slide

46. ڑ཭ͱϨΠςϯγ
    • ޫ͸50msecʹ஍ٿ൒प΋Ͱ͖ͳ͍ɻԕڑ཭ͱͷ௨৴͸ίετ͕ߴ͍
    • σʔληϯλʔؒɺΫϥ΢υؒͷڑ཭͕͋Δ৔߹ʹ͸ɺͦΕΛࠀ෰
    ͠ɺޮ཰ͷྑ͍௨৴Λߦ͏ඞཁ
    • (ੴङԕ͍໰୊)
    

    View Slide

47. ࠃ಺ͱࠃ֎ͷϨΠςϯγ
    ੴ AWS
    ౦
    GCP
    18ms
    70-100ms
    140ms
    GCP
    6ms
    ͍͍ͩͨͷ஋
    
    ଠฏ༸/๺ถେ཮/େ੢༸͸΋ͱΑΓɺੴङ΋ԕ͍
    

    View Slide

48. ߴϨΠςϯγ؀ڥͰͷHTTPS௨৴
    • ௨ৗͷTCP HandshakingʹՃ͑਺ճͷ΍ΓͱΓ͕ඞཁ
    • RTT 26msecͰHTTPSͷ௨৴Λߦͳͬͨ৔߹ɺ200msecҎ্͔͔Δ
    • RTT 100msec௒͑Δͱɺ600msecҎ্
    • ࢀߟ) mercari APIͷϨεϙϯελΠϜ(90percentile)͸ 100msec
    

    View Slide

49. ԕڑ཭઀ଓ͢ΔϢʔεέʔε
    • ΫϥΠΞϯτ͕ԕ͍ͱ͜Ζ͔ΒαʔϏεʹΞΫηε͢Δ
    • ւ֎, US౦ւ؛/੢ւ؛
    • ΞϓϦέʔγϣϯͷίʔυ͔ΒଞͷΫϥ΢υ(σʔληϯλʔ)ʹ

    ΞΫηε͢Δ
    • SaaSɺϚΠΫϩαʔϏε
    

    View Slide

50. ΫϥΠΞϯτ͔Βͷ઀ଓվળ
    • CDNΛར༻͢Δ
    • Cloudfront, Akamai, Fastly
    • ΫϥΠΞϯτ͸ۙ͘ʹ͋ΔCDNͷΤοδαʔόͱTLS Handshaking
    • CDN ͱ Origin ؒ͸ίωΫγϣϯू໿΍ઐ༻ωοτϫʔΫΛར༻͢
    Δ͜ͱͰߴ଎Խ
    • www.mercari.com ͸CDNΛར༻
    

    View Slide

51. CDNͷར༻: mercari Web
    JP US UK
    mercari.com/
    mercari.com/jp/ mercari.com/uk/
    CDN
    azon Web Services LLC or its affiliates. All rights reserved.
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    AM Add-on Example:
    IAM Add-on
    Assignment/
    Task
    Requester
    Workers
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    anagement
    nsole
    IAM Add-on Example:
    IAM Add-on
    man Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    et AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    on
    l Turk
    vice Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Reques
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights re
    User Users Client Mul
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ੴङDC
    

    View Slide

52. ΞϓϦέʔγϣϯ͔ΒΫϥ΢υ΁ΞΫηε
    • ΞϓϦέʔγϣϯͰHTTPS௨৴ͷKeepAliveΛߦ͏
    • PHP ApplicationͰͷKeepAlive͸೉͍͠
    • ϦΫΤετॲཧޙʹϝϞϦ͕ΫϦΞ͞Εɺ TCP઀ଓ΋੾ΕΔ
    • ϚϧνϓϩηεͰ͋ΓɺKeepAliveͯ͠΋ޮ཰͕ѱ͍
    • => ͦ͜Ͱ Connection PoolingΛ໨తͱͨ͠ Proxy ServerΛ։ൃ
    

    View Slide

53. chocon
    • GoͰ࣮૷ͨ͠γϯϓϧͳ

    Proxy Server
    • OSSͱͯ͠ެ։
    • github.com/kazeburo/chocon
    • ൒೥Ҏ্ͷՔಇ࣮੷
    

    View Slide

54. chocon
    % curl -H ‘Host: example.com.ccnproxy-https’
    http://10.0.0.1/v1/foo
    *.ccnproxy-https IN CNAME chocon.local.
    ಺෦DNSΛ׆༻͢ΔͱURLͷϗετ໊Λมߋ͢Δ͚ͩ
    chocon Web
    Client
    https://example.com/
    ʹproxy
    http
    http or https
    keepAlive
    Private Network
    % curl http://example.com.ccnproxy-https/v1/foo
    

    View Slide

55. Before chocon
    $ ./httpstat.sh /dev/null https://microservice.example.com/hc
    HTTP/1.1 200 OK
    Server: nginx/1.11.5
    Date: Thu, 01 Jun 2017 00:43:49 GMT
    Content-Type: application/json; charset=utf-8
    Content-Length: 22
    Expires: Thu, 01 Jun 2017 01:43:49 GMT
    Cache-Control: max-age=3600,public
    Body stored in: /tmp/httpstat-body.263264511496278239
    DNS Lookup TCP Connection SSL Handshake Server Processing Content Transfer
    [ 2ms | 24ms | 197ms | 25ms | 0ms ]
    | | | | |
    namelookup:2ms | | | |
    connect:26ms | | |
    pretransfer:223ms | |
    starttransfer:248ms |
    total:248ms
    

    View Slide

56. After chocon
    $ ./httpstat.sh /dev/null https://microservice.example.com.ccnproxy-https/hc
    HTTP/1.1 200 OK
    Cache-Control: max-age=3600,public
    Content-Length: 22
    Content-Type: application/json; charset=utf-8
    Date: Thu, 01 Jun 2017 00:43:49 GMT
    Expires: Thu, 01 Jun 2017 01:43:49 GMT
    Server: nginx/1.11.5
    X-Chocon-Req: bSCzJrCMZ9wbRN8TYhZ3wV
    Body stored in: /tmp/httpstat-body.390174181496278775
    DNS Lookup TCP Connection Server Processing Content Transfer
    [ 1ms | 1ms | 19ms | 0ms ]
    | | | |
    namelookup:1ms | | |
    connect:2ms | |
    starttransfer:21ms |
    total:21ms
    

    View Slide

57. Why chocon?
    • ࣅͨmiddleware͸ݟ͔ͭΒͳ͍
    • ୯७ͳforward proxyͰ͸HTTPS௨৴ͷू໿͸Ͱ͖ͳ͍
    • HTTPS͸end to endͰ҉߸ԽɻMITM Proxy͕ඞཁʹͳΔ
    • Goݴޠඪ४ͷHTTP/2ʹΑΓޮ཰ͷྑ͍ू໿ɺߴ଎ͳΞΫηε͕ظ଴
    

    View Slide

58. chocon in JP
    App App
    App App
    App App
    App App
    chocon
    DC(Cloud) Cloud(DC)
    Microservices
    SaaS
    Cloud API endpoint
    90msec ͕19msec ͱੴङ౦ژؒͷRTT࣮ଌ஋·Ͱվળ
    AWS SDK΋endpointΛ੾Γସ͑Δ͜ͱͰར༻Մೳ
    http or https
    keepAlive
    

    View Slide

59. chocon & Pacific Ocean
    App App
    App App
    chocon
    US Cloud
    HTTPS, HTTP/2 Keepalive
    100msecఔ౓·Ͱ஗Ԇ͕཈͑ΒΕɺଞRegionͱͷ࿈ܞͷ࣮ݱɻ
    USͷઌਐతͳΫϥ΢υαʔϏεʹΞΫηε͠΍͘͢ͳΔ
    Cloudfront/CDN ւఈέʔϒϧ
    

    View Slide

60. ·ͱΊ
    

    View Slide

61. ·ͱΊ
    • ϝϧΧϦ͸ JP/US/UK ͷ3ڌ఺ͰαʔϏεల։ɺ։ൃ΋ߦ͏
    • ֤Region͸αʔόΛத৺ͱͨ͠ڞ௨ͨ͠ΞʔΩςΫνϟ
    • άϩʔόϧͰ͸Amazon Route53, Amazon S3ͷߴ͍৴པੑʹࢧ͑ΒΕ͍ͯΔ
    • ੈքΛ݁ͿͨΊʹΫϥ΢υαʔϏε΍ಠࣗ։ൃͷιϑτ΢ΣΞΛར༻
    

    View Slide

62. We’re Hiring!
    ੈքʹ௅ΉɺϝϧΧϦ
    ݴ͍༁φγͷύϑΥʔϚϯεͱ৴པੑͰࢧ͑ΔSRE
    www.mercari.com/jp/jobs/
    

    View Slide