Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cloud connect the world as a Glue - AWS Dev Day 2017

Cloud connect the world as a Glue - AWS Dev Day 2017

kazeburo

June 02, 2017
Tweet

More Decks by kazeburo

Other Decks in Technology

Transcript

  1. Cloud connect the world as a Glue
    AWS Dev Day 2017 Track 2
    Masahiro Nagano @kazeburo

    View Slide

  2. Me
    • Masahiro Nagano / ௕໺խ޿
    • @kazeburo
    • Mercari, Inc

    Principal Engineer

    Site Reliability Engineering (SRE) Team
    • BASE, Inc Technical Advisor

    View Slide

  3. SRE Team ͷ঺հ

    View Slide

  4. SRE
    • Site Reliability Engineering ͷུ
    • Google ͷӡ༻νʔϜΛ཰͍Δ Ben Treynor ͕ఏএ
    • Google ͷ༷ʑͳϓϩμΫτɾαʔϏεΛԣஅͯ͠ɺιϑτ΢ΣΞΤ
    ϯδχΞϦϯάΑΓαΠτ/αʔϏεͷ৴པੑΛ޲্ͤ͞Δ Software
    Engineering/Teamͱͦͷ࣮ફ = Google SRE

    View Slide

  5. Google SRE
    • ιϑτ΢ΣΞΤϯδχΞ(SWE)ͱͯ͠࠾༻
    • ӡ༻ͷۀ຿Λ50%ҎԼʹ཈͑Δ
    • 50%͸ΦϖϨʔγϣϯͷࣗಈԽɺιϑτ΢ΣΞͷ৴པੑ޲্ʹ͋ͯΔ
    • ΤϥʔόδΣοτͱ͍͏ߟ͑ํ
    • SREͱSWEͷSLAΛऔΓܾΊɺར֐ΛҰகͤ͞Δ
    http://landing.google.com/sre/book.html

    View Slide

  6. View Slide

  7. Mercari SRE
    • ͍ͭͰ΋շద͔ͭ҆શʹར༻Ͱ͖Δʮ৴པੑͷߴ͍ʯαʔϏεͷ࣮ݱ
    • ʮ৽نαʔϏεͷ։ൃҎ֎ͷΤϯδχΞϦϯά͸શ෦΍Δʯ
    • 2015/11 ʮΠϯϑϥνʔϜʯ͔ΒSRE΁
    • ʮΠϯϑϥʯΑΓ΋αʔϏεࢦ޲
    • ݱࡏϝϯόʔ͸ʮ6ਓʯઈࢍืूத

    View Slide

  8. Mercari SRE ͷۀ຿ൣғ
    Operations Software Eng.
    ج൫ߏங
    OnCall (ো֐ରԠ)
    Automation
    εέʔϥϏϦςΟɾՄ༻ੑվળ
    ϛυϧ΢ΣΞߏங
    ΞϓϦέʔγϣϯͷઃܭϨϏϡʔ
    ϩάऩूɾ෼ੳج൫ͷߏஙɺӡ༻
    αʔόϓϩϏδϣχϯάɾσϓϩΠͷ੔උ
    ηΩϡϦςΟʗෆਖ਼ར༻ݕग़

    View Slide

  9. Agenda
    • ϝϧΧϦͱ͸ / ੈք3ڌ఺Ͱͷ։ൃӡ༻ମ੍ʹ͍ͭͯ
    • ϝϧΧϦͷΞʔΩςΫνϟ / Ϋϥ΢υͷར༻
    • ڑ཭Λ௒͑ΔɺੈքΛܨ͙γεςϜ

    View Slide

  10. Mercari
    • ࠃ಺࠷େڃͷϑϦϚΞϓϦ
    • 3෼Ͱ؆୯ʹग़඼
    • ҆৺҆શͳܾࡁ

    View Slide

  11. Mercari KPI
    μ΢ϯϩʔυ਺
    GMV(૯औҾֹ)
    6500ສDL(JP+US)
    ݄ؒ100ԯԁҎ্
    ग़඼਺ 1೔100ສ඼Ҏ্

    View Slide

  12. μ΢ϯϩʔυ਺ਪҠ(JP)
    JP μ΢ϯϩʔυ਺ 4000ສ (2016/11)

    View Slide

  13. ೔ຊ࠷େͷϑϦϚΞϓϦ
    1,200
    1෼ؒͷग़඼਺(peek࣌ؒଳ)
    ඼Ҏ্

    View Slide

  14. ग़඼͔Β͙͢ʹചΕΔ
    24
    ࣌ؒҎ಺
    ചΕͨ঎඼ͷ໿50%͕
    24࣌ؒҎ಺ʹऔҾ੒ཱ

    View Slide

  15. Global Service
    JP
    2016/08
    US AppStore

    US UK
    2017/03/15
    ϦϦʔε

    View Slide

  16. Global Development Team
    Tokyo
    San Francisco
    London
    San Francisco/London ʹΦϑΟε
    ݱ஍࠾༻ɺग़޲ऀɺ௕ظग़ு߹Θͤͯ
    ΤϯδχΞ͕਺໊͔Β਺े໊

    View Slide

  17. Global Development Team
    • Tokyo
    • ։ൃͷத৺ɻJPʹՃ͑ͯશͯͷregionͷ։ൃ
    • San Fransisco
    • αʔϏεͷϩʔΧϥΠζ
    • ݴޠ͚ͩͰ͸ͳ͘ɺจԽ΍श׳ͷϩʔΧϥΠζ
    • London
    • αʔϏε্ཱͪ͛ϑΣʔζ / ݱ஍ͷ๏ྩͳͲʹ߹ΘͤͨϩʔΧϥΠζ

    View Slide

  18. Global Development ͷ೉͠͞
    10:00
    1:00
    18:00
    9࣌ؒ
    7࣌ؒ
    લ೔
    Tokyo
    San Francisco
    London
    3ڌ఺ἧͬͯإΛ߹ΘͤΔ͜ͱ͕͔ͳΓ೉͍͠

    View Slide

  19. Global Development ͷਐΊํ(1)
    • Ϋϥ΢υΛ׆༻ͯ͠ίϛϡχέʔγϣϯΛਤΔ
    • ଠฏ༸ɾେ੢༸Λ·͍ͨͩPull ReqeuestϨϏϡʔ
    • Slack
    • Video Conference
    • ϦϞʔτϖΞϓϩ(εΫϦʔϯڞ༗)

    View Slide

  20. Global Development ͷਐΊํ(2)
    • ཱࣗͨ͠νʔϜͱͯ͠՝୊ղܾ͢Δ
    • ϓϩδΣΫτϚωʔδϟͱɺΫϥΠΞϯτ͔ΒαʔόαΠυΤϯ
    δχΞ·ͰϑϧελοΫͷϓϩμΫτνʔϜΛݱ஍Ͱ݁੒
    • νʔϜؙ͝ͱग़ு
    • iOS/Android͸RegionʹΑͬͯfork͋Δ͍͸ɺbranchΛ෼͚ɺޓ͍
    ͷӨڹΛݮΒ͢

    View Slide

  21. SREͷέʔε
    • 6ਓͷ͏ͪɺ1ਓ͕௕ظUSग़ுத
    • ݱ஍։ൃͷαʔϏεͷΦϖϨʔγϣϯͷ೺Ѳ
    • ि1ͰUSͱͷSync MTG
    • ே9࣌(PDT 17:00) ʹࣗ୐ʹͯ Video Conference
    • UKͱ͸Ҋ݅ϕʔεͰ༦ํʹMTG
    • OnCall ౰൪͸ே9͔࣌Βࣗ୐଴ػɻUS͔Βͷ࡞ۀґཔʹ͋ͨΔ

    View Slide

  22. Mercari Architecture

    View Slide

  23. Infrastructure
    ੴङDC
    ઐ༻αʔό
    JP
    Cloud
    US
    Cloud
    UK
    Hybrid & Multi Cloud

    View Slide

  24. Infrastructure history (1)
    • 2013/07 JP ϦϦʔε
    • ͘͞ΒΠϯλʔωοτͷVPS 1୆ʹWeb΋DB΋͢΂ͯࡌͤͨ
    • ΠϯϑϥετϥΫνϟઐ೚ऀ͍ͳ͍தͰɺ਎ۙͳٕज़Λબ୒
    • ϦϦʔεޙ2ϲ݄Ͱ͘͞ΒΫϥ΢υɺઐ༻αʔόʹҠߦ͖ͯͨ͠

    View Slide

  25. Infrastructure history (2)
    • 2014/09 US ϦϦʔε
    • AWS (Oregon) ʹͯαʔϏεߏங
    • JPϦϦʔε౰ॳʹൺ΂ͯΤϯδχΞ͕૿͑ɺAWSܦݧऀ΋ଟ͘ͳͬͨ
    • ͦΕͰ΋·ͩΠϯϑϥετϥΫνϟઐ೚ऀ͸গͳ͘ɺAWSͷϚωʔδυ
    αʔϏεΛଟ͘ར༻ͯ͠αʔϏεΛߏங
    • USࠃ಺ͷઐ༻αʔόར༻΋ݕ౼͕ͨ͠ɺUSͷεέʔϧ͸༧૝ͮ͠Β͘ɺ
    Ϋϥ΢υͷॊೈ͞Λ೔ຊΑΓ΋ॏཁࢹͨ͠

    View Slide

  26. Infrastructure history (3)
    • (2015/02 kazeburo ೖࣾ)
    • 2015/11 SREνʔϜൃ଍
    • ͘͞ΒΠϯλʔωοτͱAWSͷϋΠϒϦουͳΠϯϑϥετϥΫνϟ
    ͷ্ͷΞʔΩςΫνϟΛਐԽͤ͞ɺ৴པੑͱεέʔϥϏϦςΟͷ޲্
    • 2017/03 UK ϦϦʔε
    • ৽͍ٕ͠ज़తνϟϨϯδͱͯ͠GCPΛબ୒

    View Slide

  27. Architecture
    • ࡾ૚+αͳγϯϓϧͳΞʔΩςΫνϟ
    • Reverse Proxy = nginx

    Application = Apache+mod_php

    Database = MySQL

    Cache = memcached

    Search = Solr
    • ଟ͘Λ෺ཧαʔόʹͯߏ੒
    • εέʔϧΞοϓ΋εέʔϧΞ΢τ΋ߦ͏Diagonal Scaleࢦ޲
    • Databaseʹ͸ ioMemory ΍ NVMe Λ౥ࡌͨ͠αʔόΛ࠾༻
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    WS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    fic
    DNS-RR
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    cloud cloud
    JP

    View Slide

  28. Architecture
    JP ͷΞʔΩςΫνϟΛجຊ౿ऻ
    EC2/GCE (αʔό) Λத৺ͨ͠ߏ੒
    ɾ
    USಠࣗͷαʔϏε΍

    খن໛ʙதن໛DBʹ͸ RDS
    UKͰ͸Cloud Load BalancerΛར༻
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    anagement
    onsole
    IAM Add-on Example:
    IAM Add-on
    man Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    DNS-RR
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    US
    RDS
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2
    EC2 EC2
    EC2
    EC2
    EC2 EC2
    nginx nginx nginx
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corpora
    data cen
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    App App App
    App App App
    MySQL MySQL
    memcached
    memcached
    util util
    GCE
    cloud load balancer
    GCE GCE
    GCE GCE GCE
    GCE GCE GCE
    GCE GCE
    GCE
    GCE
    GCE GCE
    UK

    View Slide

  29. αʔόத৺ͷ Architecture
    • ϝϯςϯεϏϦςΟɾεέʔϥϏϦςΟઓུͷڞ௨Խ
    • গਓ਺Ͱͷӡ༻
    • Ansible Playbook ࠶ར༻
    • εέʔϧ͕ઌߦ͍ͯ͠ΔJPͰ࣮੷͋Δߏ੒
    • US Ͱͷ App Store ϥϯΩϯά3ҐͷτϥϑΟοΫ΋໰୊ͳ͘ӡ༻
    • EC2ͷIaaSͱͯ͠ͷύϑΥʔϚϯεɺ৴པੑ͸͔ͳΓ޲্͍ͯ͠Δ

    View Slide

  30. Mercari Architecture ·ͱΊ
    • 3ͭͷRegionͰ࠾༻͢ΔΠϯϑϥετϥΫνϟ͕ҟͳΔ
    • JP/US/UK ͸αʔόΛத৺ͱͨ͠ArchitectureΛ࠾༻
    • AWSͰ΋Ϋϥ΢υΒ͍͠ઃܭ͸ͤͣɺن໛Ͱઌߦ͢ΔJPʹ߹Θͤ
    Δ͜ͱͰɺӡ༻ͷڞ௨ԽͱলྗԽ
    • ϝϧΧϦͰ͸Ϋϥ΢υΛੵۃతʹ࢖͍ͬͯͳ͍ʁ
    • JP/US/UK ڞ௨ͷΠϯϑϥετϥΫνϟͰར༻͍ͯ͠·͢

    View Slide

  31. Mercari Global Infrastructure

    View Slide

  32. Global Infrastructure
    • Mercari JP/US/UK ͷΠϯϑϥετϥΫνϟ͸ಠཱ͍ͯ͠Δ
    • σʔλΛαʔϏεΛߦ͏Ҭ಺ʹཹΊΔඞཁੑ
    • ͍͔ͭ͘ͷΫϥ΢υαʔϏεΛڞ௨ͯ͠ར༻
    • ւ֎ͰͷΞΫηεվળ
    • Ϋϥ΢υͷߴ͍εέʔϥϏϦςΟɾ৴པੑʹΑΓαʔϏεͷՄ༻
    ੑΛอͭ

    View Slide

  33. Global Infrastructure
    JP US UK
    DNS: Amazon Route53
    CDN: Akamai, CloudFront
    Storage: Amazon S3
    Analysis: Google BigQuery
    Common Micro Services
    ܾࡁ/෺ྲྀ/Domestic Service ܾࡁ/෺ྲྀ/Domestic Service ܾࡁ/෺ྲྀ/Domestic Service
    ڞ௨ΞʔΩςΫνϟ
    αʔό͕த৺
    ֤Region
    Ϋϥ΢υ͕த৺
    ৴པੑͷߴ͍AWSͷ
    αʔϏε͕ڬΈࠐΉ

    View Slide

  34. Amazon Route53
    • ߴ͍Մ༻ੑͱ৴པੑͷDNS
    • Roadworker Λར༻
    • github.com/codenize-tools/roadworker
    • Routefile ΛGithubͰ؅ཧ
    • Pull RequestͷmergeޙɺCIΛܦ༝ͯ͠
    ࣗಈ൓ө
    hosted_zone "mercari.jp." do
    rrset "api.mercari.jp.", "CNAME" do
    ttl 30
    resource_records(
    "endpoint-api.mercari.jp"
    )
    end
    end
    #Routefile
    PR
    Github Travis-CI Route53

    View Slide

  35. Amazon Route53 + HealthCheck
    • DNS-RR ӡ༻࣌ͷ໰୊఺
    • αʔόো֐࣌ʹDNSͷॻ͖׵͑ʹ͕͔͔࣌ؒΔ
    • ϒϥ΢βͳͲͷҰ෦ΫϥΠΞϯτ͸DNS-RRͷ৔߹ɺҰ෦ͷαʔόʹ઀ଓ͕Ͱ͖
    ͳ͍৔߹ɺଞͷαʔό΁઀ଓ͠௚͢ͷͰো֐ʹΑΔӨڹ͸େ͖͘ͳΓʹ͍͘ɻ
    • ϚΠΫϩαʔϏεԽ͕ਐΉͱ༷ʑͳϒϥ΢βҎ֎ͷΫϥΠΞϯτ͕઀ଓ͢Δɻ
    ଟ͘͸DNS-RRͷো֐࣌ͷ࠶઀ଓ͸࣮૷͞Εͯͳ͍
    • Route53 ͷ Health CheckΛ࢖͍ղܾ(Λݕূத)

    View Slide

  36. Route53 + Health Check with Roadworker
    [“153.x.y.150”, "153.x.y.151"].each do |ip|
    rrset "endpoint-ha.mercari.jp.", "A" do
    ttl 30
    weight 1
    set_identifier “endpoint-ha-“ + ip.gsub(/\./,'-')
    health_check "http://#{ip}/hc", :request_interval => 30, :failure_threshold => 3
    resource_records(
    "#{ip}"
    )
    end
    end
    Health CheckʹΑΓ DNS-RR Ͱ΋Մ༻ੑΛߴΊΒΕΔ
    #Routefile

    View Slide

  37. (࿩͸ͦΕ·͕͢)
    ಺෦ DNS
    • શͯͷαʔόʹunboundΛಋೖ
    • ϩʔΧϧΩϟογϡʹΑΔύϑΥʔϚϯε޲্
    • resolv.confΑΓՄ༻ੑ্͕͕Δ
    • DNSαʔόͷunboundͰϦΫΤετΛৼΓ෼͚
    • *.local ͸BIND͕ݖҖαʔό
    • *.consul ͸consul DNS interface
    App App App
    App App App
    DNS DNS
    unbound unbound unbound
    unbound unbound unbound
    DNS
    unbound
    Consul/DNS
    BIND
    *.consul
    *.local

    View Slide

  38. (࿩͸ͦΕ·͕͢)
    ಺෦DNSͰCNAME
    • ಺෦DNSͰϚωʔδυαʔϏεͷΤϯυϙΠϯτͷCNAMEΛઃఆ
    • ΞϓϦέʔγϣϯ͔Β઀ଓ͸CNAMEܦ༝
    • ϚωʔδυαʔϏε͔ΒϚωʔδυαʔϏε΁ͷҠߦɺϚωʔδυ
    αʔϏε͔ΒEC2΁ͷҠߦɺ·ͨͦͷٯͷҠߦ͕΍Γ΍͍͢
    db-cstool-master IN CNAME cstool-db.XXXXX.us-west-2.rds.amazonaws.com.

    View Slide

  39. Amazon S3
    • ߴ͍Մ༻ੑͱ৴པੑͷετϨʔδ
    • ঎඼ը૾ɺϩάɺσʔλϕʔεͷόοΫΞοϓͳͲ͋ΒΏΔσʔλ
    Λ֨ೲ
    • IAMΛར༻ͨ͠ߴ౓ͳΞΫηε؅ཧͱૄ݁߹ͷ࣮ݱ
    • αϒγεςϜ͔ΒͷσʔλΠϯϙʔτɾΤΫεϙʔτ
    • ֎෦αʔϏεɾύʔτφʔͱͷσʔλड͚౉͠खஈ

    View Slide

  40. ͋ΒΏΔσʔλͷετϨʔδ: ঎඼ը૾
    App App App
    App App App
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    ent IAM Add-on Example:
    IAM Add-on
    ligence
    HIT)
    Assignment/
    Task
    Requester
    Workers
    ঎඼ը૾σʔλ͸ಉظతʹॖখ/Ξοϓϩʔυ
    AWS SDK for PHPΛར༻ɻෳ਺ͷը૾Λฒߦͯ͠PUTͯ͠଎౓޲্
    ग़඼!
    ਺ඦສຕ/day

    View Slide

  41. ͋ΒΏΔσʔλͷετϨʔδ: ϩά
    Log
    Log
    ΞΫηεϩάʗΤϥʔϩάͳͲ֤छϩά͸fluentܦ༝Ͱू໿ͯ͠S3ʹ֨ೲ
    aws-cli ·ͨ͸ fluent-plugin-s3
    batch + aws-cli
    > 1TB/day
    App App App
    App App App

    View Slide

  42. ͋ΒΏΔσʔλͷετϨʔδ: όοΫΞοϓ
    MySQL
    Master
    MySQL
    BackupSlave
    MySQL͸ຖ೔xtrabackup(ि1Ͱmysqldump)
    backup༻slave͔ΒbackupΛऔಘɻaws-cliͰసૹ
    > 1.2TB(ѹॖࡁ)/day
    xtrabackup + aws-cli
    MySQL
    Master
    MySQL
    BackupSlave

    View Slide

  43. Amazon S3 as a Hub
    MySQL
    SaaS / ࣾ಺
    Microservices
    + IAM
    ػցֶशʗෆਖ਼ݕ஌
    ෺ྲྀɾܾࡁ
    ৴པੑͷߴ͍S3ΛHubͱͯ͠ɺૄ݁߹Λ࣮ݱ
    SFTP Partner
    goofys
    App App App
    App App App
    nginx nginx nginx
    consulͰઃఆΛ഑෍
    σʔλͷimport/export
    ACL
    ML API ML API
    ֶशσʔλͷimport

    View Slide

  44. ػցֶश΁ͷऔΓ૊Έ
    • αʔϏεͰར༻தɾݕূத
    • ݕࡧ݁ՌͷվળɻߦಈղੳʹΑΓɺ঎඼ͷݕࡧΠϯσοΫεʹΩʔ
    ϫʔυΛ௥Ճ͠ɺΑΓݟ͚ͭ΍͘͢
    • ग़඼࣌ͷՁ֨αδΣετ
    • ػցֶशΛͩΕͰ΋ࢼ͢͜ͱ͕Ͱ͖Δ؀ڥΛ
    • Amazon ML΋ݕ౼

    View Slide

  45. ڑ཭Λ௒͑ͯੈքΛܨ͙

    View Slide

  46. ڑ཭ͱϨΠςϯγ
    • ޫ͸50msecʹ஍ٿ൒प΋Ͱ͖ͳ͍ɻԕڑ཭ͱͷ௨৴͸ίετ͕ߴ͍
    • σʔληϯλʔؒɺΫϥ΢υؒͷڑ཭͕͋Δ৔߹ʹ͸ɺͦΕΛࠀ෰
    ͠ɺޮ཰ͷྑ͍௨৴Λߦ͏ඞཁ
    • (ੴङԕ͍໰୊)

    View Slide

  47. ࠃ಺ͱࠃ֎ͷϨΠςϯγ
    ੴ AWS

    GCP
    18ms
    70-100ms
    140ms
    GCP
    6ms
    ͍͍ͩͨͷ஋

    ଠฏ༸/๺ถେ཮/େ੢༸͸΋ͱΑΓɺੴङ΋ԕ͍

    View Slide

  48. ߴϨΠςϯγ؀ڥͰͷHTTPS௨৴
    • ௨ৗͷTCP HandshakingʹՃ͑਺ճͷ΍ΓͱΓ͕ඞཁ
    • RTT 26msecͰHTTPSͷ௨৴Λߦͳͬͨ৔߹ɺ200msecҎ্͔͔Δ
    • RTT 100msec௒͑Δͱɺ600msecҎ্
    • ࢀߟ) mercari APIͷϨεϙϯελΠϜ(90percentile)͸ 100msec

    View Slide

  49. ԕڑ཭઀ଓ͢ΔϢʔεέʔε
    • ΫϥΠΞϯτ͕ԕ͍ͱ͜Ζ͔ΒαʔϏεʹΞΫηε͢Δ
    • ւ֎, US౦ւ؛/੢ւ؛
    • ΞϓϦέʔγϣϯͷίʔυ͔ΒଞͷΫϥ΢υ(σʔληϯλʔ)ʹ

    ΞΫηε͢Δ
    • SaaSɺϚΠΫϩαʔϏε

    View Slide

  50. ΫϥΠΞϯτ͔Βͷ઀ଓվળ
    • CDNΛར༻͢Δ
    • Cloudfront, Akamai, Fastly
    • ΫϥΠΞϯτ͸ۙ͘ʹ͋ΔCDNͷΤοδαʔόͱTLS Handshaking
    • CDN ͱ Origin ؒ͸ίωΫγϣϯू໿΍ઐ༻ωοτϫʔΫΛར༻͢
    Δ͜ͱͰߴ଎Խ
    • www.mercari.com ͸CDNΛར༻

    View Slide

  51. CDNͷར༻: mercari Web
    JP US UK
    mercari.com/
    mercari.com/jp/ mercari.com/uk/
    CDN
    azon Web Services LLC or its affiliates. All rights reserved.
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    AM Add-on Example:
    IAM Add-on
    Assignment/
    Task
    Requester
    Workers
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    anagement
    nsole
    IAM Add-on Example:
    IAM Add-on
    man Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    et AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    on
    l Turk
    vice Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Reques
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights re
    User Users Client Mul
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ੴङDC

    View Slide

  52. ΞϓϦέʔγϣϯ͔ΒΫϥ΢υ΁ΞΫηε
    • ΞϓϦέʔγϣϯͰHTTPS௨৴ͷKeepAliveΛߦ͏
    • PHP ApplicationͰͷKeepAlive͸೉͍͠
    • ϦΫΤετॲཧޙʹϝϞϦ͕ΫϦΞ͞Εɺ TCP઀ଓ΋੾ΕΔ
    • ϚϧνϓϩηεͰ͋ΓɺKeepAliveͯ͠΋ޮ཰͕ѱ͍
    • => ͦ͜Ͱ Connection PoolingΛ໨తͱͨ͠ Proxy ServerΛ։ൃ

    View Slide

  53. chocon
    • GoͰ࣮૷ͨ͠γϯϓϧͳ

    Proxy Server
    • OSSͱͯ͠ެ։
    • github.com/kazeburo/chocon
    • ൒೥Ҏ্ͷՔಇ࣮੷

    View Slide

  54. chocon
    % curl -H ‘Host: example.com.ccnproxy-https’
    http://10.0.0.1/v1/foo
    *.ccnproxy-https IN CNAME chocon.local.
    ಺෦DNSΛ׆༻͢ΔͱURLͷϗετ໊Λมߋ͢Δ͚ͩ
    chocon Web
    Client
    https://example.com/
    ʹproxy
    http
    http or https
    keepAlive
    Private Network
    % curl http://example.com.ccnproxy-https/v1/foo

    View Slide

  55. Before chocon
    $ ./httpstat.sh /dev/null https://microservice.example.com/hc
    HTTP/1.1 200 OK
    Server: nginx/1.11.5
    Date: Thu, 01 Jun 2017 00:43:49 GMT
    Content-Type: application/json; charset=utf-8
    Content-Length: 22
    Expires: Thu, 01 Jun 2017 01:43:49 GMT
    Cache-Control: max-age=3600,public
    Body stored in: /tmp/httpstat-body.263264511496278239
    DNS Lookup TCP Connection SSL Handshake Server Processing Content Transfer
    [ 2ms | 24ms | 197ms | 25ms | 0ms ]
    | | | | |
    namelookup:2ms | | | |
    connect:26ms | | |
    pretransfer:223ms | |
    starttransfer:248ms |
    total:248ms

    View Slide

  56. After chocon
    $ ./httpstat.sh /dev/null https://microservice.example.com.ccnproxy-https/hc
    HTTP/1.1 200 OK
    Cache-Control: max-age=3600,public
    Content-Length: 22
    Content-Type: application/json; charset=utf-8
    Date: Thu, 01 Jun 2017 00:43:49 GMT
    Expires: Thu, 01 Jun 2017 01:43:49 GMT
    Server: nginx/1.11.5
    X-Chocon-Req: bSCzJrCMZ9wbRN8TYhZ3wV
    Body stored in: /tmp/httpstat-body.390174181496278775
    DNS Lookup TCP Connection Server Processing Content Transfer
    [ 1ms | 1ms | 19ms | 0ms ]
    | | | |
    namelookup:1ms | | |
    connect:2ms | |
    starttransfer:21ms |
    total:21ms

    View Slide

  57. Why chocon?
    • ࣅͨmiddleware͸ݟ͔ͭΒͳ͍
    • ୯७ͳforward proxyͰ͸HTTPS௨৴ͷू໿͸Ͱ͖ͳ͍
    • HTTPS͸end to endͰ҉߸ԽɻMITM Proxy͕ඞཁʹͳΔ
    • Goݴޠඪ४ͷHTTP/2ʹΑΓޮ཰ͷྑ͍ू໿ɺߴ଎ͳΞΫηε͕ظ଴

    View Slide

  58. chocon in JP
    App App
    App App
    App App
    App App
    chocon
    DC(Cloud) Cloud(DC)
    Microservices
    SaaS
    Cloud API endpoint
    90msec ͕19msec ͱੴङ౦ژؒͷRTT࣮ଌ஋·Ͱվળ
    AWS SDK΋endpointΛ੾Γସ͑Δ͜ͱͰར༻Մೳ
    http or https
    keepAlive

    View Slide

  59. chocon & Pacific Ocean
    App App
    App App
    chocon
    US Cloud
    HTTPS, HTTP/2 Keepalive
    100msecఔ౓·Ͱ஗Ԇ͕཈͑ΒΕɺଞRegionͱͷ࿈ܞͷ࣮ݱɻ
    USͷઌਐతͳΫϥ΢υαʔϏεʹΞΫηε͠΍͘͢ͳΔ
    Cloudfront/CDN ւఈέʔϒϧ

    View Slide

  60. ·ͱΊ

    View Slide

  61. ·ͱΊ
    • ϝϧΧϦ͸ JP/US/UK ͷ3ڌ఺ͰαʔϏεల։ɺ։ൃ΋ߦ͏
    • ֤Region͸αʔόΛத৺ͱͨ͠ڞ௨ͨ͠ΞʔΩςΫνϟ
    • άϩʔόϧͰ͸Amazon Route53, Amazon S3ͷߴ͍৴པੑʹࢧ͑ΒΕ͍ͯΔ
    • ੈքΛ݁ͿͨΊʹΫϥ΢υαʔϏε΍ಠࣗ։ൃͷιϑτ΢ΣΞΛར༻

    View Slide

  62. We’re Hiring!
    ੈքʹ௅ΉɺϝϧΧϦ
    ݴ͍༁φγͷύϑΥʔϚϯεͱ৴པੑͰࢧ͑ΔSRE
    www.mercari.com/jp/jobs/

    View Slide