$30 off During Our Annual Pro Sale. View Details »

CDNの使い方 in Mercari/CDN in Mercari

kazeburo
October 18, 2017

CDNの使い方 in Mercari/CDN in Mercari

CDN in Mercari/CDNの使い方 in Mercari
Fastly Yamagoya Meetup 2017

kazeburo

October 18, 2017
Tweet

More Decks by kazeburo

Other Decks in Technology

Transcript

  1. CDN in Mercari
    Fastly Yamagoya Meetup 2017
    Masahiro Nagano @kazeburo

    View Slide

  2. Me
    • Masahiro Nagano / ௕໺խ޿
    • @kazeburo
    • Mercari, Inc

    Principal Engineer

    Site Reliability Engineering (SRE) Team

    View Slide

  3. Agenda
    • ϝϧΧϦͷ঺հ
    • ઌ೔ͷݸਓ৘ใྲྀग़Πϯγσϯτʹ͍ͭͯ
    • ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN
    • ࠓޙͷ՝୊ɾऔΓ૊Έ

    View Slide

  4. Mercari
    • ϑϦϚΞϓϦ
    • εϚϗͰࣸਅΛͱͬͯ؆୯ʹग़඼
    • ҆৺ɾ҆શͳܾࡁ
    • ศརͳ഑ૹ

    View Slide

  5. Mercari
    μ΢ϯϩʔυ਺
    ྲྀ௨ֹ
    7500ສDL(JP+US)
    ݄ؒ100ԯԁҎ্
    ग़඼਺ 1೔100ສ඼Ҏ্

    View Slide

  6. γεςϜ͔ΒΈͨϝϧΧϦ
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    IAM Add-on Example:
    IAM Add-on
    ence
    )
    Assignment/
    Task
    Requester
    Workers
    ग़඼!
    DB Search
    5-දࣔ
    ݕࡧ൓ө
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    େྔͷϦΫΤετ
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corp
    data c
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ϦΫΤετԠ౴
    DB Search
    ߪೖ!
    ϛϦඵʙ30ඵ ਺ඵʙ
    ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏
    ը૾ ܾࡁ
    AI

    View Slide

  7. Mercari
    JP US UK
    JP/US/UKͰల։த

    View Slide

  8. CDN੾Γସ͑ʹ൐͏
    Web൛ϝϧΧϦʹ͓͚Δݸਓ৘ใྲྀग़

    Πϯγσϯτʹ͍ͭͯ

    View Slide

  9. ଟ͘ͷօ༷ʹ͝৺഑ɺ͝໎࿭Λֻ͓͚͍ͨ͠·ͨ͠
    ਂ͓͘࿳ͼਃ͋͛͠·͢
    ଟେͳΔڠྗΛFastly༷ʹ͍͖ͨͩ·ͨ͠ɻվΊ͓ͯྱਃ্͛͠·͢

    View Slide

  10. ൃੜͨ͠ࣄ৅
    • ϝϧΧϦWeb൛΁ͷΞΫηε଎౓޲্ͱɺηΩϡϦςΟ޲্ͷͨΊɺCDN
    ͷ੾Γସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹ੾Γସ͑ઌͷCDN(Fastly)ʹ͓͚Δ
    Ωϟογϡͷಈ࡞ʹ͍ͭͯͷ೺Ѳ͕ෆ଍͓ͯ͠Γɺ͓٬͞·΁ͷϨεϙϯ
    ε͕ผͷ͓٬͞·ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖Δ৘ใ
    ΛؚΉ಺༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠

    View Slide

  11. Timeline
    • 6/22
    • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ
    • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ
    • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢
    • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑
    • 15:38ɹCDNͷ഑৴ઃఆΛdeactivate͠ɺΞΫηεΛःஅ
    • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ
    • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ
    • 20:45 Tech blogʹͯৄࡉެ։

    View Slide

  12. Timeline
    • 6/28
    • ഑৴ઃఆΛ࠶Activate
    • Cacheͷ׬શແޮԽઃఆΛߦ͍ɺFastlyࣾͷΤϯδχΞʹϨϏϡʔΛґཔ
    • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ
    • ࣾ಺Ͱݕূ
    • 6/29
    • FastlyࣾΤϯδχΞ଴ػͷ΋ͱɺ੾Γସ͑Λ࣮ࢪ

    View Slide

  13. CacheແޮԽ
    • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie"
    ͕ඞཁ
    • ”Cache-Control: no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ
    • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹
    ͸ “0ඵ” ͱͯ͠ѻΘΕΔ
    • 0ඵ͸cacheແޮͰ͸ͳ͘ʮ0ඵͷcache͕ଘࡏ͢Δʯ

    View Slide

  14. 0ඵͷcache
    • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ͡
    URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷϨεϙ
    ϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ΋ಉ͡Ϩε
    ϙϯε͕ฦ͞ΕΔ
    • `Request collapsing`
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    ©2011 Amazon Web Services LLC or its affiliates. All r
    User Users Client Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    origin
    CDN
    (1)
    (2)
    (3)
    (4)
    (5) (5)

    View Slide

  15. ରࡦ
    • VCLͷઃఆมߋ
    • ෳ਺ͷbackendʹରԠ͢ΔͨΊɺPriorityΛ௿͘ઃఆ
    # ͜͜ʹbackend੾Γସ͑ͷvcl͕ඳ͍ͯ͋Δ
    # Request Condition: always Prio: 100
    if( req.url ) { ## ඞͣtrueʹͳΔ
    if (!req.http.Fastly-FF) {
    if (req.http.X-Forwarded-For) {
    set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For ", " client.ip;
    } else {
    set req.http.Fastly-Temp-XFF = client.ip
    }
    } else {
    set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For;
    }
    return(pass);
    }

    View Slide

  16. ରࡦ
    • ϩάΛS3ʹૹ৴͠ɺlambdaʹΑΓϦΞϧλΠϜʹղੳɺmackerelͰՄࢹԽɻ
    ؂ࢹ
    • nginx/ApacheʹͯheaderΛ௥Ճ
    • header΍VCLͷมߋͷ؂ࢹ
    more_clear_headers 'Expires';
    more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache";
    add_header Set-Cookie "merCtx=\"\"; HttpOnly" always;

    View Slide

  17. https://docs.fastly.com/ja/guides/tutorials/cache-control-tutorial

    View Slide

  18. ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN

    View Slide

  19. CDN in Mercari
    • ΑΓଟ͘ͷτϥϑΟοΫΛॲཧ͢ΔͨΊ
    • Πϯϑϥӡ༻ίετͷ࡟ݮͱͯ͠CDNΛར༻
    • ը૾഑৴/ಈը഑৴
    • UX/ηΩϡϦςΟͷվળͷͨΊ
    • Clientͱͷ௨৴؀ڥ࠷దԽ
    • αΠτ/APIΛؙ͝ͱCDNԽ

    View Slide

  20. Infrastructure
    US UK
    ͦΕͧΕͷҬ಺ͷDCΛར༻
    S3 S3 S3
    ঎඼ը૾ ঎඼ը૾ ঎඼ը૾
    Frontend for Backend

    microservices
    Backend API
    JP

    View Slide

  21. Frontend for Backend/microservices
    API Gateway
    ©2011 Amazon Web Services LLC or its affiliates. All rights reserved.
    User Users Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Internet AWS Management
    Console
    IAM Add-on Example:
    IAM Add-on
    Non-Service Specific
    search
    personalization
    offer
    gRPC
    JSON over HTTPs

    View Slide

  22. App Backend Infrastructure
    JP US UK
    API͸ͦΕͧΕผυϝΠϯ / CDN͸࢖༻ͯ͠ͳ͍(2017/10ݱࡏ)
    େن໛ͳτϥϑΟοΫͱͳΔ঎඼ը૾͸CDNΛར༻ͯ͠഑৴
    Client Multimedia Corporate
    data center
    Traditional
    server
    Mobile Client
    Assignment/
    Task
    Requester
    Workers
    User Users Client Multimedia Corporate
    data center
    Traditio
    serve
    Mobile Client
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requester
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    User Users Client Multimedia
    Mobile Client
    Amazon Mechanical Turk
    On-Demand Workforce
    Human Intelligence
    Tasks (HIT)
    Assignment/
    Task
    Requ
    Workers
    Amazon
    Mechanical Turk
    Non-Service Specific
    api.example.jp api.example.com api.example.uk
    S3 S3 S3
    KQ
    DEOJ[FOFU
    VTDEOJ[FOFU
    VLDEOJ[FOFU
    CDN
    CDN CDN
    *N
    BHF'MVY

    View Slide

  23. ImageFlux
    • ͘͞ΒΠϯλʔωοτͱpixivͷڠۀ
    • ը૾ม׵ + Cache Storage
    • WebP΁ͷରԠ
    • CDNͷCache Hit Rateվળ

    View Slide

  24. Web Backend Infrastructure
    JP US UK
    Web͸γϯάϧυϝΠϯɻURIͰࢀর͢ΔBackendΛ੾Γସ͑Δ
    ͓٬༷͔Β࠷΋͍ۙEdgeͰ൑அ͢Δ͜ͱͰϨεϙϯελΠϜ/UX޲্
    S3 S3 S3
    https://www.mercari.com/
    /jp/ / /uk/

    View Slide

  25. ࠓޙͷ՝୊ɾऔΓ૊Έ

    View Slide

  26. ࠓޙͷ՝୊ɾऔΓ૊Έ
    • Security΁ͷϑΥʔΧε
    • L3-L7·Ͱେن໛ͳDoS߈ܸ͔Βͷ๷ޚ
    • WAF ͷݕূ
    • Bot੍ޚ
    • ߴ౓Խ͢ΔEdge Computing
    • Microservices Λ࣮ݱ͢Δखஈͱͯ͠ͷCDN
    • CDNͷઃఆ/VCLͷContinuous Integration
    • ίετ࠷దԽ

    View Slide

  27. Ҏ্
    Fastlyͷօ༷ɺࠓޙͱ΋ΑΖ͓͘͠ئ͍͠·͢
    => www.mercari.com/jp/jobs/

    View Slide