Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CDNの使い方 in Mercari/CDN in Mercari

kazeburo
October 18, 2017

CDNの使い方 in Mercari/CDN in Mercari

CDN in Mercari/CDNの使い方 in Mercari
Fastly Yamagoya Meetup 2017

kazeburo

October 18, 2017
Tweet

More Decks by kazeburo

Other Decks in Technology

Transcript

  1. CDN in Mercari Fastly Yamagoya Meetup 2017 Masahiro Nagano @kazeburo

  2. Me • Masahiro Nagano / ௕໺խ޿ • @kazeburo • Mercari,

    Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team
  3. Agenda • ϝϧΧϦͷ঺հ • ઌ೔ͷݸਓ৘ใྲྀग़Πϯγσϯτʹ͍ͭͯ • ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN • ࠓޙͷ՝୊ɾऔΓ૊Έ

  4. Mercari • ϑϦϚΞϓϦ • εϚϗͰࣸਅΛͱͬͯ؆୯ʹग़඼ • ҆৺ɾ҆શͳܾࡁ • ศརͳ഑ૹ

  5. Mercari μ΢ϯϩʔυ਺ ྲྀ௨ֹ 7500ສDL(JP+US) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্

  6. γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All

    rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़඼! DB Search 5-දࣔ ݕࡧ൓ө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ౴ DB Search ߪೖ! ϛϦඵʙ30ඵ ਺ඵʙ ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI
  7. Mercari JP US UK JP/US/UKͰల։த

  8. CDN੾Γସ͑ʹ൐͏ Web൛ϝϧΧϦʹ͓͚Δݸਓ৘ใྲྀग़
 Πϯγσϯτʹ͍ͭͯ

  9. ଟ͘ͷօ༷ʹ͝৺഑ɺ͝໎࿭Λֻ͓͚͍ͨ͠·ͨ͠ ਂ͓͘࿳ͼਃ͋͛͠·͢ ଟେͳΔڠྗΛFastly༷ʹ͍͖ͨͩ·ͨ͠ɻվΊ͓ͯྱਃ্͛͠·͢

  10. ൃੜͨ͠ࣄ৅ • ϝϧΧϦWeb൛΁ͷΞΫηε଎౓޲্ͱɺηΩϡϦςΟ޲্ͷͨΊɺCDN ͷ੾Γସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹ੾Γସ͑ઌͷCDN(Fastly)ʹ͓͚Δ Ωϟογϡͷಈ࡞ʹ͍ͭͯͷ೺Ѳ͕ෆ଍͓ͯ͠Γɺ͓٬͞·΁ͷϨεϙϯ ε͕ผͷ͓٬͞·ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖Δ৘ใ ΛؚΉ಺༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠

  11. Timeline • 6/22 • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢

    • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑ • 15:38ɹCDNͷ഑৴ઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։
  12. Timeline • 6/28 • ഑৴ઃఆΛ࠶Activate • Cacheͷ׬શແޮԽઃఆΛߦ͍ɺFastlyࣾͷΤϯδχΞʹϨϏϡʔΛґཔ • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ •

    ࣾ಺Ͱݕূ • 6/29 • FastlyࣾΤϯδχΞ଴ػͷ΋ͱɺ੾Γସ͑Λ࣮ࢪ
  13. CacheແޮԽ • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie" ͕ඞཁ • ”Cache-Control:

    no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹ ͸ “0ඵ” ͱͯ͠ѻΘΕΔ • 0ඵ͸cacheແޮͰ͸ͳ͘ʮ0ඵͷcache͕ଘࡏ͢Δʯ
  14. 0ඵͷcache • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ͡ URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷϨεϙ ϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ΋ಉ͡Ϩε ϙϯε͕ฦ͞ΕΔ • `Request collapsing` ©2011

    Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)
  15. ରࡦ • VCLͷઃఆมߋ • ෳ਺ͷbackendʹରԠ͢ΔͨΊɺPriorityΛ௿͘ઃఆ # ͜͜ʹbackend੾Γସ͑ͷvcl͕ඳ͍ͯ͋Δ # Request Condition:

    always Prio: 100 if( req.url ) { ## ඞͣtrueʹͳΔ if (!req.http.Fastly-FF) { if (req.http.X-Forwarded-For) { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.Fastly-Temp-XFF = client.ip } } else { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For; } return(pass); }
  16. ରࡦ • ϩάΛS3ʹૹ৴͠ɺlambdaʹΑΓϦΞϧλΠϜʹղੳɺmackerelͰՄࢹԽɻ ؂ࢹ • nginx/ApacheʹͯheaderΛ௥Ճ • header΍VCLͷมߋͷ؂ࢹ more_clear_headers 'Expires';

    more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always;
  17. https://docs.fastly.com/ja/guides/tutorials/cache-control-tutorial

  18. ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN

  19. CDN in Mercari • ΑΓଟ͘ͷτϥϑΟοΫΛॲཧ͢ΔͨΊ • Πϯϑϥӡ༻ίετͷ࡟ݮͱͯ͠CDNΛར༻ • ը૾഑৴/ಈը഑৴ •

    UX/ηΩϡϦςΟͷվળͷͨΊ • Clientͱͷ௨৴؀ڥ࠷దԽ • αΠτ/APIΛؙ͝ͱCDNԽ
  20. Infrastructure US UK ͦΕͧΕͷҬ಺ͷDCΛར༻ S3 S3 S3 ঎඼ը૾ ঎඼ը૾ ঎඼ը૾

    Frontend for Backend
 microservices Backend API JP
  21. Frontend for Backend/microservices API Gateway ©2011 Amazon Web Services LLC

    or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Non-Service Specific search personalization offer gRPC JSON over HTTPs
  22. App Backend Infrastructure JP US UK API͸ͦΕͧΕผυϝΠϯ / CDN͸࢖༻ͯ͠ͳ͍(2017/10ݱࡏ) େن໛ͳτϥϑΟοΫͱͳΔ঎඼ը૾͸CDNΛར༻ͯ͠഑৴

    Client Multimedia Corporate data center Traditional server Mobile Client Assignment/ Task Requester Workers User Users Client Multimedia Corporate data center Traditio serve Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific User Users Client Multimedia Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requ Workers Amazon Mechanical Turk Non-Service Specific api.example.jp api.example.com api.example.uk S3 S3 S3 KQ DEOJ[FOFU VTDEOJ[FOFU VLDEOJ[FOFU CDN CDN CDN *N BHF'MVY
  23. ImageFlux • ͘͞ΒΠϯλʔωοτͱpixivͷڠۀ • ը૾ม׵ + Cache Storage • WebP΁ͷରԠ

    • CDNͷCache Hit Rateվળ
  24. Web Backend Infrastructure JP US UK Web͸γϯάϧυϝΠϯɻURIͰࢀর͢ΔBackendΛ੾Γସ͑Δ ͓٬༷͔Β࠷΋͍ۙEdgeͰ൑அ͢Δ͜ͱͰϨεϙϯελΠϜ/UX޲্ S3 S3

    S3 https://www.mercari.com/ /jp/ / /uk/
  25. ࠓޙͷ՝୊ɾऔΓ૊Έ

  26. ࠓޙͷ՝୊ɾऔΓ૊Έ • Security΁ͷϑΥʔΧε • L3-L7·Ͱେن໛ͳDoS߈ܸ͔Βͷ๷ޚ • WAF ͷݕূ • Bot੍ޚ

    • ߴ౓Խ͢ΔEdge Computing • Microservices Λ࣮ݱ͢Δखஈͱͯ͠ͷCDN • CDNͷઃఆ/VCLͷContinuous Integration • ίετ࠷దԽ
  27. Ҏ্ Fastlyͷօ༷ɺࠓޙͱ΋ΑΖ͓͘͠ئ͍͠·͢ => www.mercari.com/jp/jobs/