CDNの使い方 in Mercari/CDN in Mercari

CDN in Mercari/CDNの使い方 in Mercari
Fastly Yamagoya Meetup 2017

Transcript

1. CDN in Mercari Fastly Yamagoya Meetup 2017 Masahiro Nagano @kazeburo
2. Me • Masahiro Nagano / ௕໺խ޿ • @kazeburo • Mercari, Inc
 Principal Engineer
 Site Reliability Engineering (SRE) Team
3. Agenda • ϝϧΧϦͷ঺հ • ઌ೔ͷݸਓ৘ใྲྀग़Πϯγσϯτʹ͍ͭͯ • ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN • ࠓޙͷ՝୊ɾऔΓ૊Έ
4. Mercari • ϑϦϚΞϓϦ • εϚϗͰࣸਅΛͱͬͯ؆୯ʹग़඼ • ҆৺ɾ҆શͳܾࡁ • ศརͳ഑ૹ
5. Mercari μ΢ϯϩʔυ਺ ྲྀ௨ֹ 7500ສDL(JP+US) ݄ؒ100ԯԁҎ্ ग़඼਺ 1೔100ສ඼Ҏ্
6. γεςϜ͔ΒΈͨϝϧΧϦ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. Client Multimedia Corporate data center Traditional server Mobile Client IAM Add-on Example: IAM Add-on ence ) Assignment/ Task Requester Workers ग़඼! DB Search 5-දࣔ ݕࡧ൓ө ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific େྔͷϦΫΤετ ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corp data c Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ϦΫΤετԠ౴ DB Search ߪೖ! ϛϦඵʙ30ඵ ਺ඵʙ ߴ଎ʹେྔͷτϥϯβΫγϣϯΛѻ͏ ը૾ ܾࡁ AI
7. Mercari JP US UK JP/US/UKͰల։த
8. CDN੾Γସ͑ʹ൐͏ Web൛ϝϧΧϦʹ͓͚Δݸਓ৘ใྲྀग़
 Πϯγσϯτʹ͍ͭͯ
9. ଟ͘ͷօ༷ʹ͝৺഑ɺ͝໎࿭Λֻ͓͚͍ͨ͠·ͨ͠ ਂ͓͘࿳ͼਃ͋͛͠·͢ ଟେͳΔڠྗΛFastly༷ʹ͍͖ͨͩ·ͨ͠ɻվΊ͓ͯྱਃ্͛͠·͢
10. ൃੜͨ͠ࣄ৅ • ϝϧΧϦWeb൛΁ͷΞΫηε଎౓޲্ͱɺηΩϡϦςΟ޲্ͷͨΊɺCDN ͷ੾Γସ͑Λߦ͍·ͨ͠ɻͦͷࡍʹ੾Γସ͑ઌͷCDN(Fastly)ʹ͓͚Δ Ωϟογϡͷಈ࡞ʹ͍ͭͯͷ೺Ѳ͕ෆ଍͓ͯ͠Γɺ͓٬͞·΁ͷϨεϙϯ ε͕ผͷ͓٬͞·ʹҙਤͤͣදࣔ͞Εɺ݁Ռͱͯ͠ݸਓΛಛఆͰ͖Δ৘ใ ΛؚΉ಺༰͕ຊਓҎ֎ʹӾཡ͞ΕΔঢ়ଶͱͳΓ·ͨ͠
11. Timeline • 6/22 • 9:41ɹ CDNͷ੾Γସ͑Λ࣮ࢪʢ໰୊ൃੜʣ • 14:41ɹΧελϚʔαϙʔτʹ͓ͯ٬͞·͔Βͷ໰͍߹ΘͤΛ֬ೝ͠ɺࣾ಺΁ใࠂ • 15:05ɹCDNͷ੾Γସ͑Λதࢭ͠ɺैདྷͷCDN΁໭͢ • 15:16ɹWeb൛ͷϝϧΧϦΛϝϯςφϯεϞʔυ΁੾Γସ͑ • 15:38ɹCDNͷ഑৴ઃఆΛdeactivate͠ɺΞΫηεΛःஅ • 15:47ɹWeb൛ͷϝϧΧϦϝϯςφϯεϞʔυΛऴྃ • 17:55 ίʔϙϨʔταΠτʹ͓஌ΒͤΛܝࡌ • 20:45 Tech blogʹͯৄࡉެ։
12. Timeline • 6/28 • ഑৴ઃఆΛ࠶Activate • Cacheͷ׬શແޮԽઃఆΛߦ͍ɺFastlyࣾͷΤϯδχΞʹϨϏϡʔΛґཔ • nginxͷઃఆΛߋ৽͠ɺcacheʹؔ͢ΔϔομΛมߋ • ࣾ಺Ͱݕূ • 6/29 • FastlyࣾΤϯδχΞ଴ػͷ΋ͱɺ੾Γସ͑Λ࣮ࢪ
13. CacheແޮԽ • CacheΛແޮԽ͢ΔͨΊʹ͸ “Cache-Control: private” ΋͘͠͸ "Set-Cookie" ͕ඞཁ • ”Cache-Control: no-cache” ΍ “no-store” ͸ແࢹ͞ΕΔ • Expiresϔομ΋ར༻͞ΕΔ͕ɺ೔෇ͷղऍʹࣦഊ͋Δ͍͸աڈ೔෇ͷ৔߹ ͸ “0ඵ” ͱͯ͠ѻΘΕΔ • 0ඵ͸cacheແޮͰ͸ͳ͘ʮ0ඵͷcache͕ଘࡏ͢Δʯ
14. 0ඵͷcache • CDN͔ΒΦϦδϯ΁ͷϦΫΤετͷॲཧதʹɺಉ͡ URLʹରͯ͠ϦΫΤετ͕ൃੜ͢Δͱɺ࠷ॳͷϨεϙ ϯεΛ଴ͬͯɺ2ͭ໨Ҏ߱ͷϦΫΤετʹ΋ಉ͡Ϩε ϙϯε͕ฦ͞ΕΔ • `Request collapsing` ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific ©2011 Amazon Web Services LLC or its affiliates. All r User Users Client Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Human Intelligence Tasks (HIT) Assignment/ Task Workers Amazon Mechanical Turk Non-Service Specific origin CDN (1) (2) (3) (4) (5) (5)
15. ରࡦ • VCLͷઃఆมߋ • ෳ਺ͷbackendʹରԠ͢ΔͨΊɺPriorityΛ௿͘ઃఆ # ͜͜ʹbackend੾Γସ͑ͷvcl͕ඳ͍ͯ͋Δ # Request Condition: always Prio: 100 if( req.url ) { ## ඞͣtrueʹͳΔ if (!req.http.Fastly-FF) { if (req.http.X-Forwarded-For) { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For ", " client.ip; } else { set req.http.Fastly-Temp-XFF = client.ip } } else { set req.http.Fastly-Temp-XFF = req.http.X-Forwarded-For; } return(pass); }
16. ରࡦ • ϩάΛS3ʹૹ৴͠ɺlambdaʹΑΓϦΞϧλΠϜʹղੳɺmackerelͰՄࢹԽɻ ؂ࢹ • nginx/ApacheʹͯheaderΛ௥Ճ • header΍VCLͷมߋͷ؂ࢹ more_clear_headers 'Expires'; more_set_headers "Cache-Control: private, no-cache, no-store, must-revalidate" "Pragma: no-cache"; add_header Set-Cookie "merCtx=\"\"; HttpOnly" always;
17. https://docs.fastly.com/ja/guides/tutorials/cache-control-tutorial
18. ϝϧΧϦͷΠϯϑϥετϥΫνϟͱCDN
19. CDN in Mercari • ΑΓଟ͘ͷτϥϑΟοΫΛॲཧ͢ΔͨΊ • Πϯϑϥӡ༻ίετͷ࡟ݮͱͯ͠CDNΛར༻ • ը૾഑৴/ಈը഑৴ • UX/ηΩϡϦςΟͷվળͷͨΊ • Clientͱͷ௨৴؀ڥ࠷దԽ • αΠτ/APIΛؙ͝ͱCDNԽ
20. Infrastructure US UK ͦΕͧΕͷҬ಺ͷDCΛར༻ S3 S3 S3 ঎඼ը૾ ঎඼ը૾ ঎඼ը૾ Frontend for Backend
 microservices Backend API JP
21. Frontend for Backend/microservices API Gateway ©2011 Amazon Web Services LLC or its affiliates. All rights reserved. User Users Client Multimedia Corporate data center Traditional server Mobile Client Internet AWS Management Console IAM Add-on Example: IAM Add-on Non-Service Specific search personalization offer gRPC JSON over HTTPs
22. App Backend Infrastructure JP US UK API͸ͦΕͧΕผυϝΠϯ / CDN͸࢖༻ͯ͠ͳ͍(2017/10ݱࡏ) େن໛ͳτϥϑΟοΫͱͳΔ঎඼ը૾͸CDNΛར༻ͯ͠഑৴ Client Multimedia Corporate data center Traditional server Mobile Client Assignment/ Task Requester Workers User Users Client Multimedia Corporate data center Traditio serve Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requester Workers Amazon Mechanical Turk Non-Service Specific User Users Client Multimedia Mobile Client Amazon Mechanical Turk On-Demand Workforce Human Intelligence Tasks (HIT) Assignment/ Task Requ Workers Amazon Mechanical Turk Non-Service Specific api.example.jp api.example.com api.example.uk S3 S3 S3 KQ DEOJ[FOFU VTDEOJ[FOFU VLDEOJ[FOFU CDN CDN CDN *N BHF'MVY
23. ImageFlux • ͘͞ΒΠϯλʔωοτͱpixivͷڠۀ • ը૾ม׵ + Cache Storage • WebP΁ͷରԠ • CDNͷCache Hit Rateվળ
24. Web Backend Infrastructure JP US UK Web͸γϯάϧυϝΠϯɻURIͰࢀর͢ΔBackendΛ੾Γସ͑Δ ͓٬༷͔Β࠷΋͍ۙEdgeͰ൑அ͢Δ͜ͱͰϨεϙϯελΠϜ/UX޲্ S3 S3 S3 https://www.mercari.com/ /jp/ / /uk/
25. ࠓޙͷ՝୊ɾऔΓ૊Έ
26. ࠓޙͷ՝୊ɾऔΓ૊Έ • Security΁ͷϑΥʔΧε • L3-L7·Ͱେن໛ͳDoS߈ܸ͔Βͷ๷ޚ • WAF ͷݕূ • Bot੍ޚ • ߴ౓Խ͢ΔEdge Computing • Microservices Λ࣮ݱ͢Δखஈͱͯ͠ͷCDN • CDNͷઃఆ/VCLͷContinuous Integration • ίετ࠷దԽ
27. Ҏ্ Fastlyͷօ༷ɺࠓޙͱ΋ΑΖ͓͘͠ئ͍͠·͢ => www.mercari.com/jp/jobs/