AbemaTVでネイティブエンジニア からサーバーエンジニアに挑戦した話

Battle Conference U30
http://bcu30.jp/

Transcript

1. AbemaTVͰωΠςΟϒΤϯδχΞ
 ͔ΒαʔόʔΤϯδχΞʹ௅ઓͨ͠࿩ CyberAgent, Inc. AbemaTV։ൃہ αʔόʔαΠυΤϯδχΞ LE VAN NGHIA (ΪΞ)
2. • nghialv2607 • nghialv About me • 2014/11: CyberAgent 15৽ଔೖࣾ • ~ 2015/6: AmebaΞϓϦ - iOS • ~ 2016/7: FRESH!ੜ์ૹ - iOS • ~ ࠓ : AbemaTV - Server Side ID ڵຯ • Φʔϓϯιʔε։ൃ ϓϩδΣΫτܦྺ • ΪΞ • ؔ਺ܕϓϩάϥϛϯά Hakuba, Future, MaterialKit…
3. ”20୅ͷ൴Β͕ͲΜͳ͜ͱʹʮ ௅ઓʯ͍ͯ͠Δͷ͔” ຊ೔ͷTalk BattleͷςʔϚ͸
4. ωΠςΟϒΤϯδχΞ αʔόʔΤϯδχΞ ➡
5. Ҡಈͷཧ༝ • ஌ࣝΛ෯޿͍ͨ͘͠ • Ϋϥ΢υɺωοτϫʔΫ • େྔσʔλॲཧ • ෼ࢄγεςϜɺϚΠΫϩαʔϏε
6. AbemaTVαʔόʔͷϞχλϦϯάγεςϜΛ࡞੒ ॳΊͯͷλεΫ͸
7. 1 repository 35 repositories native server AbemaTVͷίʔυ
8. AbemaTVͷαʔόαΠυ Cloud DNS Cloud Load Balancer Cloud Router Cloud CDN Pub/Sub BigQuery Cloud Storage Transcoder Kubernetes Wowza HLS MPEG-DASH DRM gRPC Microservices Protocol Buffers Varnish Alpine Linux Terraform Packer MongoDB Redis Cluster Elasticsearch Firebase
9. • ຊ • ϓϩδΣΫτͷࢿྉ • ݸਓGoogle CloudͷΞΧ΢ϯτ࡞੒ɼ ৭ʑ༡ΜͰΈΔ • AbemaTVίʔυΛಡΉ • etcd, kubernetes…ͷίʔυΛಡΉ
10. GCP + Kubernetes + gRPC Google Cloud Platform Kubernetes Service Service Service gRPC gRPC
11. AbemaTVͷΞʔΩςΫνϟ streaming-server gateway Media User Comment Share Question Purchase Firebase Watchman Transcoder Ad system encoder Live Transcoder varnish api access live-gateway Compute
 Engine MongoDB Redis Cluster Container
 Engine Cloud Storage Firebase media access live admin access stateful stateless
12. Ϩϕϧ ྫ ΞϓϦέʔγϣϯ http, grpcͷΤϥʔɼϨΠςϯγɼ಺෦ঢ়ଶ ίϯςφ ίϯςφϦιʔεঢ়ଶ ΦʔέετϨʔγϣϯ ΫϥελʔͷϦιʔεɼεέδϡʔϦϯά ϗετ (OS, Hardware) ϗετͷϦιʔεɼhardwareΤϥʔ ωοτϫʔΫ routers, switches ऩू͢Δσʔλ
13. ࣌ܥྻDBͷબ୒ • DalmatinerDB • InfluxDB • Prometheus • OpenTSDB • Graphite • … “top10 time series databases”ͷϒϩάʹΑΓ
14. • Prometheus͸Kubernetesͱ࿈ܞ͍ͯ͠Δ • αʔϏεσΟεΧόϦʔ: Ϋϥ΢υͷΑ͏ͳμΠφϛοΫͳ؀ڥʹendpoints͕มߋ͞ΕΔ • ίϯςφ, ΦʔέετϨʔγϣϯ ͷϝτϦΫεΛऔಘ͠΍͍͢ • Prometheusͷੑೳ • ύϑΥʔϚϯεɿ • ҰͭͷPrometheusαʔό͸800K samples/s • Federation • ѹॖ཰: 12.3ഒ (ѹॖͳ͠ɿ16 bytes/sample, Prometheus: 1.3 bytes/sample) • Prometheus͸Grafanaͱ࿈ܞ͍ͯ͠Δ • Prometheus͸GolangͰॻ͔Ε͍ͯΔ PrometheusΛબ୒ͨ͠ཧ༝
15. AbemaTVʹ͓͚ΔPrometheus
16. ੒Ռ
19. AlertManager Slack࿈ܞ͢ΔBot - ࣾ֎΍ϞόΠϧͰ΋ίϚϯυͰ
 γεςϜͷϝτϦΫε͕ݟ͑Δ - Τϥʔ཰, latency percentileͳͲ ͕ߴ͘ͳΔͱ௨஌͕དྷΔ
20. • ωΠςΟϒΤϯδχΞ͔ΒαʔόʔΤϯδχΞʹ௅ઓͷ࿩ • AbemaTVͷϞχλϦϯάγεςϜͷৄ͍͠৘ใ • “AbemaTVʹPrometheusͱ͍͏ϞχλϦϯάγεςϜΛಋೖͨ͠࿩”ͷϒϩάϔ • αʔόʔ΋΍Γ͍ͨωΠςΟϒΤϯδχΞ΁ • ݴޠ͡Όͳͯ͘linux, network, cloudؔ͢Δ஌ࣝͷ४උΛΦεεϝ • ·ͩ20୅ͳͷͰ৭ʑ௅ઓ͠·͠ΐ͏ʂ ·ͱΊ