AbemaTVでネイティブエンジニア
からサーバーエンジニアに挑戦した話

439d941a6e41920af9bf4d5bd5f97e3a?s=47 nghialv
March 13, 2017

 AbemaTVでネイティブエンジニア
からサーバーエンジニアに挑戦した話

Battle Conference U30
http://bcu30.jp/

439d941a6e41920af9bf4d5bd5f97e3a?s=128

nghialv

March 13, 2017
Tweet

Transcript

  1. AbemaTVͰωΠςΟϒΤϯδχΞ
 ͔ΒαʔόʔΤϯδχΞʹ௅ઓͨ͠࿩ CyberAgent, Inc. AbemaTV։ൃہ αʔόʔαΠυΤϯδχΞ LE VAN NGHIA (ΪΞ)

  2. • nghialv2607 • nghialv About me • 2014/11: CyberAgent 15৽ଔೖࣾ

    • ~ 2015/6: AmebaΞϓϦ - iOS • ~ 2016/7: FRESH!ੜ์ૹ - iOS • ~ ࠓ : AbemaTV - Server Side ID ڵຯ • Φʔϓϯιʔε։ൃ ϓϩδΣΫτܦྺ • ΪΞ • ؔ਺ܕϓϩάϥϛϯά Hakuba, Future, MaterialKit…
  3. ”20୅ͷ൴Β͕ͲΜͳ͜ͱʹʮ ௅ઓʯ͍ͯ͠Δͷ͔” ຊ೔ͷTalk BattleͷςʔϚ͸

  4. ωΠςΟϒΤϯδχΞ αʔόʔΤϯδχΞ ➡

  5. Ҡಈͷཧ༝ • ஌ࣝΛ෯޿͍ͨ͘͠ • Ϋϥ΢υɺωοτϫʔΫ • େྔσʔλॲཧ • ෼ࢄγεςϜɺϚΠΫϩαʔϏε

  6. AbemaTVαʔόʔͷϞχλϦϯάγεςϜΛ࡞੒ ॳΊͯͷλεΫ͸

  7. 1 repository 35 repositories native server AbemaTVͷίʔυ

  8. AbemaTVͷαʔόαΠυ Cloud DNS Cloud Load Balancer Cloud Router Cloud CDN

    Pub/Sub BigQuery Cloud Storage Transcoder Kubernetes Wowza HLS MPEG-DASH DRM gRPC Microservices Protocol Buffers Varnish Alpine Linux Terraform Packer MongoDB Redis Cluster Elasticsearch Firebase
  9. • ຊ • ϓϩδΣΫτͷࢿྉ • ݸਓGoogle CloudͷΞΧ΢ϯτ࡞੒ɼ ৭ʑ༡ΜͰΈΔ • AbemaTVίʔυΛಡΉ

    • etcd, kubernetes…ͷίʔυΛಡΉ
  10. GCP + Kubernetes + gRPC Google Cloud Platform Kubernetes Service

    Service Service gRPC gRPC
  11. AbemaTVͷΞʔΩςΫνϟ streaming-server gateway Media User Comment Share Question Purchase Firebase

    Watchman Transcoder Ad system encoder Live Transcoder varnish api access live-gateway Compute
 Engine MongoDB Redis Cluster Container
 Engine Cloud Storage Firebase media access live admin access stateful stateless
  12. Ϩϕϧ ྫ ΞϓϦέʔγϣϯ http, grpcͷΤϥʔɼϨΠςϯγɼ಺෦ঢ়ଶ ίϯςφ ίϯςφϦιʔεঢ়ଶ ΦʔέετϨʔγϣϯ ΫϥελʔͷϦιʔεɼεέδϡʔϦϯά ϗετ

    (OS, Hardware) ϗετͷϦιʔεɼhardwareΤϥʔ ωοτϫʔΫ routers, switches ऩू͢Δσʔλ
  13. ࣌ܥྻDBͷબ୒ • DalmatinerDB • InfluxDB • Prometheus • OpenTSDB •

    Graphite • … “top10 time series databases”ͷϒϩάʹΑΓ
  14. • Prometheus͸Kubernetesͱ࿈ܞ͍ͯ͠Δ • αʔϏεσΟεΧόϦʔ: Ϋϥ΢υͷΑ͏ͳμΠφϛοΫͳ؀ڥʹendpoints͕มߋ͞ΕΔ • ίϯςφ, ΦʔέετϨʔγϣϯ ͷϝτϦΫεΛऔಘ͠΍͍͢ •

    Prometheusͷੑೳ • ύϑΥʔϚϯεɿ • ҰͭͷPrometheusαʔό͸800K samples/s • Federation • ѹॖ཰: 12.3ഒ (ѹॖͳ͠ɿ16 bytes/sample, Prometheus: 1.3 bytes/sample) • Prometheus͸Grafanaͱ࿈ܞ͍ͯ͠Δ • Prometheus͸GolangͰॻ͔Ε͍ͯΔ PrometheusΛબ୒ͨ͠ཧ༝
  15. AbemaTVʹ͓͚ΔPrometheus

  16. ੒Ռ

  17. None
  18. None
  19. AlertManager Slack࿈ܞ͢ΔBot - ࣾ֎΍ϞόΠϧͰ΋ίϚϯυͰ
 γεςϜͷϝτϦΫε͕ݟ͑Δ - Τϥʔ཰, latency percentileͳͲ ͕ߴ͘ͳΔͱ௨஌͕དྷΔ

  20. • ωΠςΟϒΤϯδχΞ͔ΒαʔόʔΤϯδχΞʹ௅ઓͷ࿩ • AbemaTVͷϞχλϦϯάγεςϜͷৄ͍͠৘ใ • “AbemaTVʹPrometheusͱ͍͏ϞχλϦϯάγεςϜΛಋೖͨ͠࿩”ͷϒϩάϔ • αʔόʔ΋΍Γ͍ͨωΠςΟϒΤϯδχΞ΁ • ݴޠ͡Όͳͯ͘linux,

    network, cloudؔ͢Δ஌ࣝͷ४උΛΦεεϝ • ·ͩ20୅ͳͷͰ৭ʑ௅ઓ͠·͠ΐ͏ʂ ·ͱΊ