Tweet
Tweet

Slide 1

Slide 1 text

AbemaTVͰωΠςΟϒΤϯδχΞ
 ͔ΒαʔόʔΤϯδχΞʹ௅ઓͨ͠࿩ CyberAgent, Inc. AbemaTV։ൃہ αʔόʔαΠυΤϯδχΞ LE VAN NGHIA (ΪΞ)

Slide 2

Slide 2 text

• nghialv2607 • nghialv About me • 2014/11: CyberAgent 15৽ଔೖࣾ • ~ 2015/6: AmebaΞϓϦ - iOS • ~ 2016/7: FRESH!ੜ์ૹ - iOS • ~ ࠓ : AbemaTV - Server Side ID ڵຯ • Φʔϓϯιʔε։ൃ ϓϩδΣΫτܦྺ • ΪΞ • ؔ਺ܕϓϩάϥϛϯά Hakuba, Future, MaterialKit…

Slide 3

Slide 3 text

”20୅ͷ൴Β͕ͲΜͳ͜ͱʹʮ ௅ઓʯ͍ͯ͠Δͷ͔” ຊ೔ͷTalk BattleͷςʔϚ͸

Slide 4

Slide 4 text

ωΠςΟϒΤϯδχΞ αʔόʔΤϯδχΞ ➡

Slide 5

Slide 5 text

Ҡಈͷཧ༝ • ஌ࣝΛ෯޿͍ͨ͘͠ • Ϋϥ΢υɺωοτϫʔΫ • େྔσʔλॲཧ • ෼ࢄγεςϜɺϚΠΫϩαʔϏε

Slide 6

Slide 6 text

AbemaTVαʔόʔͷϞχλϦϯάγεςϜΛ࡞੒ ॳΊͯͷλεΫ͸

Slide 7

Slide 7 text

1 repository 35 repositories native server AbemaTVͷίʔυ

Slide 8

Slide 8 text

AbemaTVͷαʔόαΠυ Cloud DNS Cloud Load Balancer Cloud Router Cloud CDN Pub/Sub BigQuery Cloud Storage Transcoder Kubernetes Wowza HLS MPEG-DASH DRM gRPC Microservices Protocol Buffers Varnish Alpine Linux Terraform Packer MongoDB Redis Cluster Elasticsearch Firebase

Slide 9

Slide 9 text

• ຊ • ϓϩδΣΫτͷࢿྉ • ݸਓGoogle CloudͷΞΧ΢ϯτ࡞੒ɼ ৭ʑ༡ΜͰΈΔ • AbemaTVίʔυΛಡΉ • etcd, kubernetes…ͷίʔυΛಡΉ

Slide 10

Slide 10 text

GCP + Kubernetes + gRPC Google Cloud Platform Kubernetes Service Service Service gRPC gRPC

Slide 11

Slide 11 text

AbemaTVͷΞʔΩςΫνϟ streaming-server gateway Media User Comment Share Question Purchase Firebase Watchman Transcoder Ad system encoder Live Transcoder varnish api access live-gateway Compute
 Engine MongoDB Redis Cluster Container
 Engine Cloud Storage Firebase media access live admin access stateful stateless

Slide 12

Slide 12 text

Ϩϕϧ ྫ ΞϓϦέʔγϣϯ http, grpcͷΤϥʔɼϨΠςϯγɼ಺෦ঢ়ଶ ίϯςφ ίϯςφϦιʔεঢ়ଶ ΦʔέετϨʔγϣϯ ΫϥελʔͷϦιʔεɼεέδϡʔϦϯά ϗετ (OS, Hardware) ϗετͷϦιʔεɼhardwareΤϥʔ ωοτϫʔΫ routers, switches ऩू͢Δσʔλ

Slide 13

Slide 13 text

࣌ܥྻDBͷબ୒ • DalmatinerDB • InfluxDB • Prometheus • OpenTSDB • Graphite • … “top10 time series databases”ͷϒϩάʹΑΓ

Slide 14

Slide 14 text

• Prometheus͸Kubernetesͱ࿈ܞ͍ͯ͠Δ • αʔϏεσΟεΧόϦʔ: Ϋϥ΢υͷΑ͏ͳμΠφϛοΫͳ؀ڥʹendpoints͕มߋ͞ΕΔ • ίϯςφ, ΦʔέετϨʔγϣϯ ͷϝτϦΫεΛऔಘ͠΍͍͢ • Prometheusͷੑೳ • ύϑΥʔϚϯεɿ • ҰͭͷPrometheusαʔό͸800K samples/s • Federation • ѹॖ཰: 12.3ഒ (ѹॖͳ͠ɿ16 bytes/sample, Prometheus: 1.3 bytes/sample) • Prometheus͸Grafanaͱ࿈ܞ͍ͯ͠Δ • Prometheus͸GolangͰॻ͔Ε͍ͯΔ PrometheusΛબ୒ͨ͠ཧ༝

Slide 15

Slide 15 text

AbemaTVʹ͓͚ΔPrometheus

Slide 16

Slide 16 text

੒Ռ

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

AlertManager Slack࿈ܞ͢ΔBot - ࣾ֎΍ϞόΠϧͰ΋ίϚϯυͰ
 γεςϜͷϝτϦΫε͕ݟ͑Δ - Τϥʔ཰, latency percentileͳͲ ͕ߴ͘ͳΔͱ௨஌͕དྷΔ

Slide 20

Slide 20 text

• ωΠςΟϒΤϯδχΞ͔ΒαʔόʔΤϯδχΞʹ௅ઓͷ࿩ • AbemaTVͷϞχλϦϯάγεςϜͷৄ͍͠৘ใ • “AbemaTVʹPrometheusͱ͍͏ϞχλϦϯάγεςϜΛಋೖͨ͠࿩”ͷϒϩάϔ • αʔόʔ΋΍Γ͍ͨωΠςΟϒΤϯδχΞ΁ • ݴޠ͡Όͳͯ͘linux, network, cloudؔ͢Δ஌ࣝͷ४උΛΦεεϝ • ·ͩ20୅ͳͷͰ৭ʑ௅ઓ͠·͠ΐ͏ʂ ·ͱΊ