Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AbemaTVでネイティブエンジニア
からサーバーエンジニアに挑戦した話

nghialv
March 13, 2017

 AbemaTVでネイティブエンジニア
からサーバーエンジニアに挑戦した話

Battle Conference U30
http://bcu30.jp/

nghialv

March 13, 2017
Tweet

More Decks by nghialv

Other Decks in Technology

Transcript

  1. AbemaTVͰωΠςΟϒΤϯδχΞ

    ͔ΒαʔόʔΤϯδχΞʹ௅ઓͨ͠࿩
    CyberAgent, Inc. AbemaTV։ൃہ
    αʔόʔαΠυΤϯδχΞ
    LE VAN NGHIA (ΪΞ)

    View Slide

  2. • nghialv2607
    • nghialv
    About me
    • 2014/11: CyberAgent 15৽ଔೖࣾ
    • ~ 2015/6: AmebaΞϓϦ - iOS
    • ~ 2016/7: FRESH!ੜ์ૹ - iOS
    • ~ ࠓ : AbemaTV - Server Side
    ID
    ڵຯ
    • Φʔϓϯιʔε։ൃ
    ϓϩδΣΫτܦྺ
    • ΪΞ
    • ؔ਺ܕϓϩάϥϛϯά
    Hakuba, Future, MaterialKit…

    View Slide

  3. ”20୅ͷ൴Β͕ͲΜͳ͜ͱʹʮ
    ௅ઓʯ͍ͯ͠Δͷ͔”
    ຊ೔ͷTalk BattleͷςʔϚ͸

    View Slide

  4. ωΠςΟϒΤϯδχΞ αʔόʔΤϯδχΞ

    View Slide

  5. Ҡಈͷཧ༝
    • ஌ࣝΛ෯޿͍ͨ͘͠
    • Ϋϥ΢υɺωοτϫʔΫ
    • େྔσʔλॲཧ
    • ෼ࢄγεςϜɺϚΠΫϩαʔϏε

    View Slide

  6. AbemaTVαʔόʔͷϞχλϦϯάγεςϜΛ࡞੒
    ॳΊͯͷλεΫ͸

    View Slide

  7. 1 repository
    35 repositories
    native server
    AbemaTVͷίʔυ

    View Slide

  8. AbemaTVͷαʔόαΠυ
    Cloud DNS Cloud
    Load Balancer
    Cloud
    Router
    Cloud CDN
    Pub/Sub
    BigQuery
    Cloud
    Storage
    Transcoder
    Kubernetes
    Wowza
    HLS
    MPEG-DASH
    DRM
    gRPC
    Microservices
    Protocol Buffers
    Varnish
    Alpine Linux
    Terraform
    Packer
    MongoDB
    Redis Cluster
    Elasticsearch
    Firebase

    View Slide

  9. • ຊ
    • ϓϩδΣΫτͷࢿྉ
    • ݸਓGoogle CloudͷΞΧ΢ϯτ࡞੒ɼ
    ৭ʑ༡ΜͰΈΔ
    • AbemaTVίʔυΛಡΉ
    • etcd, kubernetes…ͷίʔυΛಡΉ

    View Slide

  10. GCP + Kubernetes + gRPC
    Google Cloud Platform
    Kubernetes
    Service
    Service
    Service
    gRPC gRPC

    View Slide

  11. AbemaTVͷΞʔΩςΫνϟ
    streaming-server
    gateway
    Media User Comment Share Question Purchase Firebase Watchman Transcoder
    Ad system
    encoder
    Live
    Transcoder
    varnish
    api access
    live-gateway
    Compute

    Engine
    MongoDB Redis Cluster
    Container

    Engine
    Cloud Storage Firebase
    media access
    live admin access
    stateful
    stateless

    View Slide

  12. Ϩϕϧ ྫ
    ΞϓϦέʔγϣϯ http, grpcͷΤϥʔɼϨΠςϯγɼ಺෦ঢ়ଶ
    ίϯςφ ίϯςφϦιʔεঢ়ଶ
    ΦʔέετϨʔγϣϯ ΫϥελʔͷϦιʔεɼεέδϡʔϦϯά
    ϗετ (OS, Hardware) ϗετͷϦιʔεɼhardwareΤϥʔ
    ωοτϫʔΫ routers, switches
    ऩू͢Δσʔλ

    View Slide

  13. ࣌ܥྻDBͷબ୒
    • DalmatinerDB
    • InfluxDB
    • Prometheus
    • OpenTSDB
    • Graphite
    • …
    “top10 time series databases”ͷϒϩάʹΑΓ

    View Slide

  14. • Prometheus͸Kubernetesͱ࿈ܞ͍ͯ͠Δ
    • αʔϏεσΟεΧόϦʔ: Ϋϥ΢υͷΑ͏ͳμΠφϛοΫͳ؀ڥʹendpoints͕มߋ͞ΕΔ
    • ίϯςφ, ΦʔέετϨʔγϣϯ ͷϝτϦΫεΛऔಘ͠΍͍͢
    • Prometheusͷੑೳ
    • ύϑΥʔϚϯεɿ
    • ҰͭͷPrometheusαʔό͸800K samples/s
    • Federation
    • ѹॖ཰: 12.3ഒ (ѹॖͳ͠ɿ16 bytes/sample, Prometheus: 1.3 bytes/sample)
    • Prometheus͸Grafanaͱ࿈ܞ͍ͯ͠Δ
    • Prometheus͸GolangͰॻ͔Ε͍ͯΔ
    PrometheusΛબ୒ͨ͠ཧ༝

    View Slide

  15. AbemaTVʹ͓͚ΔPrometheus

    View Slide

  16. ੒Ռ

    View Slide

  17. View Slide

  18. View Slide

  19. AlertManager
    Slack࿈ܞ͢ΔBot
    - ࣾ֎΍ϞόΠϧͰ΋ίϚϯυͰ

    γεςϜͷϝτϦΫε͕ݟ͑Δ
    - Τϥʔ཰, latency percentileͳͲ
    ͕ߴ͘ͳΔͱ௨஌͕དྷΔ

    View Slide

  20. • ωΠςΟϒΤϯδχΞ͔ΒαʔόʔΤϯδχΞʹ௅ઓͷ࿩
    • AbemaTVͷϞχλϦϯάγεςϜͷৄ͍͠৘ใ
    • “AbemaTVʹPrometheusͱ͍͏ϞχλϦϯάγεςϜΛಋೖͨ͠࿩”ͷϒϩάϔ
    • αʔόʔ΋΍Γ͍ͨωΠςΟϒΤϯδχΞ΁
    • ݴޠ͡Όͳͯ͘linux, network, cloudؔ͢Δ஌ࣝͷ४උΛΦεεϝ
    • ·ͩ20୅ͳͷͰ৭ʑ௅ઓ͠·͠ΐ͏ʂ
    ·ͱΊ

    View Slide