ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON

ピクシブのコンテンツ配信基盤技術
pixiv TECH SALON / 2018-03-06
https://techsalon.pixiv.co.jp/

Transcript

1. ϐΫγϒͷ഑৴ج൫ٕज़
   
   pixiv Technologies Inc.
   Harukasan / ಓҪ ढ़հ
   

   View Slide

2. Harukasan
   ಓҪ ढ़հ
   2012೥౓৽ଔೖࣾ

   Πϯϑϥ෦ˠImageFluxࣄۀ্ཱͪ͛ˠ഑৴ٕज़෦Ϛωʔδϟʔ

   ϐΫγϒςΫϊϩδʔζגࣜձࣾࣥߦ໾һ

   γχΞςοΫϦʔυ
   

   View Slide

3. ϐΫγϒͷ੒௕Λࢧ͖͑ͯͨ഑৴ٕज़
   

   View Slide

4. Infrastructure ImageFlux Live Streaming Future
   ϐΫγϒͷ੒௕Λࢧ͖͑ͯͨ഑৴ٕज़
   

   View Slide

5. Infrastructure
   

   View Slide

6. B28 ϕχϠػ
   

   View Slide

7. View Slide

8. ऩ·Βͳ͍ճઢ
   ٧·Δہࣷ
   ᫔᫓͢ΔωοτϫʔΫ
   ৴པੑ
   

   View Slide

9. スパゲッティーコード（笑）
   

   View Slide

10. 1G private lines
    !
    SERVER ROOM
    SHINJUKU DC
    1G x 8
    Internet
    Cache
    Origin
    

    View Slide

11. !
    FUKUSHIMA

    Shirakawa DC
    10G x 8
    Internet
    

    View Slide

12. pixiv͚ͩͰͳ͘શαʔϏεڞ௨Ͱ࢖͑Δը૾഑৴ج൫
    ߴूੵαʔόΛಋೖ (2U 24SSD)
    αʔϏε͔Βಠཱͨ͠ߏ੒
    ϑϧ10GωοτϫʔΫ
    ༰ྔޮ཰UP
    PXIMG
    

    View Slide

13. ImageFlux
    

    View Slide

14. PC iOS Android
    SP FP
    20+ images
    

    View Slide

15. ॲཧ͕ෳࡶͰख͕෇͚ΒΕͳ͍

    ౤ߘʹ͕͔͔࣌ؒΔ
    ը૾αΠζ͕খ͍͞
    ಈతม׵͢Δʹ΋CPUϦιʔε͕ਏ͍
    ैདྷͷը૾ม׵ͷ໰୊఺
    

    View Slide

16. go-thumber
    Gopher image was created by Takuya Ueda (https://twitter.com/tenntenn).

    Licensed under the Creative Commons 3.0 Attributions license.
    https://github.com/pixiv/go-thumber
    αϜωΠϧม׵ϓϩΩγ
    

    View Slide

17. ͏·͍
    ΍͍͢
    ͸΍͍
    

    View Slide

18. ߴ඼࣭
    লϝϞϦ
    ௿஗Ԇ
    

    View Slide

19. Faster JPEG resizing
    Decode
    Y'CbCr→RGB
    RGB→Y'CbCr
    Encode
    Resize
    Buffer
    ImageMagick
    

    View Slide

20. Faster JPEG resizing
    Decode
    Y'CbCr→RGB
    RGB→Y'CbCr
    Encode
    Resize
    Buffer
    ImageMagick
    Decode
    Encode
    Resize
    go-thumber
    Buffer
    

    View Slide

21. Faster JPEG resizing
    Decode
    Y'CbCr→RGB
    RGB→Y'CbCr
    Encode
    Resize
    Buffer
    ImageMagick
    Decode
    Encode
    Resize
    go-thumber
    Buffer
    3x

    faster
    

    View Slide

22. αϜωΠϧϚελͷੜ੒
    ΦϦδφϧը૾
    αϜωΠϧϚελJPEG
    Ξοϓϩʔυ࣌ੜ੒ go-thumberͰੜ੒
    

    View Slide

23. pixivͷશαϜωΠϧΛಈతੜ੒ʹஔ͖׵͑
    ޷͖ͳαΠζͷαϜωΠϧΛੜ੒Ͱ͖ΔΑ͏ʹ
    ଞαʔϏεʹಋೖ͢Δʹ͸αϜωΠϧϚελ͕ඞཁ
    go-thumberʹΑΔը૾ม׵
    

    View Slide

24. શαʔϏεԣஅͰ഑৴ج൫͕ར༻Ͱ͖Δঢ়ଶͰ͸ͳ͍
    ը૾ϑΥʔϚοτʹର͢Δਖ਼֬ͳ஌͕ࣝཁٻ͞ΕΔ
    αʔϏεΛ࡞Δͨͼʹը૾ʹ͍ͭͯߟ͑Δͷ͸ਏ͍
    ͦ΋ͦ΋ը૾ม׵͕΍Γ͍ͨΘ͚Ͱ͸ͳ͍
    go-thumberͷ໰୊఺
    

    View Slide

25. ϐΫγϒ͕ը૾ม׵ٕज़ʹ౤ࢿ͠ଓ͚Δʹ͸
    શαʔϏεԣஅͰར༻Ͱ͖Δ഑৴ج൫Λ։ൃ͢Δʹ͸
    ը૾ม׵͸ΠϯϑϥαʔϏεͰ͋Δ΂͖Ͱ͸
    

    View Slide

26. ։ൃαΠΫϧΛଅਐ͢ΔΫϥ΢υը૾ม׵αʔϏε
    

    View Slide

27. ։ൃαΠΫϧΛଅਐ͢ΔΫϥ΢υը૾ม׵αʔϏε
    Base:

    go-thumber

    pixiv഑৴Ϋϥελ
    Format:

    JPEG, PNG, GIF

    and WebP
    Origin:

    HTTPS and S3
    Infrastructure:
    ͘͞ΒΠϯλʔωοτ

    ίϯςϯπ഑৴ج൫
    JPEG PNG WebP HTTPS
    

    View Slide

28. γεςϜ։ൃ
    Ӧۀ/αϙʔτ
    Πϯϑϥӡ༻
    "
    ͓٬༷
    

    View Slide

29. ImageFluxΛ͝ར༻ͷ͓٬༷
    

    View Slide

30. PXIMG
    Ϋϥ΢υαʔϏε
    ࣾ಺഑৴ج൫
    ίʔυϕʔεɾϊ΢ϋ΢Λڞ༗
    ͘͞ΒΠϯλʔωοτӡ༻
    ࣾ಺Πϯϑϥ෦ӡ༻
    ߴՄ༻ੑ
    ௿ίετ
    ؅ཧίϯιʔϧ
    ϨΨγʔͳઃఆϑΝΠϧ
    

    View Slide

31. JPEGɺPNGɺGIFʹՃ͑ͯWebPग़ྗʹରԠ
    ύϥϝʔλॺ໊ػೳ
    ύε͝ͱͷσϑΥϧτύϥϝʔλ
    ΧϥʔϓϩϑΝΠϧ΁ͷରԠɺద༻ํ๏ݟ௚͠
    ෳ਺ຕͷΦʔόʔϨΠʹରԠ
    ϦϦʔεޙͷػೳ௥Ճ
    

    View Slide

32. JPEGɺPNGͷຒΊࠐΈΧϥʔϓϩϑΝΠϧʹରԠ
    ೖྗઐ༻ɺຒΊࠐΈग़ྗͰ͖ͳ͍৔߹͸sRGBʹม׵
    Color ProfileରԠ
    ICC ICC
    ICC
    ICC
    ICC
    LittleCMS2
    

    View Slide

33. type ScanLineReader interface {

    

    // Config returns the image configuration.
    Config() *Config

    

    // ReadScanLines reads scan lines into the p for each component.
    ReadScanLines(ctx context.Context, p [][]uint8) (int, error)

    }
    ֤ը૾ม׵ίϯϙʔωϯτΛؔ਺ͱ࣮ͯ͠૷
    ը૾ม׵ύΠϓϥΠϯ
    func Scale(src ScanLineReader, width, height int) ScanLineReader
    

    View Slide

34. c := Decode(src)
    c = Clip(c, image.Rect(10,10,410,310))
    c = Scale(c, 200, 150)
    Encode(rw, c)
    ը૾ม׵ύΠϓϥΠϯ
    Decode
    Clip
    Scale
    Encode
    ෳ਺ͷը૾ॲཧΛ࿈݁
    

    View Slide

35. Decode
    Scaling
    ඞཁͳॲཧΛඞཁͳ͚ͩͭͳ͛Δ
    Decode
    Y'CbCr→RGB
    Compositing
    Scaling
    RGB→Y'CbCr
    Encode
    Color transform
    

    View Slide

36. ImageFluxࣄۀ෦ͷൃ଍ʢݱ഑৴ٕज़෦ʣ
    ը૾഑৴ٕज़୯ମͰ։ൃαΠΫϧΛ·ΘͤΔΑ͏ʹ
    ར༻Ϣʔβ͕֦େ͢Δ͜ͱͰɺΑΓ඼࣭ɺػೳ͕޲্
    ߴ౓ͳը૾ม׵ٕज़ʹ௅ઓͰ͖ΔΑ͏ʹ
    ImageFluxޙͷੈք
    

    View Slide

37. Live Streaming
    

    View Slide

38. pixiv Sketch LIVE
    ͓ֆ͔͖ΛΈΜͳͰϦΞϧλΠϜϥΠϒ഑৴
    

    View Slide

39. ϦϦʔε౰ॳͷ໰୊
    WebRTC SFU͔Β௚઀ࢹௌऀʹ഑৴
    ਺ઍਓͷಉ࣌ࢹௌΛ࣮ݱͰ͖ͳ͍
    WebRTC - HLSม׵Λ࣮૷Ͱ͖ͳ͚Ε͹࣍ʹਐΊͳ͍
    

    View Slide

40. WebRTC - HLS
    WebRTC SFU Sora
    Streamer
    Transcoder
    RTP forwarding
    spawn
    GPU
    MPEG-TS
    PIPE
    M3U8
    #
    " Cache
    ഑৴ऀ
    ࢹௌऀ
    

    View Slide

41. Live Streaming
    ߴ඼࣭ͳϥΠϒ഑৴Λखܰʹ࣮ݱ͢Δ

    ϦΞϧλΠϜಈըม׵
    

    View Slide

42. ϐΫγϒΛࢧ͖͑ͯͨ഑৴ٕज़
    ௿ίετɺߴ඼࣭ͳ഑৴Λ࣮ݱͨ͠Πϯϑϥ؀ڥ
    ࣗࣾ։ൃΛଓ͚͖ͯͨը૾ɺಈըม׵ٕज़
    

    View Slide

43. Future
    

    View Slide

44. ίϯςϯπ͸ΑΓϦονʹɺΑΓߴ඼࣭ʹ
    ΑΓԕ͘ͷɺΑΓͨ͘͞ΜͷϢʔβʔʹಧ͚ΒΕΔΑ͏ʹ
    

    View Slide

45. ߴ଎ɺେ༰ྔ഑৴ͷݚڀ։ൃ
    Format:

    VP9
    HEIF (HEVC)
    AV1
    Protocol:

    WebRTC
    LHLS
    Infrastructure:

    100G network
    ෳ਺Ϧʔδϣϯ
    CDNͱͷ౷߹
    Research:

    ػցֶश
    ετϦʔϜม׵
    

    View Slide

46. Fluent Delivery
    ͳΊΒ͔ͳ഑৴
    

    View Slide

47. 1ຕ͚ͩը૾͕ग़ͳͯ͘

    ΨΫοͱͳΔͷ͸͍͢͝ετϨε
    pixivΛӾཡͨ͠ͱ͖ʹ

    ετϨεͳ͘ӾཡͰ͖Δ
    ੈքதͰۉҰͳ඼࣭Λఏڙ͢Δ
    ͳΊΒ͔ͳ഑৴
    

    View Slide

48. ߴ଎
    େ༰ྔ
    ͳΊΒ͔
    

    View Slide

49. Infrastructure ImageFlux Live Streaming Future
    

    View Slide

50. ಧ͚͍ͨ΋ͷΛɺ͋ͨΓ·͑ʹಧ͚Δ
    

    View Slide

51. ͳΊΒ͔ͳ഑৴ٕज़
    ը૾ɾಈըม׵
    ϥΠϒετϦʔϛϯά
    ΤϯδχΞΛืू͍ͯ͠·͢ʂ
    

    View Slide