Upgrade to Pro — share decks privately, control downloads, hide ads and more …

JPEG streaming in Go

JPEG streaming in Go

pixiv Night #02 - 画像処理技術(go, blender, C++ライブラリ等)
2017-03-14
https://pixiv.connpass.com/event/50284/

Harukasan

March 14, 2017
Tweet

More Decks by Harukasan

Other Decks in Technology

Transcript

  1. JPEG streaming in Go
    !IBSVLBTBO4IVOTVLF.JDIJJ
    ͋Δ͍͸+1&(ͷϒϩοΫϊΠζͱ͸ͳʹ͔

    View full-size slide

  2. Harukasan
    Shunsuke MICHII
    Lead engineer of pixiv Inc.,

    Product manager of ImageFlux,

    Infrastructure engineer, Kibana, Gopher

    View full-size slide

  3. © pixiv Inc., SAKURA Internet Inc.
    ImageFlux service
    ͓٬༷ετϨʔδ
    ίϯςϯπΩϟογϡ
    ը૾ม׵ॲཧ
    ΦϦδφϧը૾Λऔಘ
    ม׵ޙΩϟογϡΛ഑৴
    ΤϯυϢʔβ
    ։ൃऀ
    ഑৴URLͷઃఆ
    ഑৴ݩΦϦδϯͷઃఆ

    ϨϙʔτͷӾཡ
    طଘετϨʔδΛͦͷ··ར༻Մೳ
    ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔
    ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔
    ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม
    ׵͕ར༻ՄೳͰ͢ɻ
    URLΛม͑Δ͚ͩͰը૾ม׵
    ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚
    ҙͷը૾ม׵Λߦ͑·͢ɻ
    εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜
    ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾
    Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ
    ը૾ม׵ॲཧ

    View full-size slide

  4. © pixiv Inc., SAKURA Internet Inc.
    ImageFlux service
    ͓٬༷ετϨʔδ
    ίϯςϯπΩϟογϡ
    ը૾ม׵ॲཧ
    ΦϦδφϧը૾Λऔಘ
    ม׵ޙΩϟογϡΛ഑৴
    ΤϯυϢʔβ
    ։ൃऀ
    ഑৴URLͷઃఆ
    ഑৴ݩΦϦδϯͷઃఆ

    ϨϙʔτͷӾཡ
    طଘετϨʔδΛͦͷ··ར༻Մೳ
    ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔
    ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔
    ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม
    ׵͕ར༻ՄೳͰ͢ɻ
    URLΛม͑Δ͚ͩͰը૾ม׵
    ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚
    ҙͷը૾ม׵Λߦ͑·͢ɻ
    εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜
    ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾
    Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ
    ը૾ม׵ॲཧ

    View full-size slide

  5. Decoding a JPEG image
    Resizing
    Encoding into a new JPEG image
    Fetching from an origin
    Writing content into a client
    http.Transport
    libjpeg-turbo
    libswscale from ffmpeg
    http.ResponseWriter
    libjpeg-turbo

    View full-size slide

  6. streaming in Go
    • io.Reader: reading content into the given buffer
    • io.Writer: writing content from the given buffer
    typedef Reader interface {
    func Read(p []byte) (n int, err error)
    }
    typedef Writer interface {
    func Write(p []byte) (n int, err error)
    }

    View full-size slide

  7. • Copying content from a reader into a writer
    io.Copy(dst, src)
    32KB buffer
    io.Reader
    io.Writer
    Read(buffer)
    Write(buffer)
    : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

    View full-size slide

  8. • Copying content from a reader into a writer
    • To avoid allocating a buffer, use io.CopyBuffer:
    io.Copy(dst, src)
    32KB buffer
    io.Reader
    io.Writer
    Read(buffer)
    Write(buffer)
    io.CopyBuffer(dst, src, buf)
    : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

    View full-size slide

  9. func ServeHTTP(w *http.ResponseWriter, r *http.Request) {
    req, _ := http.NewRequest("GET", "http://www.example.com", nil)
    resp, _ := http.DefaultTransport.RoundTrip(req)

    io.CopyBuffer(w, resp.Body, buf)
    }
    HTTP΋ίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ͸……ʣ

    View full-size slide

  10. Decoding a JPEG image
    Resizing
    Encoding into a new JPEG image
    Fetching from an origin
    Writing content into a client
    http.Transport
    libjpeg-turbo
    libswscale from ffmpeg
    http.ResponseWriter
    libjpeg-turbo

    View full-size slide

  11. Streaming image scanlines
    • ScanlineReader interface
    typedef ScanlineReader interface {
    func Config() *Config → plane sizes, pixel format,...
    func ReadScanLine(p [][]uint8) (n int, err error)
    → reading scanlines of planes into each buffer
    → PNG: JPEG:
    }
    R
    G
    B
    Y
    Cb
    Cr
    *blue difference chroma
    *red difference chroma

    View full-size slide

  12. Decoding a JPEG image
    Resizing
    Encoding into a new JPEG image
    Fetching from an origin
    Writing content into a client
    http.Transport
    libjpeg-turbo
    libswscale from ffmpeg
    http.ResponseWriter
    libjpeg-turbo

    View full-size slide

  13. Decoding a JPEG image
    Resizing
    Encoding into a new JPEG image
    Fetching from an origin
    Writing content into a client
    http.Transport
    Decoder.ReadScanLines
    Scaler.ReadScanLines
    http.ResponseWriter
    Encoder.ReadScanLines

    View full-size slide

  14. func ServeHTTP(w *http.ResponseWriter, r *http.Request) {
    req, _ := http.NewRequest("GET", "http://www.example.com", nil)
    resp, _ := http.DefaultTransport.RoundTrip(req)
    dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader
    dec := scale.NewResize(dec ScanlineReader) → ScanlineReader
    enc := jpeg.NewEncoder()
    enc.Write(w io.Writer, dec ScanlineReader)
    }

    View full-size slide

  15. streaming image scanlines
    • ScanlineReader interfaceΛ࢖͏͜ͱͰը૾σʔλ΋ετϦʔϜͱ͠
    ͯѻ͏͜ͱ͕Ͱ͖Δ
    • interfaceΛ࢖͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ
    • ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔε͸Α͘ߟ͑ΒΕͯΔͷͰϚω͢Δ
    ͱྑ͍ײ͡ʹͳΔ

    View full-size slide

  16. ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔
    • ը૾σʔλΛ͍ͬ΃Μʹॲཧ͢Δͱશըૉ෼ͷϝϞϦ͕ඞཁ͕ͩ

    ετϦʔϜॲཧ͢Ε͹όοϑΝΛҰ౓֬อ͢Δ͚ͩͰࡁΉ
    • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม׵͕ൃੜ͠ͳ͍

    JPEGಉ࢜ͷϦαΠζͷ৔߹͸Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ
    RGB΁ͷม׵ʹΑΔϩε͕ൃੜ͠ͳ͍
    • cgo಺ͰϒϩοΫ͢Δ࣌ؒΛ୹͘͢Δ͜ͱ͕Ͱ͖Δ

    View full-size slide

  17. Streaming JPEG scanlines
    typedef ScanlineReader interface {
    func Config() *Config → plane sizes, pixel format,...
    func ReadScanLine(p [][]uint8) (n int, err error)
    → JPEG:
    }
    Y
    Cb
    Cr
    *blue difference chroma
    *red difference chroma
    ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢Ε͹JPEGΛετϦʔϜॲཧͰ͖Δ

    View full-size slide

  18. color conversion
    chroma subsampling
    huffman coding
    quantization
    DCT transformation
    ReadScanline
    io.Reader
    RGB→Y'CbCrͷ৭ۭؒม׵
    ΫϩϚαϒαϯϓϦϯάॲཧ
    DCTม׵
    ྔࢠԽ
    ϋϑϚϯѹॖ

    View full-size slide

  19. 8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ

    ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵

    →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ

    →ϋϑϚϯූ߸Խͯ͠ѹॖ
    ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ
    ߴप೾੒෼
    ௿प೾੒෼

    View full-size slide

  20. 8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ

    ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵

    →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ

    →ϋϑϚϯූ߸Խͯ͠ѹॖ
    ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ
    ߴप೾੒෼
    ௿प೾੒෼

    View full-size slide

  21. • JPEGͰ͸8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏
    • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒ΂ͨ΋ͷ
    • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰ৘ใྔΛམͱ͢ͷͰϒϩοΫڥք
    ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ
    MCU

    View full-size slide

  22. color conversion
    chroma subsampling
    huffman coding
    quantization
    DCT transformation
    ReadScanline
    io.Reader
    8x8ըૉ͝ͱॲཧ͢Δ

    →ετϦʔϛϯάॲཧͰ͖Δʂ
    1ըૉ͝ͱʹॲཧͰ͖Δ

    View full-size slide

  23. /*
    ...
    static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int stride, int height) {
    JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height);
    int i;
    for (i = 0; i < height; i++) {
    rows[i] = &buf[i * stride];
    }
    return jpeg_read_scanlines(dinfo, rows, height);
    }
    */
    import "c"
    func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) {
    n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height);
    return n, nil
    }
    libjpegͷ৔߹͸jpeg_read_scanlinesΛݺͿ͚ͩ

    View full-size slide

  24. Conclusion
    • GoͰ͸ඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ
    • JPEGͷ৔߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ

    →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢Ε͹ͪΐͬͱͣͭॲཧͰ͖Δ

    →libjpeg͸ετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢
    • ϒϩοΫϊΠζͱ͸8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ

    View full-size slide