JPEG streaming in Go

Transcript

1. JPEG streaming in Go !IBSVLBTBO
   
   4IVOTVLF
   .JDIJJ
   
   ͋Δ͍͸+1&(ͷϒϩοΫϊΠζͱ͸ͳʹ͔

2. Harukasan Shunsuke MICHII Lead engineer of pixiv Inc.,
 Product manager

   of ImageFlux,
 Infrastructure engineer, Kibana, Gopher
3. None

4. © pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ

   ը૾ม׵ॲཧ ΦϦδφϧը૾Λऔಘ ม׵ޙΩϟογϡΛ഑৴ ΤϯυϢʔβ ։ൃऀ ഑৴URLͷઃఆ ഑৴ݩΦϦδϯͷઃఆ
 ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔ ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ׵͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม׵ ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚ ҙͷը૾ม׵Λߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾ม׵ॲཧ

5. © pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ

   ը૾ม׵ॲཧ ΦϦδφϧը૾Λऔಘ ม׵ޙΩϟογϡΛ഑৴ ΤϯυϢʔβ ։ൃऀ ഑৴URLͷઃఆ ഑৴ݩΦϦδϯͷઃఆ
 ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔ ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ׵͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม׵ ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚ ҙͷը૾ม׵Λߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾ม׵ॲཧ

6. Decoding a JPEG image Resizing Encoding into a new JPEG

   image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

7. streaming in Go • io.Reader: reading content into the given

   buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }

8. • Copying content from a reader into a writer io.Copy(dst,

   src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

9. • Copying content from a reader into a writer •

   To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

10. func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",

    "http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) 
 io.CopyBuffer(w, resp.Body, buf) } HTTP΋ίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ͸……ʣ

11. Decoding a JPEG image Resizing Encoding into a new JPEG

    image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

12. Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface {

    func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma

13. Decoding a JPEG image Resizing Encoding into a new JPEG

    image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

14. Decoding a JPEG image Resizing Encoding into a new JPEG

    image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines

15. func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",

    "http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }

16. streaming image scanlines • ScanlineReader interfaceΛ࢖͏͜ͱͰը૾σʔλ΋ετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ࢖͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ •

    ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔε͸Α͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ

17. ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬ΃Μʹॲཧ͢Δͱશըૉ෼ͷϝϞϦ͕ඞཁ͕ͩ
 ετϦʔϜॲཧ͢Ε͹όοϑΝΛҰ౓֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม׵͕ൃੜ͠ͳ͍
 JPEGಉ࢜ͷϦαΠζͷ৔߹͸Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGB΁ͷม׵ʹΑΔϩε͕ൃੜ͠ͳ͍ • cgo಺ͰϒϩοΫ͢Δ࣌ؒΛ୹͘͢Δ͜ͱ͕Ͱ͖Δ

18. Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config

    → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢Ε͹JPEGΛετϦʔϜॲཧͰ͖Δ

19. color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline

    io.Reader RGB→Y'CbCrͷ৭ۭؒม׵ ΫϩϚαϒαϯϓϦϯάॲཧ DCTม׵ ྔࢠԽ ϋϑϚϯѹॖ

20. 8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ
 ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵
 →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ
 →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप೾੒෼ ௿प೾੒෼

21. 8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ
 ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵
 →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ
 →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप೾੒෼ ௿प೾੒෼

22. • JPEGͰ͸8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒ΂ͨ΋ͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰ৘ใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU

23. color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline

    io.Reader 8x8ըૉ͝ͱॲཧ͢Δ
 →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ

24. /* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int

    stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ৔߹͸jpeg_read_scanlinesΛݺͿ͚ͩ

25. Conclusion • GoͰ͸ඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ৔߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ
 →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢Ε͹ͪΐͬͱͣͭॲཧͰ͖Δ
 →libjpeg͸ετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ͸8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ