JPEG streaming in Go !IBSVLBTBO4IVOTVLF.JDIJJ ͋Δ͍͸+1&(ͷϒϩοΫϊΠζͱ͸ͳʹ͔

Harukasan Shunsuke MICHII Lead engineer of pixiv Inc.,
 Product manager of ImageFlux,
 Infrastructure engineer, Kibana, Gopher

No content

© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ ը૾ม׵ॲཧ ΦϦδφϧը૾Λऔಘ ม׵ޙΩϟογϡΛ഑৴ ΤϯυϢʔβ ։ൃऀ ഑৴URLͷઃఆ ഑৴ݩΦϦδϯͷઃఆ
 ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔ ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ׵͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม׵ ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚ ҙͷը૾ม׵Λߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾ม׵ॲཧ

© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ ը૾ม׵ॲཧ ΦϦδφϧը૾Λऔಘ ม׵ޙΩϟογϡΛ഑৴ ΤϯυϢʔβ ։ൃऀ ഑৴URLͷઃఆ ഑৴ݩΦϦδϯͷઃఆ
 ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδ΍αʔό͔ ΒҠಈ͢Δඞཁ͸͋Γ·ͤΜɻ؅ཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ׵͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม׵ ഑৴URLʹύϥϝʔλΛ௥Ճ͢Δ͚ͩͰ೚ ҙͷը૾ม׵Λߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ࢖͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ഑৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾ม׵ॲཧ

Decoding a JPEG image Resizing Encoding into a new JPEG image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

streaming in Go • io.Reader: reading content into the given buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }

• Copying content from a reader into a writer io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

• Copying content from a reader into a writer • To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ͸಺෦Ͱຖճmake

func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET", "http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) 
 io.CopyBuffer(w, resp.Body, buf) } HTTP΋ίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ͸……ʣ

Decoding a JPEG image Resizing Encoding into a new JPEG image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface { func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma

Decoding a JPEG image Resizing Encoding into a new JPEG image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo

Decoding a JPEG image Resizing Encoding into a new JPEG image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines

func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET", "http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }

streaming image scanlines • ScanlineReader interfaceΛ࢖͏͜ͱͰը૾σʔλ΋ετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ࢖͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ • ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔε͸Α͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ

ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬ΃Μʹॲཧ͢Δͱશըૉ෼ͷϝϞϦ͕ඞཁ͕ͩ
 ετϦʔϜॲཧ͢Ε͹όοϑΝΛҰ౓֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม׵͕ൃੜ͠ͳ͍
 JPEGಉ࢜ͷϦαΠζͷ৔߹͸Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGB΁ͷม׵ʹΑΔϩε͕ൃੜ͠ͳ͍ • cgo಺ͰϒϩοΫ͢Δ࣌ؒΛ୹͘͢Δ͜ͱ͕Ͱ͖Δ

Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢Ε͹JPEGΛετϦʔϜॲཧͰ͖Δ

color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline io.Reader RGB→Y'CbCrͷ৭ۭؒม׵ ΫϩϚαϒαϯϓϦϯάॲཧ DCTม׵ ྔࢠԽ ϋϑϚϯѹॖ

8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ
 ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵
 →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ
 →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप೾੒෼ ௿प೾੒෼

8x8ϐΫηϧ͝ͱʹDCTʢ཭ࢄίαΠϯม׵ʣͰ
 ௚ྲྀ੒෼ͱप೾਺͝ͱͷަྲྀ੒෼ʹม׵
 →ߴप೾੒෼΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰ৘ใྔΛ࡟ݮ
 →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप೾੒෼ ௿प೾੒෼

• JPEGͰ͸8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒ΂ͨ΋ͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰ৘ใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU

color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline io.Reader 8x8ըૉ͝ͱॲཧ͢Δ
 →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ

/* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ৔߹͸jpeg_read_scanlinesΛݺͿ͚ͩ

Conclusion • GoͰ͸ඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ৔߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ
 →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢Ε͹ͪΐͬͱͣͭॲཧͰ͖Δ
 →libjpeg͸ετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ͸8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ