Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JPEG streaming in Go
Search
Harukasan
PRO
March 14, 2017
Technology
2
2.5k
JPEG streaming in Go
pixiv Night #02 - 画像処理技術(go, blender, C++ライブラリ等)
2017-03-14
https://pixiv.connpass.com/event/50284/
Harukasan
PRO
March 14, 2017
Tweet
Share
More Decks by Harukasan
See All by Harukasan
pixivを支える技術 / 技育CAMPアカデミア
harukasan
PRO
3
260
20240401 新卒研修 - ピクシブにおける技術領域
harukasan
PRO
1
610
ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON
harukasan
PRO
5
5.2k
Goにおける画像ファイル処理 / golang.tokyo #19
harukasan
PRO
7
6.3k
WebRTC動画をトランスコードする / Transcoding video streams from WebRTC
harukasan
PRO
5
1.4k
ImageFluxを支えるリモート開発 / 20171202
harukasan
PRO
2
1.7k
YAPC::Fukuoka 前夜祭LT / Yet Another Pawoo Commit logs
harukasan
PRO
0
2.8k
YAPC::Fukuoka lunch session
harukasan
PRO
1
2.9k
マストドン会議: Pawoo / Mastodon Kaigi2
harukasan
PRO
2
420
Other Decks in Technology
See All in Technology
サーバーレスAPI(API Gateway+Lambda)とNext.jsで 個人ブログを作ろう!
shuntaka
PRO
0
560
dxd2024-生成AIに振り回された3か月間の成功と失敗/dxd2024-link-and-motivation
lmi
2
260
スレットハンティングについて知っておきたいこと
hacket
0
130
ソフトウェアエンジニアリングの知見を活かして データ基盤をいい感じにする on Snowflake [MIERUNE BBQ #10]
mtpooh
2
150
データ分析を支える技術 生成AI再入門
ishikawa_satoru
0
380
フルリモートワークはエンジニアの夢を叶えたか? #cm_odyssey
mamohacy
2
600
DDDにおける認可の扱いとKotlinにおける実装パターン / authorization-for-ddd-and-kotlin-implement-pattern
urmot
4
390
「単なる OAuth 2.0 を認証に使うと、車が通れるほどのどでかいセキュリティー・ホールができる」のか検証してみた
terara
0
380
E2Eテスト自動化プラットフォームにおけるAIの活用
shift_evolve
0
190
楽しくGoを学び合う、LayerXの勉強会文化 / LayerX's study culture of having fun and learning Go together
ar_tama
2
350
GoとアクターモデルでES+CQRSを実践! / proto_actor_es_cqrs
ytake
1
160
Scaling Technical Excellence at 104: Evolution in AWS and Developer Empowerment
scotthsieh825
1
160
Featured
See All Featured
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
44
4.7k
How GitHub (no longer) Works
holman
305
140k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
24
1.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
226
52k
It's Worth the Effort
3n
181
27k
GitHub's CSS Performance
jonrohan
1026
450k
The Brand Is Dead. Long Live the Brand.
mthomps
52
36k
Art, The Web, and Tiny UX
lynnandtonic
291
20k
Faster Mobile Websites
deanohume
303
30k
Building Adaptive Systems
keathley
34
2k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
325
21k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
Transcript
JPEG streaming in Go !IBSVLBTBO4IVOTVLF.JDIJJ ͋Δ͍+1&(ͷϒϩοΫϊΠζͱͳʹ͔
Harukasan Shunsuke MICHII Lead engineer of pixiv Inc., Product manager
of ImageFlux, Infrastructure engineer, Kibana, Gopher
None
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
streaming in Go • io.Reader: reading content into the given
buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }
• Copying content from a reader into a writer io.Copy(dst,
src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ෦Ͱຖճmake
• Copying content from a reader into a writer •
To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ෦Ͱຖճmake
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) io.CopyBuffer(w, resp.Body, buf) } HTTPίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ……ʣ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface {
func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }
streaming image scanlines • ScanlineReader interfaceΛ͏͜ͱͰը૾σʔλετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ •
ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔεΑ͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ
ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬΜʹॲཧ͢ΔͱશըૉͷϝϞϦ͕ඞཁ͕ͩ ετϦʔϜॲཧ͢ΕόοϑΝΛҰ֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม͕ൃੜ͠ͳ͍ JPEGಉ࢜ͷϦαΠζͷ߹Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGBͷมʹΑΔϩε͕ൃੜ͠ͳ͍ • cgoͰϒϩοΫ͢Δ࣌ؒΛ͘͢Δ͜ͱ͕Ͱ͖Δ
Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config
→ plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢ΕJPEGΛετϦʔϜॲཧͰ͖Δ
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader RGB→Y'CbCrͷ৭ۭؒม ΫϩϚαϒαϯϓϦϯάॲཧ DCTม ྔࢠԽ ϋϑϚϯѹॖ
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
• JPEGͰ8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒͨͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader 8x8ըૉ͝ͱॲཧ͢Δ →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ
/* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int
stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ߹jpeg_read_scanlinesΛݺͿ͚ͩ
Conclusion • GoͰඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢ΕͪΐͬͱͣͭॲཧͰ͖Δ →libjpegετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ