Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
JPEG streaming in Go
Search
Harukasan
PRO
March 14, 2017
Technology
2
2.5k
JPEG streaming in Go
pixiv Night #02 - 画像処理技術(go, blender, C++ライブラリ等)
2017-03-14
https://pixiv.connpass.com/event/50284/
Harukasan
PRO
March 14, 2017
Tweet
Share
More Decks by Harukasan
See All by Harukasan
20240401 新卒研修 - ピクシブにおける技術領域
harukasan
PRO
1
560
ピクシブのコンテンツ配信基盤技術 / pixiv TECH SALON
harukasan
PRO
5
5.1k
Goにおける画像ファイル処理 / golang.tokyo #19
harukasan
PRO
7
6.2k
WebRTC動画をトランスコードする / Transcoding video streams from WebRTC
harukasan
PRO
5
1.4k
ImageFluxを支えるリモート開発 / 20171202
harukasan
PRO
2
1.7k
YAPC::Fukuoka 前夜祭LT / Yet Another Pawoo Commit logs
harukasan
PRO
0
2.8k
YAPC::Fukuoka lunch session
harukasan
PRO
1
2.9k
マストドン会議: Pawoo / Mastodon Kaigi2
harukasan
PRO
2
410
大規模Mastodonインスタンスを運用するコツ / Inside Pawoo Mastodon infrastructure
harukasan
PRO
0
2.9k
Other Decks in Technology
See All in Technology
The depthes of profiling Ruby - RubyKaigi 2024
osyoyu
0
120
【TSkaigi】2024/05/11 当日スライド
kimitashoichi
14
3.9k
From here to resilience - a travel guide
ufried
1
160
QA経験のないエンジニアリング マネージャーがQAのカジュアル面談に出て 苦労していること・気づいたこと / scrum fest niigata 2024
yoshikiiida
2
640
Google Cloudを組織(企業)で運用する時のベストプラクティス × 健康の環境分離戦略 #まるクラ勉強会
yasumuusan
0
160
Autonomous Database Cloud 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
15
36k
Databricksの生成AI戦略
taka_aki
1
360
生成AIと産業向けソフトウェアの自動生成 〜 ハノーバーメッセ2024より〜
kioto
2
420
パスワードを保存しますか?
hanacchi
0
250
20240509 CloudWatch でいろいろなものを監視してみよう
masaruogura
1
110
DevRelによる信頼構築とデータ駆動で変わるエンジニア採用 / DevRel Trust Building to Data Driven Engineering Hiring
bobtani
1
130
技術力の伸ばし方を考える
khirata
0
140
Featured
See All Featured
Designing with Data
zakiwarfel
96
4.8k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
No one is an island. Learnings from fostering a developers community.
thoeni
16
2.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
15
1.6k
From Idea to $5000 a Month in 5 Months
shpigford
377
45k
Rails Girls Zürich Keynote
gr2m
91
13k
Optimizing for Happiness
mojombo
370
69k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
34
8.9k
Clear Off the Table
cherdarchuk
86
310k
GraphQLとの向き合い方2022年版
quramy
33
12k
The Cost Of JavaScript in 2023
addyosmani
21
4k
Transcript
JPEG streaming in Go !IBSVLBTBO4IVOTVLF.JDIJJ ͋Δ͍+1&(ͷϒϩοΫϊΠζͱͳʹ͔
Harukasan Shunsuke MICHII Lead engineer of pixiv Inc., Product manager
of ImageFlux, Infrastructure engineer, Kibana, Gopher
None
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
© pixiv Inc., SAKURA Internet Inc. ImageFlux service ͓٬༷ετϨʔδ ίϯςϯπΩϟογϡ
ը૾มॲཧ ΦϦδφϧը૾Λऔಘ มޙΩϟογϡΛ৴ ΤϯυϢʔβ ։ൃऀ ৴URLͷઃఆ ৴ݩΦϦδϯͷઃఆ ϨϙʔτͷӾཡ طଘετϨʔδΛͦͷ··ར༻Մೳ ը૾σʔλΛطଘͷετϨʔδαʔό͔ ΒҠಈ͢Δඞཁ͋Γ·ͤΜɻཧը໘͔ ΒΦϦδϯΛઃఆ͢Δ͚ͩͰ͙͢ʹը૾ม ͕ར༻ՄೳͰ͢ɻ URLΛม͑Δ͚ͩͰը૾ม ৴URLʹύϥϝʔλΛՃ͢Δ͚ͩͰ ҙͷը૾มΛߦ͑·͢ɻ εϚʔτϑΥϯɺPCͰผʑͷURLΛ͏͜ ͱͰɺ1ຕͷը૾͔Β༷ʑͳαΠζͷը૾ Λ৴͢Δ͜ͱ͕ՄೳͰ͢ɻ ը૾มॲཧ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
streaming in Go • io.Reader: reading content into the given
buffer • io.Writer: writing content from the given buffer typedef Reader interface { func Read(p []byte) (n int, err error) } typedef Writer interface { func Write(p []byte) (n int, err error) }
• Copying content from a reader into a writer io.Copy(dst,
src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) : 32KBͷόοϑΝ෦Ͱຖճmake
• Copying content from a reader into a writer •
To avoid allocating a buffer, use io.CopyBuffer: io.Copy(dst, src) 32KB buffer io.Reader io.Writer Read(buffer) Write(buffer) io.CopyBuffer(dst, src, buf) : 32KBͷόοϑΝ෦Ͱຖճmake
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) io.CopyBuffer(w, resp.Body, buf) } HTTPίϐʔ͢Δ͚ͩͰετϦʔϛϯάͰ͖ΔʢΤϥʔॲཧ……ʣ
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Streaming image scanlines • ScanlineReader interface typedef ScanlineReader interface {
func Config() *Config → plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → reading scanlines of planes into each buffer → PNG: JPEG: } R G B Y Cb Cr *blue difference chroma *red difference chroma
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport libjpeg-turbo libswscale from ffmpeg http.ResponseWriter libjpeg-turbo
Decoding a JPEG image Resizing Encoding into a new JPEG
image Fetching from an origin Writing content into a client http.Transport Decoder.ReadScanLines Scaler.ReadScanLines http.ResponseWriter Encoder.ReadScanLines
func ServeHTTP(w *http.ResponseWriter, r *http.Request) { req, _ := http.NewRequest("GET",
"http://www.example.com", nil) resp, _ := http.DefaultTransport.RoundTrip(req) dec := jpeg.NewDecoder(resp.Body io.Reader) → ScanlineReader dec := scale.NewResize(dec ScanlineReader) → ScanlineReader enc := jpeg.NewEncoder() enc.Write(w io.Writer, dec ScanlineReader) }
streaming image scanlines • ScanlineReader interfaceΛ͏͜ͱͰը૾σʔλετϦʔϜͱ͠ ͯѻ͏͜ͱ͕Ͱ͖Δ • interfaceΛ͏͜ͱͰ֤ॲཧͷར༻ํ๏Λ౷ҰͰ͖Δ •
ඪ४ϥΠϒϥϦͷΠϯλʔϑΣʔεΑ͘ߟ͑ΒΕͯΔͷͰϚω͢Δ ͱྑ͍ײ͡ʹͳΔ
ͳͥετϦʔϛϯά͢Δඞཁ͕͋Δͷ͔ • ը૾σʔλΛ͍ͬΜʹॲཧ͢ΔͱશըૉͷϝϞϦ͕ඞཁ͕ͩ ετϦʔϜॲཧ͢ΕόοϑΝΛҰ֬อ͢Δ͚ͩͰࡁΉ • ֤৭ۭؒຖʹॲཧΛهड़Ͱ͖ɺແବͳ৭ม͕ൃੜ͠ͳ͍ JPEGಉ࢜ͷϦαΠζͷ߹Y'CbCr৭ۭؒͷ··ѻ͏͜ͱͰɺ RGBͷมʹΑΔϩε͕ൃੜ͠ͳ͍ • cgoͰϒϩοΫ͢Δ࣌ؒΛ͘͢Δ͜ͱ͕Ͱ͖Δ
Streaming JPEG scanlines typedef ScanlineReader interface { func Config() *Config
→ plane sizes, pixel format,... func ReadScanLine(p [][]uint8) (n int, err error) → JPEG: } Y Cb Cr *blue difference chroma *red difference chroma ͭ·Γɺ͜ͷΠϯλʔϑΣʔεͰJPEGΛσίʔυ͢ΕJPEGΛετϦʔϜॲཧͰ͖Δ
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader RGB→Y'CbCrͷ৭ۭؒม ΫϩϚαϒαϯϓϦϯάॲཧ DCTม ྔࢠԽ ϋϑϚϯѹॖ
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
8x8ϐΫηϧ͝ͱʹDCTʢࢄίαΠϯมʣͰ ྲྀͱप͝ͱͷަྲྀʹม →ߴप΄Ͳߥ͘ͳΔΑ͏ʹྔࢠԽ͢Δ͜ͱͰใྔΛݮ →ϋϑϚϯූ߸Խͯ͠ѹॖ ΊͬͪΌࡶʹJPEGͷѹॖΛઆ໌͢Δͱ ߴप प
• JPEGͰ8x8ϐΫηϧʢ1ϒϩοΫʣ͝ͱʹDCTɺྔࢠԽΛߦ͏ • MCU: 8x8ըૉ͝ͱʹY'CbCrΛฒͨͷ • ͜ͷ8x8ըૉ͝ͱʹDCT/ྔࢠԽͰใྔΛམͱ͢ͷͰϒϩοΫڥք ʹϊΠζ͕ൃੜˠϒϩοΫϊΠζ MCU
color conversion chroma subsampling huffman coding quantization DCT transformation ReadScanline
io.Reader 8x8ըૉ͝ͱॲཧ͢Δ →ετϦʔϛϯάॲཧͰ͖Δʂ 1ըૉ͝ͱʹॲཧͰ͖Δ
/* ... static JDIMENSION read_scanlines(j_decompress_ptr dinfo, unsigned char *buf, int
stride, int height) { JSAMPROW *rows = alloca(sizeof(JSAMPROW) * height); int i; for (i = 0; i < height; i++) { rows[i] = &buf[i * stride]; } return jpeg_read_scanlines(dinfo, rows, height); } */ import "c" func (dec *Decoder) ReadScanline(p [][]uint8) (n int, err error) { n := C.read_scanline(dec.dinfo, p, dec.Stride, dec.Height); return n, nil } libjpegͷ߹jpeg_read_scanlinesΛݺͿ͚ͩ
Conclusion • GoͰඇৗʹ؆୯ʹετϦʔϜॲཧΛ࣮ݱͰ͖Δ • JPEGͷ߹8ߦͣͭσίʔυ/Τϯίʔυ͢Δ͜ͱ͕Ͱ͖Δ →ετϦʔϜͰ͖ΔΠϯλʔϑΣʔε͢ΕͪΐͬͱͣͭॲཧͰ͖Δ →libjpegετϦʔϜͰ͖ΔAPIΛ࣋ͬͯΔͷͰcgoͰݺͼग़͚ͩ͢ • ϒϩοΫϊΠζͱ8x8ըૉ͝ͱʹॲཧ͢ΔͷͰڥքʹग़ΔϊΠζ