Resizing Animated GIFs Without CGO or Third-Party Libraries

Slide 1

Slide 1 text

Takuto Nagami X: @logica0419 GitHub: @logica0419 Resizing Animated GIFs Without CGO or Third-Party Libraries

Slide 2

Slide 2 text

Why GIF resizing?

Slide 3

Slide 3 text

traQ Internal Messenger Application at traP (Student Community)

Slide 4

Slide 4 text

Animated icons & stamps (in traQ) ● Icons: enable users to express themselves better ● Stamps: more fun & live communication

Slide 5

Slide 5 text

De Facto Standard: CLI or CGO

Slide 6

Slide 6 text

Common choices ● ImageMagick ● MagickWand C API + CGO ● libvips + CGO

Slide 7

Slide 7 text

Problems https://www.docker.com/company/ https://www.irasutoya.com/2014/03/blog-post_2687.html

Slide 8

Slide 8 text

Let’s resize animated GIFs only with Pure Go! Ideally, only with standard libraries & sub repositories

Slide 9

Slide 9 text

What is GIF image?

Slide 10

Slide 10 text

● Image (not video or multimedia) ● Able to contain animation GIF ≓ Animated image https://illust55.com/1155/

Slide 11

Slide 11 text

What’s contained in the GIF image? Frame ↓ Image

Slide 12

Slide 12 text

What’s contained in the GIF image? ● In image ○ Image size (width, height) ○ Global color palette ● In frame ○ Image data ○ Delay ○ Color palette ○ Disposal

Slide 13

Slide 13 text

Processing images with Go

Slide 14

Slide 14 text

image/gif ● Standard library ○ https://pkg.go.dev/image/gif ● gif.DecodeAll() -> *gif.GIF struct type GIF struct { Image []*image.Paletted ← Each frame Delay []int ← For each frame LoopCount int Disposal []byte ← Painpoint Config image.Config ← Image size & color palette BackgroundIndex byte }

Slide 15

Slide 15 text

golang.org/x/image/draw ● One of sub repositories ● draw.Draw() function ○ Draw src image on dst image ○ op - draw.Src / draw.Over func Draw(dst Image, r image.Rectangle, src image.Image, sp image.Point, op Op)

Slide 16

Slide 16 text

golang.org/x/image/draw ● draw.Scaler interface ○ Able to resize a single image type Scaler interface { Scale(dst Image, dr image.Rectangle, src image.Image, sr image.Rectangle, op Op, opts *Options) }

Slide 17

Slide 17 text

The simplest GIF resizing

Slide 18

Slide 18 text

Prepare the destination GIF image dst := &gif.GIF{ Image: make([]*image.Paletted, 0, len(src.Image)), Delay: src.Delay, LoopCount: src.LoopCount, Disposal: src.Disposal, Config: image.Config{ ColorModel: src.Config.ColorModel, Width: width, Height: height, }, BackgroundIndex: src.BackgroundIndex, }

Slide 19

Slide 19 text

Resize each frames for _, srcFrame := range src.Image { srcFrameBounds := srcFrame.Bounds() dstFrameBounds := image.Rect(0, 0, width, height) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.CatmullRom.Scale(dstFrame, dstFrameBounds, srcFrame, srcFrameBounds, draw.Src, nil) dst.Image = append(dst.Image, dstFrame) }

Slide 20

Slide 20 text

However, there are 3 traps

Slide 21

Slide 21 text

Trap 1: Optimized GIF animation

Slide 22

Slide 22 text

Miku.gif https://piapro.jp/t/FB3J

Slide 23

Slide 23 text

After resizing ● Resized into the same size

Slide 24

Slide 24 text

Oh no! She becomes long!

Slide 25

Slide 25 text

Why? ● Due to delta frame optimization

Slide 26

Slide 26 text

Delta frame optimization ● If all frames are resized to dist size ○ frame 1: ○ frame 2~: 　

Slide 27

Slide 27 text

Again: resize each frames for _, srcFrame := range src.Image { srcFrameBounds := srcFrame.Bounds() dstFrameBounds := image.Rect(0, 0, width, height) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.CatmullRom.Scale(dstFrame, dstFrameBounds, srcFrame, srcFrameBounds, draw.Src, nil) dst.Image = append(dst.Image, dstFrame) }

Slide 28

Slide 28 text

That’s why!

Slide 29

Slide 29 text

Solution: Calculating each dst frame size

Slide 30

Slide 30 text

Calculating each dst frame size widthRatio := float64(width) / float64(srcWidth) heightRatio := float64(height) / float64(srcHeight) for _, srcFrame := range src.Image { dstFrameBounds := image.Rect( int(float64(srcFrameBounds.Min.X)*widthRatio), int(float64(srcFrameBounds.Min.Y)*heightRatio), int(float64(srcFrameBounds.Max.X)*widthRatio), int(float64(srcFrameBounds.Max.Y)*heightRatio), ) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) (Resize Image & append to dst) }

Slide 31

Slide 31 text

Calculating each dst frame size ● With frame size calculation ○ frame 1: ○ frame 2~: 　

Slide 32

Slide 32 text

Solved!

Slide 33

Slide 33 text

Trap 2: Interpolation

Slide 34

Slide 34 text

parapara.gif https://errand.jp/web/design/1023/

Slide 35

Slide 35 text

After resizing

Slide 36

Slide 36 text

What are these black jaggy noises?

Slide 37

Slide 37 text

To solve this, you need little knowledge about interpolation Let me explain

Slide 38

Slide 38 text

Abstract concept of interpolation ● Interpolation = mixing pixel colors ● Kernel = func to deﬁne how pixels mixed ○ Bicubic ○ lanczos2 / lanczos3 ● Mixing between transparent & non-transparent pixels seems to create noize

Slide 39

Slide 39 text

Seems like the noise happens when image has transparent pixels! Investigation still in progress…

Slide 40

Slide 40 text

Workaround: Piling up frames before resizing

Slide 41

Slide 41 text

Get transparent pixels out of mind Transparent pixels

Slide 42

Slide 42 text

GIF rendering: piling up frames Rendered previous frame Rendered current frame

Slide 43

Slide 43 text

What we resize now Rendered previous frame Rendered current frame

Slide 44

Slide 44 text

How about resizing this one? Rendered previous frame Rendered current frame

Slide 45

Slide 45 text

srcBounds := image.Rect(0, 0, srcWidth, srcHeight) dstBounds := image.Rect(0, 0, width, height) tempCanvas := image.NewNRGBA(srcBounds) Prepare canvas for rendering

Slide 46

Slide 46 text

Pile up current frame on canvas for _, srcFrame := range src.Image { (Calculate the dst frame size) draw.Draw(tempCanvas, srcFrameBounds, srcFrame, srcFrameBounds.Min, draw.Over)

Slide 47

Slide 47 text

Resize fittedFrame := image.NewPaletted(dstBounds, srcFrame.Palette) draw.CatmullRom.Scale(fittedFrame, dstBounds, tempCanvas, srcBounds, draw.Src, nil)

Slide 48

Slide 48 text

Trim (for delta frame optimization) dstFrame := image.NewPaletted(dstFrameBounds, srcFrame.Palette) draw.Draw(dstFrame, dstFrameBounds, fittedFrame, dstFrameBounds.Min, draw.Src) (Append resized frame to dst) }

Slide 49

Slide 49 text

But there is still a problem… https://patirabi.com/2021/10/10/061gif/

Slide 50

Slide 50 text

(Partially) Solved!

Slide 51

Slide 51 text

Trap 3: Disposal

Slide 52

Slide 52 text

Disposal ● Also called Frame Disposal Methods ○ How frames should be treated after rendering ● 3 types ○ 0x01: None ○ 0x02: Background ○ 0x03: Previous

Slide 53

Slide 53 text

0x01: Disposal None ● Do nothing, just go to next frame rendering https://legacy.imagemagick.org/Usage/anim_basics/

Slide 54

Slide 54 text

0x02: Disposal Background ● Overwrite area of frame with background color https://legacy.imagemagick.org/Usage/anim_basics/

Slide 55

Slide 55 text

● Delete current frame before next frame rendering ○ Rolling back to “previous” 0x03: Disposal Previous https://legacy.imagemagick.org/Usage/anim_basics/

Slide 56

Slide 56 text

Inappropriate disposal processing ● Mess up transparent background images https://sozai-good.com/illust/gifanimation/29065

Slide 57

Slide 57 text

Solution: Process Disposal Properly

Slide 58

Slide 58 text

bgColor := image.NewUniform( src.Config.ColorModel.(color.Palette)[src.BackgroundIndex]) for i, srcFrame := range src.Image { previous = deepCopyImage(tempCanvas) (Resizing process) switch src.Disposal[i] { case gif.DisposalNone: case gif.DisposalBackground: draw.Draw(tempCanvas, srcFrameBounds, bgColor, srcFrameBounds.Min, draw.Src) case gif.DisposalPrevious: draw.Draw(tempCanvas, srcBounds, previous, srcBounds.Min, draw.Src) } }

Slide 59

Slide 59 text

Solved!

Slide 60

Slide 60 text

A few more things to work… ● Retaining the aspect ratio ● Making resizing kernel selectable ● Concurrent processing

Slide 61

Slide 61 text

resigif ● https://github.com/logica0419/resigif ● Please use this & give me some feedback!

Slide 62

Slide 62 text

Results: ImageMagick CLI vs resigif

Slide 63

Slide 63 text

Performance assessment ● Experiment: 512px x 320px -> 256px x 160px ○ ImageMagick CLI ■ magick -ﬁlter Lanczos -scale 50% ○ CLI using resigif (single thread) ○ CLI using resigif (concurrent) ■ Using lanczos3 resizing kernel (same as ImageMagick)

Slide 64

Slide 64 text

Performance assessment ● Metrics ○ CPU: max utilization (top) ○ Memory: max utilization (top) ○ Time: using time command ■ Average of 5 trials

Slide 65

Slide 65 text

Performance assessment ● Result https://pixabay.com/gifs/cube-square-puzzle-rubiks-11588/ ImageMagick 678 KB resigif 512 KB

Slide 66

Slide 66 text

Performance assessment Max CPU Max Mem Time ImageMagick 100% 1.9% 4.64s resigif Single 120% 1.9% 4.79s resigif Concurrent 800% 1.9% 1.09s

Slide 67

Slide 67 text

● Just a reference: Two measures conbined ○ Deletion of ImageMagick ○ Migration to distroless base image ● 1/3 of the original Docker image size

Slide 68

Slide 68 text

End of the journey to resize animated GIFs only with Pure Go…

Slide 69

Slide 69 text

Special Thanks

Slide 70

Slide 70 text

Travel & hotel expenses sponsors ● Chiba Institute of Technology PPA ● pixiv Inc.

Slide 71

Slide 71 text

Reviewers / Sponsors / Supporters ● Reviewers ● Personal Sponsors ● Supporters