CAOS 2015 Summer: Hayabusa Internals

http://hayabusa.io

Transcript

1. Hayabusa Internals
   Kohei Morino(森野耕平)
   

   View Slide

2. Tech overview
   

   View Slide

3. Hayabusa modules
   ● hayabusa
   ○ On-the-fly responsive image generation and delivery
   ○ Express-compatible Node.js middleware
   ○ Plug-in architecture
   ● hayabusa-server
   ○ Express-based web server using hayabusa
   

   View Slide

4. Hayabusa plug-ins
   ● Resizing
   ● Cropping
   ● Format conversion
   ● Optimization
   ● PSD layers and slices
   ● And more
   }Using
   ImageMagick
   

   View Slide

5. Flow
   ・・・
   Original
   image
   Output
   image
   Data
   stream
   1. Parse URL
   2. Arrange modules
   Request
   File path + Params
   Modules
   

   View Slide

6. architecture
   Hayabusa
   Cache Server
   (Redis)
   Load Balancer
   CDN
   User
   GET https://hayabusa.io/foo/img1.webp
   Original Images
   S3 or HTTP server
   foo
   bar
   http://xxxxxxx/img1.png
   ● Fetch image
   ● Check changes
   ● Processed image data
   ● Metadata
   Process
   image
   Config
   

   View Slide

7. Caching (CDN)
   ● Using CloudFront (for now)
   ● ‘Vary’ header: User-Agent, Accept
   ○ Cache used only when accessed by the same URL,
   User-Agent and Accept header
   ○ Result image differs according to user’s environment
   GET https://hayabusa.io/foo/img1.auto
   png, webp…
   progressive / baseline (jpg)
   

   View Slide

8. Caching (Redis)
   ● Use ElastiCache
   ● Cache processed images
   ○ LRU cache
   ● Cache metadata
   ○ Size
   ■ for JPEG progressive/baseline decision
   ○ Header (etag)
   ■ Use for detection of original image changes
   ○ Cache time, etc.
   

   View Slide

9. Creating cache
   ● Cache is created at the first access
   ○ It takes some time for processing the image
   ● Ahead-of-time caching is supported through
   API
   ○ Post URL list
   

   View Slide

10. Optimization
    ● Wrap various modules for compression
    ○ pngquant
    ■ Reduces file sizes often as much as 70%
    ○ optipng
    ■ PNG lossless compression
    ○ zopfli
    ■ Better deflate algorithm
    ○ gifsicle
    ■ Compression for gif
    ○ jpegrecompress
    ■ Compress JPEG by re-encoding to the smallest size while
    keeping visual quality
    

    View Slide

11. Progressive vs Baseline
    Hayabusa chooses JPEG
    format by file size automatically.
    “Images of file size 10K and
    over have a better chance of
    being smaller when using
    the progressive JPEG format”
    X-axis: original image size
    Y-axis: difference when subtracting
    progressive from baseline sizes
    quoted from: http://www.bookofspeed.com/chapter5.html
    

    View Slide

12. Cascading default parameters
    Priority
    1. Params in URL
    2. Default values in directory setting
    3. Default values in origin server setting
    4. Plug-in defaults
    https://hayabusa.io/foo/img1.width-200.optipng-3.pngquant(color-256).・・・.png
    https://hayabusa.io/foo/img1.width-200.png
    simplify URL with defaults
    

    View Slide

13. Roadmap
    1. Full PSD renderer
    2. 9-slices
    3. SpriteSheets / Atlas
    4. CLI for local usage
    5. GUI for designers
    6. Unity assets
    7. Sound, Movie, Font support
    

    View Slide

14. Thank you!
    hayabusa.io
    

    View Slide