Upgrade to Pro — share decks privately, control downloads, hide ads and more …

CAOS 2015 Summer: Hayabusa Internals

CAOS 2015 Summer: Hayabusa Internals

62a74a5cefb3f6f50cdcb5ed863fd73b?s=128

Gunther Brunner

July 07, 2015
Tweet

Transcript

  1. Hayabusa Internals Kohei Morino(森野耕平)

  2. Tech overview

  3. Hayabusa modules • hayabusa ◦ On-the-fly responsive image generation and

    delivery ◦ Express-compatible Node.js middleware ◦ Plug-in architecture • hayabusa-server ◦ Express-based web server using hayabusa
  4. Hayabusa plug-ins • Resizing • Cropping • Format conversion •

    Optimization • PSD layers and slices • And more }Using ImageMagick
  5. Flow ・・・ Original image Output image Data stream 1. Parse

    URL 2. Arrange modules Request File path + Params Modules
  6. architecture Hayabusa Cache Server (Redis) Load Balancer CDN User GET

    https://hayabusa.io/foo/img1.webp Original Images S3 or HTTP server foo bar http://xxxxxxx/img1.png • Fetch image • Check changes • Processed image data • Metadata Process image Config
  7. Caching (CDN) • Using CloudFront (for now) • ‘Vary’ header:

    User-Agent, Accept ◦ Cache used only when accessed by the same URL, User-Agent and Accept header ◦ Result image differs according to user’s environment GET https://hayabusa.io/foo/img1.auto png, webp… progressive / baseline (jpg)
  8. Caching (Redis) • Use ElastiCache • Cache processed images ◦

    LRU cache • Cache metadata ◦ Size ▪ for JPEG progressive/baseline decision ◦ Header (etag) ▪ Use for detection of original image changes ◦ Cache time, etc.
  9. Creating cache • Cache is created at the first access

    ◦ It takes some time for processing the image • Ahead-of-time caching is supported through API ◦ Post URL list
  10. Optimization • Wrap various modules for compression ◦ pngquant ▪

    Reduces file sizes often as much as 70% ◦ optipng ▪ PNG lossless compression ◦ zopfli ▪ Better deflate algorithm ◦ gifsicle ▪ Compression for gif ◦ jpegrecompress ▪ Compress JPEG by re-encoding to the smallest size while keeping visual quality
  11. Progressive vs Baseline Hayabusa chooses JPEG format by file size

    automatically. “Images of file size 10K and over have a better chance of being smaller when using the progressive JPEG format” X-axis: original image size Y-axis: difference when subtracting progressive from baseline sizes quoted from: http://www.bookofspeed.com/chapter5.html
  12. Cascading default parameters Priority 1. Params in URL 2. Default

    values in directory setting 3. Default values in origin server setting 4. Plug-in defaults https://hayabusa.io/foo/img1.width-200.optipng-3.pngquant(color-256).・・・.png https://hayabusa.io/foo/img1.width-200.png simplify URL with defaults
  13. Roadmap 1. Full PSD renderer 2. 9-slices 3. SpriteSheets /

    Atlas 4. CLI for local usage 5. GUI for designers 6. Unity assets 7. Sound, Movie, Font support
  14. Thank you! hayabusa.io