Slide 1

Slide 1 text

Handling File Uploads For a modern developer

Slide 2

Slide 2 text

2007 Paperclip 2008 CarrierWave 2014 Refile 2018 ActiveStorage 2009 Dragonfly 2015 ! Shrine

Slide 3

Slide 3 text

Janko Marohnic • from Croatia ", living in Czechia # • Ruby off Rails developer • creator of Shrine @janko @jankomarohnic

Slide 4

Slide 4 text

Active Storage?

Slide 5

Slide 5 text

Active Storage Shrine Rails & Active Record only Rails, Hanami, Sinatra, Roda... Active Record, Sequel, ROM... framework library integrated experience more control more opinionated more features

Slide 6

Slide 6 text

Metadata & Validation

Slide 7

Slide 7 text

class PhotosController < AC::Base def create photo = Photo.new(photo_params) if photo.valid? photo.save # ... else # ... end end end

Slide 8

Slide 8 text

class ImageUploader < Shrine TYPES = %w[image/jpeg image/png ...] EXTENSIONS = %w[jpg jpeg png ...] Attacher.validate do validate_max_size 10*1024*1024 validate_mime_type TYPES validate_extension EXTENSIONS end end

Slide 9

Slide 9 text

\377\330\377························· (JPEG) \211PNG······························ (PNG) ········WEBP························· (WebP) MIME Type `file --mime-type -b #{file.path}` #=> "image/jpeg" MimeMagic.by_magic(file).type #=> "image/jpeg" Marcel::MimeType.for(file) #=> "image/jpeg"

Slide 10

Slide 10 text

class PhotosController < AC::Base def create file = params[:photo][:image] file.read #=> "" file.filename #=> "nature.jpg" file.content_type #=> "image/jpeg" photo = Photo.new(image: file) photo.image.mime_type #=> "text/x-php" photo.valid? #=> false end end MIME Type

Slide 11

Slide 11 text

https://www.bamsoftware.com/hacks/deflate.html $ identify spark.png ... 225000x225000 ... 6.1MB Attacher.validate do # ... if validate_mime_type IMAGE_TYPES validate_max_dimensions [5000, 5000] end end Image Dimensions

Slide 12

Slide 12 text

class VideoUploader < Shrine add_metadata :duration do |file| FFMPEG::Movie.new(file.path).duration end Attacher.validate do if file.duration > 5*60*60 errors << "is too long (max is 5h)" end end end Custom Metadata

Slide 13

Slide 13 text

{ "id": "b93777246f10e509a553.mp4", "storage": "store", "metadata": { "size": 234837, "filename": "matrix.mp4", "mime_type": "video/mp4", "duration": 9000, ... } } SELECT video_data FROM movies;

Slide 14

Slide 14 text

Processing

Slide 15

Slide 15 text

ImageMagick

Slide 16

Slide 16 text

image = MiniMagick::Image.open(file) image.resize("500x500") image.path #=> "/path/to/resized.jpg" $ gem install mini_magick

Slide 17

Slide 17 text

Resize to fit Resize to limit Resize to fill

Slide 18

Slide 18 text

magick = ImageProcessing::MiniMagick
 magick = magick.source(image) magick.resize_to_limit! 500, 500 magick.resize_to_fit! 500, 500 magick.resize_to_fill! 500, 500 magick.resize_and_pad! 500, 500 $ gem install image_processing

Slide 19

Slide 19 text

thumbnail = ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ gem install image_processing $ convert source.jpg -auto-orient -resize 500x500> -sharpen 0x1 destination.jpg

Slide 20

Slide 20 text

thumbnail = ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ gem install image_processing $ convert source.jpg -auto-orient -resize 500x500> -sharpen 0x1 destination.jpg

Slide 21

Slide 21 text

libvips⚡

Slide 22

Slide 22 text

ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ brew install imagemagick

Slide 23

Slide 23 text

ImageProcessing::Vips
 .source(image) .resize_to_limit!(500, 500) $ brew install vips

Slide 24

Slide 24 text

THUMBNAILS = { xs: [200, 200], s: [400, 400], m: [600, 600], l: [800, 800], xl: [1200, 1200], } THUMBNAILS.each do |name, (width, height)| ImageProcessing::Backend .source(original) .resize_to_limit!(width, height) end

Slide 25

Slide 25 text

0,47s 2,15s ImageMagick libvips

Slide 26

Slide 26 text

Rails 6.0

Slide 27

Slide 27 text

Processing On-the-fly

Slide 28

Slide 28 text

Active Storage https://...//... photo.image.variant( resize_to_limit: [600, 400], strip: true, quality: 85, interlace: "JPEG", )

Slide 29

Slide 29 text

Shrine https://.../thumbnail/600/400/... photo.image.derivation_url( :thumbnail, # <-- name 600, 400 # <-- args ) Shrine.derivation :thumbnail do |file, w, h| ImageProcessing::Vips.source(file) .strip .saver(quality: 85, interlace: "JPEG") .resize_to_limit!(w.to_i, h.to_i) end

Slide 30

Slide 30 text

Processing on attachment

Slide 31

Slide 31 text

class ImageUploader < Shrine derivatives_processor :thumbs do |file| vips = ImageProcessing::Vips.source(file) { large: vips.resize_to_fit!(800, 800), medium: vips.resize_to_fit!(500, 500), small: vips.resize_to_fit!(300, 300) } end end photo = Photo.new(image: file) photo.image_derivatives!(:thumbs) photo.image_derivatives #=> # { large: #, # medium: #, # small: # }

Slide 32

Slide 32 text

class VideoUploader < Shrine derivatives_processor :transcode do |file| movie = FMMPEG::Movie.new(file.path) transcoded = Tempfile.new(["", ".mp4"]) screenshot = Tempfile.new(["", ".jpg"]) movie.transcode(transcoded.path) movie.screenshot(screenshot.path) { transcoded: transcoded, screenshot: screenshot } end end

Slide 33

Slide 33 text

Direct Uploads

Slide 34

Slide 34 text

refile.js activestorage.js shrine.js ? ❌

Slide 35

Slide 35 text

jQuery-File-Upload Dropzone.js FineUploader ❌ ❌ ❌

Slide 36

Slide 36 text

Uppy

Slide 37

Slide 37 text

Uppy() .use(Dashboard, { /* ... */ })

Slide 38

Slide 38 text

Uppy() .use(DragDrop, { /* ... */ }) .use(StatusBar, { /* ... */ })

Slide 39

Slide 39 text

Uppy() .use(FileInput, { /* ... */ }) .use(ProgressBar, { /* ... */ })

Slide 40

Slide 40 text

{ "id": "xyz.jpg", ... } submit { "id": "xyz.jpg", ... } POST /upload App S3

Slide 41

Slide 41 text

mount Shrine.upload_endpoint(:cache), to: "/upload" POST /upload uppy.use(Uppy.XHRUpload, { endpoint: '/upload' })

Slide 42

Slide 42 text

{ "url": "...", "fields": "..." } submit { "id": "xyz.jpg", ... } App S3 GET /s3/params POST s3.aws.com

Slide 43

Slide 43 text

mount Shrine.presign_endpoint(:cache), to: "/s3/params" GET /s3/params uppy.use(Uppy.AwsS3, { companionUrl: '/' })

Slide 44

Slide 44 text

Resumable Uploads

Slide 45

Slide 45 text

0% ❌ 80% 80%

Slide 46

Slide 46 text

0% 0% 0% 0% 0%

Slide 47

Slide 47 text

0% 0% 0% 0% 0%

Slide 48

Slide 48 text

100% 80% ✅ ❌

Slide 49

Slide 49 text

App S3 S3 Multipart Upload

Slide 50

Slide 50 text

$ gem install uppy-s3_multipart uppy.use(Uppy.AwsS3Multipart, { companionUrl: '/' }) mount Shrine.uppy_s3_multipart(:cache), to: "/s3/multipart"

Slide 51

Slide 51 text

tus.io • HTTP protocol for resumable uploads • Numerous implementations: • Client – JavaScript, iOS, Android, ... • Server – Ruby, Node, Go, Python, Java, PHP, ...

Slide 52

Slide 52 text

S3 tus protocol storage API tus server tus client App

Slide 53

Slide 53 text

$ gem install tus-ruby-server uppy.use(Uppy.Tus, { endpoint: '/files' }) mount Tus::Server, to: "/files"

Slide 54

Slide 54 text

Recap • Validate and persist metadata • Processing "on attachment" or "on-the-fly" ImageProcessing – ImageMagick & libvips • Direct uploads Uppy simple uploads (to your app or to cloud) resumable uploads (S3 multipart upload or tus.io)

Slide 55

Slide 55 text

• https://shrinerb.com/ • https://github.com/janko/image_processing • https://libvips.github.io/libvips/ • https://uppy.io/ • https://tus.io/ • https://github.com/janko/tus-ruby-server • https://github.com/janko/uppy-s3_multipart • https://twin.github.io