Handling File Uploads for a modern developer

In Ruby we have an abundance of options for handling file attachments. Unfortunately, most file attachment gems fall short when it comes to tailoring the attachment flow to your needs and preferences, often leaving your app with suboptimal user experience.

This talk will introduce the Shrine gem and go over some of the modern best practices for handling file attachments. We will cover topics such as image processing (“Is ImageMagick always the best option?”), asynchronous file uploads (“Which JavaScript solution should I choose?”), and handling large uploads (“How do I help users that have a flaky internet connection?”).

Transcript

1. Handling File Uploads For a modern developer
2. 2007 Paperclip 2008 CarrierWave 2014 Refile 2018 ActiveStorage 2009 Dragonfly 2015 ! Shrine
3. Janko Marohnic • from Croatia ", living in Czechia # • Ruby off Rails developer • creator of Shrine @janko @jankomarohnic
4. Active Storage?
5. Active Storage Shrine Rails & Active Record only Rails, Hanami, Sinatra, Roda... Active Record, Sequel, ROM... framework library integrated experience more control more opinionated more features
6. Metadata & Validation
7. class PhotosController < AC::Base def create photo = Photo.new(photo_params) if photo.valid? photo.save # ... else # ... end end end
8. class ImageUploader < Shrine TYPES = %w[image/jpeg image/png ...] EXTENSIONS = %w[jpg jpeg png ...] Attacher.validate do validate_max_size 10*1024*1024 validate_mime_type TYPES validate_extension EXTENSIONS end end
9. \377\330\377························· (JPEG) \211PNG······························ (PNG) ········WEBP························· (WebP) MIME Type `file --mime-type -b #{file.path}` #=> "image/jpeg" MimeMagic.by_magic(file).type #=> "image/jpeg" Marcel::MimeType.for(file) #=> "image/jpeg"
10. class PhotosController < AC::Base def create file = params[:photo][:image] file.read #=> "<?php ... ?>" file.filename #=> "nature.jpg" file.content_type #=> "image/jpeg" photo = Photo.new(image: file) photo.image.mime_type #=> "text/x-php" photo.valid? #=> false end end MIME Type
11. https://www.bamsoftware.com/hacks/deflate.html $ identify spark.png ... 225000x225000 ... 6.1MB Attacher.validate do # ... if validate_mime_type IMAGE_TYPES validate_max_dimensions [5000, 5000] end end Image Dimensions
12. class VideoUploader < Shrine add_metadata :duration do |file| FFMPEG::Movie.new(file.path).duration end Attacher.validate do if file.duration > 5*60*60 errors << "is too long (max is 5h)" end end end Custom Metadata
13. { "id": "b93777246f10e509a553.mp4", "storage": "store", "metadata": { "size": 234837, "filename": "matrix.mp4", "mime_type": "video/mp4", "duration": 9000, ... } } SELECT video_data FROM movies;
14. Processing
15. ImageMagick
16. image = MiniMagick::Image.open(file) image.resize("500x500") image.path #=> "/path/to/resized.jpg" $ gem install mini_magick
17. Resize to fit Resize to limit Resize to fill
18. magick = ImageProcessing::MiniMagick
 magick = magick.source(image) magick.resize_to_limit! 500, 500 magick.resize_to_fit! 500, 500 magick.resize_to_fill! 500, 500 magick.resize_and_pad! 500, 500 $ gem install image_processing
19. thumbnail = ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ gem install image_processing $ convert source.jpg -auto-orient -resize 500x500> -sharpen 0x1 destination.jpg
20. thumbnail = ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ gem install image_processing $ convert source.jpg -auto-orient -resize 500x500> -sharpen 0x1 destination.jpg
21. libvips⚡
22. ImageProcessing::MiniMagick
 .source(image) .resize_to_limit!(500, 500) $ brew install imagemagick
23. ImageProcessing::Vips
 .source(image) .resize_to_limit!(500, 500) $ brew install vips
24. THUMBNAILS = { xs: [200, 200], s: [400, 400], m: [600, 600], l: [800, 800], xl: [1200, 1200], } THUMBNAILS.each do |name, (width, height)| ImageProcessing::Backend .source(original) .resize_to_limit!(width, height) end
25. 0,47s 2,15s ImageMagick libvips
26. Rails 6.0
27. Processing On-the-fly
28. Active Storage https://.../<base64-encoded-steps>/... photo.image.variant( resize_to_limit: [600, 400], strip: true, quality: 85, interlace: "JPEG", )
29. Shrine https://.../thumbnail/600/400/... photo.image.derivation_url( :thumbnail, # <-- name 600, 400 # <-- args ) Shrine.derivation :thumbnail do |file, w, h| ImageProcessing::Vips.source(file) .strip .saver(quality: 85, interlace: "JPEG") .resize_to_limit!(w.to_i, h.to_i) end
30. Processing on attachment
31. class ImageUploader < Shrine derivatives_processor :thumbs do |file| vips = ImageProcessing::Vips.source(file) { large: vips.resize_to_fit!(800, 800), medium: vips.resize_to_fit!(500, 500), small: vips.resize_to_fit!(300, 300) } end end photo = Photo.new(image: file) photo.image_derivatives!(:thumbs) photo.image_derivatives #=> # { large: #<Shrine::File @id="ld7.jpg">, # medium: #<Shrine::File @id="jg4.jpg">, # small: #<Shrine::File @id="02k.jpg"> }
32. class VideoUploader < Shrine derivatives_processor :transcode do |file| movie = FMMPEG::Movie.new(file.path) transcoded = Tempfile.new(["", ".mp4"]) screenshot = Tempfile.new(["", ".jpg"]) movie.transcode(transcoded.path) movie.screenshot(screenshot.path) { transcoded: transcoded, screenshot: screenshot } end end
33. Direct Uploads
34. refile.js activestorage.js shrine.js ? ❌
35. jQuery-File-Upload Dropzone.js FineUploader ❌ ❌ ❌
36. Uppy
37. Uppy() .use(Dashboard, { /* ... */ })
38. Uppy() .use(DragDrop, { /* ... */ }) .use(StatusBar, { /* ... */ })
39. Uppy() .use(FileInput, { /* ... */ }) .use(ProgressBar, { /* ... */ })
40. { "id": "xyz.jpg", ... } submit { "id": "xyz.jpg", ... } POST /upload App S3
41. mount Shrine.upload_endpoint(:cache), to: "/upload" POST /upload uppy.use(Uppy.XHRUpload, { endpoint: '/upload' })
42. { "url": "...", "fields": "..." } submit { "id": "xyz.jpg", ... } App S3 GET /s3/params POST s3.aws.com
43. mount Shrine.presign_endpoint(:cache), to: "/s3/params" GET /s3/params uppy.use(Uppy.AwsS3, { companionUrl: '/' })
44. Resumable Uploads
45. 0% ❌ 80% 80%
46. 0% 0% 0% 0% 0%
47. 0% 0% 0% 0% 0%
48. 100% 80% ✅ ❌
49. App S3 S3 Multipart Upload
50. $ gem install uppy-s3_multipart uppy.use(Uppy.AwsS3Multipart, { companionUrl: '/' }) mount Shrine.uppy_s3_multipart(:cache), to: "/s3/multipart"
51. tus.io • HTTP protocol for resumable uploads • Numerous implementations: • Client – JavaScript, iOS, Android, ... • Server – Ruby, Node, Go, Python, Java, PHP, ...
52. S3 tus protocol storage API tus server tus client App
53. $ gem install tus-ruby-server uppy.use(Uppy.Tus, { endpoint: '/files' }) mount Tus::Server, to: "/files"
54. Recap • Validate and persist metadata • Processing "on attachment" or "on-the-fly" ImageProcessing – ImageMagick & libvips • Direct uploads Uppy simple uploads (to your app or to cloud) resumable uploads (S3 multipart upload or tus.io)
55. • https://shrinerb.com/ • https://github.com/janko/image_processing • https://libvips.github.io/libvips/ • https://uppy.io/ • https://tus.io/ • https://github.com/janko/tus-ruby-server • https://github.com/janko/uppy-s3_multipart • https://twin.github.io