Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Transcript

1. Shrine File Upload Toolkit janko-m @jankomarohnic
2. • LOC = 470 (base) + 1550 (plugins) • 1 tiny dependency • streaming uploads & downloads • closes files and deletes tempfiles • asynchronous processing & uploads & deletes • simple and flexible design • support for Sequel • security
3. Design
4. Storage class MyStorage # PORO def upload(io, id) # ... end def url(id) # ... end def exists?(id) # ... end def delete(id) # ... end # ... end
5. Shrine.storages = { disk: Shrine::Storage::FileSystem.new("uploads") } uploader = Shrine.new(:disk) uploader.storage #=> #<Shrine::Storage::FileSystem> uploaded_file = uploader.upload(image) uploaded_file #=> #<Shrine::UploadedFile> uploaded_file.data #=> # { # "storage" => "disk", # "id" => "9260ea09d8effd.jpg", # "metadata" => { # "filename" => "image.jpg", # "size" => 819344, # "mime_type" => "image/jpeg", # }, # } Uploader
6. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg" uploaded_file.metadata #=> {...} uploaded_file.read #=> "..." uploaded_file.exists? #=> true uploaded_file.download #=> #<Tempfile:/tmp/sdf94ks.jpg> uploaded_file.delete ... Uploaded File
7. Shrine.storages = { cache: Shrine::Storage::FileSystem.new(...), store: Shrine::Storage::FileSystem.new(...), } Shrine.plugin :sequel # or :activerecord class ImageUploader < Shrine # ... end class User < Sequel::Model include ImageUploader[:avatar] end Attachment add_column :users, :avatar_data, :text
8. user = User.create(avatar: File.open(“avatar.jpg")) user.avatar_data #=> "{\"storage\":\"store\",\"id\": \"sg00943klsg8dd.jpg\",\"metadata\":{...}}" user.avatar #=> #<Shrine::UploadedFile> user.avatar.url #=> "uploads/9dkasd920s.jpg" user.destroy user.avatar.exists? #=> false Attacher
9. Processing
10. class ImageUploader < Shrine def process(io, context) if context[:phase] == :store # ... end end end
11. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick def process(io, context) if context[:phase] == :store resize_to_limit(io.download, 800, 800) end end end
12. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick plugin :versions, names: [:small, :medium, :large] def process(io, context) if context[:phase] == :store size_800 = resize_to_limit(io.download, 800, 800) size_500 = resize_to_limit(size_800, 500, 500) size_300 = resize_to_limit(size_500, 300, 300) {small: size_300, medium: size_500, large: size_800} end end end
13. user.avatar_data #=> # { # "large": { # "id": "lg043.jpg", # "storage": "store", # "metadata" : {"size": 94832, ...} # }, # "medium": { # "id": "kd9fk.jpg", # "storage": "store", # "metadata" : {"size": 39291, ...} # }, # "small": { # "id": "932fl.jpg", # "storage": "store", # "metadata" : {"size": 10582, ...} # } # }
14. user.avatar.class #=> Hash user.avatar #=> # { # :large => #<Shrine::UploadedFile>, # :medium => #<Shrine::UploadedFile>, # :small => #<Shrine::UploadedFile>, # } user.avatar[:large].size #=> 94832 user.avatar[:medium].size #=> 39291 user.avatar[:small].size #=> 10582
15. Direct upload
16. Choose a file Fill in other fields Submit the form Wait for file to be uploaded Get redirected
17. Choose a file Fill in other fields Submit the form No waiting Get redirected File uploads using AJAX
18. # config/routes.rb Rails.application.routes.draw do mount ImageUploader::UploadEndpoint => "/images" end Shrine.plugin :direct_upload // with jQuery-File-Upload $('[type="file"]').fileupload({ url: '/images/cache/avatar', paramName: 'file', add: (e, data) => { /* Disable submit button */ }, progress: (e, data) => { /* Add progress bar */ }, done: (e, data) => { /* Fill hidden field */ }, }); Generic
19. Shrine.storages = { cache: Shrine::Storage::S3.new(...), store: Shrine::Storage::S3.new(...), } Shrine.plugin :direct_upload, presign: true GET /images/cache/presign { "url": "https://my-bucket.s3-eu-west-1.amazonaws.com", "fields": { "key": "image.jpg", "x-amz-credential": "...", "x-amz-algorithm": "...", "x-amz-date": "...", "x-amz-signature": "..." } } Amazon S3
20. Backgrounding
21. • Processing, uploading and deleting • Graceful degradation • Supports any backgrounding library • Handles parallel jobs • Adaptation to changes • Debuggable failures • Security
22. Shrine.plugin :backgrounding Shrine::Attacher.promote { |data| UploadJob.perform_async(data) } Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) } class UploadJob include Sidekiq::Worker def perform(data) Shrine::Attacher.promote(data) end end class DeleteJob include Sidekiq::Worker def perform(data) Shrine::Attacher.delete(data) end end
23. class ImageUploader < Shrine plugin :versions, names: [...] def process(io, context) case context[:phase] when :recache # process cheap versions in foreground when :store # process expensive versions in background end end end Shrine.plugin :recache
24. Logging
25. Shrine.plugin :logging STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s) PROCESS[store]: ImageUploader[:avatar] User[29543] 1-3 files (0.22s) DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s) Shrine.plugin :logging, format: :json {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...} Shrine.plugin :logging, format: :heroku action=upload phase=cache files=1 duration=0.16 record_class=User
26. Other plugins • determine_mime_type
 – determines actual MIME type from file contents • store_dimensions
 – extracts and stores image width & height • validation_helpers
 – validation of filesize, extension, MIME type, dimensions • parallelize
 – parallelizes uploading and deleting versions • … (many more)
27. Storages • FileSystem • S3 • Cloudinary • Imgix • Fog • Flickr • SQL • GridFS
28. • http://github.com/janko-m/shrine • http://shrinerb.com (website) • http://github.com/janko-m/shrine-example • http://twin.github.io/introducing-shrine/ • http://twin.github.io/file-uploads-asynchronous- world/