Shrine - File Upload Toolkit for Ruby

Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

376e4eb9dc6c2e33d1330262edc4f109?s=128

Janko Marohnić

March 31, 2016
Tweet

Transcript

  1. Shrine File Upload Toolkit janko-m @jankomarohnic

  2. • LOC = 470 (base) + 1550 (plugins) • 1

    tiny dependency • streaming uploads & downloads • closes files and deletes tempfiles • asynchronous processing & uploads & deletes • simple and flexible design • support for Sequel • security
  3. Design

  4. Storage class MyStorage # PORO def upload(io, id) # ...

    end def url(id) # ... end def exists?(id) # ... end def delete(id) # ... end # ... end
  5. Shrine.storages = { disk: Shrine::Storage::FileSystem.new("uploads") } uploader = Shrine.new(:disk) uploader.storage

    #=> #<Shrine::Storage::FileSystem> uploaded_file = uploader.upload(image) uploaded_file #=> #<Shrine::UploadedFile> uploaded_file.data #=> # { # "storage" => "disk", # "id" => "9260ea09d8effd.jpg", # "metadata" => { # "filename" => "image.jpg", # "size" => 819344, # "mime_type" => "image/jpeg", # }, # } Uploader
  6. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg" uploaded_file.metadata #=> {...} uploaded_file.read #=> "..." uploaded_file.exists?

    #=> true uploaded_file.download #=> #<Tempfile:/tmp/sdf94ks.jpg> uploaded_file.delete ... Uploaded File
  7. Shrine.storages = { cache: Shrine::Storage::FileSystem.new(...), store: Shrine::Storage::FileSystem.new(...), } Shrine.plugin :sequel

    # or :activerecord class ImageUploader < Shrine # ... end class User < Sequel::Model include ImageUploader[:avatar] end Attachment add_column :users, :avatar_data, :text
  8. user = User.create(avatar: File.open(“avatar.jpg")) user.avatar_data #=> "{\"storage\":\"store\",\"id\": \"sg00943klsg8dd.jpg\",\"metadata\":{...}}" user.avatar #=>

    #<Shrine::UploadedFile> user.avatar.url #=> "uploads/9dkasd920s.jpg" user.destroy user.avatar.exists? #=> false Attacher
  9. Processing

  10. class ImageUploader < Shrine def process(io, context) if context[:phase] ==

    :store # ... end end end
  11. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick def process(io,

    context) if context[:phase] == :store resize_to_limit(io.download, 800, 800) end end end
  12. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick plugin :versions,

    names: [:small, :medium, :large] def process(io, context) if context[:phase] == :store size_800 = resize_to_limit(io.download, 800, 800) size_500 = resize_to_limit(size_800, 500, 500) size_300 = resize_to_limit(size_500, 300, 300) {small: size_300, medium: size_500, large: size_800} end end end
  13. user.avatar_data #=> # { # "large": { # "id": "lg043.jpg",

    # "storage": "store", # "metadata" : {"size": 94832, ...} # }, # "medium": { # "id": "kd9fk.jpg", # "storage": "store", # "metadata" : {"size": 39291, ...} # }, # "small": { # "id": "932fl.jpg", # "storage": "store", # "metadata" : {"size": 10582, ...} # } # }
  14. user.avatar.class #=> Hash user.avatar #=> # { # :large =>

    #<Shrine::UploadedFile>, # :medium => #<Shrine::UploadedFile>, # :small => #<Shrine::UploadedFile>, # } user.avatar[:large].size #=> 94832 user.avatar[:medium].size #=> 39291 user.avatar[:small].size #=> 10582
  15. Direct upload

  16. Choose a file Fill in other fields Submit the form

    Wait for file to be uploaded Get redirected
  17. Choose a file Fill in other fields Submit the form

    No waiting Get redirected File uploads using AJAX
  18. # config/routes.rb Rails.application.routes.draw do mount ImageUploader::UploadEndpoint => "/images" end Shrine.plugin

    :direct_upload // with jQuery-File-Upload $('[type="file"]').fileupload({ url: '/images/cache/avatar', paramName: 'file', add: (e, data) => { /* Disable submit button */ }, progress: (e, data) => { /* Add progress bar */ }, done: (e, data) => { /* Fill hidden field */ }, }); Generic
  19. Shrine.storages = { cache: Shrine::Storage::S3.new(...), store: Shrine::Storage::S3.new(...), } Shrine.plugin :direct_upload,

    presign: true GET /images/cache/presign { "url": "https://my-bucket.s3-eu-west-1.amazonaws.com", "fields": { "key": "image.jpg", "x-amz-credential": "...", "x-amz-algorithm": "...", "x-amz-date": "...", "x-amz-signature": "..." } } Amazon S3
  20. Backgrounding

  21. • Processing, uploading and deleting • Graceful degradation • Supports

    any backgrounding library • Handles parallel jobs • Adaptation to changes • Debuggable failures • Security
  22. Shrine.plugin :backgrounding Shrine::Attacher.promote { |data| UploadJob.perform_async(data) } Shrine::Attacher.delete { |data|

    DeleteJob.perform_async(data) } class UploadJob include Sidekiq::Worker def perform(data) Shrine::Attacher.promote(data) end end class DeleteJob include Sidekiq::Worker def perform(data) Shrine::Attacher.delete(data) end end
  23. class ImageUploader < Shrine plugin :versions, names: [...] def process(io,

    context) case context[:phase] when :recache # process cheap versions in foreground when :store # process expensive versions in background end end end Shrine.plugin :recache
  24. Logging

  25. Shrine.plugin :logging STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s) PROCESS[store]: ImageUploader[:avatar]

    User[29543] 1-3 files (0.22s) DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s) Shrine.plugin :logging, format: :json {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...} Shrine.plugin :logging, format: :heroku action=upload phase=cache files=1 duration=0.16 record_class=User
  26. Other plugins • determine_mime_type
 – determines actual MIME type from

    file contents • store_dimensions
 – extracts and stores image width & height • validation_helpers
 – validation of filesize, extension, MIME type, dimensions • parallelize
 – parallelizes uploading and deleting versions • … (many more)
  27. Storages • FileSystem • S3 • Cloudinary • Imgix •

    Fog • Flickr • SQL • GridFS
  28. • http://github.com/janko-m/shrine • http://shrinerb.com (website) • http://github.com/janko-m/shrine-example • http://twin.github.io/introducing-shrine/ •

    http://twin.github.io/file-uploads-asynchronous- world/