Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shrine - File Upload Toolkit for Ruby

Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Janko Marohnić

March 31, 2016
Tweet

More Decks by Janko Marohnić

Other Decks in Programming

Transcript

  1. Shrine File Upload Toolkit janko-m @jankomarohnic

  2. • LOC = 470 (base) + 1550 (plugins) • 1

    tiny dependency • streaming uploads & downloads • closes files and deletes tempfiles • asynchronous processing & uploads & deletes • simple and flexible design • support for Sequel • security
  3. Design

  4. Storage class MyStorage # PORO def upload(io, id) # ...

    end def url(id) # ... end def exists?(id) # ... end def delete(id) # ... end # ... end
  5. Shrine.storages = { disk: Shrine::Storage::FileSystem.new("uploads") } uploader = Shrine.new(:disk) uploader.storage

    #=> #<Shrine::Storage::FileSystem> uploaded_file = uploader.upload(image) uploaded_file #=> #<Shrine::UploadedFile> uploaded_file.data #=> # { # "storage" => "disk", # "id" => "9260ea09d8effd.jpg", # "metadata" => { # "filename" => "image.jpg", # "size" => 819344, # "mime_type" => "image/jpeg", # }, # } Uploader
  6. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg" uploaded_file.metadata #=> {...} uploaded_file.read #=> "..." uploaded_file.exists?

    #=> true uploaded_file.download #=> #<Tempfile:/tmp/sdf94ks.jpg> uploaded_file.delete ... Uploaded File
  7. Shrine.storages = { cache: Shrine::Storage::FileSystem.new(...), store: Shrine::Storage::FileSystem.new(...), } Shrine.plugin :sequel

    # or :activerecord class ImageUploader < Shrine # ... end class User < Sequel::Model include ImageUploader[:avatar] end Attachment add_column :users, :avatar_data, :text
  8. user = User.create(avatar: File.open(“avatar.jpg")) user.avatar_data #=> "{\"storage\":\"store\",\"id\": \"sg00943klsg8dd.jpg\",\"metadata\":{...}}" user.avatar #=>

    #<Shrine::UploadedFile> user.avatar.url #=> "uploads/9dkasd920s.jpg" user.destroy user.avatar.exists? #=> false Attacher
  9. Processing

  10. class ImageUploader < Shrine def process(io, context) if context[:phase] ==

    :store # ... end end end
  11. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick def process(io,

    context) if context[:phase] == :store resize_to_limit(io.download, 800, 800) end end end
  12. require "image_processing/mini_magick" class ImageUploader < Shrine include ImageProcessing::MiniMagick plugin :versions,

    names: [:small, :medium, :large] def process(io, context) if context[:phase] == :store size_800 = resize_to_limit(io.download, 800, 800) size_500 = resize_to_limit(size_800, 500, 500) size_300 = resize_to_limit(size_500, 300, 300) {small: size_300, medium: size_500, large: size_800} end end end
  13. user.avatar_data #=> # { # "large": { # "id": "lg043.jpg",

    # "storage": "store", # "metadata" : {"size": 94832, ...} # }, # "medium": { # "id": "kd9fk.jpg", # "storage": "store", # "metadata" : {"size": 39291, ...} # }, # "small": { # "id": "932fl.jpg", # "storage": "store", # "metadata" : {"size": 10582, ...} # } # }
  14. user.avatar.class #=> Hash user.avatar #=> # { # :large =>

    #<Shrine::UploadedFile>, # :medium => #<Shrine::UploadedFile>, # :small => #<Shrine::UploadedFile>, # } user.avatar[:large].size #=> 94832 user.avatar[:medium].size #=> 39291 user.avatar[:small].size #=> 10582
  15. Direct upload

  16. Choose a file Fill in other fields Submit the form

    Wait for file to be uploaded Get redirected
  17. Choose a file Fill in other fields Submit the form

    No waiting Get redirected File uploads using AJAX
  18. # config/routes.rb Rails.application.routes.draw do mount ImageUploader::UploadEndpoint => "/images" end Shrine.plugin

    :direct_upload // with jQuery-File-Upload $('[type="file"]').fileupload({ url: '/images/cache/avatar', paramName: 'file', add: (e, data) => { /* Disable submit button */ }, progress: (e, data) => { /* Add progress bar */ }, done: (e, data) => { /* Fill hidden field */ }, }); Generic
  19. Shrine.storages = { cache: Shrine::Storage::S3.new(...), store: Shrine::Storage::S3.new(...), } Shrine.plugin :direct_upload,

    presign: true GET /images/cache/presign { "url": "https://my-bucket.s3-eu-west-1.amazonaws.com", "fields": { "key": "image.jpg", "x-amz-credential": "...", "x-amz-algorithm": "...", "x-amz-date": "...", "x-amz-signature": "..." } } Amazon S3
  20. Backgrounding

  21. • Processing, uploading and deleting • Graceful degradation • Supports

    any backgrounding library • Handles parallel jobs • Adaptation to changes • Debuggable failures • Security
  22. Shrine.plugin :backgrounding Shrine::Attacher.promote { |data| UploadJob.perform_async(data) } Shrine::Attacher.delete { |data|

    DeleteJob.perform_async(data) } class UploadJob include Sidekiq::Worker def perform(data) Shrine::Attacher.promote(data) end end class DeleteJob include Sidekiq::Worker def perform(data) Shrine::Attacher.delete(data) end end
  23. class ImageUploader < Shrine plugin :versions, names: [...] def process(io,

    context) case context[:phase] when :recache # process cheap versions in foreground when :store # process expensive versions in background end end end Shrine.plugin :recache
  24. Logging

  25. Shrine.plugin :logging STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s) PROCESS[store]: ImageUploader[:avatar]

    User[29543] 1-3 files (0.22s) DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s) Shrine.plugin :logging, format: :json {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...} Shrine.plugin :logging, format: :heroku action=upload phase=cache files=1 duration=0.16 record_class=User
  26. Other plugins • determine_mime_type
 – determines actual MIME type from

    file contents • store_dimensions
 – extracts and stores image width & height • validation_helpers
 – validation of filesize, extension, MIME type, dimensions • parallelize
 – parallelizes uploading and deleting versions • … (many more)
  27. Storages • FileSystem • S3 • Cloudinary • Imgix •

    Fog • Flickr • SQL • GridFS
  28. • http://github.com/janko-m/shrine • http://shrinerb.com (website) • http://github.com/janko-m/shrine-example • http://twin.github.io/introducing-shrine/ •

    http://twin.github.io/file-uploads-asynchronous- world/