Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Transcript

1. Shrine
   File Upload Toolkit
   janko-m @jankomarohnic
   

   View Slide

2. • LOC = 470 (base) + 1550 (plugins)
   • 1 tiny dependency
   • streaming uploads & downloads
   • closes files and deletes tempfiles
   • asynchronous processing & uploads & deletes
   • simple and flexible design
   • support for Sequel
   • security
   

   View Slide

3. Design
   

   View Slide

4. Storage
   class MyStorage # PORO
   def upload(io, id)
   # ...
   end
   def url(id)
   # ...
   end
   def exists?(id)
   # ...
   end
   def delete(id)
   # ...
   end
   # ...
   end
   

   View Slide

5. Shrine.storages = {
   disk: Shrine::Storage::FileSystem.new("uploads")
   }
   uploader = Shrine.new(:disk)
   uploader.storage #=> #
   uploaded_file = uploader.upload(image)
   uploaded_file #=> #
   uploaded_file.data #=>
   # {
   # "storage" => "disk",
   # "id" => "9260ea09d8effd.jpg",
   # "metadata" => {
   # "filename" => "image.jpg",
   # "size" => 819344,
   # "mime_type" => "image/jpeg",
   # },
   # }
   Uploader
   

   View Slide

6. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg"
   uploaded_file.metadata #=> {...}
   uploaded_file.read #=> "..."
   uploaded_file.exists? #=> true
   uploaded_file.download #=> #
   uploaded_file.delete
   ...
   Uploaded File
   

   View Slide

7. Shrine.storages = {
   cache: Shrine::Storage::FileSystem.new(...),
   store: Shrine::Storage::FileSystem.new(...),
   }
   Shrine.plugin :sequel # or :activerecord
   class ImageUploader < Shrine
   # ...
   end
   class User < Sequel::Model
   include ImageUploader[:avatar]
   end
   Attachment
   add_column :users, :avatar_data, :text
   

   View Slide

8. user = User.create(avatar: File.open(“avatar.jpg"))
   user.avatar_data #=> "{\"storage\":\"store\",\"id\":
   \"sg00943klsg8dd.jpg\",\"metadata\":{...}}"
   user.avatar #=> #
   user.avatar.url #=> "uploads/9dkasd920s.jpg"
   user.destroy
   user.avatar.exists? #=> false
   Attacher
   

   View Slide

9. Processing
   

   View Slide

10. class ImageUploader < Shrine
    def process(io, context)
    if context[:phase] == :store
    # ...
    end
    end
    end
    

    View Slide

11. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    def process(io, context)
    if context[:phase] == :store
    resize_to_limit(io.download, 800, 800)
    end
    end
    end
    

    View Slide

12. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    plugin :versions, names: [:small, :medium, :large]
    def process(io, context)
    if context[:phase] == :store
    size_800 = resize_to_limit(io.download, 800, 800)
    size_500 = resize_to_limit(size_800, 500, 500)
    size_300 = resize_to_limit(size_500, 300, 300)
    {small: size_300, medium: size_500, large: size_800}
    end
    end
    end
    

    View Slide

13. user.avatar_data #=>
    # {
    # "large": {
    # "id": "lg043.jpg",
    # "storage": "store",
    # "metadata" : {"size": 94832, ...}
    # },
    # "medium": {
    # "id": "kd9fk.jpg",
    # "storage": "store",
    # "metadata" : {"size": 39291, ...}
    # },
    # "small": {
    # "id": "932fl.jpg",
    # "storage": "store",
    # "metadata" : {"size": 10582, ...}
    # }
    # }
    

    View Slide

14. user.avatar.class #=> Hash
    user.avatar #=>
    # {
    # :large => #,
    # :medium => #,
    # :small => #,
    # }
    user.avatar[:large].size #=> 94832
    user.avatar[:medium].size #=> 39291
    user.avatar[:small].size #=> 10582
    

    View Slide

15. Direct upload
    

    View Slide

16. Choose a file
    Fill in other fields
    Submit the form
    Wait for file to be uploaded
    Get redirected
    

    View Slide

17. Choose a file
    Fill in other fields
    Submit the form
    No waiting
    Get redirected
    File uploads
    using AJAX
    

    View Slide

18. # config/routes.rb
    Rails.application.routes.draw do
    mount ImageUploader::UploadEndpoint => "/images"
    end
    Shrine.plugin :direct_upload
    // with jQuery-File-Upload
    $('[type="file"]').fileupload({
    url: '/images/cache/avatar',
    paramName: 'file',
    add: (e, data) => { /* Disable submit button */ },
    progress: (e, data) => { /* Add progress bar */ },
    done: (e, data) => { /* Fill hidden field */ },
    });
    Generic
    

    View Slide

19. Shrine.storages = {
    cache: Shrine::Storage::S3.new(...),
    store: Shrine::Storage::S3.new(...),
    }
    Shrine.plugin :direct_upload, presign: true
    GET /images/cache/presign
    {
    "url": "https://my-bucket.s3-eu-west-1.amazonaws.com",
    "fields": {
    "key": "image.jpg",
    "x-amz-credential": "...",
    "x-amz-algorithm": "...",
    "x-amz-date": "...",
    "x-amz-signature": "..."
    }
    }
    Amazon S3
    

    View Slide

20. Backgrounding
    

    View Slide

21. • Processing, uploading and deleting
    • Graceful degradation
    • Supports any backgrounding library
    • Handles parallel jobs
    • Adaptation to changes
    • Debuggable failures
    • Security
    

    View Slide

22. Shrine.plugin :backgrounding
    Shrine::Attacher.promote { |data| UploadJob.perform_async(data) }
    Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
    class UploadJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.promote(data)
    end
    end
    class DeleteJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.delete(data)
    end
    end
    

    View Slide

23. class ImageUploader < Shrine
    plugin :versions, names: [...]
    def process(io, context)
    case context[:phase]
    when :recache
    # process cheap versions in foreground
    when :store
    # process expensive versions in background
    end
    end
    end
    Shrine.plugin :recache
    

    View Slide

24. Logging
    

    View Slide

25. Shrine.plugin :logging
    STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s)
    PROCESS[store]: ImageUploader[:avatar] User[29543] 1-3 files (0.22s)
    DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s)
    Shrine.plugin :logging, format: :json
    {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...}
    Shrine.plugin :logging, format: :heroku
    action=upload phase=cache files=1 duration=0.16 record_class=User
    

    View Slide

26. Other plugins
    • determine_mime_type

    – determines actual MIME type from file contents
    • store_dimensions

    – extracts and stores image width & height
    • validation_helpers

    – validation of filesize, extension, MIME type, dimensions
    • parallelize

    – parallelizes uploading and deleting versions
    • … (many more)
    

    View Slide

27. Storages
    • FileSystem
    • S3
    • Cloudinary
    • Imgix
    • Fog
    • Flickr
    • SQL
    • GridFS
    

    View Slide

28. • http://github.com/janko-m/shrine
    • http://shrinerb.com (website)
    • http://github.com/janko-m/shrine-example
    • http://twin.github.io/introducing-shrine/
    • http://twin.github.io/file-uploads-asynchronous-
    world/
    

    View Slide