Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shrine - File Upload Toolkit for Ruby

Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Janko Marohnić

March 31, 2016
Tweet

More Decks by Janko Marohnić

Other Decks in Programming

Transcript

  1. Shrine
    File Upload Toolkit
    janko-m @jankomarohnic

    View Slide

  2. • LOC = 470 (base) + 1550 (plugins)
    • 1 tiny dependency
    • streaming uploads & downloads
    • closes files and deletes tempfiles
    • asynchronous processing & uploads & deletes
    • simple and flexible design
    • support for Sequel
    • security

    View Slide

  3. Design

    View Slide

  4. Storage
    class MyStorage # PORO
    def upload(io, id)
    # ...
    end
    def url(id)
    # ...
    end
    def exists?(id)
    # ...
    end
    def delete(id)
    # ...
    end
    # ...
    end

    View Slide

  5. Shrine.storages = {
    disk: Shrine::Storage::FileSystem.new("uploads")
    }
    uploader = Shrine.new(:disk)
    uploader.storage #=> #
    uploaded_file = uploader.upload(image)
    uploaded_file #=> #
    uploaded_file.data #=>
    # {
    # "storage" => "disk",
    # "id" => "9260ea09d8effd.jpg",
    # "metadata" => {
    # "filename" => "image.jpg",
    # "size" => 819344,
    # "mime_type" => "image/jpeg",
    # },
    # }
    Uploader

    View Slide

  6. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg"
    uploaded_file.metadata #=> {...}
    uploaded_file.read #=> "..."
    uploaded_file.exists? #=> true
    uploaded_file.download #=> #
    uploaded_file.delete
    ...
    Uploaded File

    View Slide

  7. Shrine.storages = {
    cache: Shrine::Storage::FileSystem.new(...),
    store: Shrine::Storage::FileSystem.new(...),
    }
    Shrine.plugin :sequel # or :activerecord
    class ImageUploader < Shrine
    # ...
    end
    class User < Sequel::Model
    include ImageUploader[:avatar]
    end
    Attachment
    add_column :users, :avatar_data, :text

    View Slide

  8. user = User.create(avatar: File.open(“avatar.jpg"))
    user.avatar_data #=> "{\"storage\":\"store\",\"id\":
    \"sg00943klsg8dd.jpg\",\"metadata\":{...}}"
    user.avatar #=> #
    user.avatar.url #=> "uploads/9dkasd920s.jpg"
    user.destroy
    user.avatar.exists? #=> false
    Attacher

    View Slide

  9. Processing

    View Slide

  10. class ImageUploader < Shrine
    def process(io, context)
    if context[:phase] == :store
    # ...
    end
    end
    end

    View Slide

  11. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    def process(io, context)
    if context[:phase] == :store
    resize_to_limit(io.download, 800, 800)
    end
    end
    end

    View Slide

  12. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    plugin :versions, names: [:small, :medium, :large]
    def process(io, context)
    if context[:phase] == :store
    size_800 = resize_to_limit(io.download, 800, 800)
    size_500 = resize_to_limit(size_800, 500, 500)
    size_300 = resize_to_limit(size_500, 300, 300)
    {small: size_300, medium: size_500, large: size_800}
    end
    end
    end

    View Slide

  13. user.avatar_data #=>
    # {
    # "large": {
    # "id": "lg043.jpg",
    # "storage": "store",
    # "metadata" : {"size": 94832, ...}
    # },
    # "medium": {
    # "id": "kd9fk.jpg",
    # "storage": "store",
    # "metadata" : {"size": 39291, ...}
    # },
    # "small": {
    # "id": "932fl.jpg",
    # "storage": "store",
    # "metadata" : {"size": 10582, ...}
    # }
    # }

    View Slide

  14. user.avatar.class #=> Hash
    user.avatar #=>
    # {
    # :large => #,
    # :medium => #,
    # :small => #,
    # }
    user.avatar[:large].size #=> 94832
    user.avatar[:medium].size #=> 39291
    user.avatar[:small].size #=> 10582

    View Slide

  15. Direct upload

    View Slide

  16. Choose a file
    Fill in other fields
    Submit the form
    Wait for file to be uploaded
    Get redirected

    View Slide

  17. Choose a file
    Fill in other fields
    Submit the form
    No waiting
    Get redirected
    File uploads
    using AJAX

    View Slide

  18. # config/routes.rb
    Rails.application.routes.draw do
    mount ImageUploader::UploadEndpoint => "/images"
    end
    Shrine.plugin :direct_upload
    // with jQuery-File-Upload
    $('[type="file"]').fileupload({
    url: '/images/cache/avatar',
    paramName: 'file',
    add: (e, data) => { /* Disable submit button */ },
    progress: (e, data) => { /* Add progress bar */ },
    done: (e, data) => { /* Fill hidden field */ },
    });
    Generic

    View Slide

  19. Shrine.storages = {
    cache: Shrine::Storage::S3.new(...),
    store: Shrine::Storage::S3.new(...),
    }
    Shrine.plugin :direct_upload, presign: true
    GET /images/cache/presign
    {
    "url": "https://my-bucket.s3-eu-west-1.amazonaws.com",
    "fields": {
    "key": "image.jpg",
    "x-amz-credential": "...",
    "x-amz-algorithm": "...",
    "x-amz-date": "...",
    "x-amz-signature": "..."
    }
    }
    Amazon S3

    View Slide

  20. Backgrounding

    View Slide

  21. • Processing, uploading and deleting
    • Graceful degradation
    • Supports any backgrounding library
    • Handles parallel jobs
    • Adaptation to changes
    • Debuggable failures
    • Security

    View Slide

  22. Shrine.plugin :backgrounding
    Shrine::Attacher.promote { |data| UploadJob.perform_async(data) }
    Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
    class UploadJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.promote(data)
    end
    end
    class DeleteJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.delete(data)
    end
    end

    View Slide

  23. class ImageUploader < Shrine
    plugin :versions, names: [...]
    def process(io, context)
    case context[:phase]
    when :recache
    # process cheap versions in foreground
    when :store
    # process expensive versions in background
    end
    end
    end
    Shrine.plugin :recache

    View Slide

  24. Logging

    View Slide

  25. Shrine.plugin :logging
    STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s)
    PROCESS[store]: ImageUploader[:avatar] User[29543] 1-3 files (0.22s)
    DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s)
    Shrine.plugin :logging, format: :json
    {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...}
    Shrine.plugin :logging, format: :heroku
    action=upload phase=cache files=1 duration=0.16 record_class=User

    View Slide

  26. Other plugins
    • determine_mime_type

    – determines actual MIME type from file contents
    • store_dimensions

    – extracts and stores image width & height
    • validation_helpers

    – validation of filesize, extension, MIME type, dimensions
    • parallelize

    – parallelizes uploading and deleting versions
    • … (many more)

    View Slide

  27. Storages
    • FileSystem
    • S3
    • Cloudinary
    • Imgix
    • Fog
    • Flickr
    • SQL
    • GridFS

    View Slide

  28. • http://github.com/janko-m/shrine
    • http://shrinerb.com (website)
    • http://github.com/janko-m/shrine-example
    • http://twin.github.io/introducing-shrine/
    • http://twin.github.io/file-uploads-asynchronous-
    world/

    View Slide