Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Shrine - File Upload Toolkit for Ruby

Shrine - File Upload Toolkit for Ruby

Accepting file uploads is very common in web applications, but also very delicate; we have to manage the fact that it's slow, and that it comes with a lot of security implications.

Shrine is a new Ruby library for file uploads, which aims to address limitations and issues of existing libraries (CarrierWave, Paperclip, Dragonfly and Refile).

In this presentation I show how Shrine's design differs, and how it makes Shrine simple where other libraries are complex, while also making some very advanced features possible.

Janko Marohnić

March 31, 2016
Tweet

More Decks by Janko Marohnić

Other Decks in Programming

Transcript

  1. Shrine
    File Upload Toolkit
    janko-m @jankomarohnic

    View full-size slide

  2. • LOC = 470 (base) + 1550 (plugins)
    • 1 tiny dependency
    • streaming uploads & downloads
    • closes files and deletes tempfiles
    • asynchronous processing & uploads & deletes
    • simple and flexible design
    • support for Sequel
    • security

    View full-size slide

  3. Storage
    class MyStorage # PORO
    def upload(io, id)
    # ...
    end
    def url(id)
    # ...
    end
    def exists?(id)
    # ...
    end
    def delete(id)
    # ...
    end
    # ...
    end

    View full-size slide

  4. Shrine.storages = {
    disk: Shrine::Storage::FileSystem.new("uploads")
    }
    uploader = Shrine.new(:disk)
    uploader.storage #=> #
    uploaded_file = uploader.upload(image)
    uploaded_file #=> #
    uploaded_file.data #=>
    # {
    # "storage" => "disk",
    # "id" => "9260ea09d8effd.jpg",
    # "metadata" => {
    # "filename" => "image.jpg",
    # "size" => 819344,
    # "mime_type" => "image/jpeg",
    # },
    # }
    Uploader

    View full-size slide

  5. uploaded_file.url #=> "uploads/9260ea09d8effd.jpg"
    uploaded_file.metadata #=> {...}
    uploaded_file.read #=> "..."
    uploaded_file.exists? #=> true
    uploaded_file.download #=> #
    uploaded_file.delete
    ...
    Uploaded File

    View full-size slide

  6. Shrine.storages = {
    cache: Shrine::Storage::FileSystem.new(...),
    store: Shrine::Storage::FileSystem.new(...),
    }
    Shrine.plugin :sequel # or :activerecord
    class ImageUploader < Shrine
    # ...
    end
    class User < Sequel::Model
    include ImageUploader[:avatar]
    end
    Attachment
    add_column :users, :avatar_data, :text

    View full-size slide

  7. user = User.create(avatar: File.open(“avatar.jpg"))
    user.avatar_data #=> "{\"storage\":\"store\",\"id\":
    \"sg00943klsg8dd.jpg\",\"metadata\":{...}}"
    user.avatar #=> #
    user.avatar.url #=> "uploads/9dkasd920s.jpg"
    user.destroy
    user.avatar.exists? #=> false
    Attacher

    View full-size slide

  8. class ImageUploader < Shrine
    def process(io, context)
    if context[:phase] == :store
    # ...
    end
    end
    end

    View full-size slide

  9. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    def process(io, context)
    if context[:phase] == :store
    resize_to_limit(io.download, 800, 800)
    end
    end
    end

    View full-size slide

  10. require "image_processing/mini_magick"
    class ImageUploader < Shrine
    include ImageProcessing::MiniMagick
    plugin :versions, names: [:small, :medium, :large]
    def process(io, context)
    if context[:phase] == :store
    size_800 = resize_to_limit(io.download, 800, 800)
    size_500 = resize_to_limit(size_800, 500, 500)
    size_300 = resize_to_limit(size_500, 300, 300)
    {small: size_300, medium: size_500, large: size_800}
    end
    end
    end

    View full-size slide

  11. user.avatar_data #=>
    # {
    # "large": {
    # "id": "lg043.jpg",
    # "storage": "store",
    # "metadata" : {"size": 94832, ...}
    # },
    # "medium": {
    # "id": "kd9fk.jpg",
    # "storage": "store",
    # "metadata" : {"size": 39291, ...}
    # },
    # "small": {
    # "id": "932fl.jpg",
    # "storage": "store",
    # "metadata" : {"size": 10582, ...}
    # }
    # }

    View full-size slide

  12. user.avatar.class #=> Hash
    user.avatar #=>
    # {
    # :large => #,
    # :medium => #,
    # :small => #,
    # }
    user.avatar[:large].size #=> 94832
    user.avatar[:medium].size #=> 39291
    user.avatar[:small].size #=> 10582

    View full-size slide

  13. Direct upload

    View full-size slide

  14. Choose a file
    Fill in other fields
    Submit the form
    Wait for file to be uploaded
    Get redirected

    View full-size slide

  15. Choose a file
    Fill in other fields
    Submit the form
    No waiting
    Get redirected
    File uploads
    using AJAX

    View full-size slide

  16. # config/routes.rb
    Rails.application.routes.draw do
    mount ImageUploader::UploadEndpoint => "/images"
    end
    Shrine.plugin :direct_upload
    // with jQuery-File-Upload
    $('[type="file"]').fileupload({
    url: '/images/cache/avatar',
    paramName: 'file',
    add: (e, data) => { /* Disable submit button */ },
    progress: (e, data) => { /* Add progress bar */ },
    done: (e, data) => { /* Fill hidden field */ },
    });
    Generic

    View full-size slide

  17. Shrine.storages = {
    cache: Shrine::Storage::S3.new(...),
    store: Shrine::Storage::S3.new(...),
    }
    Shrine.plugin :direct_upload, presign: true
    GET /images/cache/presign
    {
    "url": "https://my-bucket.s3-eu-west-1.amazonaws.com",
    "fields": {
    "key": "image.jpg",
    "x-amz-credential": "...",
    "x-amz-algorithm": "...",
    "x-amz-date": "...",
    "x-amz-signature": "..."
    }
    }
    Amazon S3

    View full-size slide

  18. Backgrounding

    View full-size slide

  19. • Processing, uploading and deleting
    • Graceful degradation
    • Supports any backgrounding library
    • Handles parallel jobs
    • Adaptation to changes
    • Debuggable failures
    • Security

    View full-size slide

  20. Shrine.plugin :backgrounding
    Shrine::Attacher.promote { |data| UploadJob.perform_async(data) }
    Shrine::Attacher.delete { |data| DeleteJob.perform_async(data) }
    class UploadJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.promote(data)
    end
    end
    class DeleteJob
    include Sidekiq::Worker
    def perform(data)
    Shrine::Attacher.delete(data)
    end
    end

    View full-size slide

  21. class ImageUploader < Shrine
    plugin :versions, names: [...]
    def process(io, context)
    case context[:phase]
    when :recache
    # process cheap versions in foreground
    when :store
    # process expensive versions in background
    end
    end
    end
    Shrine.plugin :recache

    View full-size slide

  22. Shrine.plugin :logging
    STORE[cache] ImageUploader[:avatar] User[29543] 1 file (0.1s)
    PROCESS[store]: ImageUploader[:avatar] User[29543] 1-3 files (0.22s)
    DELETE[destroy]: ImageUploader[:avatar] User[29543] 3 files (0.07s)
    Shrine.plugin :logging, format: :json
    {“action”:”upload”, “phase”:”cache”, “duration”: 0.16, ...}
    Shrine.plugin :logging, format: :heroku
    action=upload phase=cache files=1 duration=0.16 record_class=User

    View full-size slide

  23. Other plugins
    • determine_mime_type

    – determines actual MIME type from file contents
    • store_dimensions

    – extracts and stores image width & height
    • validation_helpers

    – validation of filesize, extension, MIME type, dimensions
    • parallelize

    – parallelizes uploading and deleting versions
    • … (many more)

    View full-size slide

  24. Storages
    • FileSystem
    • S3
    • Cloudinary
    • Imgix
    • Fog
    • Flickr
    • SQL
    • GridFS

    View full-size slide

  25. • http://github.com/janko-m/shrine
    • http://shrinerb.com (website)
    • http://github.com/janko-m/shrine-example
    • http://twin.github.io/introducing-shrine/
    • http://twin.github.io/file-uploads-asynchronous-
    world/

    View full-size slide