Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Handling File Uploads for a modern developer

Handling File Uploads for a modern developer

In Ruby we have an abundance of options for handling file attachments. Unfortunately, most file attachment gems fall short when it comes to tailoring the attachment flow to your needs and preferences, often leaving your app with suboptimal user experience.

This talk will introduce the Shrine gem and go over some of the modern best practices for handling file attachments. We will cover topics such as image processing (“Is ImageMagick always the best option?”), asynchronous file uploads (“Which JavaScript solution should I choose?”), and handling large uploads (“How do I help users that have a flaky internet connection?”).

Janko Marohnić

September 07, 2019
Tweet

More Decks by Janko Marohnić

Other Decks in Programming

Transcript

  1. Handling File Uploads
    For a modern developer

    View full-size slide

  2. 2007
    Paperclip 2008
    CarrierWave
    2014
    Refile
    2018
    ActiveStorage
    2009
    Dragonfly
    2015
    ! Shrine

    View full-size slide

  3. Janko Marohnic
    • from Croatia ", living in Czechia #
    • Ruby off Rails developer
    • creator of Shrine
    @janko
    @jankomarohnic

    View full-size slide

  4. Active Storage?

    View full-size slide

  5. Active Storage Shrine
    Rails & Active Record only
    Rails, Hanami, Sinatra, Roda...
    Active Record, Sequel, ROM...
    framework library
    integrated experience more control
    more opinionated more features

    View full-size slide

  6. Metadata

    &

    Validation

    View full-size slide

  7. class PhotosController < AC::Base
    def create
    photo = Photo.new(photo_params)
    if photo.valid?
    photo.save
    # ...
    else
    # ...
    end
    end
    end

    View full-size slide

  8. class ImageUploader < Shrine
    TYPES = %w[image/jpeg image/png ...]
    EXTENSIONS = %w[jpg jpeg png ...]
    Attacher.validate do
    validate_max_size 10*1024*1024
    validate_mime_type TYPES
    validate_extension EXTENSIONS
    end
    end

    View full-size slide

  9. \377\330\377························· (JPEG)
    \211PNG······························ (PNG)
    ········WEBP························· (WebP)
    MIME Type
    `file --mime-type -b #{file.path}`
    #=> "image/jpeg"
    MimeMagic.by_magic(file).type
    #=> "image/jpeg"
    Marcel::MimeType.for(file)
    #=> "image/jpeg"

    View full-size slide

  10. class PhotosController < AC::Base
    def create
    file = params[:photo][:image]
    file.read #=> ""
    file.filename #=> "nature.jpg"
    file.content_type #=> "image/jpeg"
    photo = Photo.new(image: file)
    photo.image.mime_type #=> "text/x-php"
    photo.valid? #=> false
    end
    end
    MIME Type

    View full-size slide

  11. https://www.bamsoftware.com/hacks/deflate.html
    $ identify spark.png
    ... 225000x225000 ... 6.1MB

    Attacher.validate do
    # ...
    if validate_mime_type IMAGE_TYPES
    validate_max_dimensions [5000, 5000]
    end
    end
    Image Dimensions

    View full-size slide

  12. class VideoUploader < Shrine
    add_metadata :duration do |file|
    FFMPEG::Movie.new(file.path).duration
    end
    Attacher.validate do
    if file.duration > 5*60*60
    errors << "is too long (max is 5h)"
    end
    end
    end
    Custom Metadata

    View full-size slide

  13. {
    "id": "b93777246f10e509a553.mp4",
    "storage": "store",
    "metadata": {
    "size": 234837,
    "filename": "matrix.mp4",
    "mime_type": "video/mp4",
    "duration": 9000,
    ...
    }
    }
    SELECT video_data FROM movies;

    View full-size slide

  14. image = MiniMagick::Image.open(file)
    image.resize("500x500")
    image.path #=> "/path/to/resized.jpg"
    $ gem install mini_magick

    View full-size slide

  15. Resize to fit
    Resize to limit
    Resize to fill

    View full-size slide

  16. magick = ImageProcessing::MiniMagick

    magick = magick.source(image)
    magick.resize_to_limit! 500, 500
    magick.resize_to_fit! 500, 500
    magick.resize_to_fill! 500, 500
    magick.resize_and_pad! 500, 500
    $ gem install image_processing

    View full-size slide

  17. thumbnail = ImageProcessing::MiniMagick

    .source(image)
    .resize_to_limit!(500, 500)
    $ gem install image_processing
    $ convert source.jpg -auto-orient -resize
    500x500> -sharpen 0x1 destination.jpg

    View full-size slide

  18. thumbnail = ImageProcessing::MiniMagick

    .source(image)
    .resize_to_limit!(500, 500)
    $ gem install image_processing
    $ convert source.jpg -auto-orient -resize
    500x500> -sharpen 0x1 destination.jpg

    View full-size slide

  19. ImageProcessing::MiniMagick

    .source(image)
    .resize_to_limit!(500, 500)
    $ brew install imagemagick

    View full-size slide

  20. ImageProcessing::Vips

    .source(image)
    .resize_to_limit!(500, 500)
    $ brew install vips

    View full-size slide

  21. THUMBNAILS = {
    xs: [200, 200],
    s: [400, 400],
    m: [600, 600],
    l: [800, 800],
    xl: [1200, 1200],
    }
    THUMBNAILS.each do |name, (width, height)|
    ImageProcessing::Backend
    .source(original)
    .resize_to_limit!(width, height)
    end

    View full-size slide

  22. 0,47s
    2,15s
    ImageMagick libvips

    View full-size slide

  23. Processing

    On-the-fly

    View full-size slide

  24. Active Storage
    https://...//...
    photo.image.variant(
    resize_to_limit: [600, 400],
    strip: true,
    quality: 85,
    interlace: "JPEG",
    )

    View full-size slide

  25. Shrine
    https://.../thumbnail/600/400/...
    photo.image.derivation_url(
    :thumbnail, # <-- name
    600, 400 # <-- args
    )
    Shrine.derivation :thumbnail do |file, w, h|
    ImageProcessing::Vips.source(file)
    .strip
    .saver(quality: 85, interlace: "JPEG")
    .resize_to_limit!(w.to_i, h.to_i)
    end

    View full-size slide

  26. Processing

    on attachment

    View full-size slide

  27. class ImageUploader < Shrine
    derivatives_processor :thumbs do |file|
    vips = ImageProcessing::Vips.source(file)
    { large: vips.resize_to_fit!(800, 800),
    medium: vips.resize_to_fit!(500, 500),
    small: vips.resize_to_fit!(300, 300) }
    end
    end
    photo = Photo.new(image: file)
    photo.image_derivatives!(:thumbs)
    photo.image_derivatives #=>
    # { large: #,
    # medium: #,
    # small: # }

    View full-size slide

  28. class VideoUploader < Shrine
    derivatives_processor :transcode do |file|
    movie = FMMPEG::Movie.new(file.path)
    transcoded = Tempfile.new(["", ".mp4"])
    screenshot = Tempfile.new(["", ".jpg"])
    movie.transcode(transcoded.path)
    movie.screenshot(screenshot.path)
    { transcoded: transcoded,
    screenshot: screenshot }
    end
    end

    View full-size slide

  29. Direct
    Uploads

    View full-size slide

  30. refile.js
    activestorage.js
    shrine.js ?

    View full-size slide

  31. jQuery-File-Upload
    Dropzone.js
    FineUploader



    View full-size slide

  32. Uppy()
    .use(Dashboard, { /* ... */ })

    View full-size slide

  33. Uppy()
    .use(DragDrop, { /* ... */ })
    .use(StatusBar, { /* ... */ })

    View full-size slide

  34. Uppy()
    .use(FileInput, { /* ... */ })
    .use(ProgressBar, { /* ... */ })

    View full-size slide

  35. { "id": "xyz.jpg", ... }
    submit
    { "id": "xyz.jpg", ... }

    POST /upload
    App S3

    View full-size slide

  36. mount Shrine.upload_endpoint(:cache),
    to: "/upload"
    POST /upload
    uppy.use(Uppy.XHRUpload, {
    endpoint: '/upload'
    })

    View full-size slide

  37. { "url": "...", "fields": "..." }
    submit
    { "id": "xyz.jpg", ... }
    App S3
    GET /s3/params

    POST s3.aws.com

    View full-size slide

  38. mount Shrine.presign_endpoint(:cache),
    to: "/s3/params"
    GET /s3/params
    uppy.use(Uppy.AwsS3, {
    companionUrl: '/'
    })

    View full-size slide

  39. Resumable
    Uploads

    View full-size slide

  40. 0% 0% 0% 0% 0%

    View full-size slide

  41. 100%
    80%


    View full-size slide

  42. App
    S3

    S3 Multipart Upload

    View full-size slide

  43. $ gem install uppy-s3_multipart
    uppy.use(Uppy.AwsS3Multipart, {
    companionUrl: '/'
    })
    mount Shrine.uppy_s3_multipart(:cache),
    to: "/s3/multipart"

    View full-size slide

  44. tus.io
    • HTTP protocol for resumable uploads
    • Numerous implementations:
    • Client – JavaScript, iOS, Android, ...
    • Server – Ruby, Node, Go, Python, Java, PHP, ...

    View full-size slide

  45. S3

    tus protocol storage API
    tus server
    tus client
    App

    View full-size slide

  46. $ gem install tus-ruby-server
    uppy.use(Uppy.Tus, {
    endpoint: '/files'
    })
    mount Tus::Server, to: "/files"

    View full-size slide

  47. Recap
    • Validate and persist metadata
    • Processing "on attachment" or "on-the-fly"
    ImageProcessing – ImageMagick & libvips
    • Direct uploads
    Uppy
    simple uploads (to your app or to cloud)
    resumable uploads (S3 multipart upload or tus.io)

    View full-size slide

  48. • https://shrinerb.com/
    • https://github.com/janko/image_processing
    • https://libvips.github.io/libvips/
    • https://uppy.io/
    • https://tus.io/
    • https://github.com/janko/tus-ruby-server
    • https://github.com/janko/uppy-s3_multipart
    • https://twin.github.io

    View full-size slide