Share
Embed
un-uploader-demo
An example full-stack upload workflow for Ember applications
Mattia Gheda
June 06, 2014
Transcript
18 slides
-
Server Client S3 Pusher 3. Upload to signed URL 2.
Generate signed URL for S3 4. Persist image model 6. Notify of completed / failed processing 5. Fetch uploaded file for processing The Flow 1. User selects files -
• Provides file upload workflow management • Displays processed content
The Parts Server S3 Pusher • Generates signed S3 upload URL • Processes uploaded files via worker queue • Provides REST API for Ember Data • Very cost effective redundant storage • Handles direct upload from browser • Stores processed versions of uploads • Simple and robust pub/sub message platform • Notifies client of processing events -
Advantages • Direct upload from client to S3 • Upload
progress notification • Async processing on the server • Notify client on completed processing • Notify other connected clients of newly created objects -
Client / 1 - File-picker component • File-input (with multiple
files) • Sends an event to the controller with the list of images • Styled to be hidden behind a button -
Client / 2 - ArrayController • Has the list of
the uploaded images • Wraps images with an item-controller • Every image is handled independently • Handles the action from the component ◦ Creates Ember Data records ◦ Sends the startUpload action to each added image -
Client / 3 - Item Controller uploading Upload to S3
signing Get signed url processing Transcode file processed failed initialized start upload process complete error error (if file is invalid) (if processing fails) When a state is entered an action is performed -
Client / 4 - Notes and Caveats • IE 10+
Uses native XHR2 upload, CORS • Possible to support older browsers with extra code • Basic drag-n-drop thanks to Chrome/Firefox • Pusher is available to the controller via an initializer, fancier solutions like ember-pusher exist • Support for drop file outside of add image button also possible, not implemented in example • You might want to use separate arrays for the existing images and the ones under upload, moving them from one list to the other on complete -
Server / 1 - Rails • POST /api/uploads endpoint to
get a signed-url for the upload • POST /api/images endpoint to create the server-side model when the upload is completed • After the Image is created a job for processing is enqueued • Rails can be easily swapped out for another server side technology (Node, Go, etc.) -
Server / 2 - Sidekiq and Pusher • Sidekiq processes
the images (with ImageMagick) • A Pusher event is sent when the processing succeeds/fails • (Replace Pusher with Faye, Socket.io, message_bus) -
Server / 3 - Notes and Caveats • Ember app
is currently built by ember-cli and served by Nginx / Rails • A better approach would be to push assets to S3 and serve only the index from Rails (See Luke Melia’s great talk) • S3 images sometimes are not immediately available, the current background job implementation does polling, web-hooks are another viable option -
Final Notes • This is a mid-level complexity system •
Designed for more than resizing images: ◦ Audio processing ◦ Video processing ◦ Data analysis ◦ … • If S3 is a concern, direct uploads are possible -
The Future • Solid testing for upload process (state machine
is tested and UI too) • Libraryification (omg gross I just typed that) of the client and server ◦ {{un-uploader}} ◦ Un.Uploader.configure({ … }) • Add other Server-side options (PRs are welcome :) )