Active Storage

Inside Active Storage: a code review of Rails' new framework

Presented at RailsConf, April 2018
http://railsconf.com/program/sessions#session-562

Rails 5.2 ships with a new library to support file uploads: Active Storage.

In this talk, we will analyze its internal code, learning how it integrates with Active Record and Action Pack to supply amazing features out of the box, from image previews to cloud storage.

We will review its class architecture and delve into the implementation of several Active Storage methods, such as has_one_attached and upload. On the way, we will better understand some Ruby patterns frequently used within Rails: meta-programming, macros, auto-loading, initializers, class attributes.

---

Links from this presentation:

* https://www.clutter.com
* http://guides.rubyonrails.org/active_storage_overview.html
* https://github.com/rails/rails/tree/master/activestorage
* https://evilmartians.com/chronicles/rails-5-2-active-storage-and-beyond

Transcript

1. Inside Active Storage:
   a code review of Rails'
   new framework
   speakerdeck.com/claudiob
   

   View Slide

2. I work for a storage company
   clutter.com
   

   View Slide

3. I will talk about Active Storage
   guides.rubyonrails.org/active_storage_overview.html
   

   View Slide

4. Inside Active Storage
   - How to use in a Rails app (5 min)
   - The main classes of the library (10 min)
   - How the library works (10 min)
   

   View Slide

5. Inside Active Storage
   - How to use in a Rails app (5 min)
   - The main classes of the library (10 min)
   - How the library works (10 min)
   

   View Slide

6. How to use Active Storage
   Let's add an image uploader to a Rails 5.2 scaffold app
   

   View Slide

7. How to use Active Storage
   Start with a Rails scaffold app
   rails new catalog
   cd catalog
   bin/rails generate scaffold cat name:string
   bin/rails db:create db:migrate
   bin/rails server
   

   View Slide

8. How to use Active Storage
   Start with a Rails scaffold app
   

   View Slide

9. How to use Active Storage
   Step 1: add a file field called “picture”
   
   <%= form.label :picture %>
   <%= form.file_field :picture %>
   
   

   View Slide

10. How to use Active Storage
    Step 2: add “picture” to the whitelisted params
    class CatsController < ApplicationController
    def index
    […]
    def cat_params
    params.require(:cat).permit(:name, :picture)
    end
    end
    

    View Slide

11. How to use Active Storage
    Step 3: add “has_one_attached” to the model
    class Cat < ApplicationRecord
    […]
    has_one_attached :picture
    […]
    end
    

    View Slide

12. How to use Active Storage
    Bonus step: display the picture
    <%= image_tag @cat.picture %>
    

    View Slide

13. How to use Active Storage
    More features: variants, previewers, analyzers
    @cat.picture.variant(monochrome: true, flip: "90")
    has_one_attached :document
    @cat.document.preview(resize: "100x100")
    has_many_attached :videos
    @cat.videos.map(&:metadata) # size, angle, duration
    

    View Slide

14. Inside Active Storage
    - How to use in a Rails app (5 min)
    - The main classes of the library (10 min)
    - How the library works (10 min)
    

    View Slide

15. The main classes of Active Storage
    github.com/rails/rails/tree/master/activestorage
    

    View Slide

16. The main classes of Active Storage
    bundle open activestorage
    

    View Slide

17. The main classes of Active Storage
    1. ActiveStorage::Service
    2. ActiveStorage::Blob
    3. ActiveStorage::Attachment
    

    View Slide

18. The main classes of Active Storage
    1. ActiveStorage::Service
    2. ActiveStorage::Blob
    3. ActiveStorage::Attachment
    

    View Slide

19. ActiveStorage::Service
    Moves bytes from memory to disk
    Disk Storage (5.8ms) Uploaded file to key: nty3U4Bzagqj3ekd (checksum: LPzlVe+CBQ==)
    #{Rails.root}/storage/
    nt/y3/nty3U4Bzagqj3ekd
    ActionDispatch::
    Http::UploadedFile
    

    View Slide

20. What is ActiveStorage::Service
    An abstract class serving as an interface for implementations
    class ActiveStorage::Service
    def upload(key, io, checksum: nil)
    raise NotImplementedError
    end
    def download(key)
    raise NotImplementedError
    end
    end
    

    View Slide

21. Inside ActiveStorage::Service (disk)
    Moves bytes from memory to local disk and back
    class ActiveStorage::Service::DiskService < Service
    def upload(key, io, checksum: nil)
    IO.copy_stream(io, make_path_for(key))
    end
    def download(key)
    File.binread path_for(key)
    end
    end
    

    View Slide

22. Inside ActiveStorage::Service (S3)
    Moves bytes from memory to AWS S3 and back
    class ActiveStorage::Service::S3Service < Service
    def upload(key, io, checksum: nil)
    object_for(key).put(upload_options.merge(body: io))
    end
    def download(key)
    object_for(key).get.body.read
    end
    end
    

    View Slide

23. The main classes of Active Storage
    1. ActiveStorage::Service
    2. ActiveStorage::Blob
    3. ActiveStorage::Attachment
    

    View Slide

24. ActiveStorage::Blob
    Stores a reference to the uploaded file
    ActiveStorage::Blob Create (5.8ms) INSERT INTO active_storage_blobs (key, filename, …
    key: "nty3U4Bzagqj3ekd"
    filename: "fluffy.jpg"
    content_type: "image/jpeg"
    byte_size: 1565751
    ActionDispatch::
    Http::UploadedFile
    

    View Slide

25. What is ActiveStorage::Blob
    A record in the database with information about one file
    create_table "active_storage_blobs", do |t|
    t.string "key", null: false
    t.string "filename", null: false
    t.string "content_type"
    t.text "metadata"
    t.bigint "byte_size", null: false
    t.string "checksum", null: false
    t.index ["key"], unique: true
    end
    

    View Slide

26. What is ActiveStorage::Blob
    Migrations are provided to generate the required tables
    

    View Slide

27. Inside ActiveStorage::Blob
    Extracts file metadata and invokes the service upload
    class ActiveStorage::Blob < ActiveRecord::Base
    def upload(io)
    self.checksum = compute_checksum_in_chunks(io)
    self.content_type = extract_content_type(io)
    self.byte_size = io.size
    service.upload(key, io, checksum: checksum)
    end
    end
    

    View Slide

28. The main classes of Active Storage
    1. ActiveStorage::Service
    2. ActiveStorage::Blob
    3. ActiveStorage::Attachment
    

    View Slide

29. ActiveStorage::Attachment
    Associates a blob with a record
    ActiveStorage::Blob Create (5.8ms) INSERT INTO active_storage_attachments (name, …
    name: "picture"
    record_type: "Cat"
    record_id: 4
    blob_id: 2
    ActionDispatch::
    Http::UploadedFile
    

    View Slide

30. What is ActiveStorage::Attachment
    A record in the database associating a blob to its owner
    create_table "active_storage_attachments" do |t|
    t.string "name", null: false
    t.string "record_type", null: false
    t.integer "record_id", null: false
    t.integer "blob_id", null: false
    t.index ["blob_id"]
    t.index ["record_type", "record_id", "name", "blob_id"],
    unique: true
    end
    

    View Slide

31. Inside ActiveStorage::Attachment
    A record in the database associating a blob to its owner
    class ActiveStorage::Attachment < ActiveRecord::Base
    self.table_name = "active_storage_attachments"
    belongs_to :record, polymorphic: true, touch: true
    belongs_to :blob, class_name: "ActiveStorage::Blob"
    end
    

    View Slide

32. The main classes of Active Storage
    1. ActiveStorage::Service
    2. ActiveStorage::Blob
    3. ActiveStorage::Attachment
    

    View Slide

33. Inside Active Storage
    - How to use in a Rails app (5 min)
    - The main classes of the library (10 min)
    - How the library works (10 min)
    

    View Slide

34. How Active Storage works
    How does a single method add this whole behavior?
    class Cat < ApplicationRecord
    […]
    has_one_attached :picture
    […]
    end
    

    View Slide

35. How Active Storage works
    When loading Active Record, extend it with new methods
    class ActiveStorage::Engine < Rails::Engine
    initializer "active_storage.attached" do
    ActiveSupport.on_load(:active_record) do
    extend ActiveStorage::Attached::Macros
    end
    end
    end
    

    View Slide

36. How Active Storage works
    Meta-programming: define methods with a given name
    module ActiveStorage::Attached::Macros
    def has_one_attached(name)
    class_eval <<-CODE, __FILE__, __LINE__ + 1
    def #{name}=(attachable)
    #{name}.attach(attachable)
    end
    [… 5 more methods…]
    

    View Slide

37. How Active Storage works
    Meta-programming: define methods with a given name
    module ActiveStorage::Attached::Macros
    def has_one_attached(:picture)
    class_eval <<-CODE, __FILE__, __LINE__ + 1
    def picture=(attachable)
    picture.attach(attachable)
    end
    [… 5 more methods…]
    

    View Slide

38. How Active Storage works
    When a picture is attached to a cat…
    class Cat < ApplicationRecord
    has_one_attached :picture
    def picture=(attachable)
    picture.attach(attachable)
    end
    end
    

    View Slide

39. How Active Storage works
    …a picture is instantiated…
    class Cat < ApplicationRecord
    has_one_attached :picture
    def picture
    ActiveStorage::Attached::One.new(
    "picture", self, dependent: :purge_later)
    end
    end
    

    View Slide

40. How Active Storage works
    …and a blob and an attachment are added to the database.
    class ActiveStorage::Attached::One < Attached
    def attach(attachable)
    blob = create_blob_from(attachable)
    write_attachment build_attachment(blob: blob)
    […]
    end
    end
    

    View Slide

41. How Active Storage works
    When a cat is destroyed…
    class Cat < ApplicationRecord
    has_one_attached :picture
    if dependent == :purge_later # true by default
    after_destroy_commit { picture.purge_later }
    end
    end
    

    View Slide

42. How Active Storage works
    …the picture purges the attachment…
    class ActiveStorage::Attached::One < Attached
    def purge_later
    attachment.purge_later if attached?
    end
    end
    

    View Slide

43. How Active Storage works
    …the attachment purges the blob…
    class ActiveStorage::Attachment < ActiveRecord::Base
    def purge_later
    blob.purge_later
    destroy
    end
    end
    

    View Slide

44. How Active Storage works
    …the blob enqueues a purge job…
    class ActiveStorage::Blob < Attached
    def purge_later
    ActiveStorage::PurgeJob.perform_later(self)
    end
    end
    

    View Slide

45. How Active Storage works
    …the job purges the blob…
    class ActiveStorage::PurgeJob < ActiveJob::Base
    def perform(blob)
    blob.purge
    end
    end
    

    View Slide

46. How Active Storage works
    …the job purges the blob…
    class ActiveStorage::Blob < Attached
    def purge
    delete
    destroy
    end
    end
    

    View Slide

47. How Active Storage works
    …the blob asks the service to delete the file…
    class ActiveStorage::Blob < Attached
    def delete
    service.delete(key)
    end
    end
    

    View Slide

48. How Active Storage works
    …the service deletes the file.
    class ActiveStorage::DiskService
    def delete(key)
    File.delete path_for(key)
    end
    end
    

    View Slide

49. More to explore about Active Storage
    - JavaScript module
    - ActiveSupport::Notification
    - Direct and resolve routes
    

    View Slide

50. Inside Active Storage:
    a code review of Rails'
    new framework
    speakerdeck.com/claudiob
    

    View Slide