Active Storage

Inside Active Storage: a code review of Rails' new framework

Presented at RailsConf, April 2018
http://railsconf.com/program/sessions#session-562

Rails 5.2 ships with a new library to support file uploads: Active Storage.

In this talk, we will analyze its internal code, learning how it integrates with Active Record and Action Pack to supply amazing features out of the box, from image previews to cloud storage.

We will review its class architecture and delve into the implementation of several Active Storage methods, such as has_one_attached and upload. On the way, we will better understand some Ruby patterns frequently used within Rails: meta-programming, macros, auto-loading, initializers, class attributes.

---

Links from this presentation:

* https://www.clutter.com
* http://guides.rubyonrails.org/active_storage_overview.html
* https://github.com/rails/rails/tree/master/activestorage
* https://evilmartians.com/chronicles/rails-5-2-active-storage-and-beyond

Transcript

1. Inside Active Storage: a code review of Rails' new framework speakerdeck.com/claudiob
2. I work for a storage company clutter.com
3. I will talk about Active Storage guides.rubyonrails.org/active_storage_overview.html
4. Inside Active Storage - How to use in a Rails app (5 min) - The main classes of the library (10 min) - How the library works (10 min)
5. Inside Active Storage - How to use in a Rails app (5 min) - The main classes of the library (10 min) - How the library works (10 min)
6. How to use Active Storage Let's add an image uploader to a Rails 5.2 scaffold app
7. How to use Active Storage Start with a Rails scaffold app rails new catalog cd catalog bin/rails generate scaffold cat name:string bin/rails db:create db:migrate bin/rails server
8. How to use Active Storage Start with a Rails scaffold app
9. How to use Active Storage Step 1: add a file field called “picture” <div class="field"> <%= form.label :picture %> <%= form.file_field :picture %> </div>
10. How to use Active Storage Step 2: add “picture” to the whitelisted params class CatsController < ApplicationController def index […] def cat_params params.require(:cat).permit(:name, :picture) end end
11. How to use Active Storage Step 3: add “has_one_attached” to the model class Cat < ApplicationRecord […] has_one_attached :picture […] end
12. How to use Active Storage Bonus step: display the picture <%= image_tag @cat.picture %>
13. How to use Active Storage More features: variants, previewers, analyzers @cat.picture.variant(monochrome: true, flip: "90") has_one_attached :document @cat.document.preview(resize: "100x100") has_many_attached :videos @cat.videos.map(&:metadata) # size, angle, duration
14. Inside Active Storage - How to use in a Rails app (5 min) - The main classes of the library (10 min) - How the library works (10 min)
15. The main classes of Active Storage github.com/rails/rails/tree/master/activestorage
16. The main classes of Active Storage bundle open activestorage
17. The main classes of Active Storage 1. ActiveStorage::Service 2. ActiveStorage::Blob 3. ActiveStorage::Attachment
18. The main classes of Active Storage 1. ActiveStorage::Service 2. ActiveStorage::Blob 3. ActiveStorage::Attachment
19. ActiveStorage::Service Moves bytes from memory to disk Disk Storage (5.8ms) Uploaded file to key: nty3U4Bzagqj3ekd (checksum: LPzlVe+CBQ==) #{Rails.root}/storage/ nt/y3/nty3U4Bzagqj3ekd ActionDispatch:: Http::UploadedFile
20. What is ActiveStorage::Service An abstract class serving as an interface for implementations class ActiveStorage::Service def upload(key, io, checksum: nil) raise NotImplementedError end def download(key) raise NotImplementedError end end
21. Inside ActiveStorage::Service (disk) Moves bytes from memory to local disk and back class ActiveStorage::Service::DiskService < Service def upload(key, io, checksum: nil) IO.copy_stream(io, make_path_for(key)) end def download(key) File.binread path_for(key) end end
22. Inside ActiveStorage::Service (S3) Moves bytes from memory to AWS S3 and back class ActiveStorage::Service::S3Service < Service def upload(key, io, checksum: nil) object_for(key).put(upload_options.merge(body: io)) end def download(key) object_for(key).get.body.read end end
23. The main classes of Active Storage 1. ActiveStorage::Service 2. ActiveStorage::Blob 3. ActiveStorage::Attachment
24. ActiveStorage::Blob Stores a reference to the uploaded file ActiveStorage::Blob Create (5.8ms) INSERT INTO active_storage_blobs (key, filename, … key: "nty3U4Bzagqj3ekd" filename: "fluffy.jpg" content_type: "image/jpeg" byte_size: 1565751 ActionDispatch:: Http::UploadedFile
25. What is ActiveStorage::Blob A record in the database with information about one file create_table "active_storage_blobs", do |t| t.string "key", null: false t.string "filename", null: false t.string "content_type" t.text "metadata" t.bigint "byte_size", null: false t.string "checksum", null: false t.index ["key"], unique: true end
26. What is ActiveStorage::Blob Migrations are provided to generate the required tables
27. Inside ActiveStorage::Blob Extracts file metadata and invokes the service upload class ActiveStorage::Blob < ActiveRecord::Base def upload(io) self.checksum = compute_checksum_in_chunks(io) self.content_type = extract_content_type(io) self.byte_size = io.size service.upload(key, io, checksum: checksum) end end
28. The main classes of Active Storage 1. ActiveStorage::Service 2. ActiveStorage::Blob 3. ActiveStorage::Attachment
29. ActiveStorage::Attachment Associates a blob with a record ActiveStorage::Blob Create (5.8ms) INSERT INTO active_storage_attachments (name, … name: "picture" record_type: "Cat" record_id: 4 blob_id: 2 ActionDispatch:: Http::UploadedFile
30. What is ActiveStorage::Attachment A record in the database associating a blob to its owner create_table "active_storage_attachments" do |t| t.string "name", null: false t.string "record_type", null: false t.integer "record_id", null: false t.integer "blob_id", null: false t.index ["blob_id"] t.index ["record_type", "record_id", "name", "blob_id"], unique: true end
31. Inside ActiveStorage::Attachment A record in the database associating a blob to its owner class ActiveStorage::Attachment < ActiveRecord::Base self.table_name = "active_storage_attachments" belongs_to :record, polymorphic: true, touch: true belongs_to :blob, class_name: "ActiveStorage::Blob" end
32. The main classes of Active Storage 1. ActiveStorage::Service 2. ActiveStorage::Blob 3. ActiveStorage::Attachment
33. Inside Active Storage - How to use in a Rails app (5 min) - The main classes of the library (10 min) - How the library works (10 min)
34. How Active Storage works How does a single method add this whole behavior? class Cat < ApplicationRecord […] has_one_attached :picture […] end
35. How Active Storage works When loading Active Record, extend it with new methods class ActiveStorage::Engine < Rails::Engine initializer "active_storage.attached" do ActiveSupport.on_load(:active_record) do extend ActiveStorage::Attached::Macros end end end
36. How Active Storage works Meta-programming: define methods with a given name module ActiveStorage::Attached::Macros def has_one_attached(name) class_eval <<-CODE, __FILE__, __LINE__ + 1 def #{name}=(attachable) #{name}.attach(attachable) end [… 5 more methods…]
37. How Active Storage works Meta-programming: define methods with a given name module ActiveStorage::Attached::Macros def has_one_attached(:picture) class_eval <<-CODE, __FILE__, __LINE__ + 1 def picture=(attachable) picture.attach(attachable) end [… 5 more methods…]
38. How Active Storage works When a picture is attached to a cat… class Cat < ApplicationRecord has_one_attached :picture def picture=(attachable) picture.attach(attachable) end end
39. How Active Storage works …a picture is instantiated… class Cat < ApplicationRecord has_one_attached :picture def picture ActiveStorage::Attached::One.new( "picture", self, dependent: :purge_later) end end
40. How Active Storage works …and a blob and an attachment are added to the database. class ActiveStorage::Attached::One < Attached def attach(attachable) blob = create_blob_from(attachable) write_attachment build_attachment(blob: blob) […] end end
41. How Active Storage works When a cat is destroyed… class Cat < ApplicationRecord has_one_attached :picture if dependent == :purge_later # true by default after_destroy_commit { picture.purge_later } end end
42. How Active Storage works …the picture purges the attachment… class ActiveStorage::Attached::One < Attached def purge_later attachment.purge_later if attached? end end
43. How Active Storage works …the attachment purges the blob… class ActiveStorage::Attachment < ActiveRecord::Base def purge_later blob.purge_later destroy end end
44. How Active Storage works …the blob enqueues a purge job… class ActiveStorage::Blob < Attached def purge_later ActiveStorage::PurgeJob.perform_later(self) end end
45. How Active Storage works …the job purges the blob… class ActiveStorage::PurgeJob < ActiveJob::Base def perform(blob) blob.purge end end
46. How Active Storage works …the job purges the blob… class ActiveStorage::Blob < Attached def purge delete destroy end end
47. How Active Storage works …the blob asks the service to delete the file… class ActiveStorage::Blob < Attached def delete service.delete(key) end end
48. How Active Storage works …the service deletes the file. class ActiveStorage::DiskService def delete(key) File.delete path_for(key) end end
49. More to explore about Active Storage - JavaScript module - ActiveSupport::Notification - Direct and resolve routes
50. Inside Active Storage: a code review of Rails' new framework speakerdeck.com/claudiob