Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How Sprockets works

How Sprockets works

Almost all applications have assets like CSS, JavaScript and others. That means the asset pipeline is an integral part of the Ruby on Rails framework. In this talk we'll show you how the asset pipeline works, and how you can take full advantage of the asset pipeline's features. Ever wondered how to convert an SVG to PNG automatically? Wanted to know what exactly happens to your CoffeeScript files? We'll explore that, and more.

0525b332aafb83307b32d9747a93de03?s=128

Rafael França

May 04, 2016
Tweet

Transcript

  1. How Sprockets works

  2. Rafael França rafaelfranca rafaelfranca

  3. Core Member

  4. None
  5. Maintainer

  6. Ruby on Rails Bot rails-bot

  7. Spring Turbolinks jquery-ujs Action View Active Record Active Job Action

    Controller Action Mailer web-console Sprockets rails-html-sanitizer
  8. Agenda • Why do we need an assets pipeline? •

    The gems responsible • How it works in a Rails application • How to extend it
  9. Why do we need an assets pipeline?

  10. Where should I put my assets?

  11. public/*

  12. Code organization X Performance

  13. Small self-contained files

  14. Fewer assets request

  15. Legible code

  16. Fewer bytes being transferred

  17. New technologies • CoffeeScript • Sass • ES6

  18. Rails assets pipeline

  19. How it works in Rails

  20. Assets live in app/assets • app/assets/stylesheet • app/assets/javascript • app/assets/images

    • There are also: lib/assets/ and vendor/assets/
  21. Assets are compiled on-the-fly in development and need to be

    precompiled in production
  22. Assets names are generated with digest to cache busting •

    public/assets/application- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322 950ec65b6.js
  23. The gems responsible

  24. Gems • sprockets • sprockets-rails • sass-rails • execjs •

    coffee-rails
  25. sprockets

  26. Compiles and serves assets

  27. Processors pipeline

  28. Sprockets' key components • processors • transformers • compressors •

    directives • environment • manifest • pipelines
  29. Processor

  30. Most important component in Sprockets

  31. Any callable. Accepts an input Hash and returns a Hash

    of metadata
  32. -> (input) { data = input[:data].gsub(";", "") { data: data

    } }
  33. Input hash • :data - String asset contents • :environment

    - Current Sprockets::Environment instance. • :cache - A Sprockets::Cache instance. • :uri - String Asset URI. • :source_path - String full path to original file. • :load_path - String current load path for filename. • :name - String logical path for filename. • :content_type - String content type of the output asset. • :metadata - Hash of processor metadata.
  34. Return hash • :data - Replaces the assets input[:data] to

    the next processor in the chain. • :required - A Set of String Asset URIs that the Bundle processor should concatenate together. • :stubbed - A Set of String Asset URIs that will be omitted from the :required set. • :links - A Set of String Asset URIs that should be compiled along with this asset. • :dependencies - A Set of String Cache URIs that should be monitored for caching. • :map - An Array of source maps for the asset. • :charset - The mime charset for an asset.
  35. Builtin processors • BabelProcessor • CoffeScriptProcessor • SassProcessor

  36. Bundler processor

  37. A processor that run concatenated assets rather than individual files

  38. register_bundle_processor 'application/javascript', Bundle register_bundle_processor 'text/css', Bundle

  39. Bundle processor takes a single file asset and prepends all

    the `:required` URIs to the contents.
  40. Transformer

  41. A processor that convert a file from one format to

    another
  42. register_transformer ‘text/coffeescript', ‘application/javascript', CoffeeScriptProcessor

  43. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js,

    map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
  44. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js,

    map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
  45. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js,

    map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
  46. Compressor

  47. Special kind of bundle processor

  48. register_compressor ‘application/javascript’, :uglify, UglifierCompressor

  49. env.js_compressor = :uglify

  50. Directives

  51. Special comments that declares bundlers and their dependencies

  52. // app/assets/javascripts/application.js //= require jquery //= require jquery-ui //= require

    users //= require_tree .
  53. Rails.application.config.assets.precompile << %w(application.js application.css)

  54. LOOSE_APP_ASSETS = lambda do |logical_path, filename| filename.start_with?(::Rails.root.join("app/assets").to_s) && !['.js', '.css',

    ''].include?(File.extname(logical_path)) end config.assets.precompile = [LOOSE_APP_ASSETS, /(?:\/|\\|\A)application\.(css|js)$/]
  55. // app/assets/config/manifest.js //= link_tree ../images //= link_directory ../javascripts .js //=

    link_directory ../stylesheets .css //= link my_engine // my_engine/app/assets/config/my_engine.js //= link_tree ../images/bootstrap
  56. Directives • require • require_directory • require_tree • require_self •

    link • link_directory • link_tree • depend_on • depend_on_asset • stub
  57. Environment

  58. It has methods to retrieve and serve assets, manipulating load

    path, and registering processors.
  59. environment = Sprockets::Environment.new environment.find_asset('application.js')

  60. environment = Sprockets::Environment.new environment.register_transformer 'image/svg+xml', 'image/png', SVGTransformer.new

  61. What is the manifest

  62. Logs the contents of assets compiled to a single directory.

    It records basic attributes about the asset for fast lookup without having to compile. A pointer from each logical path indicates which fingerprinted asset is the current one.
  63. Points the asset path to the fingerprinted version

  64. javascript_include_tag 'application' # <script # src="/assets/application.debug- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322950ec65b6.js" # ></script>

  65. { "application.js" => "application-2e8e9a7c6b0aafa0c9bdeec90ea30213.js", "jquery.js" => "jquery-ae0908555a245f8266f77df5a8edca2e.js" }

  66. { "application-2e8e9a7c6b0aafa0c9bdeec90ea30213.js" => { 'logical_path' => "application.js", 'mtime' => "2011-12-13T21:47:08-06:00",

    'digest' => "2e8e9a7c6b0aafa0c9bdeec90ea30213" } }
  67. More • mime types • dependency resolver • tranformer suffix

    • bundle metadata reducer
  68. sprockets-rails

  69. Integrates Sprockets to the Rails application

  70. Defines helpers

  71. • javascript_include_tag • stylesheet_link_tag

  72. Configures the Sprockets environment

  73. Checks the precompile list

  74. None
  75. sass-rails

  76. Integrates the Sass processor with the Rails application

  77. Defines generators

  78. Creates a importer that knows how to handle globs and

    ERB
  79. @import "foo/*"; // bar.scss.erb @import "bar";

  80. Configures the Sass processor

  81. execjs

  82. It allows you to run JavaScript code from Ruby

  83. It uses the JavaScript environment available in the machine

  84. • therubyracer - Google V8 embedded within Ruby • therubyrhino

    - Mozilla Rhino embedded within JRuby • Duktape.rb - Duktape JavaScript interpreter • Node.js • Apple JavaScriptCore - Included with Mac OS X • Microsoft Windows Script Host (JScript) • Google V8
  85. require "execjs" require "open-uri" source = open("http://coffeescript.org/extras/coffee-script.js").read context = ExecJS.compile(source)

    context.call("CoffeeScript.compile", "square = (x) -> x * x", bare: true) # => "var square;\nsquare = function(x) {\n return x * x;\n};"
  86. It is used by the coffee-script gem to compile CoffeeScript

    code
  87. coffee-rails

  88. Configures generators

  89. Defines a template handler

  90. How the assets are generated in development.

  91. <%= javascript_include_tag 'application' %>

  92. javascript_include_tag 'application' # <script # src="/assets/application.debug- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322950ec65b6.js" # ></script>

  93. GET /assets/application.debug- *.js

  94. sprockets-rails use the Sprockets’ pipeline `debug`

  95. register_pipeline :debug do [SourceMapCommentProcessor] end

  96. The SourceMapCommentProccessor will generate the asset bundle and add the

    source map comment
  97. //# sourceMappingURL=application.js- bf4c2d805fe200d419d3ea539cd105732f1e1b2848689fd449bce22a6be39c88.map

  98. To build the asset bundle the default pipeline is used

  99. register_pipeline :default do |env, type, file_type| env.default_processors_for(type, file_type) end

  100. def default_processors_for(type, file_type) bundled_processors = config[:bundle_processors][type] if bundled_processors.any? bundled_processors else

    self_processors_for(type, file_type) end end
  101. def default_processors_for(type, file_type) bundled_processors = config[:bundle_processors][type] if bundled_processors.any? bundled_processors else

    self_processors_for(type, file_type) end end
  102. The bundle pipeline will compile all required files and merge

    them
  103. To compile the required files the self pipeline is used

  104. register_pipeline :self do |env, type, file_type| env.self_processors_for(type, file_type) end

  105. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type

    != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
  106. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type

    != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
  107. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type

    != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
  108. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type

    != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
  109. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type

    != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
  110. None
  111. The bundler processor merge all of them

  112. And the result is sent to the browser

  113. What are the key difference between development and production

  114. In production all of this happens in the precompile task

    and only a static asset is returned
  115. Extending it

  116. Creating new directives

  117. class NpmDirectiveProcessor < Sprockets::DirectiveProcessor def process_npm_directive(path) dirs = node_modules_paths(@filename) uri,

    deps = @environment.resolve!( path, accept: @content_type, pipeline: :self, load_paths: dirs ) @dependencies.merge(deps) @required << uri end # ... end
  118. register_preprocessor 'application/javascript', NpmDirectiveProcessor.new(comments: ["//", ["/*", "*/"]])

  119. // app/assets/javascripts/my_component.js //= npm lodash

  120. How to create a new processor to transform SVGs to

    PNGs
  121. environment.register_transformer 'image/svg+xml', 'image/png', SVGTransformer.new

  122. // app/assets/config/manifest.js // Given you have foo.svg //= link foo.png

    // or //= link_tree ../images .png
  123. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) {

    self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
  124. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) {

    self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
  125. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) {

    self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
  126. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) {

    self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
  127. Sprockets is used in many Rails applications

  128. Many users don’t know it exists

  129. Many users don’t know how it works

  130. Understand your tools

  131. Document your understanding

  132. Share with the community

  133. Saving Sprockets Richard Schneeman Thursday, May 5 Room 3501 EF

    2:40PM
  134. We’re hiring! 70+ open positions in a wide range of

    disciplines Ottawa, Montreal, Toronto, Waterloo, Remote https://shopify.com/careers
  135. Testing Rails at Scale Emil Stolarsky May 4, 4:30PM Room

    3501 AB Rails 5 Features You Haven't Heard About Sean Griffin May 5, 11:40AM Room 3501 DC
  136. Rafael França rafaelfranca rafaelfranca Thank you