Pro Yearly is on sale from $80 to $50! »

Basecamp Next: Code Spelunking

Basecamp Next: Code Spelunking

Heard about the big Basecamp launch this March? Wondering what's new, how it's shaping Rails, and the tech behind it? We're going to go over some the practices and patterns in the new Basecamp's code base and you can learn how to improve your app with them.

Eb8975af8e49e19e3dd6b6b84a542e26?s=128

Nick Quaranto

April 25, 2012
Tweet

Transcript

  1. code spelunking basecamp next @qrush

  2. i’m @qrush

  3. @ i live here

  4. i work at 37signals

  5. old & new patterns neato gems how stuff works™

  6. 50 days months commits code-test 12 14k 1:1

  7. architecture overview

  8. rails services

  9. rails mysql

  10. rails mysql memcached

  11. rails mysql redis memcached

  12. rails mysql redis memcached elastic search

  13. rails mysql redis memcached depot elastic search

  14. rails mysql redis memcached portfolio depot elastic search

  15. rails launchpad mysql redis memcached portfolio depot elastic search

  16. rails launchpad mysql redis memcached queenbee portfolio depot elastic search

  17. None
  18. None
  19. None
  20. rails frontends

  21. poller rails

  22. poller rails rainbows

  23. poller rails rainbows nginx

  24. poller rails unicorn rainbows nginx

  25. poller rails unicorn nginx rainbows nginx

  26. poller rails unicorn unicorn nginx nginx rainbows ... ... nginx

    rails
  27. rails local

  28. rails mysql

  29. pow http://pow.cx

  30. rails mysql pow

  31. rails mysql pow launchpad

  32. rails mysql pow launchpad pow

  33. rails mysql pow launchpad portfolio pow

  34. rails mysql pow launchpad portfolio pow pow

  35. bcx.dev launchpad.dev 37img.dev

  36. testing

  37. None
  38. Test::Unit mocha capybara m http://github.com/qrush/m testing

  39. functional pow unit integration

  40. functional pow unit integration api

  41. functional pow unit integration api poller

  42. functional pow unit integration api poller 37id

  43. functional pow unit integration api poller 37id acceptance client

  44. chain rake tasks

  45. require 'rails/test_unit/sub_test_task' Rails::SubTestTask.new api: 'test:prepare' do |t| t.libs << 'test'

    t.pattern = 'test/api/**/*_test.rb' end
  46. Rake::Task['test:run'].enhance do Rake::Task['test:api'].invoke end

  47. development setup

  48. None
  49. the reset button

  50. one bash script

  51. ./script/setup

  52. ./script/setup; rake

  53. 1. setup app 2. reset database 3. restart server 4.

    custom wrangling
  54. keep it quiet

  55. echo "Installing libraries..." { (gem list -i bundler || gem

    install bundler) && rbenv rehash; bundle install && rbenv rehash } >&3 2>&1
  56. use flags to speed up

  57. if [ -z "$KEEPDB" ]; then echo "Reloading the database"

    { # lots o’ rake tasks } >&3 2>&1 fi
  58. concerns everywhere

  59. sharing code is hard

  60. use a concern!

  61. module YourConcern extend ActiveSupport::Concern included do # class methods here

    end # instance methods here end
  62. module Ajax extend ActiveSupport::Concern included do before_filter :set_ajax_cookies end private

    def set_ajax_cookies
  63. models trashing searching buckets permissions

  64. controllers rendering assets authorization authentication current person, account

  65. group alike methods

  66. class ApplicationController include CurrentAccount include CurrentPerson include ExceptionNotification include Ajax

    include Mobile include EnsureCompatibleBrowser include WrongContentTypeFallback
  67. None
  68. modules != classes

  69. still a ball of mud

  70. assets lots of them

  71. ~9700 ruby tests coffeescript ~10800 ~9600 lines of:

  72. lots of SCSS too!

  73. None
  74. vendor/assets jquery backbone underscore node’s EventEmitter wysihtml5 eco

  75. eco http://github.com/sstephenson/eco

  76. wysihtml5 http://github.com/xing/wysihtml5

  77. jquery plugins scrollTo easing embedly cookies md5 outsideEvents

  78. javascript patterns

  79. extend jQuery

  80. $.fn.highlight = -> $(this). effect("highlight", {}, 1500)

  81. $.fn.resetForm = -> @removeClass('submitting'). get(0). reset() this

  82. window.bcx

  83. // application.js.coffee.erb window.bcx ?= {} _.extend bcx, models: {} collections:

    {} views: {} env: <%= Rails.env.to_json %>
  84. data-behaviors

  85. access-toggle account_name_cancel account_name_form account_name_header account_name_header_name account_name_link activate add_calendar add_to_all_calendars add_to_all_projects

    admin_permission alt_date_field assigned_to assignee_name assignee_options attachments_required autoresize autosave backbone_collection backbone_model bounce bounce_nav bucket_selector caching_classic_projects calendar_accesses calendar_alert calendar_event_drag_area calendar_invitees calendar_todo can_create_projects_permissi on collapse_on_click complete confirm confirm_action create_project date_entry date_picker delete dirty_tracking document_version_link due_date edit edit_bucket edit_calendar_event_form edit_calendar_todo edit_identity edit_project_header editable_field_prompt email_preferences enlargeable expand_exclusively expand_on_click expandable expandable expand_exclusively file_drop_target filter_and filter_due format_timeline grant-all has_hover_content input_change_emitter invite invite_notice invitees jump_to_month jump_to_new_calendar_event lazy_invitees lazy_load_subscribers link_container load_assignee_options load_completed_todos member members migrate_account move_action move_operation_form navigate new new_calendar_event new_group new_message new_project new_project_dialog new_subgroup new_todolist new_upload no_due_date no_reset notification_email nubbin pending_attachments project_palette project_star read_only refresh_timeline remove remove_duplicates remove_group remove_invitee remove_member rename_group resend_invitation resubscribe reveal_event_subsc revoke revoke-all save_document save_group scroll_content scroll_forward scroll_reverse scroll_view select_on_focus select_suggestion set_start_of_week show_all_todos show_link_if_acces slide slider sortable sortable_container sortable_handle
  86. access-toggle account_name_cancel account_name_form account_name_header account_name_header_name account_name_link activate add_calendar add_to_all_calendars add_to_all_projects

    admin_permission alt_date_field assigned_to assignee_name assignee_options attachments_required autoresize autosave backbone_collection backbone_model bounce bounce_nav bucket_selector caching_classic_projects calendar_accesses calendar_alert calendar_event_drag_area calendar_invitees calendar_todo can_create_projects_permissi on collapse_on_click complete confirm confirm_action create_project date_entry date_picker delete dirty_tracking document_version_link due_date edit edit_bucket edit_calendar_event_form edit_calendar_todo edit_identity edit_project_header editable_field_prompt email_preferences enlargeable expand_exclusively expand_on_click expandable expandable expand_exclusively file_drop_target filter_and filter_due format_timeline grant-all has_hover_content input_change_emitter invite invite_notice invitees jump_to_month jump_to_new_calendar_event lazy_invitees lazy_load_subscribers link_container load_assignee_options load_completed_todos member members migrate_account move_action move_operation_form navigate new new_calendar_event new_group new_message new_project new_project_dialog new_subgroup new_todolist new_upload no_due_date no_reset notification_email nubbin pending_attachments project_palette project_star read_only refresh_timeline remove remove_duplicates remove_group remove_invitee remove_member rename_group resend_invitation resubscribe reveal_event_subsc revoke revoke-all save_document save_group scroll_content scroll_forward scroll_reverse scroll_view select_on_focus select_suggestion set_start_of_week show_all_todos show_link_if_acces slide slider sortable sortable_container sortable_handle
  87. out the wazoo! keeps javascript logic out of views stops

    tying behavior to css classes still easy to query for
  88. $('[data-behavior~=date_picker]'). live 'focus', -> $(this).datepicker()

  89. rails-behaviors http://josh.github.com/rails-behaviors/ * not using this yet!

  90. page events

  91. use custom events! _.bind() is your new best friend trigger

    custom events for what you need keep namespacing consistent
  92. None
  93. Stacker basics intercepts most clicks on <a> requests the page

    via $.ajax renders the response as a new “page” smart about errors too!
  94. history.js https://github.com/balupton/History.js/

  95. bcx.on 'page:beforechange', 'set bcx.currentBucket', -> bcx.on 'page:change', 'lazily load invitees',

    -> bcx.on 'page:update', 'install invitee field lists', ->
  96. page:beforechange setting the current project/calendar apply classes to <body> start

    performance timing
  97. page:change fired when a new “page” is loaded install views

    hide/show elements based on roles lazily load data
  98. page:update fired after change, on every ajaxSuccess install views, behavior

    moved loaded data into place stop “busy” animations
  99. None
  100. None
  101. console mastery

  102. console.count() +1

  103. console.count("rendered a todo") 1 console.count("rendered a todo") 2

  104. console.count("rendered a todo") 1 console.count("rendered a todo") 2

  105. console.warn() console.error() make messages stand out

  106. console.group() console.groupEnd() clump together logs

  107. None
  108. None
  109. None
  110. console.profile() console.profileEnd() see what’s slow

  111. None
  112. None
  113. None
  114. console.profile("something slow") console.profileEnd()

  115. anywhere in your code: debugger when stopped: console.trace()

  116. chrome.csi()

  117. console.csi

  118. chrome.csi() see how long it’s been since page load

  119. None
  120. None
  121. api building with love

  122. Repo for API docs http://github.com/37signals/bcx-api

  123. jbuilder http://github.com/rails/jbuilder

  124. json.(@document, :id, :title, :content, :created_at, :updated_at) json.last_updater @document.last_updater, :id, :name

    json.partial! "api/comments/comments", comments: @document.comments
  125. { "id": 963979453, "title": "By Jove!", "content": "I’ve figured it

    out!", "created_at": "2012-03-27T13:19:29-05:0 "updated_at": "2012-03-27T13:53:24-05:0 "last_updater": { "id": 149087659, "name": "Sherlock Holmes" }, "comments": [ ] }
  126. why not to_json? keep view data in the views use

    partials! public vs private JSON
  127. strong_parameters http://github.com/rails/strong_parameters

  128. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show

    end def document_params params.required(:document). permit(:title, :content) end end
  129. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show

    end def document_params params.required(:document). permit(:title, :content) end end
  130. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show

    end def document_params params.required(:document). permit(:title, :content) end end
  131. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show

    end def document_params params.required(:document). permit(:title, :content) end end
  132. why not attr_accessible? controller, not a model problem can still

    share via concerns easier to return error, status codes
  133. HTTP 204 No Content http://httpstatus.es/204 Use on create, destroy actions!

  134. can’t read headers from HTTP 204 http://bugs.jquery.com/ticket/1450

  135. None
  136. None
  137. logging is worth it

  138. None
  139. log EVERYTHING unique ID per request account ID for log

    splitting controller, actions on each SQL query
  140. ActiveSupport::TaggedLogger new in Rails 3.2!

  141. YourApp::Application.configure do config.log_tags = [:uuid] end [71ba53fd717c67a6677a058f4a5acdf4] Processing by ProjectsController#index

    as HTML
  142. BCX::Application.configure do config.log_tags = [ -> request { request.env['bcx.account.queenbee_id'] }

    ] end
  143. That just happened. <3 Ruby 1.9 lambdas Extremely configurable logging

    Use Rack’s environment for sharing data
  144. # github.com/rails/rails # 3-2-stable # railties/lib/rails/application.rb middleware.use ::Rails::Rack::Logger, config.log_tags

  145. module Rails::Rack::Logger # if tags are defined... Rails.logger.tagged(compute_tags(env)) { call_app(env)

    } # else call_app(env)
  146. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag

    when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
  147. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag

    when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
  148. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag

    when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
  149. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag

    when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
  150. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag

    when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
  151. None
  152. marginalia http://github.com/37signals/marginalia

  153. Account Load (0.3ms) SELECT `accounts`.* FROM `accounts` WHERE `accounts`.`queenbee_id` =

    1234567890 LIMIT 1 /*application:BCX, controller:project_imports,action:show*/
  154. # in your Gemfile gem 'query_comments' # in an initializer

    Marginalia.application_name = "BCX"
  155. MORE logging? Know where slow queries came from Immediately identify

    web or job query MySQL, sqlite, maybe Postgres now?
  156. stats count everything

  157. statsd http://github.com/etsy/statsd http://github.com/jeremy/statsd-ruby

  158. tracks: usage, performance successful, failed logins processed emails people invited

  159. ActiveSupport::Notifications http://api.rubyonrails.org/classes/ ActiveSupport/Notifications.html

  160. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded"

  161. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded" event name

  162. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded" hash payload

  163. ActiveSupport::Notifications.subscribe(/usage/) do |name, start, finish, id, payload| Statsd.increment(payload[:measurement]) end

  164. resque-statsd https://github.com/jamster/resque-statsd

  165. None
  166. None
  167. tracks: enqueue count completion count failure count time to process

  168. we’re still learning don’t stop evolving

  169. thanks for listening! @qrush