Basecamp Next: Code Spelunking

Heard about the big Basecamp launch this March? Wondering what's new, how it's shaping Rails, and the tech behind it? We're going to go over some the practices and patterns in the new Basecamp's code base and you can learn how to improve your app with them.

Transcript

1. code spelunking basecamp next @qrush
2. i’m @qrush
3. @ i live here
4. i work at 37signals
5. old & new patterns neato gems how stuff works™
6. 50 days months commits code-test 12 14k 1:1
7. architecture overview
8. rails services
9. rails mysql
10. rails mysql memcached
11. rails mysql redis memcached
12. rails mysql redis memcached elastic search
13. rails mysql redis memcached depot elastic search
14. rails mysql redis memcached portfolio depot elastic search
15. rails launchpad mysql redis memcached portfolio depot elastic search
16. rails launchpad mysql redis memcached queenbee portfolio depot elastic search
20. rails frontends
21. poller rails
22. poller rails rainbows
23. poller rails rainbows nginx
24. poller rails unicorn rainbows nginx
25. poller rails unicorn nginx rainbows nginx
26. poller rails unicorn unicorn nginx nginx rainbows ... ... nginx rails
27. rails local
28. rails mysql
29. pow http://pow.cx
30. rails mysql pow
31. rails mysql pow launchpad
32. rails mysql pow launchpad pow
33. rails mysql pow launchpad portfolio pow
34. rails mysql pow launchpad portfolio pow pow
35. bcx.dev launchpad.dev 37img.dev
36. testing
38. Test::Unit mocha capybara m http://github.com/qrush/m testing
39. functional pow unit integration
40. functional pow unit integration api
41. functional pow unit integration api poller
42. functional pow unit integration api poller 37id
43. functional pow unit integration api poller 37id acceptance client
44. chain rake tasks
45. require 'rails/test_unit/sub_test_task' Rails::SubTestTask.new api: 'test:prepare' do |t| t.libs << 'test' t.pattern = 'test/api/**/*_test.rb' end
46. Rake::Task['test:run'].enhance do Rake::Task['test:api'].invoke end
47. development setup
49. the reset button
50. one bash script
51. ./script/setup
52. ./script/setup; rake
53. 1. setup app 2. reset database 3. restart server 4. custom wrangling
54. keep it quiet
55. echo "Installing libraries..." { (gem list -i bundler || gem install bundler) && rbenv rehash; bundle install && rbenv rehash } >&3 2>&1
56. use flags to speed up
57. if [ -z "$KEEPDB" ]; then echo "Reloading the database" { # lots o’ rake tasks } >&3 2>&1 fi
58. concerns everywhere
59. sharing code is hard
60. use a concern!
61. module YourConcern extend ActiveSupport::Concern included do # class methods here end # instance methods here end
62. module Ajax extend ActiveSupport::Concern included do before_filter :set_ajax_cookies end private def set_ajax_cookies
63. models trashing searching buckets permissions
64. controllers rendering assets authorization authentication current person, account
65. group alike methods
66. class ApplicationController include CurrentAccount include CurrentPerson include ExceptionNotification include Ajax include Mobile include EnsureCompatibleBrowser include WrongContentTypeFallback
68. modules != classes
69. still a ball of mud
70. assets lots of them
71. ~9700 ruby tests coffeescript ~10800 ~9600 lines of:
72. lots of SCSS too!
74. vendor/assets jquery backbone underscore node’s EventEmitter wysihtml5 eco
75. eco http://github.com/sstephenson/eco
76. wysihtml5 http://github.com/xing/wysihtml5
77. jquery plugins scrollTo easing embedly cookies md5 outsideEvents
78. javascript patterns
79. extend jQuery
80. $.fn.highlight = -> $(this). effect("highlight", {}, 1500)
81. $.fn.resetForm = -> @removeClass('submitting'). get(0). reset() this
82. window.bcx
83. // application.js.coffee.erb window.bcx ?= {} _.extend bcx, models: {} collections: {} views: {} env: <%= Rails.env.to_json %>
84. data-behaviors
85. access-toggle account_name_cancel account_name_form account_name_header account_name_header_name account_name_link activate add_calendar add_to_all_calendars add_to_all_projects admin_permission alt_date_field assigned_to assignee_name assignee_options attachments_required autoresize autosave backbone_collection backbone_model bounce bounce_nav bucket_selector caching_classic_projects calendar_accesses calendar_alert calendar_event_drag_area calendar_invitees calendar_todo can_create_projects_permissi on collapse_on_click complete confirm confirm_action create_project date_entry date_picker delete dirty_tracking document_version_link due_date edit edit_bucket edit_calendar_event_form edit_calendar_todo edit_identity edit_project_header editable_field_prompt email_preferences enlargeable expand_exclusively expand_on_click expandable expandable expand_exclusively file_drop_target filter_and filter_due format_timeline grant-all has_hover_content input_change_emitter invite invite_notice invitees jump_to_month jump_to_new_calendar_event lazy_invitees lazy_load_subscribers link_container load_assignee_options load_completed_todos member members migrate_account move_action move_operation_form navigate new new_calendar_event new_group new_message new_project new_project_dialog new_subgroup new_todolist new_upload no_due_date no_reset notification_email nubbin pending_attachments project_palette project_star read_only refresh_timeline remove remove_duplicates remove_group remove_invitee remove_member rename_group resend_invitation resubscribe reveal_event_subsc revoke revoke-all save_document save_group scroll_content scroll_forward scroll_reverse scroll_view select_on_focus select_suggestion set_start_of_week show_all_todos show_link_if_acces slide slider sortable sortable_container sortable_handle
86. access-toggle account_name_cancel account_name_form account_name_header account_name_header_name account_name_link activate add_calendar add_to_all_calendars add_to_all_projects admin_permission alt_date_field assigned_to assignee_name assignee_options attachments_required autoresize autosave backbone_collection backbone_model bounce bounce_nav bucket_selector caching_classic_projects calendar_accesses calendar_alert calendar_event_drag_area calendar_invitees calendar_todo can_create_projects_permissi on collapse_on_click complete confirm confirm_action create_project date_entry date_picker delete dirty_tracking document_version_link due_date edit edit_bucket edit_calendar_event_form edit_calendar_todo edit_identity edit_project_header editable_field_prompt email_preferences enlargeable expand_exclusively expand_on_click expandable expandable expand_exclusively file_drop_target filter_and filter_due format_timeline grant-all has_hover_content input_change_emitter invite invite_notice invitees jump_to_month jump_to_new_calendar_event lazy_invitees lazy_load_subscribers link_container load_assignee_options load_completed_todos member members migrate_account move_action move_operation_form navigate new new_calendar_event new_group new_message new_project new_project_dialog new_subgroup new_todolist new_upload no_due_date no_reset notification_email nubbin pending_attachments project_palette project_star read_only refresh_timeline remove remove_duplicates remove_group remove_invitee remove_member rename_group resend_invitation resubscribe reveal_event_subsc revoke revoke-all save_document save_group scroll_content scroll_forward scroll_reverse scroll_view select_on_focus select_suggestion set_start_of_week show_all_todos show_link_if_acces slide slider sortable sortable_container sortable_handle
87. out the wazoo! keeps javascript logic out of views stops tying behavior to css classes still easy to query for
88. $('[data-behavior~=date_picker]'). live 'focus', -> $(this).datepicker()
89. rails-behaviors http://josh.github.com/rails-behaviors/ * not using this yet!
90. page events
91. use custom events! _.bind() is your new best friend trigger custom events for what you need keep namespacing consistent
93. Stacker basics intercepts most clicks on <a> requests the page via $.ajax renders the response as a new “page” smart about errors too!
94. history.js https://github.com/balupton/History.js/
95. bcx.on 'page:beforechange', 'set bcx.currentBucket', -> bcx.on 'page:change', 'lazily load invitees', -> bcx.on 'page:update', 'install invitee field lists', ->
96. page:beforechange setting the current project/calendar apply classes to <body> start performance timing
97. page:change fired when a new “page” is loaded install views hide/show elements based on roles lazily load data
98. page:update fired after change, on every ajaxSuccess install views, behavior moved loaded data into place stop “busy” animations
101. console mastery
102. console.count() +1
103. console.count("rendered a todo") 1 console.count("rendered a todo") 2
104. console.count("rendered a todo") 1 console.count("rendered a todo") 2
105. console.warn() console.error() make messages stand out
106. console.group() console.groupEnd() clump together logs
110. console.profile() console.profileEnd() see what’s slow
114. console.profile("something slow") console.profileEnd()
115. anywhere in your code: debugger when stopped: console.trace()
116. chrome.csi()
117. console.csi
118. chrome.csi() see how long it’s been since page load
121. api building with love
122. Repo for API docs http://github.com/37signals/bcx-api
123. jbuilder http://github.com/rails/jbuilder
124. json.(@document, :id, :title, :content, :created_at, :updated_at) json.last_updater @document.last_updater, :id, :name json.partial! "api/comments/comments", comments: @document.comments
125. { "id": 963979453, "title": "By Jove!", "content": "I’ve figured it out!", "created_at": "2012-03-27T13:19:29-05:0 "updated_at": "2012-03-27T13:53:24-05:0 "last_updater": { "id": 149087659, "name": "Sherlock Holmes" }, "comments": [ ] }
126. why not to_json? keep view data in the views use partials! public vs private JSON
127. strong_parameters http://github.com/rails/strong_parameters
128. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show end def document_params params.required(:document). permit(:title, :content) end end
129. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show end def document_params params.required(:document). permit(:title, :content) end end
130. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show end def document_params params.required(:document). permit(:title, :content) end end
131. class Api::DocumentsController < Api::BaseController def update @document.update_attributes! document_params render :show end def document_params params.required(:document). permit(:title, :content) end end
132. why not attr_accessible? controller, not a model problem can still share via concerns easier to return error, status codes
133. HTTP 204 No Content http://httpstatus.es/204 Use on create, destroy actions!
134. can’t read headers from HTTP 204 http://bugs.jquery.com/ticket/1450
137. logging is worth it
139. log EVERYTHING unique ID per request account ID for log splitting controller, actions on each SQL query
140. ActiveSupport::TaggedLogger new in Rails 3.2!
141. YourApp::Application.configure do config.log_tags = [:uuid] end [71ba53fd717c67a6677a058f4a5acdf4] Processing by ProjectsController#index as HTML
142. BCX::Application.configure do config.log_tags = [ -> request { request.env['bcx.account.queenbee_id'] } ] end
143. That just happened. <3 Ruby 1.9 lambdas Extremely configurable logging Use Rack’s environment for sharing data
144. # github.com/rails/rails # 3-2-stable # railties/lib/rails/application.rb middleware.use ::Rails::Rack::Logger, config.log_tags
145. module Rails::Rack::Logger # if tags are defined... Rails.logger.tagged(compute_tags(env)) { call_app(env) } # else call_app(env)
146. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
147. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
148. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
149. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
150. def compute_tags(env) request = ActionDispatch::Request.new(env) @tags.collect do |tag| case tag when Proc tag.call(request) when Symbol request.send(tag) else tag end end end
152. marginalia http://github.com/37signals/marginalia
153. Account Load (0.3ms) SELECT `accounts`.* FROM `accounts` WHERE `accounts`.`queenbee_id` = 1234567890 LIMIT 1 /*application:BCX, controller:project_imports,action:show*/
154. # in your Gemfile gem 'query_comments' # in an initializer Marginalia.application_name = "BCX"
155. MORE logging? Know where slow queries came from Immediately identify web or job query MySQL, sqlite, maybe Postgres now?
156. stats count everything
157. statsd http://github.com/etsy/statsd http://github.com/jeremy/statsd-ruby
158. tracks: usage, performance successful, failed logins processed emails people invited
159. ActiveSupport::Notifications http://api.rubyonrails.org/classes/ ActiveSupport/Notifications.html
160. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded"
161. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded" event name
162. ActiveSupport::Notifications.instrument :usage, measurement: "logins.succeeded" hash payload
163. ActiveSupport::Notifications.subscribe(/usage/) do |name, start, finish, id, payload| Statsd.increment(payload[:measurement]) end
164. resque-statsd https://github.com/jamster/resque-statsd
167. tracks: enqueue count completion count failure count time to process
168. we’re still learning don’t stop evolving
169. thanks for listening! @qrush