Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Refactoring with Science

Refactoring with Science

Using data and Science to refactor with greater confidence.

Wynn Netherland

February 21, 2014
Tweet

More Decks by Wynn Netherland

Other Decks in Programming

Transcript

  1. Refactoring GitHub
    with Science
    !
    #BigRubyConf

    View full-size slide

  2. !
    Hello, my name is Wynn.

    View full-size slide

  3. !
    Folks call me @pengwynn.

    View full-size slide

  4. !
    http://wynn.fm

    View full-size slide

  5. Changing code is EASY!
    !

    View full-size slide

  6. Changing code with CONFIDENCE isn't.
    !

    View full-size slide

  7. How do we gain CONFIDENCE?
    !

    View full-size slide

  8. Write lots of TESTS.
    !
    ❯ testrb test/integration/api/repos_test.rb
    Loading suite in dotcom mode ...
    Run options:
    # Running tests:
    Finished tests in 30.638101s, 4.7979 tests/s, 64.7886 assertions/s.
    147 tests, 1985 assertions, 0 failures, 0 errors, 0 skips
    ruby -v: ruby 2.1.0p0-github (development) [x86_64-darwin13.0]

    View full-size slide

  9. Use data to inform decisions.
    !

    View full-size slide

  10. !
    /graph me -1hr @api.queries.serialized
    >

    View full-size slide

  11. The Graph Store !
    !

    View full-size slide

  12. The Graph Store !
    !
    There's a graph for that™

    View full-size slide

  13. The Graph Store !
    !
    There's a graph for that™
    …if not, make one.

    View full-size slide

  14. !
    Built on Graphite.

    View full-size slide

  15. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View full-size slide

  16. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View full-size slide

  17. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View full-size slide

  18. Deploy with confidence. "
    !

    View full-size slide

  19. How# has "'d over the years.
    !

    View full-size slide

  20. 1989
    10 PRINT "WYNN"
    20 GOTO 10

    View full-size slide

  21. 2009
    !
    # General configuration settings, required for all recipes!
    set :application, "project-name"
    set :domain, "project-domain"
    set :extra_domains, %w() # Add something like www.project-domain.com
    role :app, domain
    role :web, domain
    role :db, domain, :primary => true
    !
    # set :user, "user"
    !
    # Deployment Settings
    set :repository, "repository url"
    set :deploy_to, "/absolute/path/to/location/your/server"
    set :deploy_via, :checkout
    set :config_files, %w()
    !
    # SSH Keys for caching (you must generate these first.)
    ssh_options[:keys] = %w(~/.ssh/mykey1 ~/.ssh/mykey2)
    !
    # Change this to :thin if you want to use Thin instead.
    set :app_server, :mongrel
    !
    # Change this to :merb if you want to use Merb instead. (experimental)
    set :app_framework, :rails
    !
    # =============================================================
    # Application Server Settings (Thin or Mongrel)
    # =============================================================
    set :app_servers, 1
    set :app_server_port, 7007
    set :app_environment, 'production'
    set :app_server_address, '127.0.0.1'
    set :app_server_conf, "#{shared_path}/config/thin.yml"
    !
    # =============================================================
    # Nginx Settings

    View full-size slide

  22. !
    The GitHub Flow

    View full-size slide

  23. !
    Start with a Pull Request.

    View full-size slide

  24. !
    Create a branch.

    View full-size slide

  25. !
    Add commits.

    View full-size slide

  26. !
    Open a pull request.

    View full-size slide

  27. !
    Discuss and review.

    View full-size slide

  28. !
    Merge and deploy.

    View full-size slide

  29. !
    Or deploy and merge. $

    View full-size slide

  30. !
    The following deployments are based on actual events. Branch names
    and SHAs have been changed to protect the innocent.

    View full-size slide

  31. !
    hubot where can I deploy?
    >
    Environments for github
    -----------------------------------------------------
    production: unlocked
    lab-one: locked 6 hours ago by jasonrudolph
    lab-two: unlocked
    sekret-lab: unlocked
    playground: unlocked

    View full-size slide

  32. !
    hubot deploy github/api-my-feature to production
    >

    View full-size slide

  33. !
    hubot deploy github/api-my-feature to production
    >
    project

    View full-size slide

  34. !
    hubot deploy github/api-my-feature to production
    >
    branch

    View full-size slide

  35. !
    hubot deploy github/api-my-feature to production
    >
    environment

    View full-size slide

  36. !
    hubot deploy github/api-my-feature to production
    >
    Merge master into api-my-feature for pengwynn - hubot github/github@deadbeef

    View full-size slide

  37. !
    Get up to date with master. ⏳

    View full-size slide

  38. !
    Wait on the CI server. '

    View full-size slide

  39. !
    hubot tv me
    >

    View full-size slide

  40. !
    hubot deploy github/api-my-feature to production
    >
    Build #8675309 (deadbeef) of github/api-my-feature
    was successful (203s)

    View full-size slide

  41. !
    Release the kraken! (

    View full-size slide

  42. !
    hubot deploy github/api-my-feature to production
    >
    pengwynn is deploying github/api-my-feature (beefdead..deadbeef) to production
    pengwynn's production deployment of github is NOW )))! (22s)

    View full-size slide

  43. "The first transport is away!"

    View full-size slide

  44. ## TODO
    - [x] Deploy "
    - [ ] Observe * !

    View full-size slide

  45. !
    Problem? ABORT. +

    View full-size slide

  46. !
    hubot deploy! github
    >
    pengwynn is deploying github/master to production

    View full-size slide

  47. !
    ✨Everything -? MERGE.✨

    View full-size slide

  48. !
    >
    pengwynn: it looks like you merged "api-my-feature" branch
    into master, so I've unlocked production.

    View full-size slide

  49. !
    >
    pengwynn is deploying github/master to production
    pengwynn is deploying github/master to lab-one
    pengwynn is deploying github/master to lab-two
    pengwynn is deploying github/master to sekret-lab

    View full-size slide

  50. Prune with confidence. ✂️
    !

    View full-size slide

  51. !
    “Are we even calling this anywhere?”

    View full-size slide

  52. !
    Find and remove vestigial code.

    View full-size slide

  53. !
    GitHub::Backscatter

    View full-size slide

  54. !
    GitHub::Backscatter
    not open source,
    but you can roll your own

    View full-size slide

  55. !
    *69 for Ruby methods

    View full-size slide

  56. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View full-size slide

  57. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View full-size slide

  58. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View full-size slide

  59. The calls are
    coming from
    INSIDE the APP.
    0

    View full-size slide

  60. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure 1000
    # ...
    end
    end

    View full-size slide

  61. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure 1000
    # ...
    end
    end
    trace 1 out of every 1000 calls

    View full-size slide

  62. Experiment with confidence. 1
    !

    View full-size slide

  63. !
    github/dat-science

    View full-size slide

  64. !
    Safely observe alternate code paths.

    View full-size slide

  65. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  66. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  67. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  68. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  69. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  70. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View full-size slide

  71. !
    Fancy a DSL?

    View full-size slide

  72. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    science "widget-permissions" do |e|
    e.control { model.check_user(user).valid? }
    e.candidate { user.can? :read, model }
    end
    end
    end

    View full-size slide

  73. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    science "widget-permissions" do |e|
    e.control { model.check_user(user).valid? }
    e.candidate { user.can? :read, model }
    end
    end
    end

    View full-size slide

  74. !
    Create and run.

    View full-size slide

  75. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View full-size slide

  76. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View full-size slide

  77. !
    Ramping up experiments.

    View full-size slide

  78. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def enabled?
    rand(100) < 10 # Run 10% of the time
    end
    end
    end

    View full-size slide

  79. !
    jnunemaker/flipper 2

    View full-size slide

  80. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def enabled?
    MyApp.flipper[name].enabled?
    end
    end
    end

    View full-size slide

  81. !
    Publishing results.

    View full-size slide

  82. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def publish
    MyApp.instrument "science.#{event}", payload
    end
    end
    end

    View full-size slide

  83. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def publish
    MyApp.instrument "science.#{event}", payload
    end
    end
    end

    View full-size slide

  84. !
    Adding custom context.

    View full-size slide

  85. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.context :post => post
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View full-size slide

  86. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.context :post => post
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View full-size slide

  87. Mad science.

    View full-size slide

  88. module GitHub
    class Experiment < Dat::Science::Experiment
    class << self
    # Public: enable mad science mode: returns the candidate values by default
    # instead of the control.
    def mad_science=(val)
    @mad_science = val
    end
    !
    # Internal: whether or not to always run the candidate instead of the
    # control.
    def candidate_instead_of_control?
    @mad_science
    end
    end

    end

    View full-size slide

  89. module GitHub
    class Experiment < Dat::Science::Experiment
    class << self
    # Public: enable mad science mode: returns the candidate values by default
    # instead of the control.
    def mad_science=(val)
    @mad_science = val
    end
    !
    # Internal: whether or not to always run the candidate instead of the
    # control.
    def candidate_instead_of_control?
    @mad_science
    end
    end

    end

    View full-size slide

  90. !
    Analyzing results.

    View full-size slide

  91. !
    github/dat-analysis

    View full-size slide

  92. Thanks, Jim.

    View full-size slide