Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Refactoring with Science

Refactoring with Science

Using data and Science to refactor with greater confidence.

Wynn Netherland

February 21, 2014
Tweet

More Decks by Wynn Netherland

Other Decks in Programming

Transcript

  1. Refactoring GitHub
    with Science
    !
    #BigRubyConf

    View Slide

  2. !
    Hello, my name is Wynn.

    View Slide

  3. !
    Folks call me @pengwynn.

    View Slide

  4. !
    http://wynn.fm

    View Slide

  5. Changing code is EASY!
    !

    View Slide

  6. Changing code with CONFIDENCE isn't.
    !

    View Slide

  7. How do we gain CONFIDENCE?
    !

    View Slide

  8. Write lots of TESTS.
    !
    ❯ testrb test/integration/api/repos_test.rb
    Loading suite in dotcom mode ...
    Run options:
    # Running tests:
    Finished tests in 30.638101s, 4.7979 tests/s, 64.7886 assertions/s.
    147 tests, 1985 assertions, 0 failures, 0 errors, 0 skips
    ruby -v: ruby 2.1.0p0-github (development) [x86_64-darwin13.0]

    View Slide

  9. Use data to inform decisions.
    !

    View Slide

  10. !
    /graph me -1hr @api.queries.serialized
    >

    View Slide

  11. The Graph Store !
    !

    View Slide

  12. The Graph Store !
    !
    There's a graph for that™

    View Slide

  13. The Graph Store !
    !
    There's a graph for that™
    …if not, make one.

    View Slide

  14. !
    Built on Graphite.

    View Slide

  15. !

    View Slide

  16. !

    View Slide

  17. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View Slide

  18. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View Slide

  19. !
    # counter
    GitHub.stats.increment("the-thing.the-action")
    !
    # gauge
    GitHub.stats.gauge("the-thing.the-gauge", thing.current_reading)
    !
    # timing
    GitHub.stats.time("the-thing.the-operation") { the_operation }

    View Slide

  20. Deploy with confidence. "
    !

    View Slide

  21. How# has "'d over the years.
    !

    View Slide

  22. 1989
    10 PRINT "WYNN"
    20 GOTO 10

    View Slide

  23. 1994

    View Slide

  24. 1999
    !

    View Slide

  25. 2004
    !

    View Slide

  26. 2009
    !
    # General configuration settings, required for all recipes!
    set :application, "project-name"
    set :domain, "project-domain"
    set :extra_domains, %w() # Add something like www.project-domain.com
    role :app, domain
    role :web, domain
    role :db, domain, :primary => true
    !
    # set :user, "user"
    !
    # Deployment Settings
    set :repository, "repository url"
    set :deploy_to, "/absolute/path/to/location/your/server"
    set :deploy_via, :checkout
    set :config_files, %w()
    !
    # SSH Keys for caching (you must generate these first.)
    ssh_options[:keys] = %w(~/.ssh/mykey1 ~/.ssh/mykey2)
    !
    # Change this to :thin if you want to use Thin instead.
    set :app_server, :mongrel
    !
    # Change this to :merb if you want to use Merb instead. (experimental)
    set :app_framework, :rails
    !
    # =============================================================
    # Application Server Settings (Thin or Mongrel)
    # =============================================================
    set :app_servers, 1
    set :app_server_port, 7007
    set :app_environment, 'production'
    set :app_server_address, '127.0.0.1'
    set :app_server_conf, "#{shared_path}/config/thin.yml"
    !
    # =============================================================
    # Nginx Settings

    View Slide

  27. Today
    !

    View Slide

  28. !
    The GitHub Flow

    View Slide

  29. !
    Start with a Pull Request.

    View Slide

  30. !

    View Slide

  31. !
    Create a branch.

    View Slide

  32. !
    Add commits.

    View Slide

  33. !
    Open a pull request.

    View Slide

  34. !
    Discuss and review.

    View Slide

  35. !
    Merge and deploy.

    View Slide

  36. !
    Or deploy and merge. $

    View Slide

  37. !
    %

    View Slide

  38. !

    View Slide

  39. !
    The following deployments are based on actual events. Branch names
    and SHAs have been changed to protect the innocent.

    View Slide

  40. !
    hubot where can I deploy?
    >
    Environments for github
    -----------------------------------------------------
    production: unlocked
    lab-one: locked 6 hours ago by jasonrudolph
    lab-two: unlocked
    sekret-lab: unlocked
    playground: unlocked

    View Slide

  41. !
    hubot deploy github/api-my-feature to production
    >

    View Slide

  42. !
    hubot deploy github/api-my-feature to production
    >
    project

    View Slide

  43. !
    hubot deploy github/api-my-feature to production
    >
    branch

    View Slide

  44. !
    hubot deploy github/api-my-feature to production
    >
    environment

    View Slide

  45. !
    hubot deploy github/api-my-feature to production
    >
    Merge master into api-my-feature for pengwynn - hubot github/github@deadbeef

    View Slide

  46. !
    Get up to date with master. ⏳

    View Slide

  47. !
    Wait on the CI server. '

    View Slide

  48. !

    View Slide

  49. !
    hubot tv me
    >

    View Slide

  50. !
    hubot deploy github/api-my-feature to production
    >
    Build #8675309 (deadbeef) of github/api-my-feature
    was successful (203s)

    View Slide

  51. !
    Release the kraken! (

    View Slide

  52. !
    hubot deploy github/api-my-feature to production
    >
    pengwynn is deploying github/api-my-feature (beefdead..deadbeef) to production
    pengwynn's production deployment of github is NOW )))! (22s)

    View Slide

  53. "The first transport is away!"

    View Slide

  54. ## TODO
    - [x] Deploy "
    - [ ] Observe * !

    View Slide

  55. !

    View Slide

  56. !

    View Slide

  57. !

    View Slide

  58. !

    View Slide

  59. !
    Problem? ABORT. +

    View Slide

  60. !
    hubot deploy! github
    >
    pengwynn is deploying github/master to production

    View Slide

  61. !
    ✨Everything -? MERGE.✨

    View Slide

  62. !

    View Slide

  63. !
    >
    pengwynn: it looks like you merged "api-my-feature" branch
    into master, so I've unlocked production.

    View Slide

  64. !
    >
    pengwynn is deploying github/master to production
    pengwynn is deploying github/master to lab-one
    pengwynn is deploying github/master to lab-two
    pengwynn is deploying github/master to sekret-lab

    View Slide

  65. Prune with confidence. ✂️
    !

    View Slide

  66. !
    “Are we even calling this anywhere?”

    View Slide

  67. !
    Find and remove vestigial code.

    View Slide

  68. !
    GitHub::Backscatter

    View Slide

  69. !
    GitHub::Backscatter
    not open source,
    but you can roll your own

    View Slide

  70. !
    *69 for Ruby methods

    View Slide

  71. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View Slide

  72. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View Slide

  73. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure
    # ...
    end
    end

    View Slide

  74. View Slide

  75. The calls are
    coming from
    INSIDE the APP.
    0

    View Slide

  76. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure 1000
    # ...
    end
    end

    View Slide

  77. class Foo
    include GitHub::Backscatter
    !
    def the_method(args)
    backscatter_measure 1000
    # ...
    end
    end
    trace 1 out of every 1000 calls

    View Slide

  78. Experiment with confidence. 1
    !

    View Slide

  79. !
    1

    View Slide

  80. !
    github/dat-science

    View Slide

  81. !
    @jbarnette

    View Slide

  82. !
    @rick

    View Slide

  83. !

    View Slide

  84. !

    View Slide

  85. !
    Safely observe alternate code paths.

    View Slide

  86. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  87. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  88. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  89. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  90. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  91. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    experiment = Dat::Science::Experiment.new \
    "widget-permissions" do |e|
    !
    e.control { old_method } # old way
    e.candidate { new_method } # new way
    end
    !
    experiment.run
    end
    end

    View Slide

  92. !
    Fancy a DSL?

    View Slide

  93. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    science "widget-permissions" do |e|
    e.control { model.check_user(user).valid? }
    e.candidate { user.can? :read, model }
    end
    end
    end

    View Slide

  94. require "dat/science"
    !
    class MyApp::Widget
    def allows?(user)
    science "widget-permissions" do |e|
    e.control { model.check_user(user).valid? }
    e.candidate { user.can? :read, model }
    end
    end
    end

    View Slide

  95. !
    Create and run.

    View Slide

  96. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View Slide

  97. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View Slide

  98. !
    Ramping up experiments.

    View Slide

  99. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def enabled?
    rand(100) < 10 # Run 10% of the time
    end
    end
    end

    View Slide

  100. !
    jnunemaker/flipper 2

    View Slide

  101. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def enabled?
    MyApp.flipper[name].enabled?
    end
    end
    end

    View Slide

  102. !
    Publishing results.

    View Slide

  103. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def publish
    MyApp.instrument "science.#{event}", payload
    end
    end
    end

    View Slide

  104. require "dat/science"
    !
    module Myapp
    class Experiment < Dat::Science::Experiment
    def publish
    MyApp.instrument "science.#{event}", payload
    end
    end
    end

    View Slide

  105. View Slide

  106. View Slide

  107. !
    Adding custom context.

    View Slide

  108. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.context :post => post
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View Slide

  109. require "dat/science"
    !
    class Blog::Post
    def permalink(user)
    science "permalink" do |e|
    e.context :post => post
    e.control { post[:permalink] }
    e.candidate { post.slug }
    e.comparator {|a, b| a.downcase == b.downcase }
    end
    end
    end

    View Slide

  110. Mad science.

    View Slide

  111. module GitHub
    class Experiment < Dat::Science::Experiment
    class << self
    # Public: enable mad science mode: returns the candidate values by default
    # instead of the control.
    def mad_science=(val)
    @mad_science = val
    end
    !
    # Internal: whether or not to always run the candidate instead of the
    # control.
    def candidate_instead_of_control?
    @mad_science
    end
    end

    end

    View Slide

  112. module GitHub
    class Experiment < Dat::Science::Experiment
    class << self
    # Public: enable mad science mode: returns the candidate values by default
    # instead of the control.
    def mad_science=(val)
    @mad_science = val
    end
    !
    # Internal: whether or not to always run the candidate instead of the
    # control.
    def candidate_instead_of_control?
    @mad_science
    end
    end

    end

    View Slide

  113. !
    Analyzing results.

    View Slide

  114. !
    github/dat-analysis

    View Slide

  115. Thanks, Jim.

    View Slide

  116. !
    EOF

    View Slide