Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dr Sheldon Cooper Presents: Fun with Flags

Dr Sheldon Cooper Presents: Fun with Flags

No no, not country flags, feature flags! Feature flags are a powerful technique that allows teams to modify a system’s behaviour without changing code. They can be used several reasons – canary releases and A/B testing to name a few.

This talk will show you how you’re already using feature flags in your application without realising it. Next, we’ll take a look at some of the best tooling out there to help you take feature flags to the next level. Finally, we’ll cover strategies for removing feature flags before they become technical debt that you have to manage.

Michael Heap

May 26, 2018
Tweet

More Decks by Michael Heap

Other Decks in Technology

Transcript

  1. Michael Heap

    View full-size slide

  2. Fun with Country Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  3. Fun with Country Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  4. Fun with Country Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  5. Fun with Country Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  6. Fun with Feature Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  7. Michael
    @mheap
    Developer Relations
    NexmoDev

    View full-size slide

  8. Fun with Feature Flags!
    Presented by @mheap at #phpsrb

    View full-size slide

  9. Today’s Plan

    View full-size slide

  10. 1. What are feature flags?
    2. Implementation
    3. Flag management
    4. Tips and tricks
    5. Conclusion

    View full-size slide

  11. 1. What are feature flags?
    2. Implementation
    3. Flag management
    4. Tips and tricks
    5. Conclusion

    View full-size slide

  12. 1. What are feature flags?
    2. Implementation
    3. Flag management
    4. Tips and tricks
    5. Conclusion

    View full-size slide

  13. 1. What are feature flags?
    2. Implementation
    3. Flag management
    4. Tips and tricks
    5. Conclusion

    View full-size slide

  14. 1. What are feature flags?
    2. Implementation
    3. Flag management
    4. Tips and tricks
    5. Conclusion

    View full-size slide

  15. Feature Flags?

    View full-size slide

  16. Feature Toggles?

    View full-size slide

  17. Feature Flip?

    View full-size slide

  18. Feature Gate?

    View full-size slide

  19. Feature Switches?

    View full-size slide

  20. Separate feature release
    from code deployment

    View full-size slide

  21. http://code.flickr.net/2009/12/02/flipping-out/

    View full-size slide

  22. Feature flags aren’t new

    View full-size slide

  23. #ifdef NEW_ALGORITHM
    NEW_ALGORITHM(input)
    #else
    ALGORITHM(input)
    #endif
    $ make build -DNEW_ALGORITHM

    View full-size slide

  24. ./make_call.sh
    --use-new-algorithm
    ./make_call.sh
    --use-original-algorithm

    View full-size slide

  25. if (in_array($user->id, $whitelistedUsers)) {
    return $service->useNewAlgorithm($input);
    }
    return $service->useAlgorithm($input);

    View full-size slide

  26. 1. Permissions

    View full-size slide

  27. 1. Permissions
    2. Early-access

    View full-size slide

  28. 1. Permissions
    2. Early-access
    3. Opt-in

    View full-size slide

  29. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out

    View full-size slide

  30. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users

    View full-size slide

  31. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users
    6. Calendar driven

    View full-size slide

  32. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users
    6. Calendar driven
    7. Kill switch

    View full-size slide

  33. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users
    6. Calendar driven
    7. Kill switch
    8. Power user

    View full-size slide

  34. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users
    6. Calendar driven
    7. Kill switch
    8. Power user
    9. Maintenance

    View full-size slide

  35. 1. Permissions
    2. Early-access
    3. Opt-in
    4. Incremental roll-out
    5. Block users
    6. Calendar driven
    7. Kill switch
    8. Power user
    9. Maintenance
    10. Sunset

    View full-size slide

  36. Release
    Toggles
    Ops
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles

    View full-size slide

  37. Release
    Toggles
    Ops
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles
    Lifetime
    Days
    Dynamism
    Per deployment

    View full-size slide

  38. Release
    Toggles
    Ops
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles
    Lifetime
    Days - Weeks
    Dynamism
    Human Controlled

    View full-size slide

  39. Release
    Toggles
    Ops
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles
    Lifetime
    Years
    Dynamism
    Per Request

    View full-size slide

  40. Release
    Toggles
    Ops
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles
    Lifetime
    Weeks - Months
    Dynamism
    Per Request

    View full-size slide

  41. https://martinfowler.com/articles/feature-toggles.html

    View full-size slide

  42. Ownership
    Release toggles owned by devs
    Ops toggles owned by ops
    Permission toggles
    Experiment toggles
    owned by product

    View full-size slide

  43. On/Off
    Switch
    Enable or
    disable
    entirely

    View full-size slide

  44. On/Off
    Switch
    Percentage
    of users
    Enable or
    disable
    entirely
    Toggle
    for a
    percentage
    of users

    View full-size slide

  45. On/Off
    Switch
    Percentage
    of users
    Individuals
    Enable or
    disable
    entirely
    Toggle
    for a
    percentage
    of users
    Toggle for
    specific
    users

    View full-size slide

  46. On/Off
    Switch
    Percentage
    of users
    Individuals Groups
    Enable or
    disable
    entirely
    Toggle
    for a
    percentage
    of users
    Toggle for
    specific
    users
    Enable for
    specific
    groups
    e.g.
    Power Users

    View full-size slide

  47. On/Off
    Switch
    Percentage
    of users
    Individuals Groups Request
    information
    Enable or
    disable
    entirely
    Toggle
    for a
    percentage
    of users
    Toggle for
    specific
    users
    Enable for
    specific
    groups
    e.g.
    Power Users
    Enable for
    specific
    request
    headers or
    bodies

    View full-size slide

  48. Implementation

    View full-size slide

  49. Build vs Buy

    View full-size slide

  50. 1. Dashboard

    View full-size slide

  51. 1. Dashboard
    2. Access control

    View full-size slide

  52. 1. Dashboard
    2. Access control
    3. Complex rules

    View full-size slide

  53. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags

    View full-size slide

  54. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags
    5. Multivariate rules

    View full-size slide

  55. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags
    5. Multivariate rules
    6. Auditing

    View full-size slide

  56. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags
    5. Multivariate rules
    6. Auditing
    7. Speed

    View full-size slide

  57. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags
    5. Multivariate rules
    6. Auditing
    7. Speed
    8. Scale

    View full-size slide

  58. 1. Dashboard
    2. Access control
    3. Complex rules
    4. Cascading flags
    5. Multivariate rules
    6. Auditing
    7. Speed
    8. Scale
    9. 100% uptime

    View full-size slide

  59. Rollout (Ruby)
    # Is a feature active?
    $rollout.active?(:chat)
    # Activate a specific user
    $rollout.activate_user(:chat, @user)
    # Define a group
    $rollout.define_group(:admins) do |user|
    user.admin?
    end
    # Roll out to a percentage of users
    $rollout.activate_percentage(:chat, 20)

    View full-size slide

  60. Rollout (PHP)
    # Is a feature active?
    $rollout->isActive('chat');
    # Activate a specific user
    $rollout->activateUser('chat', $user);
    # Define a group
    $rollout->defineGroup('admins', function($user = null) {
    return $user->isAdmin();
    });
    # Roll out to a percentage of users
    $rollout->activatePercentage('chat', 20);

    View full-size slide

  61. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  62. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  63. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  64. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  65. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_AFFIRMATIVE);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  66. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_MAJORITY);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  67. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_AFFIRMATIVE);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  68. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  69. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  70. Qandidate Toggle
    $manager = new ToggleManager(new InMemoryCollection());
    $afterSix = new GreaterThanEqual(18);
    $beforeTen = new LessThanEqual(22);
    $conditions = [];
    $conditions[] = new OperatorCondition('time', $afterSix);
    $conditions[] = new OperatorCondition('time', $beforeTen);
    $toggle = new Toggle('disable-work-email', $conditions, Toggle::STRATEGY_UNANIMOUS);
    $manager->add($toggle);
    $context = new Context();
    $context->set('time', (int) date('G'));
    echo 'Everyone is ' . ($manager->active('disable-work-email', $context))
    ? ‘happy!' : 'sad :(';

    View full-size slide

  71. $manager = new ToggleManager(new InMemoryCollection());
    $toggle = new Toggle('some-feature', [
    new OperatorCondition('user_id', new GreaterThan(41))
    ]);
    $manager->add($toggle);
    Configuration: Code

    View full-size slide

  72. Configuration: Array
    $data = array(
    'some-feature' => array(
    'name' => 'toggling',
    'conditions' => array(
    array(
    'name' => 'operator-condition',
    'key' => 'user_id',
    'operator' => array('name' => 'greater-than', 'value' => 41),
    ),
    ),
    ),
    );
    $serializer = new InMemoryCollectionSerializer();
    $collection = $serializer->deserialize($data);
    $manager = new ToggleManager($collection);

    View full-size slide

  73. Configuration: JSON/YAML
    $yaml = <<toggles:
    - name: some-feature
    conditions:
    - name: operator-condition
    key: user_id
    operator:
    name: greater-than
    value: 41
    YML;
    $array = Yaml::parse($yaml);
    $serializer = new InMemoryCollectionSerializer();
    $collection = $serializer->deserialize($array);
    $manager = new ToggleManager($collection);

    View full-size slide

  74. Configuration: Redis
    $predis = new Predis\Client();
    $collection = new PredisCollection('toggle_demo', $predis);
    $manager = new ToggleManager($collection);

    View full-size slide

  75. Qandidate Toggle API
    {
    "conditions" : [
    {
    "name" : "operator-condition",
    "operator" : {
    "name" : "less-than",
    "value" : "1337"
    },
    "key" : "user_id"
    }
    ],
    "name" : "foo",
    "status" : "conditionally-active",
    "originalName" : "foo"
    }
    PUT /toggles/{name}
    https://github.com/qandidate-labs/qandidate-toggle-api

    View full-size slide

  76. Qandidate Toggle UI
    https://github.com/qandidate-labs/qandidate-toggle-ui

    View full-size slide

  77. Build (Pros)
    1. Free!
    2. Total control
    3. Complex rules
    4. API and UI available

    View full-size slide

  78. Build (Cons)
    1. Self hosted
    2. PHP only
    3. Complex rules

    View full-size slide

  79. Multi-Language

    View full-size slide

  80. Prebuilt Dashboard

    View full-size slide

  81. Environment Aware

    View full-size slide

  82. Buy (Pros)
    1. Lovely dashboard
    2. Multi-language
    3. Built in audit log
    4. Environment aware
    5. A/B testing conversion tracking

    View full-size slide

  83. Buy (Cons)
    1. It costs (at least) $79/month
    2. Can’t modify the product
    3. Dependent on 3rd party

    View full-size slide

  84. Build vs Buy

    View full-size slide

  85. Show me code

    View full-size slide

  86. use Qandidate\Bundle\ToggleBundle\Annotations\Toggle;
    /**
    * @Toggle("profile-management")
    */
    class ProfileController
    {
    public function updatePassword()
    {
    }
    public function updateEmail()
    {
    }
    }

    View full-size slide

  87. use Qandidate\Bundle\ToggleBundle\Annotations\Toggle;
    class ProfileController
    {
    /**
    * @Toggle("profile-edit-password")
    */
    public function updatePassword()
    {
    }
    public function updateEmail()
    {
    }
    }

    View full-size slide

  88. public function onKernelRequest($event) {
    $flag = $event->getRequest->attributes->get('_flag');
    $alt = $event->getRequest->attributes->get('_alt');
    if ($toggleManager->active($flag, $context)) {
    $event->getRequest->attributes->set('controller', $alt);
    }
    }
    profile:
    path: /profile/{user}
    defaults:
    _controller: AcmeBundle:Profile:update
    _alt: AcmeBundle:NewProfile:update
    _flag: new-profile-controller

    View full-size slide

  89. {% if feature_is_active('new-profile-image-picker') %}
    {# Show new picker #}
    {% else %}
    {# Use old flash-based picker #}
    {% endif %}

    View full-size slide

  90. class ProfileFactory {
    public function build($args, $context) {
    if ($this->toggleManager->active('new-profile', $context)) {
    return new ShinyNewProfile($args);
    }
    return new Profile($args);
    }
    }

    View full-size slide

  91. Controller View Factory
    Enable a
    route for
    specific users
    Show different
    UI elements
    Branch by
    abstraction for
    refactoring

    View full-size slide

  92. Toggle Configuratin

    View full-size slide

  93. Hard
    Coded
    Change
    the code
    Dynamic
    Static

    View full-size slide

  94. Hard
    Coded
    Deploy
    Config
    Change
    the code
    Deploy
    your
    application
    Dynamic
    Static

    View full-size slide

  95. Hard
    Coded
    Deploy
    Config
    Parameterised
    Config
    Change
    the code
    Deploy
    your
    application
    Edit config,
    restart app
    Dynamic
    Static

    View full-size slide

  96. Hard
    Coded
    Deploy
    Config
    Parameterised
    Config
    Database
    Config
    Change
    the code
    Deploy
    your
    application
    Edit config,
    restart app
    Edit via
    UI
    Dynamic
    Static

    View full-size slide

  97. Hard
    Coded
    Deploy
    Config
    Parameterised
    Config
    Database
    Config
    Distributed
    Config
    Change
    the code
    Deploy
    your
    application
    Edit config,
    restart app
    Edit via
    UI
    e.g. Consul
    or
    Zookeeper
    Dynamic
    Static

    View full-size slide

  98. Feature Dashboard

    View full-size slide

  99. Role Based Access

    View full-size slide

  100. Non-technical Users

    View full-size slide

  101. Technical Debt

    View full-size slide

  102. Cleanup based on activity

    View full-size slide

  103. Cleanup based on activity

    View full-size slide

  104. Create & Cleanup

    View full-size slide

  105. Expiration date

    View full-size slide

  106. Explosion date

    View full-size slide

  107. Tips & Tricks

    View full-size slide

  108. Testing Flags

    View full-size slide

  109. Testing Flags
    Feature A Feature B
    ✅ ✅
    ✅ ☠
    ☠ ✅
    ☠ ☠

    View full-size slide

  110. Testing Flags
    Feature A Feature B Feature C
    ✅ ✅ ✅
    ✅ ☠ ✅
    ✅ ✅ ☠
    ☠ ✅ ✅
    ✅ ☠ ☠
    ☠ ✅ ☠
    ☠ ☠ ✅
    ☠ ☠ ☠

    View full-size slide

  111. Testing Flags
    Feature A Feature B Feature C Feature D
    ✅ ✅ ✅ ✅
    ✅ ✅ ✅ ☠
    ✅ ✅ ☠ ✅
    ✅ ☠ ✅ ✅
    ☠ ✅ ✅ ✅
    ✅ ✅ ☠ ☠
    ✅ ☠ ✅ ☠
    ✅ ☠ ☠ ✅
    ☠ ✅ ✅ ☠
    ☠ ✅ ☠ ✅
    ☠ ☠ ✅ ✅
    ✅ ☠ ☠ ☠
    ☠ ✅ ☠ ☠
    ☠ ☠ ✅ ☠
    ☠ ☠ ☠ ✅
    ☠ ☠ ☠ ☠

    View full-size slide

  112. Testing Flags
    Number of flags Possible permutations
    2 4
    3 8
    4 16
    5 32
    10 1024
    25 33,554,432
    50 1.12589991E+15
    100 1.2676506E+30

    View full-size slide

  113. Homepage
    Carousel
    Homepage CTA
    ✅ ✅
    ✅ ☠
    ☠ ✅
    ☠ ☠
    Forum Edit
    Permission
    Forum Delete
    Permission
    ✅ ✅
    ✅ ☠
    ☠ ✅
    ☠ ☠
    Upload Profile
    Image


    New Interesting
    Algorithm


    High Post Length


    Show Debug
    Info?


    Enable
    Audit Log


    Auto Logout


    API Access API v2
    ✅ ✅
    ✅ ☠
    ☠ ✅
    ☠ ☠

    View full-size slide

  114. Expose State
    {
    "api_v2": true,
    "audit_log": true,
    "new_recommendation_algo": false
    }
    GET /toggle_status

    View full-size slide

  115. Flag Descriptions
    basic-rec-algo: true



    View full-size slide

  116. Flag Descriptions
    basic-rec-algo:

    enabled: true

    description: Use a simplistic recommendation algorithm. This is fast and
    produces less load on backend systems, but is way less accurate than
    our standard algorithm.

    View full-size slide

  117. Clear Naming
    allow_user_delete and allow_user_edit
    not
    user_control

    View full-size slide

  118. Audit Log
    Feature State User Date
    allow_user_edit
    ✅ Sarah 2018-05-11 12:22:08
    enhanced_rec_algo
    ✅ Ashley 2018-05-04 15:33:16
    allow_user_signup
    ☠ Ashley 2018-05-04 15:27:33
    enhanced_rec_algo
    ☠ system:load_manager 2018-05-04 15:16:30
    allow_user_signup
    ✅ Bob 2018-05-04 15:12:01
    enable_api
    ✅ Sarah 2018-04-30 09:56:59

    View full-size slide

  119. Fail Safe
    Do positively enable features e.g. enable-ibm-
    analysis, enhanced-recomendation-algo

    Don’t use flags for suppressing new features e.g.

    disable-shiny-new-feature

    View full-size slide

  120. Flags are great!

    View full-size slide

  121. Ops
    Toggles
    Release
    Toggles
    Flag Classes
    Permission
    Toggles
    Experiment
    Toggles

    View full-size slide

  122. Build vs Buy

    View full-size slide

  123. Qandidate Toggle

    View full-size slide

  124. One More Thing

    View full-size slide

  125. Knight Capital

    View full-size slide

  126. High frequency trader

    View full-size slide

  127. Mistakes were made

    View full-size slide

  128. Knight Capital
    1. Reused flags

    View full-size slide

  129. Knight Capital
    1. Reused flags
    2. Didn’t delete obsolete code

    View full-size slide

  130. Knight Capital
    1. Reused flags
    2. Didn’t delete obsolete code
    3. Couldn’t be disabled without a deployment

    View full-size slide

  131. Knight Capital
    1. Reused flags
    2. Didn’t delete obsolete code
    3. Couldn’t be disabled without a deployment
    4. The deploy actually enabled the old code on
    all 8 servers, instead of disabling it

    View full-size slide

  132. @mheap / @NexmoDev
    [email protected]
    https://joind.in/talk/583e8

    View full-size slide