Implementing Feature Flags in Serverless Environments

Implementing Feature Flags in Serverless Environments

With feature flags, we can change our software's behavior at runtime without modifying or re-deploying our code. There are many good reasons to use feature flags. Among other things, they allow us to separate the risk of deployment from that of rolling out new functionality. However, implementing feature flags in serverless environments like AWS Lambda poses some challenges. In this presentation, Mathias will show you how to overcome these difficulties by building a serverless flag storage pipeline for the popular feature management platform LaunchDarkly. All using Go, of course.

2190d7a468f51fa3be5eabfc9397a28b?s=128

Mathias Lafeldt

June 26, 2018
Tweet

Transcript

  1. 4.

    "A feature flag is a way to change your software's

    functionality without changing and re-deploying your code."
  2. 6.

    Use Cases of Feature Flags • Separate deployment from feature

    rollout • Canary releases • Production testing • Turning things off with a kill switch • A/B testing, Chaos Engineering, and other experiments • Migrations • Trunk-based development
  3. 7.

    Open Source and SaaS Solutions • Unleash - Enterprise ready

    feature toggles service (OSS) • Rollout - A secure feature management system for the enterprise • Split - Feature experimentation platform for engineering and product teams • LaunchDarkly - Feature management platform for modern development
  4. 8.
  5. 9.
  6. 10.
  7. 11.

    LaunchDarkly Go SDK Example import ld "gopkg.in/launchdarkly/go-client.v4" ldClient, err :=

    ld.MakeClient("some-sdk-key", 5*time.Second) if err != nil { ... } defer ldClient.Close() ldUser := ld.NewUser("some-user") flagA, _ := ldClient.BoolVariation("some.bool.flag", ldUser, false) if flagA { ... } flagB, _ := ldClient.IntVariation("some.int.flag", ldUser, 2018) doSomething(flagB)
  8. 12.

    Go Serverless, not Flagless: Implementing Feature Flags in Serverless Environments

    https://blog.launchdarkly.com/go-serveless-not-flagless- implementing-feature-flags-in-serverless-environments (April 14, 2017)
  9. 13.

    "[Lambda functions] will have poor cold start performance because we

    need to call out to LaunchDarkly whenever a new Execution Context is created."
  10. 14.
  11. 15.

    Using the Redis Feature Store import ld "gopkg.in/launchdarkly/go-client.v4" import redis

    "gopkg.in/launchdarkly/go-client.v4/redis" config := ld.DefaultConfig config.FeatureStore = redis.NewRedisFeatureStoreFromUrl( "redis://localhost:6379", "launchdarkly", 30*time.Second, nil) // Enable daemon mode to only read flags from Redis config.UseLdd = true ldClient, err := ld.MakeCustomClient("some-sdk-key", config, 5*time.Second) if err != nil { ... }
  12. 16.
  13. 17.
  14. 18.

    "From various reports around the web, [Lambda] cold starts within

    VPCs could add up to 10 seconds of latency!" https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-latency-killer-5408323278dd
  15. 19.

    If only there were some serverless key-value store that we

    can easily access from AWS Lambda... !
  16. 20.
  17. 21.

    Let's Write a DynamoDB Feature Store! type FeatureStore interface {

    Get(kind VersionedDataKind, key string) (VersionedData, error) All(kind VersionedDataKind) (map[string]VersionedData, error) Init(map[VersionedDataKind]map[string]VersionedData) error Delete(kind VersionedDataKind, key string, version int) error Upsert(kind VersionedDataKind, item VersionedData) error Initialized() bool } var VersionedDataKinds = [...]VersionedDataKind{ Features, Segments, }
  18. 22.

    Diving into the DynamoDB API and AWS SDK • GetItem

    • PutItem • Scan • Query • BatchWriteItem • ...
  19. 23.

    $ curl -s -H 'Authorization: some-sdk-key' https://app.launchdarkly.com/sdk/latest-flags | jq .

    { "some.bool.flag": { "key": "some.bool.flag", "version": 2, "on": true, "prerequisites": [], "salt": "57c59e2cfb724c749ff438332035ccea", "sel": "cedb458484144ead94864dfff58cee51", "targets": [], "rules": [], "fallthrough": { "variation": 0 }, "offVariation": 1, "variations": [ true, false ], "trackEvents": true, "debugEventsUntilDate": null, "deleted": false }, "some.int.flag": ... }
  20. 24.

    dynamodbattribute.MarshalMap/UnmarshalMap type FeatureFlag struct { Key string `json:"key" bson:"key"` Version

    int `json:"version" bson:"version"` On bool `json:"on" bson:"on"` Prerequisites []Prerequisite `json:"prerequisites" bson:"prerequisites"` Salt string `json:"salt" bson:"salt"` Sel string `json:"sel" bson:"sel"` Targets []Target `json:"targets" bson:"targets"` Rules []Rule `json:"rules" bson:"rules"` Fallthrough VariationOrRollout `json:"fallthrough" bson:"fallthrough"` OffVariation *int `json:"offVariation" bson:"offVariation"` Variations []interface{} `json:"variations" bson:"variations"` TrackEvents bool `json:"trackEvents" bson:"trackEvents"` DebugEventsUntilDate *uint64 `json:"debugEventsUntilDate" bson:"debugEventsUntilDate"` Deleted bool `json:"deleted" bson:"deleted"` }
  21. 25.

    How to Store Everything in the Same DynamoDB Table? •

    Use the item's kind as the primary partition key, e.g. features or segments • Use the item's key as the primary sort key, e.g. feature.foo • Figure out how to do this without modifying LaunchDarkly's structs • ! Add a namespace attribute after using MarshalMap
  22. 26.

    Making the Test Suite Happy import ldtest "gopkg.in/launchdarkly/go-client.v4/shared_test" func TestDynamoDBFeatureStore(t

    *testing.T) { table := os.Getenv("LAUNCHDARKLY_DYNAMODB_TABLE") if table == "" { t.Skip("LAUNCHDARKLY_DYNAMODB_TABLE not set in environment") } ldtest.RunFeatureStoreTests(t, func() ld.FeatureStore { store, err := NewDynamoDBFeatureStore(table, nil) if err != nil { t.Fatal(err) } return store }) }
  23. 29.

    Using the DynamoDB Feature Store import ld "gopkg.in/launchdarkly/go-client.v4" import "github.com/mlafeldt/launchdarkly-dynamo-store/dynamodb"

    store, err := dynamodb.NewDynamoDBFeatureStore("some-table", nil) if err != nil { ... } config := ld.DefaultConfig config.FeatureStore = store config.UseLdd = true ldClient, err := ld.MakeCustomClient("some-sdk-key", config, 5*time.Second) if err != nil { ... }
  24. 30.

    Wait - we are not done yet. ! How to

    propagate flag updates?
  25. 32.
  26. 33.

    Takeaways • Use feature flags! • Take advantage of existing

    feature flag tooling • Cache flags in environments sensitive to cold starts • Choose the right key-value store for the job • Use a (serverless) flag storage pipeline to propagate changes