Implementing Feature Flags in Serverless Environments - GDG Go & Cloud User Group Hamburg - 

Hello I'm Mathias Lafeldt Freelance Solutions Architect @mlafeldt 

Feature Flags 101 The Ultimate Feature Flag Guide ! https://rollout.io/blog/ultimate-feature-flag-guide 

"A feature flag is a way to change your software's functionality without changing and re-deploying your code." 

A Very Contrived Example if (feature.isActive("holiday-greeting")) { print("Happy holidays," + user.name + "!"); } 

Use Cases of Feature Flags • Separate deployment from feature rollout • Canary releases • Production testing • Turning things off with a kill switch • A/B testing, Chaos Engineering, and other experiments • Migrations • Trunk-based development 

Open Source and SaaS Solutions • Unleash - Enterprise ready feature toggles service (OSS) • Rollout - A secure feature management system for the enterprise • Split - Feature experimentation platform for engineering and product teams • LaunchDarkly - Feature management platform for modern development 

No content

No content

No content

LaunchDarkly Go SDK Example import ld "gopkg.in/launchdarkly/go-client.v4" ldClient, err := ld.MakeClient("some-sdk-key", 5*time.Second) if err != nil { ... } defer ldClient.Close() ldUser := ld.NewUser("some-user") flagA, _ := ldClient.BoolVariation("some.bool.flag", ldUser, false) if flagA { ... } flagB, _ := ldClient.IntVariation("some.int.flag", ldUser, 2018) doSomething(flagB) 

Go Serverless, not Flagless: Implementing Feature Flags in Serverless Environments https://blog.launchdarkly.com/go-serveless-not-flagless- implementing-feature-flags-in-serverless-environments (April 14, 2017) 

"[Lambda functions] will have poor cold start performance because we need to call out to LaunchDarkly whenever a new Execution Context is created." 

No content

Using the Redis Feature Store import ld "gopkg.in/launchdarkly/go-client.v4" import redis "gopkg.in/launchdarkly/go-client.v4/redis" config := ld.DefaultConfig config.FeatureStore = redis.NewRedisFeatureStoreFromUrl( "redis://localhost:6379", "launchdarkly", 30*time.Second, nil) // Enable daemon mode to only read flags from Redis config.UseLdd = true ldClient, err := ld.MakeCustomClient("some-sdk-key", config, 5*time.Second) if err != nil { ... } 

No content

No content

"From various reports around the web, [Lambda] cold starts within VPCs could add up to 10 seconds of latency!" https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-latency-killer-5408323278dd 

If only there were some serverless key-value store that we can easily access from AWS Lambda... ! 

No content

Let's Write a DynamoDB Feature Store! type FeatureStore interface { Get(kind VersionedDataKind, key string) (VersionedData, error) All(kind VersionedDataKind) (map[string]VersionedData, error) Init(map[VersionedDataKind]map[string]VersionedData) error Delete(kind VersionedDataKind, key string, version int) error Upsert(kind VersionedDataKind, item VersionedData) error Initialized() bool } var VersionedDataKinds = [...]VersionedDataKind{ Features, Segments, } 

Diving into the DynamoDB API and AWS SDK • GetItem • PutItem • Scan • Query • BatchWriteItem • ... 

$ curl -s -H 'Authorization: some-sdk-key' https://app.launchdarkly.com/sdk/latest-flags | jq . { "some.bool.flag": { "key": "some.bool.flag", "version": 2, "on": true, "prerequisites": [], "salt": "57c59e2cfb724c749ff438332035ccea", "sel": "cedb458484144ead94864dfff58cee51", "targets": [], "rules": [], "fallthrough": { "variation": 0 }, "offVariation": 1, "variations": [ true, false ], "trackEvents": true, "debugEventsUntilDate": null, "deleted": false }, "some.int.flag": ... } 

dynamodbattribute.MarshalMap/UnmarshalMap type FeatureFlag struct { Key string `json:"key" bson:"key"` Version int `json:"version" bson:"version"` On bool `json:"on" bson:"on"` Prerequisites []Prerequisite `json:"prerequisites" bson:"prerequisites"` Salt string `json:"salt" bson:"salt"` Sel string `json:"sel" bson:"sel"` Targets []Target `json:"targets" bson:"targets"` Rules []Rule `json:"rules" bson:"rules"` Fallthrough VariationOrRollout `json:"fallthrough" bson:"fallthrough"` OffVariation *int `json:"offVariation" bson:"offVariation"` Variations []interface{} `json:"variations" bson:"variations"` TrackEvents bool `json:"trackEvents" bson:"trackEvents"` DebugEventsUntilDate *uint64 `json:"debugEventsUntilDate" bson:"debugEventsUntilDate"` Deleted bool `json:"deleted" bson:"deleted"` } 

How to Store Everything in the Same DynamoDB Table? • Use the item's kind as the primary partition key, e.g. features or segments • Use the item's key as the primary sort key, e.g. feature.foo • Figure out how to do this without modifying LaunchDarkly's structs • ! Add a namespace attribute after using MarshalMap 

Making the Test Suite Happy import ldtest "gopkg.in/launchdarkly/go-client.v4/shared_test" func TestDynamoDBFeatureStore(t *testing.T) { table := os.Getenv("LAUNCHDARKLY_DYNAMODB_TABLE") if table == "" { t.Skip("LAUNCHDARKLY_DYNAMODB_TABLE not set in environment") } ldtest.RunFeatureStoreTests(t, func() ld.FeatureStore { store, err := NewDynamoDBFeatureStore(table, nil) if err != nil { t.Fatal(err) } return store }) } 

Making LaunchDarkly and AWS users happy (and this little puppy) 

Now Open Source: DynamoDB Store for LaunchDarkly's Go SDK github.com/mlafeldt/launchdarkly-dynamo-store 

Using the DynamoDB Feature Store import ld "gopkg.in/launchdarkly/go-client.v4" import "github.com/mlafeldt/launchdarkly-dynamo-store/dynamodb" store, err := dynamodb.NewDynamoDBFeatureStore("some-table", nil) if err != nil { ... } config := ld.DefaultConfig config.FeatureStore = store config.UseLdd = true ldClient, err := ld.MakeCustomClient("some-sdk-key", config, 5*time.Second) if err != nil { ... } 

Wait - we are not done yet. ! How to propagate flag updates? 

Serverless Flag Storage Pipeline 

! Demo ! 

Takeaways • Use feature flags! • Take advantage of existing feature flag tooling • Cache flags in environments sensitive to cold starts • Choose the right key-value store for the job • Use a (serverless) flag storage pipeline to propagate changes 

Thank you. @mlafeldt [email protected] github.com/mlafeldt/launchdarkly-dynamo-store