Slide 1

Slide 1 text

Sean Sullivan February 19, 2019 Serverless Meetup Portland Oregon SAFE LAMBDA DEPLOYMENTS

Slide 2

Slide 2 text

www.stackery.io

Slide 3

Slide 3 text

software engineer Portland Oregon e-commerce systems Hudson’s Bay Company About me

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

saksfifthavenue.com saksoff5th.com lordandtaylor.com thebay.com

Slide 6

Slide 6 text

saksfifthavenue.com

Slide 7

Slide 7 text

“Your deploys should be as boring, straightforward, and stress-free as possible” Zach Holman March 2016 https://zachholman.com/posts/deploying-software

Slide 8

Slide 8 text

“You don't need release managers, you don't need special deploy days, you don't need all hands on deck for every single deploy.” Zach Holman March 2016 https://zachholman.com/posts/deploying-software

Slide 9

Slide 9 text

“We love to deploy new code incrementally hundreds of times a day. And there's good reason for that: it's safer overall. Incremental deploys are easier to understand and fix than one gigantic deploy once a year.” https://zachholman.com/talk/move-fast-break-nothing/ Zach Holman October 2014

Slide 10

Slide 10 text

https://twitter.com/gwenshap/status/1059870433736937472 QCon San Francisco November 2018

Slide 11

Slide 11 text

“You go to production with the code you have, not the code you might want or wish to have at a later time.” Donald Rumsfeld December 2004 * fake quote

Slide 12

Slide 12 text

https://twitter.com/CodeWisdom/status/1097852555118432257

Slide 13

Slide 13 text

undifferentiated heavy lifting AWS

Slide 14

Slide 14 text

provisioning servers AWS serverless

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

system plumbing business logic

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

HBC 2019 microservice architecture multi-tenant services increased adoption of AWS Lambda safe deployments

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

HBC email types • order acknowledgement • order shipment partial • order shipment full • order cancel • ready for pickup in store

Slide 21

Slide 21 text

Queue consumer Event queue Email Dashboard UI Email Sender Email Render External Email Provider Feature Flag service

Slide 22

Slide 22 text

Email Render Product Detail service Delivery Date Estimation service request HTML Feature Flag service

Slide 23

Slide 23 text

HBC email system • multi-tenant design • Cloudwatch Metrics • Cloudwatch Alerts

Slide 24

Slide 24 text

HBC email system • code coverage threshold • components deployed independently • safe deployments

Slide 25

Slide 25 text

Mark Zuckerberg

Slide 26

Slide 26 text

Move Fast and Break Things Move Slow and Don’t Break Things

Slide 27

Slide 27 text

Move Fast and Don’t Break Things Move Purposefully and Fix Things

Slide 28

Slide 28 text

manual testing automated testing

Slide 29

Slide 29 text

how often do you release code to production?

Slide 30

Slide 30 text

safe AWS Lambda deployments?

Slide 31

Slide 31 text

https://www.youtube.com/watch?v=UIp6sLyvoC0 Gerardo Estaba August 2018

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

Traffic Shifting

Slide 35

Slide 35 text

You can now shift incoming traffic between two AWS Lambda function versions based on pre-assigned weights. You can now also use AWS CodeDeploy to automatically manage the rollout of new function versions. This allows you to gradually shift traffic between two versions source: aws.amazon.com

Slide 36

Slide 36 text

Traffic Shifting configurations AllAtOnce Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes

Slide 37

Slide 37 text

Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes Traffic Shifting configurations

Slide 38

Slide 38 text

all-at-once traffic shifting gradual traffic shifting

Slide 39

Slide 39 text

AWS command line tools

Slide 40

Slide 40 text

$ brew update $ brew tap aws/tap $ brew install aws/tap/aws-sam-cli $ brew install aws-cli

Slide 41

Slide 41 text

Live demo

Slide 42

Slide 42 text

aws cloudformation package \ --template-file my-template.yaml \ --s3-bucket mycorp-codedeploy \ --s3-prefix my-lambda-func/jars \ --output-template-file package/2325bd1e.yaml

Slide 43

Slide 43 text

aws cloudformation deploy \ --template-file package/2325bd1e.yaml \ --stack-name mylambda-func—xyz \ --parameter-overrides Stage=xyz \ --region us-east-1 \ --capabilities CAPABILITY_IAM \ --notification-arns arn:aws:sns:us- east-1:123456789:myteam-cf-notifications

Slide 44

Slide 44 text

aws cloudformation deploy \ --template-file package/2325bd1e.yaml \ --stack-name mylambda-func—xyz \ --parameter-overrides Stage=xyz \ --parameter-overrides DeploymentPref=Canary10Percent5Minutes \ --region us-east-1 \ --capabilities CAPABILITY_IAM \ --notification-arns arn:aws:sns:us- east-1:123456789:myteam-cf-notifications

Slide 45

Slide 45 text

Traffic Hooks

Slide 46

Slide 46 text

AWS traffic hooks • PreTraffic hook • PostTraffic hook

Slide 47

Slide 47 text

AWS CodeDeploy UI https://console.aws.amazon.com/codesuite/codedeploy/deployments

Slide 48

Slide 48 text

https://github.com/awslabs/serverless-application-model/pull/605

Slide 49

Slide 49 text

AWS re:Invent 2018

Slide 50

Slide 50 text

Clare Liguori November 2018 https://www.youtube.com/watch?v=01ewawuL-IY

Slide 51

Slide 51 text

Clare Liguori November 2018 https://www.youtube.com/watch?v=01ewawuL-IY

Slide 52

Slide 52 text

additional validation logic in Pre Traffic hook implement Post Traffic hook utilize AWS Cloud9 IDE Future work at HBC

Slide 53

Slide 53 text

Final thoughts

Slide 54

Slide 54 text

traffic shifting

Slide 55

Slide 55 text

small incremental releases

Slide 56

Slide 56 text

engineering velocity is a competitive advantage

Slide 57

Slide 57 text

The end

Slide 58

Slide 58 text

No content