Safe Lambda Deployments - San Francisco - 2019-06-13

AWS Lambda
safe deployments
Serverless Nights San Francisco
June 13, 2019

Transcript

1. Sean Sullivan June 13, 2019 San Francisco Serverless Nights SAFE LAMBDA DEPLOYMENTS
2. software engineer Portland Oregon e-commerce systems Hudson’s Bay Company About me
5. saksfifthavenue.com saksoff5th.com lordandtaylor.com thebay.com
6. saksfifthavenue.com
7. legacy systems cloud native systems
8. servers serverless
9. Trying to build serverless applications and chase rings. That’s all I'm about #DubNation
10. @StephenCurry30 LOL 1-800-SERVERLESS 1-800-SERVERLESS
11. @Drake I probably relate more to the casual developer who writes JavaScript and is not super athletic
12. @StephenCurry30 LET’S GO!!! 1-800-RAPTORS 1-800-RAPTORS
13. @Drake deploying code is the most important part of the game
14. @Drake I try to make production deployments look easy, but the behind-the-scenes stuff is a challenge
15. production deployments
16. Losing game 1 to the Raptors was frustrating, but not as frustrating as the bad production deployments that I've seen
17. Extended production outages are pretty devastating. That feeling never goes away ☹
18. Zach Holman March 2016 https://zachholman.com/posts/deploying-software
19. “Your deploys should be as boring, straightforward, and stress-free as possible” Zach Holman March 2016 https://zachholman.com/posts/deploying-software
20. “You don't need release managers, you don't need special deploy days, you don't need all hands on deck for every single deploy.” Zach Holman March 2016 https://zachholman.com/posts/deploying-software
21. “We love to deploy new code incrementally hundreds of times a day. And there's good reason for that: it's safer overall. Incremental deploys are easier to understand and fix” https://zachholman.com/talk/move-fast-break-nothing/ Zach Holman October 2014
22. deployment process
23. I never wanted to play basketball, I just wanted to write code and deploy to production
24. autonomy
26. bureaucracy freedom and responsibility
27. HBC 2019 microservice architecture multi-tenant services increased adoption of AWS Lambda safe deployments
28. system plumbing business logic
30. HBC email types • order acknowledgement • order shipment partial • order shipment full • order cancel • ready for pickup in store
31. Queue consumer Event queue Email Dashboard UI Email Sender Email Render External Email Provider Feature Flag service
32. Email Render Product Detail service Delivery Date Estimation service request HTML Feature Flag service
33. HBC email system multi-tenant design Cloudwatch Metrics Cloudwatch Alarms
34. HBC email system code coverage threshold components deployed independently safe deployments
35. Move Fast and Break Things Move Slow and Don’t Break Things Move Fast and Don’t Break Things Move Purposefully and Fix Things
36. manual testing automated testing
37. safe AWS Lambda deployments?
39. Safe Lambda deployments validation hooks traffic shifting
40. YAML
42. Traffic Shifting
43. You can now shift incoming traffic between two AWS Lambda function versions based on pre-assigned weights. You can now also use AWS CodeDeploy to automatically manage the rollout of new function versions. This allows you to gradually shift traffic between two versions source: aws.amazon.com
44. Traffic Shifting configurations AllAtOnce Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes
45. Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes Traffic Shifting configurations
46. all-at-once traffic shifting gradual traffic shifting
47. AWS command line tools
48. $ brew update $ brew tap aws/tap $ brew install aws/tap/aws-sam-cli $ brew install aws-cli
49. aws cloudformation package \ --template-file my-template.yaml \ --s3-bucket mycorp-codedeploy \ --s3-prefix my-lambda-func/jars \ --output-template-file package/2325bd1e.yaml
50. aws cloudformation deploy \ --template-file package/2325bd1e.yaml \ --stack-name mylambda-func—xyz \ --parameter-overrides Stage=xyz \ --parameter-overrides DeploymentPref=Canary10Percent5Minutes \ --region us-east-1 \ --capabilities CAPABILITY_IAM \ --notification-arns arn:aws:sns:us- east-1:123456789:myteam-cf-notifications
51. Traffic Hooks
52. AWS traffic hooks PreTraffic hook PostTraffic hook
53. AWS CodeDeploy UI https://console.aws.amazon.com/codesuite/codedeploy/deployments
54. https://github.com/awslabs/serverless-application-model/pull/605
55. additional validation logic in Pre Traffic hook implement Post Traffic hook utilize AWS Cloud9 IDE Future work at HBC
56. Final thoughts
57. traffic shifting
58. small incremental releases
59. engineering velocity is a competitive advantage
60. I'd rather be hated for the code that I’ve deployed than loved for winning an NBA title
61. THANK YOU!
63. Bonus slides
64. To win MVP was a dream come true for sure, but I'd rather deploy code to production
65. Always Be Deploying