Taking Serverless to the Next Level

Taking Serverless to the Next Level

ServerlessDays, Cardiff, January 30th, 2019

What are the top best practices to build and run applications without thinking about servers? Model your serverless components as templates and deploy your infrastructure as config to leverage best practices such as code reviews. Step-through and debug your code with a local execution environment. Coordinate and manage the state of each distributed component. Continuously deliver your code in production using application lifecycle management tools. Enable gradual, safe deployments. To support our findings, we’ll review customer case studies to see what they did, why, and which benefits they got most.

7c9b8b368924556d8642bdaed3ded1f5?s=128

Danilo Poccia

January 30, 2019
Tweet

Transcript

  1. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Taking Serverless to the Next Level Danilo Poccia Principal Evangelist, Serverless, AWS @danilop danilop
  2. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark “I know how to build a serverless function, now what?”
  3. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code
  4. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code ✓ Make infrastructure changes repeatable and predictable ✓ Release infrastructure changes using the same tools as code changes ✓ Replicate production environment in a staging environment to enable continuous testing
  5. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Declarative I tell you what I need I tell you what to do Imperative
  6. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code best practices ✓ Infrastructure and application in the same source repository For example: AWS CloudFormation HashiCorp Terraform ✓ Deployments include infrastructure updates
  7. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code for serverless apps For example: AWS Serverless Application Model (SAM) Serverless Framework
  8. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark SAM template AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetFunction: Type: AWS::Serverless::Function Properties: Handler: index.get Runtime: nodejs8.10 CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref MyTable Events: GetResource: Type: Api Properties: Path: /resource/{resourceId} Method: get MyTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway • DynamoDB table
  9. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark SAM CLI https://github.com/awslabs/aws-sam-cli
  10. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Nested apps to simplify solving recurring problems Standard Component TweetSource: Type: AWS::Serverless::Application Properties: Location: ApplicationId: arn:aws:serverlessrepo:... SemanticVersion: 2.0.0 Parameters: TweetProcessorFunctionName: !Ref MyFunction SearchText: '#serverless -filter:nativeretweets’ Custom Business Logic
  11. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark AWS Cloud Development Kit (CDK) https://awslabs.github.io/aws-cdk AWS CDK Toolkit + AWS Construct Library + @aws-cdk/aws-serverless D eveloper Preview
  12. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments
  13. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Source Build Test Production Continuous Integration / Continuous Deployment
  14. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Serverless deployments Code Stack Package Deploy Template
  15. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Serverless deployments with a test environment Code Test Stack Package Deploy Template Feedback Loop Production Stack Deploy
  16. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark CodeDeploy – Lambda deployments Enable in your serverless application template Resources: GetFunction: Type: AWS::Serverless::Function Properties: DeploymentPreference: Type: Canary10Percent10Minutes Alarms: - !Ref ErrorsAlarm Hooks: PreTraffic: !Ref PreTrafficHook Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce
  17. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 Lambda function code 100%
  18. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code 100% Run PreTraffic hook against v2 code before it receives traffic v2 code 0%
  19. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code 90% Wait for 10 minutes, roll back in case of alarm v2 code 10%
  20. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark CodeDeploy – Lambda canary deployment API Gateway Lambda function weighted alias “live” v1 code 0% Run PostTraffic hook and complete deployment v2 code 100%
  21. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark API Gateway canary stage API Gateway Production stage v1 code v2 code 99.5% 0.5% Canary stage
  22. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark BUSINESS LOGIC LIB B Before BUSINESS LOGIC LIB A LIB B BUSINESS LOGIC LIB A LIB B BUSINESS LOGIC LIB A LIB B LIB A Use Lambda Layers for shared code that doesn’t change frequently
  23. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark BUSINESS LOGIC BUSINESS LOGIC BUSINESS LOGIC BUSINESS LOGIC LIB A LIB B Use Lambda Layers for shared code that doesn’t change frequently Focus on your business logic and speed up function deployments After
  24. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Code reviews 1. Review infrastructure changes 2. Understand architectural impact 3. How effective is to exchange feedback? 4. How effective is to review code history? 5. Using pull requests? 6. Is manual approval really required? 7. Look for different perspectives (cross team) X X X X
  25. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments Project to product
  26. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Project Product v1 v2 v3 Customer needs
  27. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Project Product Reach milestone Customer value Lifecycle costs Cost to reach milestone Backward looking Forward looking
  28. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark “The process becomes the proxy for the result you want. You stop looking at outcomes and just make sure you’re doing the process right.” Jeff Bezos 2016 Letter to Shareholders “Jeff, what does Day 2 look like?” That’s a question I just got at our most recent all-hands meeting. I’ve been reminding people that it’s Day 1 for a couple of decades. I work in an Amazon building named Day 1, and when I moved buildings, I took the name with me. I spend time thinking about this topic. “Day 2 is stasis. Followed by irrelevance. Followed by excruciating, painful decline. Followed by death. And that is why it is always Day 1.” To be sure, this kind of decline would happen in extreme slow motion. An established company might harvest Day 2 for decades, but the final result would still come. I’m interested in the question, how do you fend off Day 2? What are the techniques and tactics? How do you keep the vitality of Day 1, even inside a large organization? Such a question can’t have a simple answer. There will be many elements, multiple paths, and many traps. I don’t know the whole answer, but I may know bits of it. Here’s a starter pack of essentials for Day 1 defense: customer obsession, a skeptical view of proxies, the eager adoption of external trends, and high-velocity decision making. True Customer Obsession There are many ways to center a business. You can be competitor focused, you can be product focused, you can be technology focused, you can be business model focused, and there are more. But in my view, obsessive customer focus is by far the most protective of Day 1 vitality. Why? There are many advantages to a customer-centric approach, but here’s the big one: customers are always beautifully, wonderfully dissatisfied, even when they report being happy and business is great. Even when they don’t yet know it, customers want something better, and your desire to delight customers will drive you to invent on their behalf. No customer ever asked Amazon to create the Prime membership program, but it sure turns out they wanted it, and I could give you many such examples. Staying in Day 1 requires you to experiment patiently, accept failures, plant seeds, protect saplings, and double down when you see customer delight. A customer-obsessed culture best creates the conditions where all of that can happen. Resist Proxies As companies get larger and more complex, there’s a tendency to manage to proxies. This comes in many shapes and sizes, and it’s dangerous, subtle, and very Day 2. A common example is process as proxy. Good process serves you so you can serve customers. But if you’re not watchful, the process can become the thing. This can happen very easily in large organizations. The process becomes the proxy for the result you want. You stop looking at outcomes and just make sure you’re doing the process right. Gulp. It’s not that rare to hear a junior leader defend a bad outcome with something like, “Well, we followed the process.” A more experienced leader will use it as an opportunity to investigate and improve the process. The process is not the thing. It’s always worth asking, do we own the process or does the process own us? In a Day 2 company, you might find it’s the second. Another example: market research and customer surveys can become proxies for customers – something that’s especially dangerous when you’re inventing and designing products. “Fifty-five percent of beta testers report being satisfied with this feature. That is up from 47% in the first survey.” That’s hard to interpret and could unintentionally mislead. Resist proxies
  29. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Feedback to business Code Test Stack Package Deploy Template Feedback Loop Production Stack Deploy Business
  30. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Product Features Defects Risks Debts Product development Business Customers Security & Compliance Developers & Architects Avoid Overutilization
  31. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments Project to product Microservices
  32. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark “Complexity arises when the dependencies among the elements become important.” Scott E. Page, John H. Miller Complex Adaptive Systems
  33. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Add correlation IDs to logs – CloudWatch Logs + Insights
  34. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark End-to-end tracing – AWS X-Ray Service Map
  35. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark End-to-end tracing – AWS X-Ray Traces
  36. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Understand latency… and percentiles! P50 P90 P99 P99.9 P100
  37. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Client Application Get Content Index Get Content and Thumbnails Upload Content with Metadata Update Content Metadata Get Content Metadata Files (Multimedia) Database Build Thumbnails Update Content Index Extract and Update Metadata Event: new or updated file Event: new or updated content metadata Lambda Functions User Resources Think event-driven – Sample media sharing application
  38. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark “Antifragility is beyond resilience or robustness. The resilient resists shocks and stays the same; the antifragile gets better.” Nassim Nicholas Taleb Antifragile
  39. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments Project to product Microservices Team structure
  40. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Separable Vs complex tasks Separable task Complex task
  41. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark A model of abilities as a collection of cognitive tools Adam Ability = 5 { A, B, C, D, E } For example: A – mobile development on iOS B – back end development in Java C – data analytics in Python D – complex SQL queries E – …
  42. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark How can I build a 2 people team out of these 3 candidates? Adam Carl Betsy { C, D, G } Ability = 5 Ability = 4 Ability = 3 { A, B, E, F } { A, B, C, D, E }
  43. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Team with best abilities Adam Carl Betsy { C, D, G } Ability = 5 Ability = 4 Ability = 3 Team Ability = 6 { A, B, E, F } { A, B, C, D, E }
  44. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Team with more distinct cognitive tools – Diversity bonus Adam Carl Betsy { A, B, E, F } { A, B, C, D, E } { C, D, G } Ability = 5 Ability = 4 Ability = 3 Team Ability = 7
  45. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark No diversity, no bonus Adam Carl Betsy { A, B, D, E } { A, B, C, D, E } { B, D, E } Ability = 5 Ability = 4 Ability = 3
  46. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Some cognitive tools should be learned in order Adam Carl Betsy { A, B, C, D } { A, B, C, D, E } { A, B, C } Ability = 5 Ability = 4 Ability = 3
  47. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark 2,092 people who worked on 474 musicals from 1945 to 1989 Small world networks & creativity AJS Volume 111 Number 2 (September 2005): 000–000 PROOF 1 ᭧ 2005 by The University of Chicago. All rights reserved. 0002-9602/2005/11102-0003$10.00 Thursday Oct 13 2005 11:31 AM AJS v111n2 090090 VSJ Collaboration and Creativity: The Small World Problem1 Brian Uzzi Northwestern University Jarrett Spiro Stanford University Small world networks have received disproportionate notice in di- verse fields because of their suspected effect on system dynamics. The authors analyzed the small world network of the creative artists who made Broadway musicals from 1945 to 1989. Based on original arguments, new statistical methods, and tests of construct validity, they found that the varying “small world” properties of the systemic- level network of these artists affected their creativity in terms of the financial and artistic performance of the musicals they produced. The small world network effect was parabolic; performance in- creased up to a threshold after which point the positive effects reversed. Creativity aids problem solving, innovation, and aesthetics, yet our un- derstanding of it is still forming. We know that creativity is spurred when diverse ideas are united or when creative material in one domain inspires or forces fresh thinking in another. These structural preconditions suggest 1 Our thanks go out to Duncan Watts; Huggy Rao; Peter Murmann; Ron Burt; Matt Bothner; Frank Dobbin; Bruce Kogut; Lee Fleming; David Stark; John Padgett; Dan Diermeier; Stuart Oken; Jerry Davis; Woody Powell; workshop participants at the University of Chicago, University of California at Los Angeles, Harvard, Cornell, New York University, the Northwestern University Institute for Complex Organizations (NICO); and the excellent AJS reviewers, especially the reviewer who provided a remarkable 15, single-spaced pages of superb commentary. We particularly wish to thank Mark Newman for his advice and help in developing and interpreting the bipartite-affiliation network statistics. We also wish to give very special thanks to the Santa Fe Institute for creating a rich collaborative environment wherein these ideas first emerged, and to John Padgett, the organizer of the States and Markets group at the Santa Fe Institute. Direct correspondence to Brian Uzzi, Kellog School of Man- agement, Northwestern University, Evanston, Illinois 60208. E-mail: Uzzi@northwestern.edu
  48. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Two pizza teams Photo by Kristina Bratko on Unsplash = "(" − 1) 2 Communication paths in a team of N people 1. Maximize team abilities 2. Mix new and existing relationships 3. Minimize communication paths
  49. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Zappos’ collisions Collisions, or serendipitous encounters, according to the Zappos CEO, are a good thing and he’d like to see more people in his company’s new headquarters’ community having them. Also unique, Hsieh says, is that, like Zappos, the Downtown Project strives to “institutionalize return on luck.” By that, he means it enables those collisions among people in the community. Studies show that such random encounters increase innovation and productivity, he says. https://www.forbes.com/sites/techonomy/2013/11/15/why-zappos-ceo-hsieh-wants-to-enable-more-collisions-in-vegas
  50. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments Project to product Microservices Team structure Don’t reinvent the wheel
  51. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Capital One – Credit Offers API serverless architecture Affiliates www.capitalone.com/ credit-cards/prequalify AWS Cloud Capital One API Gateway VPC Lambda Function Traces Logs Production Support Command Center COAT Credit Offers API Team Lambda Function S3 Bucket TTL Third-Party API
  52. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Capital One – Credit Offers API CI/CD pipeline Continuous Improvement, Continuous Delivery! GitHub LGTM Bot Jenkins AWS SAM S3 Bucket (Versioning) Lambda Function DeploymentType: dev: AllAtOnce qa: AllAtOnce qaw: AllAtOnce prod: Canary10Percent10Minutes prodw: Canary10Percent10Minutes canary5xxGetProductsAlarm: Type: AWS::CloudFormation::Alarm Properties: AlarmActions: - !FindInMap: - params - AdminSNSTopic - !Ref Environment AlarmDescription: 500 error from product listing Lambda. ComparisonOperator: GreatherThanOrEqualTothreshold Period: 300 Statistic: Sum Threshold: 1 EvaluationPeriod: 1
  53. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Capital One – Benefits from taking the API serverless Performance gains From the time the request is received by lambda to the time to send the response back 70% Cost savings By removing EC2, ELB and RDS from our solution 90% Increase in team velocity Reduce investment in team’s time on DevOps and dedicate back to feature development! 30%
  54. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Guardian News & Media Workflows • automating subscriber account deletions across many distributed systems • receiving customer orders while external billing and payment services are offline • running an extract, transform, and load (ETL) newspaper-fulfillment pipeline through a series of Lambda functions • orchestrating a self-service account-deletion process to free up time for technical support staff “Developers do not have to be experts in workflow, and the developers who wrote the Lambda function can easily create the Step Functions workflow.”
  55. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Financial Engines The integer programming optimizer (IPO) server uses a native, third-party solver library to perform the calculations used to evaluate and optimize investment portfolios. The company replaced 50 IPO servers running in 21 different environments with four Lambda endpoints that handle all the traffic. Each time an IPO request is received, a Lambda function is triggered and runs the IPO code. Since the functions are stateless, AWS Lambda can run as many copies of the function as needed in parallel, dynamically allocating compute capacity to match the rate of incoming requests. “We see spikes of up to 1,000 requests per second, with an average duration of 100 milliseconds—which is about 100 concurrent executors.”
  56. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark
  57. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Infrastructure as code Deployments Project to product Microservices Team structure Don’t reinvent the wheel
  58. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark “Learn and be curious!”
  59. © 2019, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Thank you! Danilo Poccia Principal Evangelist, Serverless, AWS @danilop danilop