Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Think Big, Start Small, Scale Fast: Develop a Marketplace app using AWS Lambda

Think Big, Start Small, Scale Fast: Develop a Marketplace app using AWS Lambda

Are you planning to develop a new Cloud app? Serverless might be right for you! In this talk, we'll show you what it takes to run an app using a serverless architecture based on AWS Lambda, from design and deployment, to testing, debugging, and most importantly – how to scale. We'll go beyond the basics to demonstrate real use cases, best practices, and explain why serverless might be a good fit for you.

Talk is available online: https://www.youtube.com/watch?v=XfpskCjfBBw

023594dc361cab87c8e40d03ba0f3b18?s=128

sebastianhesse

September 07, 2018
Tweet

Transcript

  1. Think Big, Start Small, Scale Fast Develop a Marketplace app

    using AWS Lambda SEBASTIAN HESSE | SOFTWARE ENGINEER | K15T SOFTWARE | @SEEEBIII
  2. Start Setup Develop Scale Goal

  3. Start Setup Develop Scale Goal

  4. Start Setup Develop Scale Goal

  5. Backbone Issue Sync for Jira Sync issue data across company

    borders
  6. After Implemented native cloud to cloud sync. Before Sync initiated

    from on-prem to another on-prem or cloud.
  7. TYPICAL CLOUD ARCHITECTURE SERVERLESS Lambda Function Lambda Function Lambda Function

  8. Our Cloud Setup + Amazon Web Services AWS Lambda

  9. module.exports = function(event, context, cb) { cb(null, "Hello World."); }

  10. Long Running Tasks Perform bigger tasks, e.g. to initialize data.

    Our Use Cases For Serverless Scheduled Tasks Schedule functions for cleaning up data or other tasks. Webhooks Receive webhooks from Jira, Confluence, etc. and process them.
  11. Start Setup Develop Scale Goal

  12. Start Setup Develop Scale Goal DEVELOPMENT

  13. Local Machine Execute and debug + test your code on

    your local machine. Cloud Upload your code to the cloud and test it there. Testing & Debugging
  14. No pain, no gain Log as much as necessary and

    test it in the cloud.
  15. Quick code updates Wrote our own library lambda-updater to quickly

    update the Lambda code. Write tests Writing unit and integration tests can already identify lots of problems. Real environment No service mocks, latencies don't explode and a better feeling about the execution. Why?!
  16. Quick code updates Wrote our own library lambda-updater to quickly

    update the Lambda code. Write tests Writing unit and integration tests can already identify lots of problems. Real environment No service mocks, latencies don't explode and a better feeling about the execution. Why?!
  17. Quick code updates Wrote our own library lambda-updater to quickly

    update the Lambda code. Write tests Writing unit and integration tests can already identify lots of problems. Real environment No service mocks, latencies don't explode and a better feeling about the execution. Why?!
  18. Start Setup Develop Scale Goal DEPLOYMENT

  19. CloudFormation Infrastructure as Code

  20. "webhooks": [ { "event": "jira:issue_updated", "url": "/webhooks?appVersion={{version}}" } ] Different

    Versions atlassian-connect.json:
  21. + </> DevOps

  22. Start Setup Develop Scale Goal

  23. Start Setup Develop Scale Goal FUNCTION SCOPE

  24. Task #2 Check content. Function Scope Task #3 Do something

    with it. Task #1 Receive webhook.
  25. Task #2 Check content. Function Scope Task #3 Do something

    with it. Task #1 Receive webhook.
  26. Keep it simple, stupid. KELLY JOHNSON

  27. Reusability Reuse your functions from multiple contexts. Performance Single purpose

    functions execute faster. Multiple Scopes
  28. Lambda Response Times 0 500 1000 1500 2000 1 2

    3 4 5
  29. Lambda Response Times 0 500 1000 1500 2000 1 2

    3 4 5
  30. Programming Language Java for background functions, NodeJS for functions which

    are performance relevant. Function Scope One scope per function. For example, one webhook receiver function, one email sending function, etc. Consider
  31. Can we use AWS Lambda as a REST API? SOME

    DEVELOPERS
  32. Lambda Response Times 0 500 1000 1500 2000 1 2

    3 4 5
  33. Start Setup Develop Scale Goal COMMUNICATION

  34. Function Payload Communicate Function 1 Function 2

  35. Function Payload Communicate Function 1 Function 2

  36. Solution #1 Function 1 Function 2 Function 2

  37. Solution #2 KEY Function 1 Function 2 Function Payload S3

  38. Start Setup Develop Scale Goal

  39. Start Setup Develop Scale Goal WEBHOOKS

  40. 1000 webhooks at the same time

  41. Scaling for AWS Lambda is done automatically... right? MANY DEVELOPERS

  42. Other services are not necessarily scaled. Lambda functions are scaled

    by AWS. YES NO
  43. DynamoDB S3

  44. DynamoDB S3

  45. Buffer Webhooks Function 1 AWS Kinesis Webhook

  46. Buffer Webhooks Function 1 Function 2 AWS Kinesis Webhook

  47. Start Setup Develop Scale Goal LONG RUNNING TASKS

  48. 5 minute execution time limit

  49. 5 minutes are enough for the task my function is

    performing. A FEW DEVELOPERS
  50. Collect tenant information from database. < 1 sec 350ms /

    100 issues Retrieve all issues from a project in Jira. Find issues matching your configuration. < 1 sec Example: Retrieve 800 Issues For Sync Perform further steps and update issue data. 350ms / 1 issue 10 sec Store relevant information in database.
  51. Solution #1 Call yourself before function reaches execution limit.

  52. Solution #2 Use other services like an EC2 server to

    execute your tasks.
  53. Solution #3 Use AWS Step Functions to define an execution

    order of your functions. AWS Step Functions
  54. Example Step Function "Retrieve issues": { "Type": "Task", "Resource": "arn:aws:lambda:...",

    "Next": "All issues?", "Retry": [ { ... } ] } Retrieve issues All issues? Process issues Failed START END
  55. Start Setup Develop Scale Goal SCHEDULED TASKS

  56. Examples Webhooks Webhooks are not 100% reliable. Catch up missed

    webhooks every day. Weekly Updates Send a status update every week and let your customers know about some numbers. Clean Up Clean up data in your environment, e.g. in your database.
  57. Trigger Function AWS CloudWatch Lambda Function

  58. Monitoring Enable monitoring and notification about failures. Learn from it

    and adapt your architecture. Failures Functions can fail quite easily and you might never know, because you don't check the logs or any metrics.
  59. Start Setup Develop Scale Goal

  60. Trade Offs Potentially less operational costs, but more complex architecture.

    Please Remember Scalability Consider scalability for all of your services. Function Scope Focus on one scope per function. Think about the use case of a function.
  61. Start Setup Develop Scale Goal

  62. Start Setup Develop Scale Goal

  63. Start now! SEBASTIAN HESSE | SOFTWARE ENGINEER | K15T SOFTWARE

    | @SEEEBIII