Serverless Integration Patterns

Enterprise Integration Patterns is an awesome book about how to make software systems talk to each other. AWS Lambda is an interesting new way to run code without managing conventional servers. Serverless framework is a way to use Lambda without tearing your hair out. This talk introduces AWS Lambda and Serverless framework, examines a few classic patterns from EIP, and explores using Lambda as an integration toolkit.

Transcript

1. Software is our craft.TM 8th Light, Inc. SCNY June 29, 2017 Serverless Integration Patterns
5. http://www.enterpriseintegrationpatterns.com/
6. Read the dang book, Bjorn!
7. http://www.enterpriseintegrationpatterns.com/
8. Enterprise Integration Patterns
9. Enterprise Integration Patterns
10. Patterns Books
11. Patterns Books
12. Patterns Books
13. Patterns Books
14. Patterns Books Read patterns books as histories
15. Patterns Books It’s worth it: patterns give us a shared language to talk about design and ideas to clarify and articulate our thoughts.
16. Messaging
17. Messaging “Messaging is a technology that enables high-speed, asynchronous, program-to- program communication with reliable delivery. Programs communicate by sending packets of data called messages to each other. Channels, also known as queues, are logical pathways that connect the programs and convey messages. A channel behaves like a collection or array of messages, but one that is magically shared across multiple computers and can be used concurrently by multiple applications. A sender or producer is a program that sends a message by writing the message to a channel. A receiver or consumer is a program that receives a message by reading (and deleting) it from a channel.” C Application A Application B
18. Integration Styles: File Transfer Application A Application B
19. Integration Styles: Shared Database Application A Application C Application B
20. Integration Styles: Remote Procedure Invocation Application A Application B
21. Integration Styles: Messaging C Application A Application C Application B
22. Integration Styles What about RESTful JSON APIs?
23. Patterns: Pipes and Filters How can we perform complex processing on a message while maintaining independence and flexibility? Use the Pipes and Filters architectural style to divide a larger processing task into a sequence of smaller, independent processing steps (filters) that are connected by channels (pipes). Decrypt Auth De-dupe
24. Patterns: Messaging Gateway C How do you encapsulate access to the messaging system from the rest of the application? Use a Messaging Gateway, a class that wraps messaging-specific method calls and exposes domain-specific methods to the application.
25. Patterns: Router Router Gadgets Widgets How can you decouple individual processing steps so that messages can be passed to different filters depending on a set of conditions? Insert a special filter, a Message Router, which consumes a message from one message channel and republishes it to a different output channel depending on a set of conditions.
26. Patterns: Publish-Subscribe Subscriber Publisher How can the sender broadcast an event to all interested receivers? Send the message to a Publish-Subscribe Channel, which delivers a copy of a particular event to each receiver. Subscriber Subscriber
27. Serverless Architecture
28. Backend-as-a-Service
29. Function-as-a-Service “Serverless can also mean applications where some amount of server-side logic is still written by the application developer but unlike traditional architectures is run in stateless compute containers that are event-triggered, ephemeral (may only last for one invocation), and fully managed by a 3rd party.” — Mike Roberts, “Serverless Architectures”
30. Function-as-a-Service
31. Serverless Architecture Serverless
32. Serverless Architecture “Serverless”
33. Serverless Architecture “““Serverless”””
34. Serverless Architecture https://charity.wtf/2016/05/31/operational-best-practices-serverless/
35. Serverless Architecture Physical Servers Virtualization Containers and PaaS Serverless http://www.enterpriseintegrationpatterns.com/ramblings/google_cloud_functions.html
36. Serverless Architecture Physical Servers Virtualization Containers and PaaS “““Serverless””” http://www.enterpriseintegrationpatterns.com/ramblings/google_cloud_functions.html
37. Serverless Architecture
38. Serverless Architecture
39. AWS Lambda
40. AWS Lambda • Launched in 2014, so it’s still pretty new • Supports Node, Python, Java, and C# • Runs your app code in an ephemeral container • Configure memory (and proportional compute) • Charged based on compute time https://docs.aws.amazon.com/lambda/latest/dg/welcome.html
41. AWS Lambda • Launched in 2014, so it’s still pretty new • Supports Node, Python, Java, and C# • Runs your app code in an ephemeral container • Configure memory (and proportional compute) • Charged based on compute time
42. Limitations • App code has to be stateless • Can’t run longer than 300 seconds • 500 MB of temporary disk space • Soft limit of 1000 concurrently running Lambdas • Limited access to the underlying infrastructure
43. Anatomy of a Lambda def hello(event, context): print('Hello, world!') return { "message": "Lambda function ran successfully", "event": event }
44. Anatomy of a Lambda def hello(event, context): print('Hello, world!') return { "message": "Lambda function ran successfully", "event": event } JSON data. Different event triggers have different formats.
45. Anatomy of a Lambda def hello(event, context): print('Hello, world!') return { "message": "Lambda function ran successfully", "event": event } Context object. Stores information about the Lambda/AWS config.
46. Anatomy of a Lambda def hello(event, context): print('Hello, world!') return { "message": "Lambda function ran successfully", "event": event } Return a dict. Different format if your Lambda is connected to an API Gateway route.
47. AWS Lambda Demo
48. Lambda as an Integration Toolkit
49. Lambda as an Integration Toolkit
50. Lambda as an Integration Toolkit
51. Lambda as an Integration Toolkit C
52. Lambda as an Integration Toolkit C
53. Lambda as an Integration Toolkit C • S3 operations
54. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests
55. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests • DynamoDB Triggers •DynamoDB Triggers
56. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests • DynamoDB Triggers • SNS Notifications •DynamoDB Triggers
57. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests • DynamoDB Triggers • SNS Notifications • SQS Messages •DynamoDB Triggers
58. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests • DynamoDB Triggers • SNS Notifications • SQS Messages • Kinesis Stream Events •DynamoDB Triggers
59. Lambda as an Integration Toolkit C • S3 operations • HTTP Requests • HTTP Requests • DynamoDB Triggers • SNS Notifications • SQS Messages • Kinesis Stream Events •DynamoDB Triggers Plus direct invocation, scheduled events, CloudWatch, Cognito, email, Alexa…
60. SNS as a Message Channel C import boto3 import json import os def send_message(message, topic): client = boto3.client('sns') prefix = os.environ.get('SNS_ARN_PREFIX') topic_arn = '{}:{}'.format(prefix, topic) client.publish( TopicArn=topic_arn, Message=json.dumps({"message": message}) ) import sns def say_hello(event, context): sns.send_message( ‘Hello, world!’, ‘’ ) return { "message": “Lambda function ran successfully", "event": event } sns.py handler.py https://github.com/ecmendenhall/serverless-integration-patterns/blob/master/lib/sns.py
61. SNS as a Message Channel C Do you still need queues?
62. SNS as a Message Channel C Do you still need queues?
63. API Gateway as Messaging Gateway C λ λ
64. Lambdas as Pipes, Filters, Routers, Adapters… λ λ
65. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ
66. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ
67. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ λ
68. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ λ λ
69. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ λ λ
70. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ λ λ
71. Lambdas as Pipes, Filters, Routers, Adapters… λ λ λ λ λ λ
72. Serverless Framework
73. Serverless Framework • Handles packaging, deployment, configuration, multiple environments • Uses Cloudformation YAML for extra infrastructure • Supports multiple languages and FaaS providers https://serverless.com/framework/docs/
74. Serverless Framework Serverless Framework
75. Serverless Framework “Serverless” “Framework” • Very lightweight: the framework-y parts are about provisioning resources and deploying code more than structuring your application • Moving fast • Still need to know AWS, especially Cloudformation
76. def hello(event, context): print('Hello, world!') return { "message": “Lambda function ran successfully", "event": event } Serverless Framework serverless.yml service: hello-world provider: name: aws runtime: python2.7 stage: dev region: us-east-1 environment: variable1: value1 functions: hello: handler: handler.hello events: - schedule: rate(10 minutes) resources: Resources: MyCoolBucket: Type: AWS::S3::Bucket Properties: BucketName: my-new-bucket handler.py
77. Serverless Framework Demo
78. Examples
79. Examples https://github.com/ecmendenhall/serverless-integration-patterns ⚡
80. Connor Mendenhall connor@8thlight.com Thank You