This is sparta: AWS Lambda and Go

Transcript

1. AWS Lambda and Go: A perfect match Andreas Mohrhard
 andreas.mohrhard@cosee.biz @amohrhard
2. What is serverless?
3. $ cat serverless_manifesto.md 1. Function are the unit of deployment and scaling. 2. No machines, VMs, or containers visible in the programming model. 3. Permanent storage lives elsewhere. 4. Scales per request; Users cannot over- or under-provision capacity. 5. Never pay for idle (no cold servers/containers or their costs). 6. Implicitly fault-tolerant because functions can run anywhere. 7. BYOC - Bring Your Own Code. 8. Metrics and logging are a universal right. http://blog.rowanudell.com/the-serverless-compute-manifesto/
4. AWS Lambda Function as a Service (FaaS)
 cost estimates: serverlesscalc.com
5. Simple Queueing Service Simple Notification Service Step Functions DynamoDB Simple Storage Service (S3) API Gateway Lambda CloudWatch Events
6. $ cat where_to_use.md Driving APIs Deployments Sending mails Transforming logs Running stress tests on APIs Running stress tests on API Automating long-running
 workflows Periodic tasks Resizing images Custom resources
 for CloudFormation Running Tests in CodePipeline Data pipelines and much more… Reacting to 
 infrastructure changes Monitoring databases
 (running SQL) Analysing CloudTrail
 logs
7. $ cat requirements.md • Fast to develop (fast builds, good IDEs) • Easy to deploy • Lightweight solution • Good performance • Easy to learn • Availability of good libraries and SDKs for the ☁
8. Framework? Platform?
9. Static Types Deployment Startup
 Performance Runtime
 Performance Eco system NodeJS ❌ ✅✅ ✅✅ ✅ ✅ Python ❌ ✅✅ ✅✅ ✅ ✅✅ Java ✅✅ ❌ ❌ ✅✅ ❌ C# ✅✅ ✅ ❌ ✅✅ ❌ Go ✅✅ ✅ ✅ ✅✅ ✅✅ * Classification highly subjective.
10. $ cat why.go Static type system Fast tooling, fast compiles Easy deployment (static binaries) Popular for systems programming
 (also “DevOps tools”) Easy to learn
11. Framework? Platform? Go
13. […] AMIIDLookup: Type: "AWS!::Lambda!::Function" Properties: Handler: "index.handler" Role: Fn!::GetAtt: - "LambdaExecutionRole" - "Arn" Code: S3Bucket: "lambda-functions" S3Key: "amilookup.zip" Runtime: "nodejs4.3" Timeout: "25" TracingConfig: Mode: "Active" AWSTemplateFormatVersion: "2010-09-09" Resources: RootRole: Type: "AWS!::IAM!::Role" Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: - "ec2.amazonaws.com" Action: - "sts:AssumeRole" Path: "/" RolePolicies: Type: "AWS!::IAM!::Policy" Properties: PolicyName: "root" PolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Action: "*" Resource: "*" $ cat cloudformation.yaml
14. AWSTemplateFormatVersion : '2010-09-09' Transform: AWS!::Serverless-2016-10-31 Description: A hello world application. Parameters: Bucket: Type: String CodeZipKey: Type: String Resources: HelloWorldFunction: Type: AWS!::Serverless!::Function Properties: Handler: index.handler Runtime: nodejs6.10 CodeUri: Bucket: !Ref Bucket Key: !Ref CodeZipKey $ cat SAM.yaml
15. Frameworks
16. Lambda and Go?
17. $ cat goWithLambda/main.go https://github.com/aws/aws-lambda-go/
18. $ cat goWithLambda/events_s3.go https://github.com/aws/aws-lambda-go/
19. $ cat goWithLambda/events_apigw.go https://github.com/aws/aws-lambda-go/
20. $ cat goWithLambda/events_sns.go https://github.com/aws/aws-lambda-go/
21. $ open architecture.svg Go Binary AWS Lambda HTTP Server Invocation Response
22. Framework? gosparta.io Platform? Go
23. $ go get github.com/mweagle/sparta
24. Demo
26. HelloWorld resource command Usage: main [command] Available Commands: delete Delete service describe Describe service execute Execute help Help about any command profile Interactively examine service pprof output provision Provision service version Sparta framework version […] Use "main [command] !--help" for more information about a command. $ go run main.go
27. Archive: HelloWorld.zip Length Date Time Name --------- ---------- ----- ---- 20286097 02-12-2018 21:50 Sparta.lambda.amd64 --------- ------- 20286097 1 file $ unzip -l HelloWorld.zip
28. $ cat sparta_vision.md Everthing as Code!
29. $ cat example_iam_role.go lambdaFn.RoleDefinition.Privileges = append(lambdaFn.RoleDefinition.Privileges, sparta.IAMRolePrivilege{ Actions: []string{"s3:GetObject", "s3:HeadObject"}, Resource: "arn:aws:s3!!:::MyS3Bucket", })
30. $ cat example_vault.go package main import ( "fmt" spartaVault "github.com/mweagle/SpartaVault/encrypt" ) var testKey = &spartaVault.KMSEncryptedValue{ KMSKeyARNOrGuid: "4f2f62e1-41e0-49e2-8da4-3a7ec511f498", PropertyName: "testKey", Key: “……wfAYJKoZIhvcNAQcGoG8wbQIBADse8m/G4G6iI=“, Nonce: "VDS+3LffkcSUGEpc", Value: "U4RQWOVsYyGiaJ2VhGXeWhO5Gd3+6uhaiqcg", Created: "2016-09-20T05:57:42-07:00", } func main() { plaintextValue, _ !:= testKey.Decrypt() fmt.Printf("Decrypted: %s\n", plaintextValue) }
31. func appendS3Lambda(api *sparta.API, lambdaFunctions []*sparta.LambdaAWSInfo) []*sparta.LambdaAWSInfo { lambdaFn !:= sparta.HandleAWSLambda(sparta.LambdaName(echoS3Event), http.HandlerFunc(echoS3Event), sparta.IAMRoleDefinition{}) apiGatewayResource, _ !:= api.NewResource("/hello/world/test", lambdaFn) apiGatewayResource.NewMethod("GET", http.StatusOK) lambdaFn.Permissions = […] return append(lambdaFunctions, lambdaFn) } func main() { stage !:= sparta.NewStage("prod") apiGateway !:= sparta.NewAPIGateway("MySpartaAPI", stage) apiGateway.CORSEnabled = true stackName !:= spartaCF.UserScopedStackName("SpartaApplication") sparta.Main(stackName, "Simple Sparta application", spartaLambdaData(apiGateway), apiGateway, nil) } $ cat example_apigateway.go
32. $ cat example_s3permission.go s3Lambda !:= HandleAWSLambda(LambdaName(s3LambdaProcessor), http.HandlerFunc(s3LambdaProcessor), IAMRoleDefinition{}) s3Lambda.Permissions = append(s3Lambda.Permissions, S3Permission{ BasePermission: BasePermission{ SourceArn: s3Bucket, }, Events: []string{"s3:ObjectCreated:*", "s3:ObjectRemoved:*"}, })
33. $ cat example_stepfunc.go lambdaTaskState !:= step.NewTaskState("lambdaRollDie", lambdaFn) successState !:= step.NewSuccessState("success") delayState !:= step.NewWaitDelayState("tryAgainShortly", 3*time.Second) lambdaChoices !:= []step.ChoiceBranch{ &step.Not{ Comparison: &step.NumericGreaterThan{ Variable: "$.roll", Value: 3, }, Next: delayState, }, } choiceState !:= step.NewChoiceState("checkRoll", lambdaChoices!!...). WithDefault(successState) lambdaTaskState.Next(choiceState) delayState.Next(lambdaTaskState)
34. $ cat example_custominfra.go lambdaFn.Decorator = func(lambdaResourceName string, lambdaResource gocf.LambdaFunction, template *gocf.Template, logger *logrus.Logger) error { cfResource !:= template.AddResource(s3BucketResourceName, &gocf.S3Bucket{ AccessControl: gocf.String("PublicRead"), }) cfResource.DeletionPolicy = "Delete" return nil } go-cloudformation: type-safe CloudFormation!
35. $ go test • Integrates in go test workflow (httptest.NewServer) • Manual testing with http server on localhost (go run main.go explore)
36. $ cat ci_cd.md github.com/mweagle/SpartaCodePipeline var pipelineProvisionCommand = &cobra.Command{ Use: "provisionPipeline", Short: "Provision a CI/CD pipeline for this stack", RunE: func(cmd *cobra.Command, args []string) error { validate !:= validator.New() cliErrors !:= validate.Struct(&pipelineOptions) if cliErrors !!= nil { return cliErrors } return pipeline.Provision(&pipelineOptions) }, }
37. $ cat monitoring.md [http://gosparta.io/docs/faq/]
38. $ cat extensibility.md • Custom resources • Extra commands / flags • Environment specific configuration (via tags) • Workflow hooks
39. $ cat sparta_conclusion.md • Full featured solution • Everything as (Go) code: Application code, infrastructure and monitoring • Sparta helps with best practices: Testing, CI/CD, Principle of Least Privileges • Alternatives • github.com/eawsy/aws-lambda-go-shim, github.com/apex/apex
40. AWS Lambda and Go: A perfect match
41. AWS Cloud Infrastructure and Go: A perfect match
42. func main() { template !:= cloudformation.NewTemplate() ports !:= [3]int{22, 80, 443} var ingressRules []cloudformation.AWSEC2SecurityGroup_Ingress for _, port !:= range ports { ingressRules = append(ingressRules, cloudformation.AWSEC2SecurityGroup_Ingress{ CidrIp: "0.0.0.0/0", FromPort: port, ToPort: port, }) } template.Resources["SecurityGroup"] = &cloudformation.AWSEC2SecurityGroup{ GroupDescription: "Illustrating generating security group entries", GroupName: "testgroup", VpcId: "vpc-1234beef", SecurityGroupIngress: ingressRules, } y, _ !:= template.YAML() fmt.Printf("%s\n", string(y)) } $ go get github.com/awslabs/goformation
43. $ cat goformation_example.go AWSTemplateFormatVersion: 2010-09-09 Resources: SecurityGroup: Properties: GroupDescription: Illustrating generating security group entries GroupName: testgroup SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 ToPort: 80 - CidrIp: 0.0.0.0/0 FromPort: 443 ToPort: 443 VpcId: vpc-1234beef Type: AWS!::EC2!::SecurityGroup
44. $ cat conclusion.md • Easy choice for DevOps automation (response times do not matter that much) • Fast startup times, performance of a compiled language • Good tooling • Easy to use, easy to integrate, easy to develop, easy to maintain
45. $ cat WhatsNext.md • github.com/ewindisch/lambda-transport • Aurora Serverless - Serverless SQL database • Fargate - serverless containers
46. > blog.cosee.biz > talks.cosee.biz > jobs.cosee.biz Thanks to @mweagle for gosparta.io!