Upgrade to Pro — share decks privately, control downloads, hide ads and more …

All-in-one Serverless mit AWS und Go

All-in-one Serverless mit AWS und Go

Die Trendsprache im Bereich DevOps Tooling ist Go. Ob für Service Discovery, Container-Orchestrierung, Monitoring-Lösungen oder andere Tools, Go findet man in vielen Bereichen wieder. Der andere große Trend sind Serverless Architekturen, also Code einfach in die Cloud deployen, ohne sich um Server zu kümmern. Hier hat Amazon Web Services mit Lambda den Startschuss gegeben. Aktuell lässt sich Go leider nur über Umwege mit AWS Lambda zusammenbringen. Das Go-Framework Sparta (gosparta.io) bringt die beiden Trends zusammen und bietet eine integrierte Lösung für Entwicklung und Deployment von Serverless Anwendungen. Alles aus einem Guss. Alles in Go.

Avatar for Andreas Mohrhard

Andreas Mohrhard

December 11, 2017
Tweet

More Decks by Andreas Mohrhard

Other Decks in Programming

Transcript

  1. Simple Queueing Service Simple Notification Service Step Functions DynamoDB Simple

    Storage Service (S3) API Gateway Lambda CloudWatch Events
  2. $ cat where_to_use.md Driving APIs Deployments Sending mails Transforming logs

    Running stress tests on APIs Running stress tests on API Automating long-running
 workflows Periodic tasks Resizing images Custom resources
 for CloudFormation Running Tests in CodePipeline Data pipelines and much more… Reacting to 
 infrastructure changes Monitoring databases
 (running SQL) Analysing CloudTrail
 logs
  3. $ cat requirements.md • Fast to develop (fast builds, good

    IDEs) • Easy to deploy • Lightweight solution • Good performance • Easy to learn • Availability of good libraries and SDKs for the ☁
  4. $ cat why.go Static type system Fast tooling, fast compiles

    Easy deployment (static binaries) Popular for systems programming
 (also “DevOps tools”) Easy to learn
  5. Configuration Serverless Build IAM Policies / Security Unit Tests Integration

    Tests Versioning Rollback Stress Testing Events / Triggers Deployment Complex Infrastructure
  6. […] AMIIDLookup: Type: "AWS!::Lambda!::Function" Properties: Handler: "index.handler" Role: Fn!::GetAtt: -

    "LambdaExecutionRole" - "Arn" Code: S3Bucket: "lambda-functions" S3Key: "amilookup.zip" Runtime: "nodejs4.3" Timeout: "25" TracingConfig: Mode: "Active" AWSTemplateFormatVersion: "2010-09-09" Resources: RootRole: Type: "AWS!::IAM!::Role" Properties: AssumeRolePolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Principal: Service: - "ec2.amazonaws.com" Action: - "sts:AssumeRole" Path: "/" RolePolicies: Type: "AWS!::IAM!::Policy" Properties: PolicyName: "root" PolicyDocument: Version: "2012-10-17" Statement: - Effect: "Allow" Action: "*" Resource: "*" $ cat cloudformation.yaml
  7. * At least in theory. const exec = require('child_process').exec; exports.handler

    = function(event, context) { const child = exec('./goLambdaGo ' + ''' + JSON.stringify(event) + ''', (error) !=> { context.done(error, 'Process complete!'); }); child.stdout.on('data', console.log); child.stderr.on('data', console.error); } $ cat goWithLambda/index.js
  8. Archive: HelloWorld.zip Length Date Time Name --------- ---------- ----- ----

    20849801 11-12-2017 09:58 bin/Sparta.lambda.amd64 11822 00-00-1980 00:00 index.js 0 00-00-1980 00:00 node_modules/google-protobuf/ [more protobuf stuff] 1553 00-00-1980 00:00 sparta_utils.js 42309 00-00-1980 00:00 proto/proxy_pb.js 1421 00-00-1980 00:00 golang-constants.json --------- ------- 21562781 24 files $ unzip -l HelloWorld.zip
  9. $ open architecture.svg Go Binary NodeJS HTTP Server Invocation Response

    protobuf Python python module built with cgo Invocation Response python + cgo about 50% faster!
  10. $ cat example_vault.go package main import ( "fmt" spartaVault "github.com/mweagle/SpartaVault/encrypt"

    ) var testKey = &spartaVault.KMSEncryptedValue{ KMSKeyARNOrGuid: "4f2f62e1-41e0-49e2-8da4-3a7ec511f498", PropertyName: "testKey", Key: “……wfAYJKoZIhvcNAQcGoG8wbQIBADse8m/G4G6iI=“, Nonce: "VDS+3LffkcSUGEpc", Value: "U4RQWOVsYyGiaJ2VhGXeWhO5Gd3+6uhaiqcg", Created: "2016-09-20T05:57:42-07:00", } func main() { plaintextValue, _ !:= testKey.Decrypt() fmt.Printf("Decrypted: %s\n", plaintextValue) }
  11. func appendS3Lambda(api *sparta.API, lambdaFunctions []*sparta.LambdaAWSInfo) []*sparta.LambdaAWSInfo { lambdaFn !:= sparta.HandleAWSLambda(sparta.LambdaName(echoS3Event),

    http.HandlerFunc(echoS3Event), sparta.IAMRoleDefinition{}) apiGatewayResource, _ !:= api.NewResource("/hello/world/test", lambdaFn) apiGatewayResource.NewMethod("GET", http.StatusOK) lambdaFn.Permissions = […] return append(lambdaFunctions, lambdaFn) } func main() { stage !:= sparta.NewStage("prod") apiGateway !:= sparta.NewAPIGateway("MySpartaAPI", stage) apiGateway.CORSEnabled = true stackName !:= spartaCF.UserScopedStackName("SpartaApplication") sparta.Main(stackName, "Simple Sparta application", spartaLambdaData(apiGateway), apiGateway, nil) } $ cat example_apigateway.go
  12. $ cat example_s3permission.go s3Lambda !:= HandleAWSLambda(LambdaName(s3LambdaProcessor), http.HandlerFunc(s3LambdaProcessor), IAMRoleDefinition{}) s3Lambda.Permissions =

    append(s3Lambda.Permissions, S3Permission{ BasePermission: BasePermission{ SourceArn: s3Bucket, }, Events: []string{"s3:ObjectCreated:*", "s3:ObjectRemoved:*"}, })
  13. $ cat example_stepfunc.go lambdaTaskState !:= step.NewTaskState("lambdaRollDie", lambdaFn) successState !:= step.NewSuccessState("success")

    delayState !:= step.NewWaitDelayState("tryAgainShortly", 3*time.Second) lambdaChoices !:= []step.ChoiceBranch{ &step.Not{ Comparison: &step.NumericGreaterThan{ Variable: "$.roll", Value: 3, }, Next: delayState, }, } choiceState !:= step.NewChoiceState("checkRoll", lambdaChoices!!...). WithDefault(successState) lambdaTaskState.Next(choiceState) delayState.Next(lambdaTaskState)
  14. $ cat example_custominfra.go lambdaFn.Decorator = func(lambdaResourceName string, lambdaResource gocf.LambdaFunction, template

    *gocf.Template, logger *logrus.Logger) error { cfResource !:= template.AddResource(s3BucketResourceName, &gocf.S3Bucket{ AccessControl: gocf.String("PublicRead"), }) cfResource.DeletionPolicy = "Delete" return nil } go-cloudformation: type-safe CloudFormation!
  15. $ go test • Integrates in go test workflow (httptest.NewServer)

    • Manual testing with http server on localhost (go run main.go explore)
  16. $ cat ci_cd.md github.com/mweagle/SpartaCodePipeline var pipelineProvisionCommand = &cobra.Command{ Use: "provisionPipeline",

    Short: "Provision a CI/CD pipeline for this stack", RunE: func(cmd *cobra.Command, args []string) error { validate !:= validator.New() cliErrors !:= validate.Struct(&pipelineOptions) if cliErrors !!= nil { return cliErrors } return pipeline.Provision(&pipelineOptions) }, }
  17. $ cat sparta_conclusion.md • Full featured solution • Everything as

    (Go) code: Application code, infrastructure and monitoring • Sparta helps with best practices: Testing, CI/CD, Principle of Least Privileges • Alternatives • github.com/eawsy/aws-lambda-go-shim, github.com/apex/apex
  18. func main() { template !:= cloudformation.NewTemplate() ports !:= [3]int{22, 80,

    443} var ingressRules []cloudformation.AWSEC2SecurityGroup_Ingress for _, port !:= range ports { ingressRules = append(ingressRules, cloudformation.AWSEC2SecurityGroup_Ingress{ CidrIp: "0.0.0.0/0", FromPort: port, ToPort: port, }) } template.Resources["SecurityGroup"] = &cloudformation.AWSEC2SecurityGroup{ GroupDescription: "Illustrating generating security group entries", GroupName: "testgroup", VpcId: "vpc-1234beef", SecurityGroupIngress: ingressRules, } y, _ !:= template.YAML() fmt.Printf("%s\n", string(y)) } $ go get github.com/awslabs/goformation
  19. $ cat goformation_example.go AWSTemplateFormatVersion: 2010-09-09 Resources: SecurityGroup: Properties: GroupDescription: Illustrating

    generating security group entries GroupName: testgroup SecurityGroupIngress: - CidrIp: 0.0.0.0/0 FromPort: 22 ToPort: 22 - CidrIp: 0.0.0.0/0 FromPort: 80 ToPort: 80 - CidrIp: 0.0.0.0/0 FromPort: 443 ToPort: 443 VpcId: vpc-1234beef Type: AWS!::EC2!::SecurityGroup
  20. $ cat conclusion.md • Easy choice for DevOps automation (response

    times do not matter that much) • Fast startup times, performance of a compiled language • Good tooling • Easy to use, easy to integrate, easy to develop, easy to maintain
  21. $ cat NEWS.md • Aurora Serverless - serverless sql database

    • Fargate - serverless containers • …. and Go!