Slide 1

Slide 1 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Modern Application Development in the Cloud Danilo Poccia Principal Evangelist, Serverless AWS @danilop Iñigo Etxabe Founder & CTO Datik @ietxabe

Slide 2

Slide 2 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Best practices for modern application development • Enable experimentation • Componentize applications • Update applications and infrastructure quickly • Model and provision application resources • Simplify infrastructure management • Improve application performance • Secure the entire application lifecycle

Slide 3

Slide 3 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Best practices for modern application development • Enable experimentation by creating a culture of ownership • Componentize applications using microservices • Update applications and infrastructure quickly by automating the release pipeline • Model and provision application resources using infrastructure as code • Simplify infrastructure management with serverless technologies • Improve application performance by increasing observability • Secure the entire application lifecycle by automating security

Slide 4

Slide 4 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Microservices

Slide 5

Slide 5 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T How Amazon SQS works Front End Back End Metadata Amazon DynamoDB Load Manager

Slide 6

Slide 6 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Serverless

Slide 7

Slide 7 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS operational responsibility models On-Premises Cloud Less More Compute Virtual Machine EC2 Elastic Beanstalk AWS Lambda Fargate Databases MySQL MySQL on EC2 RDS MySQL RDS Aurora Aurora Serverless DynamoDB Storage Storage S3 Messaging ESBs Amazon MQ Kinesis SQS / SNS Analytics Hadoop Hadoop on EC2 EMR Elasticsearch Service Athena

Slide 8

Slide 8 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T How Amazon MQ works Amazon API Gateway DynamoDB Control Plane Data Plane AWS Lambda

Slide 9

Slide 9 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Some AWS services that use containers …more…

Slide 10

Slide 10 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Databases

Slide 11

Slide 11 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Databases in modern applications RDBMS (RDS) NoSQL (DynamoDB)

Slide 12

Slide 12 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Relational or not? NOT!

Slide 13

Slide 13 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Databases in Modern Applications RDBMS (RDS) NoSQL (DynamoDB) Amazon Quantum Ledger Database

Slide 14

Slide 14 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T • Transactions are SQL-ish • Query the Summary with SQL • “Records” are ION (JSON superset) documents • Journal is a cryptographically chained immutable ledger • Journal is also a database table • It’s serverless! Amazon Quantum Ledger Database Summary Journal Transactions

Slide 15

Slide 15 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Integration patterns

Slide 16

Slide 16 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Integration options from AWS Amazon API Gateway Queues Simple Fully- managed Any volume Amazon SQS Pub/sub Simple Fully-managed Flexible Amazon SNS Orchestration Powerful Fully-managed Low code AWS Step Functions Connect Efficient Fully-managed Real-time Client-to-Service Messaging Orchestration

Slide 17

Slide 17 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Keep orchestration out of code Track status of data and execution Remove redundant code

Slide 18

Slide 18 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T

Slide 19

Slide 19 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Serverless applications Services (anything) Changes in data state Requests to endpoints Changes in resource state Event source Function Node.js Python Java / JVM C# / PowerShell Go Ruby Runtime API

Slide 20

Slide 20 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Anatomy of a Lambda function Handler() function Function to be executed upon invocation Event object Data sent during Lambda function Invocation Context object Methods available to interact with runtime information (request ID, log group, more) import json def lambda_handler(event, context): # TODO implement return { 'statusCode': 200, 'body': json.dumps('Hello World!') }

Slide 21

Slide 21 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Lambda Layers Lets functions easily share code: Upload layer once, reference within any function Promote separation of responsibilities, lets developers iterate faster on writing business logic Built in support for secure sharing by ecosystem

Slide 22

Slide 22 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Using Lambda Layers • Put common components in a ZIP file and upload it as a Lambda Layer • Layers are immutable and can be versioned to manage updates • When a version is deleted or permissions to use it are revoked, functions that used it previously will continue to work, but you won’t be able to create new ones • You can reference up to five layers, one of which can optionally be a custom runtime Lambda Layers arn:aws:lambda:region:accountId:layer:shared-lib Lambda Layers arn:aws:lambda:region:accountId:layer:shared-lib:2 Lambda Layers arn:aws:lambda:region:accountId:layer:shared-lib:3

Slide 23

Slide 23 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Lambda Runtime API Bring any Linux compatible language runtime Powered by new Runtime API - Codifies the runtime calling conventions and integration points At launch, custom runtimes powering Ruby support in AWS Lambda, more runtimes from partners (like Erlang) Custom runtimes distributed as “layers” Rule Stack

Slide 24

Slide 24 text

Build PCI and HIPAA compliant serverless applications! Serverless platform services that can be used in both:

Slide 25

Slide 25 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Start with a framework AWS Chalice AWS Amplify AWS SAM AWS: Third-party: Serverless Framework

Slide 26

Slide 26 text

Meet SAM!

Slide 27

Slide 27 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Serverless Application Model (SAM) AWSTemplateFormatVersion: '2010-09-09’ Transform: AWS::Serverless-2016-10-31 Resources: GetFunction: Type: AWS::Serverless::Function Properties: Handler: index.get Runtime: nodejs8.10 CodeUri: src/ Policies: - DynamoDBReadPolicy: TableName: !Ref MyTable Events: GetResource: Type: Api Properties: Path: /resource/{resourceId} Method: get MyTable: Type: AWS::Serverless::SimpleTable Just 20 lines to create: • Lambda function • IAM role • API Gateway • DynamoDB table O pen Source

Slide 28

Slide 28 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T SAM CLI sam init --name my-function --runtime python cd my-function/ sam build sam package --s3-bucket my-packages-bucket \ --output-template-file packaged.yaml sam deploy --template-file packaged.yaml \ --stack-name my-function-prod sam publish # To the AWS Serverless Application Repository

Slide 29

Slide 29 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T

Slide 30

Slide 30 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T TweetSource: Type: AWS::Serverless::Application Properties: Location: ApplicationId: arn:aws:serverlessrepo:... SemanticVersion: 2.0.0 Parameters: TweetProcessorFunctionName: !Ref MyFunction SearchText: '#serverless - filter:nativeretweets' Nested apps to simplify solving recurring problems Standard Component Custom Business Logic Polling schedule (CloudWatch Events rule) trigger TwitterProcessor SearchCheckpoint TwitterSearchPoller Twitter Search API

Slide 31

Slide 31 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS SAM Template Capabilities • Can mix in other non-SAM CloudFormation resources in the same template • i.e. Amazon S3, Amazon Kinesis, AWS Step Functions • Supports use of Parameters, Mappings, Outputs, etc • Supports Intrinsic Functions • Can use ImportValue (exceptions for RestApiId, Policies, StageName attributes) • YAML or JSON

Slide 32

Slide 32 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Using AWS CloudFormation Export/ImportValue Outputs: WebServerSecurityGroup: Description: Security group for public web servers Value: Fn::GetAtt: - WebServerSecurityGroup - GroupId Export: Name: Fn::Sub: "${AWS::StackName}-SecurityGroupID” PublicSubnet: Description: Subnet for public web servers Value: Ref: PublicSubnet Export: Name: Fn::Sub: "${AWS::StackName}-SubnetID" Resources: WebServerInstance: Type: AWS::EC2::Instance Properties: InstanceType: t2.micro ImageId: ami-a1b23456 NetworkInterfaces: - GroupSet: - Fn::ImportValue: Fn::Sub: "${NetworkStackName}-SecurityGroupID" AssociatePublicIpAddress: 'true' DeviceIndex: '0' DeleteOnTermination: 'true' SubnetId: Fn::ImportValue: Fn::Sub: "${NetworkStackName}-SubnetID" Stack A – Network Stack B – Web Servers This is a Parameter

Slide 33

Slide 33 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T CodeDeploy – Lambda deployments in SAM templates Resources: GetFunction: Type: AWS::Serverless::Function Properties: AutoPublishAlias: live DeploymentPreference: Type: Canary10Percent10Minutes Alarms: - !Ref ErrorsAlarm - !Ref LatencyAlarm Hooks: PreTraffic: !Ref PreTrafficHookFunction PostTraffic: !Ref PostTrafficHookFunction Canary10Percent30Minutes Canary10Percent5Minutes Canary10Percent10Minutes Canary10Percent15Minutes Linear10PercentEvery10Minutes Linear10PercentEvery1Minute Linear10PercentEvery2Minutes Linear10PercentEvery3Minutes AllAtOnce

Slide 34

Slide 34 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Globals: Function: Runtime: nodejs6.10 CodeUri: s3://code-artifacts/pet_app1234.zip MemorySize: 1024 Timeout: 30 AutoPublishAlias: !Ref ENVIRONMENT getDogsFunction: Type: AWS::Serverless::Function Properties: Handler: getdogs.handler Events: GetDogs: Type: Api Properties: Path: /Dogs Method: ANY getCatsFunction: Type: AWS::Serverless::Function Properties: Handler: getCats.handler Events: GetCats: Type: Api Properties: Path: /Cats Method: ANY getBirdsFunction: Type: AWS::Serverless::Function Properties: Handler: getBirds.handler Timeout: 15 Events: GetBirds: Type: Api Properties: Path: /Birds Method: ANY AWS SAM Globals

Slide 35

Slide 35 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Lambda permissions model Fine grained security controls for both execution and invocation: Execution policies: • Define what AWS resources/API calls can this function access via IAM • Used in streaming invocations • E.g. “Lambda function A can read from DynamoDB table users” Function policies: • Used for sync and async invocations • E.g. “Actions on bucket X can invoke Lambda function Z” • Resource policies allow for cross account configst access

Slide 36

Slide 36 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS SAM Policy Templates MyQueueFunction: Type: AWS::Serverless::Function Properties: ... Policies: # Gives permissions to poll an SQS Queue - SQSPollerPolicy: queueName: !Ref MyQueue ... MyQueue: Type: AWS::SQS::Queue ...

Slide 37

Slide 37 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T SAM Policy Templates 45+ predefined policies All found here: https://bit.ly/2xWycnj

Slide 38

Slide 38 text

S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Slide 39

Slide 39 text

IÑIGO ETXABE FOUNDER & CTO @ietxabe SMART INFORMATION FOR A MOVING WORLD!

Slide 40

Slide 40 text

5.000 +100 10.311.573L 180.000 PLATAFORMA ESCALABLE MÁS DE 5.000 VEHÍCULOS INSTALADOS START-UP DE EXITO MÁS DE 100 EMPRESAS DE TRANSPORTE HAN CONFIADO EN DATIK APORTAMOS VALOR DESDE EL AHORRO MÁS DE 10 MILLONES DE AHORRO PROPUESTOS A NUESTROS CLIENTES EXPERIENCIA INTERNACIONAL INFORMACIÓN DESDE MÁS DE 15 PAÍSES ESPECIALISTAS EN SEGURIDAD MÁS DE 180.00 HORAS DE VÍDEO ALMACENADOS Smart Information for a moving world! 15 DATIK EN CIFRAS...

Slide 41

Slide 41 text

Smart Information for a moving world! NUESTRA VISIÓN Los sistemas de transporte son cada vez más complejos y costosos Telematics Passenger WiFi CCTV system Driver Assistance System

Slide 42

Slide 42 text

Smart Information for a moving world! EQUIPO INTEROPERABLE Datik Computing Brain iPanelVideo iPanelFleet iPanelDriver iPanelWiFi FMS Cameras GPS/3G DSM Display WiFi iPanelSAE iPanelSales

Slide 43

Slide 43 text

MONITORIZACIÓN DEL CONDUCTOR INFORMACIÓN A PASAJEROS TELEMATICS VIDEO VIGILANCIA CONTROL OPERACIONAL MANTENIMIENTO DESCARGA REMOTA DEL TACÓGRAFO TICKETING WIFI A BORDO CLOUD PLATFORM COSTES OPERATIVOS SEGURIDAD OPERACIÓN SERVICIOS AL PASAJERO Smart Information for a moving world!

Slide 44

Slide 44 text

CUANTO MÁS VEMOS, MÁS CONECTADOS NOS SENTIMOS Nuestra aplicación para la videovigilancia es un paso más en la conectividad total del vehículo. VIDEO SURVEILLANCE Smart Information for a moving world!

Slide 45

Slide 45 text

0,598-1,579 0,598-1,579 0,598-1,579 0,598-1,579 AYUDA A LOS CONDUCTORES A EVITAR ACCIDENTES. MONITORIZACIÓN DEL CONDUCTOR Smart Information for a moving world!

Slide 46

Slide 46 text

Smart Information for a moving world! 65 INFORMACIÓN A PASAJEROS Monitores multimedia de exterior Aplicación móvil Monitores multimedia de interior y anunciador de audio El 360º de la información a pasajeros. • La aplicación móvil, los paneles multimedia para paradas, los paneles LED para el exterior de los vehículos y los anunciadores interiores informan al viajero en todo momento.

Slide 47

Slide 47 text

Smart Information for a moving world! Sheffield, United Kingdom San Sebastian Spain Buenos Aires, Argentina Querétaro Mexico North America and LATAM EUROPE Santiago de Chile Chile 66 DATIK EN EL MUNDO

Slide 48

Slide 48 text

Smart Information for a moving world! 67 NUESTROS CLIENTES Operadores Manufacturers

Slide 49

Slide 49 text

Smart Information for a moving world! 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Primeros proyectos para trenes Primer empleado Transición hacia buses Irizar Group Gen 1 cloud Datik MX Datik UK Puntualidad e incidencias Conducción eficiente Video vigilancia Vehículos eléctricos Predictivo puertas y baterías Migration to AWS Gen 4 cloud Gen 2 cloud Gen 3 cloud Información a pasajeros Fatiga del conductor BREVE HISTORIA

Slide 50

Slide 50 text

Smart Information for a moving world! 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 GENERACIÓN 2 ● Servidores balanceados Activo-Pasivo ● Cambio a Angular.js y APIs REST ● Alojador local ● Arquitectura sencilla ● Pocos componentes software ● Desárrollo ágil ● Desarrolladores de frontend y backend ● Poco escalable y resiliente ● Baja productividad (<50%) ● Despliegues lentos ● Gestión muy manual Gen 4 cloud Gen 2 cloud Gen 3 cloud ● Único servidor en alojador local ● Aplicación monolítica ● Desarrollo basado en GWT Gen 1 cloud

Slide 51

Slide 51 text

Smart Information for a moving world! ● Se popularizan metodologías DevOps ● Migración a AWS ● Breaking the monolith: Primeros micro servicios ● Necesidad de almacenamiento masivo económico (Amazon S3 vs. Openstack Swift) ● Agilidad en el aprovisionamiento de servicios ● Servicios gestionados (Amazon RDS) ● Muchas instancias AWS Elastic Beanstalk (hasta 120) ● Incertidumbre de seguridad: gestión de Security Groups complicada ● Instancias con poco uso (productividad del 60%) Motivos del cambio GENERACIÓN 3 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Gen 1 cloud Gen 4 cloud Gen 2 cloud Gen 3 cloud AWS Elastic Beanstalk Amazon S3 Amazon DynamoDB Amazon Elasticsearch Service AWS Lambda AWS IoT Core Amazon CloudWatch Amazon SQS Amazon SNS

Slide 52

Slide 52 text

Smart Information for a moving world! Elastic Beanstalk Application Amazon RDS Elastic Beanstalk Application S3 Bucket Amazon Elasticsearch Service Lambda Function AWS IoT Core Amazon SNS/SQS Amazon SNS/SQS MQTT EC2 Instance GENERACIÓN 3 Telem etría

Slide 53

Slide 53 text

Smart Information for a moving world! Amazon RDS S3 Bucket Amazon Elasticsearch Service Lambda Function AWS IoT Core Amazon SNS/SQS Amazon SNS/SQS MQTT Telem etría Elastic Load Balancer GENERACIÓN 3 1 EBS = 1 Docker 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 Gen 1 cloud Gen 4 cloud Gen 2 cloud Gen 3 cloud

Slide 54

Slide 54 text

Smart Information for a moving world! GENERACIÓN 4 Imagen yaml Diagnóstico Metricas Logs Elastic Load Balancer yaml Imagen Imagen On Premises Gitlab Registry Rancher Docker Traefik Traefik Kubernates Prometheus Graphana

Slide 55

Slide 55 text

Smart Information for a moving world! ● Despliegues muy ágiles y automatizables ● Misma arquitectura y misma gestión en on-premises y en AWS ● Ligero ahorro de costes (10%). Con reserva de instancias hasta 50% de ahorro ● Alta productividad de la infraestructura (>80%) ● Alta escalabilidad y auto-escalable ● Alta resiliencia ● Alta gestionabilidad (de 120 a 20 instancias) GENERACIÓN 4

Slide 56

Slide 56 text

Smart Information for a moving world! ● Motivación del cambio a AWS ○ Necesidad de disponer de un almacenamiento masivo ○ Servicios gestionados ○ Mayor autonomía y agilidad en la gestión de instancias ● Por qué AWS: ○ Líder del sector ○ Disponibilidad de tutoriales, documentación y formación ○ Red de empresas asociadas: consultores, asesoramiento ● Logros obtenidos: ○ Romper monolito e implementar arquitectura de micro servicios ○ Pasar de 120 instancias (small-medium) EC2 a 20 (xlarge) ○ 5 despliegues por semana (limitado por los recursos de desarrollo) ○ Migración de Elastic Beanstalk a Kubernetes, reducción de costes del 10% (depende de uso) ○ Con reserva de instancias, reducción de costes del 50% ○ Arquitectura escalable, resiliente y optimizado en costes CONCLUSIONES

Slide 57

Slide 57 text

@ITS_Datik @ietxabe

Slide 58

Slide 58 text

S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.

Slide 59

Slide 59 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Positive Chat – Serverless architecture Amazon DynamoDB Amazon Cognito Amazon API Gateway WebSocket connection PositiveChat Lambda function Connections table Conversations table Topics table Web browser AWS Cloud S3 bucket for static assets (HTML, CSS, JS) Authentication Authorization To be implemented Amazon Comprehend Amazon Translate Amazon Rekognition To be implemented https://github.com/danilop/serverless-positive-chat D em o

Slide 60

Slide 60 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T $ wc -l positive-chat/app.js 326 positive-chat/app.js $ wc -l www/index.js 204 www/index.js backend + frontend ≃ 460 lines of code removing empty lines and comments D em o

Slide 61

Slide 61 text

© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Positive Chat https://pchat.demo.danilop.net/?room=AWSSummitMadrid D em o

Slide 62

Slide 62 text

Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Danilo Poccia @danilop AWS Iñigo Etxabe @ietxabe Datik