Slide 1

Slide 1 text

@SigNarvaez | @MongoDB | @BigDataDayLA On-Prem, IaaS, etc. Serverless

Slide 2

Slide 2 text

Serverless Architectures with AWS Lambda and MongoDB Atlas Sig Narváez Sr. Solutions Architect [email protected] @SigNarvaez

Slide 3

Slide 3 text

Serverless

Slide 4

Slide 4 text

Big Iron Commodity Hardware Virtualized Containers Functions Where will my code run? @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 5

Slide 5 text

Thoughtworks Technology Radar

Slide 6

Slide 6 text

AWS re:Invent 2016: The State of Serverless Computing (SVR311)

Slide 7

Slide 7 text

AWS re:Invent 2016: The State of Serverless Computing (SVR311)

Slide 8

Slide 8 text

https://www.slideshare.net/AmazonWebServices/aws-reinvent-2016-how-thermo- fisher-is-reducing-mass-spectrometry-experiment-times-from-days-to-minutes-with- mongodb-aws-dat204 @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 9

Slide 9 text

Microservices

Slide 10

Slide 10 text

Serverless Microservice @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 11

Slide 11 text

Shape • Person • Insurance Policies • Shape changes per policy type • Addresses Operations via API • GET Customers with soon-to-expire policies, within a geo radius • GET Customers / by SSN, id, etc. • PATCH Update basic contact info (cell, email, …) Customer Single View - Insurance Industry (hypothetical) High-level architecture of a single view platform @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 12

Slide 12 text

CQRS on Serverless Microservices COMMAND Update basic info QUERY Soon to Expire GEO By SSN, ID, … CUD API Key Read API Key Lambda Function(s) Majority Writes Secondary Reads Lambda Function(s) VPC Peering @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 13

Slide 13 text

MongoDB Atlas & AWS Build it!

Slide 14

Slide 14 text

Automated Service On-demand DBaaS Secure Highly Available Disaster Recovery Elastic Scalability MongoDB Atlas

Slide 15

Slide 15 text

Setup MongoDB Atlas - M10+ Need assigned AWS region @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 16

Slide 16 text

mgeneratejs • https://github.com/rueckstiess/mgeneratejs • npm install -g mgeneratejs • Create template – generate data • Upload to Atlas via mongoimport • Hint: get connection string from Atlas UI! • Browse with Compass Generate dataset (optional) Template (CustomerSingleView.json) mgeneratejs -n 100 CustomerSingleView.json | mongoimport --host ”YOUR ATLAS CLUSTER" --numInsertionWorkers 4 --db SingleView--collection Customers --authenticationDatabase admin -- ssl --username YOURUSER --password YOURPASSWORD @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 17

Slide 17 text

IAM • Role with Lambda execute policies VPC • VPC Peering Connection • Security Group Required AWS Services Lambda • Set VPC, Security Group and IAM role • Upload deployment package (.zip) API Gateway • API definition (Resources & HTTP Methods) • Map Routes to Lambda functions • API Keys & Usage Plans @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 18

Slide 18 text

VPC Peering

Slide 19

Slide 19 text

VPC Peering Atlas

Slide 20

Slide 20 text

VPC Peering Atlas AWS

Slide 21

Slide 21 text

VPC Peering Atlas AWS

Slide 22

Slide 22 text

VPC Peering Atlas AWS

Slide 23

Slide 23 text

VPC Peering Atlas AWS

Slide 24

Slide 24 text

VPC Peering Atlas AWS

Slide 25

Slide 25 text

Lambda

Slide 26

Slide 26 text

Connections & Containers http://docs.aws.amazon.com/lambda/latest/dg/lambda-introduction.html … AWS Lambda maintains the container for some time in anticipation of another Lambda function invocation. … the service freezes the container after a function completes, and thaws the container for reuse. If AWS Lambda chooses to reuse the container, this has the following implications: - Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. You can add logic in your code to check if a connection already exists before creating one. @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 27

Slide 27 text

Python lambda function skeleton @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 28

Slide 28 text

Node.js lambda function skeleton @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 29

Slide 29 text

Local Emulators : Test on EC2 instance against Atlas python-lambda-local at https://pypi.python.org/pypi/python-lambda-local lambda-local (node.js) at https://www.npmjs.com/package/lambda-local

Slide 30

Slide 30 text

Code packaging http://docs.aws.amazon.com/lambd a/latest/dg/deployment-package- v2.html @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 31

Slide 31 text

Lambda functions @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 32

Slide 32 text

Upload & configure function The handler function The role with lambda permissions The VPC (peered with Atlas) The security group that allows traffic At least 2 subnets

Slide 33

Slide 33 text

API Gateway

Slide 34

Slide 34 text

Read API – GET /api/v1/customers

Slide 35

Slide 35 text

CUD API - PATCH /api/v1/customers

Slide 36

Slide 36 text

Deploying the API

Slide 37

Slide 37 text

Access and throttling via API Keys

Slide 38

Slide 38 text

Test!

Slide 39

Slide 39 text

Test with Postman @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 40

Slide 40 text

Load test too!

Slide 41

Slide 41 text

AWS CloudWatch @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 42

Slide 42 text

MongoDB Atlas Monitoring and Alerts @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 43

Slide 43 text

MongoDB Compass @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 44

Slide 44 text

Serverless Web too!!

Slide 45

Slide 45 text

RetroPie – Old-school games, killer tiny hardware @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 46

Slide 46 text

Retro Web App based on aws-serverless-express https://github.com/snarvaez/bddla17/t ree/master/singleViewWebApp https://github.com/awslabs/aws- serverless-express @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 47

Slide 47 text

Done! … But what about?

Slide 48

Slide 48 text

Scaling? Scaling Lambda No user intervention required - Default safety throttle of 100 concurrent executions per account per region. Functions invoked synchronously throw 429 error code. Functions invoked asynchronously can absorb reasonable bursts for approx. 15-30 minutes. If exhausted, consider using Simple Queue Service (SQS) or Simple Notification Service (SNS) as the Dead Letter Queue (DLQ). Read more at https://aws.amazon.com/lambda/faqs/ Scaling MongoDB Atlas On-Demand Zero downtime Upscale/Downscale: • Instance size • Storage size • IOPS • Replication factor. @SigNarvaez | @MongoDB | @BigDataDayLA

Slide 49

Slide 49 text

Serverless Architectures with AWS Lambda and MongoDB Atlas Sig Narváez Sr. Solutions Architect [email protected] @SigNarvaez Q & A https://github.com/snarvaez/bddla17 https://resources.mongodb.com/serverles s-architectures @SigNarvaez | @MongoDB | @BigDataDayLA