FaaS is the Least Important Thing About Serverless

The serverless trend emerged from the introduction of FaaS from providers like AWS Lambda. Moving compute from instances and containers to on-demand FaaS has massive appeal. But functions are one small part of the serverless story! I present a way of building serverless applications to minimise code and maximise use of managed services. This approach shifts the emphasis from coding to the development of powerful, scalable architectures. I show ways of replacing many of your Lambdas with alternative managed services and explain why code should first and foremost be a tool for building on-demand infrastructure.

I will cover five common uses for serverless functions today; orchestration, delegation, data reshaping, database storage or querying and service integration.

In each case, I will demonstrate how system complexity, cost and maintenance effort can be reduced by adopting client-side orchestration, an event bus, managed data stores and cloud-native data transformation solutions such as AWS Glue, S3 Select, Kinesis, Sagemaker and API Gateway.

Eoin is Co-Founder and CTO of fourTheorem, a modern application development consultancy based in Ireland and New York.

Transcript

1. Why FaaS is the Least Important Thing About Serverless Eoin Shanaghy, CTO @fourTheorem @eoins
2. fourTheorem
3. Modern Application Development / Modernisation Serverless and Microservices DevOps Machine Learning
4. Eoin Shanaghy @eoins
5. http://bit.ly/aiservice 40% discount twmicroxchg19 + 3 FREE BOOKS! !
6. Software Development has been in its Infancy Half-baked Solutions Poor Planning Dubious Quality
7. This time it’s different
9. Why is Now Different?
10. Era of the Product Mindset LEAN LOW COST EXPERIMENTATION FEEDBACK LOOPS CONTINUOUS DEPLOYMENT PRODUCT DISCOVERY BUILD THE RIGHT PRODUCT
11. PRODUCT MANAGEMENT CUSTOMER EXPERIENCE ENTERPRISE BUILDING PRODUCTS THAT MATTER
12. SERVERLESS
13. EARLY ENTERPRISE JAVA CORBA OOP UML ARCHITECTURE SPECIFICATIONS RDBMS STORED PROCS FUNCTIONAL SPECIFICATIONS 2000
14. Deployment every 3-6 months
15. Microservice Era Microservices Containers Distributed Systems Organisational Change Faster Adaptable Build your own PaaS Infrastructure Investment Scalable Reliable Focus Disposable CD Immutable
16. λ
17. Serverless !== FaaS The same rules don’t necessarily apply Don’t just divide existing code into Lambdas
18. Take a Greenfield perspective Forget legacy constraints Disregard current implementation Establish a technology vision Serverless Adoption – First Steps
20. What does all our code actually do?
21. What do we spend our time on?
22. Differentiating Code Integration Transformation Delegation Data Access Orchestration
23. Non Functional Considerations Monitoring Security Scalability Reliability Governance / Compliance
24. Data Transformation
25. Integration
26. Data Access
27. Orchestration and Routing
28. Lambdify Everything?
29. Lambdify Everything? Developer Flow Feedback Loop Local Development Debugging Size Limits Latency
30. Data Locality Access to Files (S3) Performance Synchronous Fetch Cost No EBS or EFS Storage Cost < Compute Cost Move the Code to the Data
31. There are other ways
32. Serverless is true Utility Computing • FaaS is one of these utilities. There are many. • Favourites (AWS) • S3 • Cognito • DynamoDB • AppSync • Aurora Serverless • Step Functions • IAM • Lambda The Holistic Serverless Definition
33. The Effect of Managed Services State is Federated Distributed State Management Data in the Right Form > Fetch on Demand
34. Distributed State – Understand Access Patterns SQL DynamoDB Event Streams Structured Normalised Arbitrary access patterns Unstructured Single table Predictable access patterns Event sourcing Event-driven Real-time requirement
35. • Optimise for access – polyglot persistence • Downsample • Aggregate • Keep raw data • Iterate on your approach • Keep code close to the data – remember the power of SQL Store with Intent
36. S3 SELECT Read from S3, Parquet or JSON SELECT temp, timestamp, device_id FROM s3Object s WHERE s.date BETWEEN ‘2019- 04-01' AND ‘2019-04-30' 400% faster
37. ATHENA Arbitrary analytics Aggregations Complex queries on big data volumes Nice for archival log querying
38. GLUE Serverless ETL Build ETL pipelines with Scala or Python Shape data for analytics / machine learning
39. SAGEMAKER
40. KINESIS STREAMS & ANALYTICS
41. Authentication and Authorisation
42. ORCHESTRATION STRATEGIES
43. EXPLICIT Lambda AWS Step Functions Synchronously Invoke other services Pay while you wait Declarative state machine Pay per transition
44. AWS Step Functions Declarative state machine Pay per transition
45. IMPLICIT Event-driven Isolated Steps Reactive
46. EMBRACE THE POWER OF THE CLIENT
47. DELEGATION
48. API Gateway Service Integration
49. API Gateway Service Integration
50. FAAS IS NOT THE MOST IMPORTANT THING
51. Important Things! Product Iteration Speed Automation – CI/CD Expendable components Managed services Everything that happens after deployment
52. One more thing… Full Stack Full Test Pyramid Open Source CD Pipeline SLIC Starter Use it – Contribute – Talk to me! @eoins Project Structure Access Control
53. THANK YOU! http://bit.ly/aiservice @eoins