FaaS is the Least Important Thing About Serverless

FaaS is the Least Important Thing About Serverless

The serverless trend emerged from the introduction of FaaS from providers like AWS Lambda. Moving compute from instances and containers to on-demand FaaS has massive appeal. But functions are one small part of the serverless story! I present a way of building serverless applications to minimise code and maximise use of managed services. This approach shifts the emphasis from coding to the development of powerful, scalable architectures. I show ways of replacing many of your Lambdas with alternative managed services and explain why code should first and foremost be a tool for building on-demand infrastructure.

I will cover five common uses for serverless functions today; orchestration, delegation, data reshaping, database storage or querying and service integration.

In each case, I will demonstrate how system complexity, cost and maintenance effort can be reduced by adopting client-side orchestration, an event bus, managed data stores and cloud-native data transformation solutions such as AWS Glue, S3 Select, Kinesis, Sagemaker and API Gateway.

Eoin is Co-Founder and CTO of fourTheorem, a modern application development consultancy based in Ireland and New York.

43f9d2a530cd32d7d8622e9f3e533e40?s=128

Eoin Shanaghy

April 02, 2019
Tweet

Transcript

  1. Why FaaS is the Least Important Thing About Serverless Eoin

    Shanaghy, CTO @fourTheorem @eoins
  2. fourTheorem

  3. Modern Application Development / Modernisation Serverless and Microservices DevOps Machine

    Learning
  4. Eoin Shanaghy @eoins

  5. http://bit.ly/aiservice 40% discount twmicroxchg19 + 3 FREE BOOKS! !

  6. Software Development has been in its Infancy Half-baked Solutions Poor

    Planning Dubious Quality
  7. This time it’s different

  8. None
  9. Why is Now Different?

  10. Era of the Product Mindset LEAN LOW COST EXPERIMENTATION FEEDBACK

    LOOPS CONTINUOUS DEPLOYMENT PRODUCT DISCOVERY BUILD THE RIGHT PRODUCT
  11. PRODUCT MANAGEMENT CUSTOMER EXPERIENCE ENTERPRISE BUILDING PRODUCTS THAT MATTER

  12. SERVERLESS

  13. EARLY ENTERPRISE JAVA CORBA OOP UML ARCHITECTURE SPECIFICATIONS RDBMS STORED

    PROCS FUNCTIONAL SPECIFICATIONS 2000
  14. Deployment every 3-6 months

  15. Microservice Era Microservices Containers Distributed Systems Organisational Change Faster Adaptable

    Build your own PaaS Infrastructure Investment Scalable Reliable Focus Disposable CD Immutable
  16. λ

  17. Serverless !== FaaS The same rules don’t necessarily apply Don’t

    just divide existing code into Lambdas
  18. Take a Greenfield perspective Forget legacy constraints Disregard current implementation

    Establish a technology vision Serverless Adoption – First Steps
  19. None
  20. What does all our code actually do?

  21. What do we spend our time on?

  22. Differentiating Code Integration Transformation Delegation Data Access Orchestration

  23. Non Functional Considerations Monitoring Security Scalability Reliability Governance / Compliance

  24. Data Transformation

  25. Integration

  26. Data Access

  27. Orchestration and Routing

  28. Lambdify Everything?

  29. Lambdify Everything? Developer Flow Feedback Loop Local Development Debugging Size

    Limits Latency
  30. Data Locality Access to Files (S3) Performance Synchronous Fetch Cost

    No EBS or EFS Storage Cost < Compute Cost Move the Code to the Data
  31. There are other ways

  32. Serverless is true Utility Computing • FaaS is one of

    these utilities. There are many. • Favourites (AWS) • S3 • Cognito • DynamoDB • AppSync • Aurora Serverless • Step Functions • IAM • Lambda The Holistic Serverless Definition
  33. The Effect of Managed Services State is Federated Distributed State

    Management Data in the Right Form > Fetch on Demand
  34. Distributed State – Understand Access Patterns SQL DynamoDB Event Streams

    Structured Normalised Arbitrary access patterns Unstructured Single table Predictable access patterns Event sourcing Event-driven Real-time requirement
  35. • Optimise for access – polyglot persistence • Downsample •

    Aggregate • Keep raw data • Iterate on your approach • Keep code close to the data – remember the power of SQL Store with Intent
  36. S3 SELECT Read from S3, Parquet or JSON SELECT temp,

    timestamp, device_id FROM s3Object s WHERE s.date BETWEEN ‘2019- 04-01' AND ‘2019-04-30' 400% faster
  37. ATHENA Arbitrary analytics Aggregations Complex queries on big data volumes

    Nice for archival log querying
  38. GLUE Serverless ETL Build ETL pipelines with Scala or Python

    Shape data for analytics / machine learning
  39. SAGEMAKER

  40. KINESIS STREAMS & ANALYTICS

  41. Authentication and Authorisation

  42. ORCHESTRATION STRATEGIES

  43. EXPLICIT Lambda AWS Step Functions Synchronously Invoke other services Pay

    while you wait Declarative state machine Pay per transition
  44. AWS Step Functions Declarative state machine Pay per transition

  45. IMPLICIT Event-driven Isolated Steps Reactive

  46. EMBRACE THE POWER OF THE CLIENT

  47. DELEGATION

  48. API Gateway Service Integration

  49. API Gateway Service Integration

  50. FAAS IS NOT THE MOST IMPORTANT THING

  51. Important Things! Product Iteration Speed Automation – CI/CD Expendable components

    Managed services Everything that happens after deployment
  52. One more thing… Full Stack Full Test Pyramid Open Source

    CD Pipeline SLIC Starter Use it – Contribute – Talk to me! @eoins Project Structure Access Control
  53. THANK YOU! http://bit.ly/aiservice @eoins