Serverless computing for Java developers

Java hasn’t been a popular language in AWS Lambda or any other Serverless computing solution. There are some reasons like lack of easy to use libraries, cold starts, and monitoring. I’ll talk about these issues and show some use-cases from Opsgenie on how we leverage this new paradigm. I’ll close the talk with some specific suggestions for Java devs.

Transcript

1. @srhtcn Serverless computing for Java developers The Opsgenie Experience Serhat Can @srhtcn
2. @srhtcn Who am I? • Ex-Software Engineer Technical Evangelist at • AWS Community Hero • Devopsdays core team member • Organizer Serverless Turkey meetup and Devops Turkey meetup • Podcast Turuncu Pasaport • @srhtcn on Twitter
3. @srhtcn You want to run code on cloud. Your options: Bare metal IaaS (VM) CaaS (container) PaaS (app) Serverless (function) More control, more code Less control, less code
4. @srhtcn
5. @srhtcn Making thoughtful decisions about tools and architecture can help; well-considered constraints can free us from the decisions that aren't bringing us distinguishable benefit. Bridget Kromhout https://queue.acm.org/detail.cfm?id=3185224
6. @srhtcn What is Serverless?
7. @srhtcn Defining Serverless Serverless is an event driven, utility based, stateless, code execution environment. Simon Wardley @swardley
8. @srhtcn Defining Serverless Event driven: Code is initiated and run after an event like HTTP request or storage of a file triggers.
9. @srhtcn Defining Serverless Event driven: Code is initiated and run after an event like HTTP request or storage of a file triggers. Utility based: No payment for idle time or hosting. You pay for the resources you use when your code is triggered.
10. @srhtcn Defining Serverless Event driven: Code is initiated and run after an event like HTTP request or storage of a file triggers. Utility based: No payment for idle time or hosting. You pay for the resources you use when your code is triggered. Stateless: Code execution environment is deconstructed after sometime. No information is guaranteed to stay in the environment after function execution is completed.
11. @srhtcn Defining Serverless Event driven: Code is initiated and run after an event like HTTP request or storage of a file triggers. Utility based: No payment for idle time or hosting. You pay for the resources you use when your code is triggered. Stateless: Code execution environment is deconstructed after sometime. No information is guaranteed to stay in the environment after function execution is completed. Code execution: Just code, not servers / VMs / containers etc.
12. @srhtcn Why should I go Serverless? Two main reasons
13. @srhtcn ServerLess is more Less code to maintain, less ops, less toil • Scaling • Provisioning • OS or Language updates • Resource utilization • Network monitoring • Fault tolerance • Shipping logs https://landing.google.com/sre/book/chapters/eliminating-toil.html
14. @srhtcn
15. @srhtcn Economics No payment for idle time or hosting Easy to get started Faster time to market
16. @srhtcn Java in a Serverless World • AWS Lambda: Java 8 and more details soon!
17. @srhtcn Java in a Serverless World • AWS Lambda: Java 8 and more details soon! • Google Cloud Functions: no Java support
18. @srhtcn Java in a Serverless World • AWS Lambda: Java 8 and more details soon! • Google Cloud Functions: no Java support • Azure Functions: Java 8 for Azure Functions is currently in preview (GA 3 days ago!)
19. @srhtcn Java in a Serverless World • AWS Lambda: Java 8 and more details soon! • Google Cloud Functions: no Java support • Azure Functions: Java 8 for Azure Functions is currently in preview (GA 3 days ago!) • IBM Cloud Functions: Java 8 and bring your own language via Docker
20. @srhtcn Java in a Serverless World • AWS Lambda: Java 8 and more details soon! • Google Cloud Functions: no Java support • Azure Functions: Java 8 for Azure Functions is currently in preview (GA 3 days ago!) • IBM Cloud Functions: Java 8 and bring your own language via Docker • Oracle’s FN Project: container native, cloud agnostic serverless platform
21. @srhtcn AWS Lambda
22. @srhtcn How it works?
23. @srhtcn How it works internally? https://engineering.opsgenie.com/what-is-different-in-the-serverless-world-b9e0f68de191
24. @srhtcn Language support Node.js (JavaScript) Python, Java (Java 8 compatible), C# (.NET Core), Golang, Ruby NEW: Bring your own language!
25. @srhtcn Bring your own language: Lambda Runtime API Allows you to use different versions of Java
26. @srhtcn Pricing You choose memory size % of CPU core and network capacity increases proportionally with memory More memory doesn’t always mean you pay more
27. @srhtcn https://www.slideshare.net/ChrisMunns/aws-startup-day-boston-2018-the-best-practices-and-hard-lessons-learned-of-serverless-applications
28. @srhtcn Supported event sources 20 different services can trigger AWS Lambda functions including. Event sources that aren't stream-based: Synchronous invocation: Asynchronous invocation: Poll-based (or pull model) event sources that are stream-based: Poll-based event sources that are not stream-based:
29. @srhtcn Toolkit around AWS Lambda Orchestration: Step Functions Deployment: SAM, Serverless.js, CloudFormation, Apex, Terraform Monitoring: X-Ray, Thundra Marketplace: AWS Serverless Application Repository
30. @srhtcn AWS Lambda at OpsGenie AWS Lambda with Java 8 DynamoDB SQS SNS VPC Serverless.js
31. @srhtcn Fast scaling under immediate high load Under-utilized machines Pricing (still not a huge concern) Operational complexity Learning curve - kubernetes? AWS Fargate - YES Why did we consider AWS Lambda?
32. @srhtcn OpsGenie’s Serverless journey 2015 Writing small scale custom integrations At this point, we started leveraging AWS Lambda to help our customer run custom code
33. @srhtcn OpsGenie’s Serverless journey 2015 Writing small scale custom integrations At this point, we started leveraging AWS Lambda to help our customer run custom code First production usage Started using AWS Lambda for leveraging async / not business critical jobs such as DynamoDB autoscale 2016
34. @srhtcn OpsGenie’s Serverless journey 2015 Writing small scale custom integrations At this point, we started leveraging AWS Lambda to help our customer run custom code First production usage Started using AWS Lambda for leveraging async / not business critical jobs such as DynamoDB autoscale 2016 Service and Incident Management A new customer facing feature running on AWS Lambda integrated with the rest of the code base. 2017
35. @srhtcn OpsGenie’s Serverless journey 2015 Writing small scale custom integrations At this point, we started leveraging AWS Lambda to help our customer run custom code First production usage Started using AWS Lambda for leveraging async / not business critical jobs such as DynamoDB autoscale 2016 Service and Incident Management A new customer facing feature running on AWS Lambda integrated with the rest of the code base. 2017 A Spin Off: Thundra Observability for AWS Lambda 2018
36. @srhtcn Serverless architectures
37. @srhtcn Custom integrations AWS Lambda is a life saver for custom solutions, because; ◦ Customers do not need to manage servers ◦ Easy to get started and deploy (give a .zip file) ◦ Real pay what you use pricing
38. @srhtcn Create Alerts from Slack Messages Source: https://github.com/opsgenie/slack-to-opsgenie-alert-creator
39. @srhtcn Alert Enrichment
40. @srhtcn Elasticsearch data indexing
41. @srhtcn DynamoDB Cross Region Replication
42. @srhtcn DynamoDB Auto Scale
43. @srhtcn Service and Incident Management
44. Challenges
45. @srhtcn Challenges • Monitoring and debugging • Cold start • Local development • Concurrent execution limit • No well-known good practices
46. @srhtcn Fixing “it is slow” is harder in AWS Lambda Too many moving pieces No way to attach an agent Even how to send the monitoring data is a discussion point
47. @srhtcn Determine the latency in different levels Automatic instrumentation GC, Thread counts & durations, CPU usage details Get the stack trace in case of an error and drill down See logs, traces, and metrics in one view thundra.io What we needed was
48. @srhtcn An incident of $40,000
49. @srhtcn Lessons learned: An incident of $40,000 Avoid infinite retries Monitor and alert for pricing (no pricing metric for AWS Lambda) Think of Cloudwatch cost and sample logs & metrics
50. @srhtcn Cold start When: - memory size - code size - VPC - the language https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76
51. @srhtcn Solving cold start problem Wait for AWS to improve it Increase memory (and pay more) Lightweight application framework instead of Spring Do some smart warm-up
52. @srhtcn Account level concurrent execution limit Lambda concurrent execution count for non stream based events: events (or requests) per second * function duration
53. @srhtcn Account level concurrent execution limit Lambda concurrent execution count for non stream based events: events (or requests) per second * function duration Hard to deal with peaks in request numbers Takes time to increase the limit Functions affect each other’s scalability
54. @srhtcn Latency in a third party can bring your whole system down https://read.acloud.guru/does-aws-lambda-keep-its-serverless-marketing-promise-of-continuous-scaling-e990114bb379
55. @srhtcn Function level concurrent execution limit Limit the scalability of non-critical functions Reserved capacity is subtracted from the global limit
56. @srhtcn Don’t put your functions in a VPC unless you have to You need sufficient IP addresses in your subnet and ENI to scale https://docs.aws.amazon.com/lambda/latest/dg/vpc.html Determine the ENI capacity you need: Concurrent executions * (Memory in GB / 3 GB)
57. @srhtcn Serverless computing will become the default computing paradigm of the Cloud Era. @srhtcn https://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-3.pdf
58. @srhtcn Thank you! Serhat Can twitter.com/srhtcn linkedin.com/in/serhatcan medium.com/@serhatcan