Serverless in Java Lessons learnt

Serverless has gained a lot on popularity recently and changed the way we develop the applications. We no longer need to care about setting up and managing the servers, scalability and deployment is simplified. Serverless is very often referred to as the approach which will let you to shift focus to implementing business logic when writing the code. But where has the complexity moved to now? How performant is Java code in serverless solution? Is serverless good for complex solutions? What are the benefits? During my talk I’d like to answer those questions based on our experiences of working on serverless solution written fully in Java.

Transcript

1. Serverless in Java Lessons learnt Krzysztof Pawlowski krzysztof.pawlowski@merapar.com JEE Conf, 27.04.2019 Kyiv
2. About me • Helping clients create value with innovative cloud solutions @ Merapar • Teaching students good programming practices @ Polish- Japanese Academy of IT • 10+ years of experience with Java • 2 years of experience with AWS • Certified AWS Solutions Architect and AWS Developer
3. Agenda • What is serverless? • Why did we choose serverless and Java? • What did we learn when using Java for serverless solution? • Best practices

4. –Techopedia “Serverless computing is a type of cloud computing where the customer does not have to provision servers for the back-end code to run on, but (…) cloud provider starts and stops a container platform as a service as requests come (…).”
5. How does serverless work? Function A Function A code / jars storage Serverless Computing Platform Execution Environment Server Execution Environment Server Execution Environment Server Execution Environment Server Client Function A 1. Execute Function A 2. Retrieve Function A 5. Return result 3. Deploy Function A 4. Run Function A
6. How does serverless work? EC2 instance (server) Execution environment JVM Request Handler
7. Our use-case: migration project • Migration of several million users and devices to a new platform • Users decide when they want to migrate • Users and data are migrated to a new backoffice • Firmware is upgraded • How to orchestrate the migration process?

8. Our use-case: migration project
9. Why serverless? • No servers to manage • Simplified deployment and packaging (AWS Cloudformation) • Automatic continuous scaling • No idle / cold servers = no costs when not used • Pay per request • Availability and fault tolerance build in 

10. Why Java? • Competencies in a team • Well tested libraries • Strongly typed language • AWS SDK for Java • Tooling: IntelliJ IDEA, Maven etc.
11. Lesson 1: What should be the project structure?
12. Project structure or One big Maven project Maven project per AWS Lambda
13. Project structure 
 One big maven project
 
 Pros: • easier building process • code re-usage
 Cons: • longer time needed to 
 download/unpack package 
 before invocation 
 Maven project per lambda Pros: • single functionality deployment • lambda code separation / loose coupling • shorter time needed to download/ unpack package before invocation Cons: • code duplication
14. Lesson 2: Where has the complexity moved to?
15. Endpoint definition in Spring Boot @RequestMapping(method = RequestMethod.POST)
 @ResponseStatus(HttpStatus.CREATED)
 @ResponseBody
 @Path(“/foo”)
 public Long create(@RequestBody Foo resource) {
 return service.create(resource);
 }
16. Endpoint definition in AWS AWS IAM AWS API Gateway AWS Lambda • complexity moved from one codebase to multiple services • initial bootstrapping is more work

17. Endpoint definition in AWS 1. implement AWS Lambda handler import com.amazonaws.services.lambda.runtime.Context import com.amazonaws.services.lambda.runtime.RequestHandler; public class Hello implements RequestHandler<Foo, Long> {
 public Long myHandler(Foo resource, Context context) {
 return service.create(resource);
 }
 }
18. Endpoint definition in AWS 2. create API Gateway endpoint definition • a long json definition including: • OpenAPI definition (aka Swagger) of the endpoint • all responses mappings • all errors mappings
19. Endpoint definition in AWS 3. create IAM role for API Gateway to call AWS Lambda "LambdaPermission": {
 "Type": "AWS::Lambda::Permission",
 "Properties": {
 "Action": "lambda:invokeFunction",
 "FunctionName": “arn:aws:lambda:<region>
 :<account-id>:function:hello”,
 "Principal": "apigateway.amazonaws.com",
 “SourceArn”:”arn:aws:execute-api:<region>
 :<account_id>:HelloApi"
 }
 }
20. Developer now needs to know more about infrastructure/platform
21. Lesson 3: How to debug/troubleshoot?
22. Debugging image: https://www.jetbrains.com/help/idea/migrating-from-eclipse-to-intellij-idea.html ? No remote debugging
23. Running your code locally • AWS SAM (Serverless Application Model) Local • simulates AWS cloud on your machine • Eclipse/IntelliJ plugin - possible to debug locally • not all the services are implemented
24. IntelliJ AWS toolkit source: https://aws.amazon.com/blogs/developer/aws-toolkit-for-intellij-now-generally-available/
25. IntelliJ AWS toolkit source: https://aws.amazon.com/blogs/developer/aws-toolkit-for-intellij-now-generally-available/
26. IntelliJ AWS toolkit source: https://aws.amazon.com/blogs/developer/aws-toolkit-for-intellij-now-generally-available/
27. Metrics and logs • Metrics and monitoring • CloudWatch • X-Ray
28. Metrics and logs • Logs • CloudWatch
 Logs Insights • ElasticSearch / Kibana • All managed by AWS
29. Lesson 4: What is the performance?
30. Java performance in AWS • AWS Lambda performance test based simple application exposing GET endpoint returning “Hello world” response written in: • Java • Node.JS • C# (.net core sdk 2.0) • F# • Go • Python • Run in with 1024MB MEM • Cold start not taken into account
31. Java performance in AWS Java code had average performance
32. Java performance in AWS Java code had consistent performance
33. Java performance in AWS Java endpoint had similar performance to other languages
34. Java performance in AWS Java endpoint had a consistent performance
35. Performance tuning AWS Lambda • you pay per 100 ms • price depends on amount of memory • cpu depends on amount of memory
36. Performance tuning
37. Performance tuning
38. Lesson 5: Cold start might be an issue
39. Cold start Downloading the code Starting the execution environment Bootstrapping the runtime Running the code AWS optimisation User optimisation Cold start Total execution time Warm start
40. Cold start 1st call Cold start Running the code Running the code Running the code Running the code 2nd call 3rd call 4th call First execution of lambda function
41. Cold start Cold start Running the code Cold start Running the code > 20..45mins Big interval between lambdas invocation
42. Cold start Cold start Running the code Running the code Running the code Running the code Cold start Running the code Running the code Cold start Running the code Running the code Running the code Concurrent lambda invocations (can be limited in lambda definition) - new execution environment created
43. Cold start • In case of Java cold start might take couple of seconds • Less harmful in dynamically typed languages (Python, Node.JS) • Bigger jar file increases cold start
44. Decreasing cold start time • Increase memory/cpu for lambda function • might increase the cost • Separate jar file for each lambda function • Go for dynamically typed language
45. Eliminating cold start time • Pre-warm lambda • call lambda every ~20 min to keep it warm (cronjob) • cost assuming 3GB mem lambda running for 1s every 20 min is about $1 per year • for n concurrent lambda executions - run at once n concurrent lambdas with the same interval
46. Cold start • Other option how to handle cold start: • do nothing! • it might not be an issue in your case
47. Lesson 6: Understand and use AWS best practices
48. –Best Practices for Working with AWS Lambda Functions (https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html) “Take advantage of Execution Context reuse to improve the performance of your function.”
49. Reusing execution context • Execution context is created during bootstrapping the environment (part of cold start) • use static initialisation/constructor, global/static variables and singletons • reuse connections: database, http, etc. • do not create long-initialised objects in the handleRequest method
50. –Best Practices for Working with AWS Lambda Functions (https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html) “Use AWS Lambda Environment Variables to pass operational parameters to your function.”
51. AWS Lambda Environment Variables
52. –Best Practices for Working with AWS Lambda Functions (https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html) “Be familiar with AWS Lambda Limits.”
53. AWS Lambda limits • Concurrent executions: 1000 • Function timeout: 900 seconds • Function memory allocation: 128MB to 3008 MB • Function environment variables: 4KB • /tmp directory storage: 512MB
54. Conclusion • Is Java good for serverless? • Definitely yes! • Is Java always good for serverless? • Definitely not!
55. Conclusion • When to use serverless? • Unpredictable load of your app • Low load of the app -> low cost • No long running processes • When to use Java in serverless? • Cold start is not an issue • Java is a language of preference for the team
56. Thank you!
 Q & A Krzysztof Pawlowski krzychpawlowski krzysztof.pawlowski@merapar.com