Practical AWS Step Functions. Batch systems built with no code.

https://jawspankration2024.jaws-ug.jp/ja/timetable/TT-48/

Transcript

1. Practical AWS Step Functions. Batch systems built with no code. Masanori YAMAGUCHI JAWS PANKRATION 2024 #jawspankration2024 #jawsugchiba
2. Masanori YAMAGUCHI ForgeVision Inc, Head of AWS Business and Technology. @kinunori JAWS-UG Chiba branch, JAWS DAYS 2021Committee Chair, AWS Community HERO, AWS Samurai, APAC Community Award 2022 AWS Ambassador, APN ALL AWS Certifications Engineers, AWS Gold Jacket Club. #コミュニティ界の三冠王
3. #jawspankration2024 #jawsugchiba Content ・Overview and requirements of the batch system covered in this session. ・How to meet each requirement using Step Functions without writing code. ・Key Takeaways.
4. #jawspankration2024 #jawsugchiba What is Step Functions? ・Serverless workflow orchestration service. ・A workflow is defined as a state machine, with each step represented as a specific state. This allows for flexible branching, parallel processing, and error handling. ・It seamlessly integrates with AWS services like AWS Lambda, Amazon S3, and DynamoDB, enabling the development of serverless applications and backend automation.
5. #jawspankration2024 #jawsugchiba Requirements for the batch system to be built: Automated EC2 image retrieval system. Obtain the AMI of specific EC2 instances existing on the AWS account. ・All processing must be completed within the state machine (no need for code execution in Lambda functions). ・The AMI name should follow the format. “ami-{SpecificTagValue}-{EnvironmentName}-YYYYMMDD-hhmm-{uuid}” ・The system must be able to determine the successful completion of the AMI creation process. In case of an error, the state machine should terminate abnormally.
6. #jawspankration2024 #jawsugchiba Requirement 1: All processing must be completed within the state machine (no need for code execution in Lambda functions). Answer : Use AWS service SDK integrations in Step Functions. ・By using SDK Integration, you can directly invoke AWS service API actions from the workflow. ・This allows you to send instructions to AWS services from the state machine without writing code to execute the API.
7. #jawspankration2024 #jawsugchiba SDK integrations vs Lambda function(via state machine) Pros Cons SDK integrations It’s easy to understand the processing flow of the state machine. Processes that are not provided by the built-in functions cannot be performed. Even people who cannot read programming languages can understand the process. There are differences between the API provided by the API documentation and the API available through SDK integration. Lambda function (via state machine) Capable of executing flexible processes. Only those who understand programming languages can perform maintenance. You can execute the API according to the API documentation. You need to refer to various services to understand the processing of the state machine. Synchronous processing is required (e.g., waiting for the execution results of a Lambda function). Key decision factors.
8. Requirement 2: The AMI name should follow the format ami-{SpecificTagValue}-{EnvironmentName}-YYYYMMDD-hhmm-{uuid} Answer : Use built-in functions to obtain dates and format strings. Date retrieval. -(dash) to split hour and minute. { “NamePrefix.$”: “States.Format(‘{}-{}-{}_{}’, ‘app’, $.Service, $.Env, ‘{}’)”, “DateTSplited.$”: “States.StringSplit($$.Execution.StartTime, ‘T’)” } { "NamePrefix.$": "$.GetTimeAndDate.NamePrefix", "DateHyphenSplited.$": "States.StringSplit($.GetTimeAndDate.DateTSplited[0], '-')" }
9. #jawspankration2024 #jawsugchiba AWS Step Functions provides built-in functions. These built-in functions allow you to perform a wide range of general-purpose operations, including but not limited to the following: ・Array manipulation ・Encoding and decoding ・String and mathematical operations ・UUID generation For more details, please refer to the documentation: https://docs.aws.amazon.com/ja_jp/step-functions/latest/dg/amazon-states-language-intrinsic- functions.html
10. Requirement 3: ・The system must be able to determine the successful completion of the AMI creation process. ・In case of an error, the state machine should terminate abnormally. Answer : Use workflow states to check processing status and handle errors. #jawspankration2024 #jawsugchiba
11. Choice state (one of the workflow states.)
12. Choice state (one of the workflow states.) cont’d. ・We can execute specific actions when a particular path(JSONPath) in the state machine data is in a certain state. ・It is also possible to configure multiple choices. ・Similar to a case statement. If the JSONPath DescribeImagesResult.Images[0].State is available, the AMI creation process is considered complete, and the workflow moves on to the subsequent steps. By using the Choice state, you can catch errors in the AMI creation process and transition to error handling.
13. https://aws.amazon.com/jp/builders-flash/202306/implement-step-functions-error-handling/ #jawspankration2024 #jawsugchiba Let’s implement error handling in AWS Step Functions. (AWS Step Functions のエラーハンドリングを実装しよう) For additional error handling methods, please refer to my article on the official AWS website.
14. Key Takeaway Many tasks can be accomplished using Step Functions’ built-in features. It is important to carefully consider whether it is truly necessary to invoke a Lambda function from the state machine. Consider point: • The skill level of the maintenance team. • Cost (consider the expenses of running Lambda functions) • Visibility requirements. (keeping the architecture simple) • Whether the required processing can be performed using built-in functions. • The complexity of error handling. (error handling is simpler when kept within the state machine) #jawspankration2024 #jawsugchiba
15. Thank you ! #jawspankration2024 #jawsugchiba @kinunori Masanori Yamaguchi