Enhancing AXA Data Platform with Terraform

JAWS-UG東京 IaC Night 〜入門から上級まで！AWSをコードで構築しよう〜
https://jawsug.connpass.com/event/344954/

Transcript

1. Enhancing AXA Data Platform with Terraform Making Development and Operations Easier AXA Life Insurance Corporation Data Platform Engineering Division Keisuke Taniguchi
2. AXA Life Insurance Corporation Data Platform Engineering Division Keisuke Taniguchi • Develop AXA Data Platform. Previously, developed Data & MA Platform at an e-commerce company keisuke-taniguchi-6b7503188 @csk_pos
3. Agenda 1. Introduction：AXA 
 2. AXA Data Platform 
 3. Terraform Use Case 
 4. Terraform Implementation Tips 
 5. Summary 
 
 

4. AXA
5. 5 すべての人々のより良い未来のために。 私たちはみなさんの大切なものを守ります。
6. • A French-Global insurance and Asset Management Group, serving 93 million customers in 51 countries with 145,000 employees • The origin of AXA was an insurance company founded in 1817 in Normandy, Ancienne Mutuelle de Rouen, which was rebranded to AXA in 1985 • AXA entered Japan in 1994, and expanded by acquisition of Nichidan, established in 1934 • AXA acquired XL Group in 2018 to create the #1 global P&C commercial lines insurance platform S&P Rating AA- ※Figures are actual results for AXA Group in 2024, Ratings are as of February 25, 2025 About AXA Asset Under Management Revenues €110bn Underlying Earnings €8.1bn €879bn
7. AXA Group companies in Japan As of December 31, 2024 Key Indicators AXA Life Insurance AXA General Insurance AXA Holdings Japan 7.12Mn Number of Policies 8,980 Employees 839.8Bn Premiums (3Q,2024) AXA Group in Japan
8. AXA Data Platform
9. AXA Data Platform Manage AWS & Databricks Progressing Databricks migration
10. Terraform Use Case
11. Use Case #1：Development Productivity Improvement • Issue ◦ Managing Over 10,000 Lines in a Single CloudFormation Template ◦ Manual operations are required from code changes to deployment ◦ AWS and Databricks use different IaC tools • What we wanted to solve ◦ Split the FAT template to make development easier ◦ Automate from code changes to deployment to improve development productivity ◦ Manage AWS and Databricks with a common IaC tool
12. Use Case #2：Operational Improvement • Issue ◦ CloudFormation resource inconsistencies due to manual changes ◦ Databricks was deployed without an IaC tool Without IaC, updates risk overwriting changes without detecting differences ◦ Many manually created Lambda packages are approaching EOL • What we wanted to solve ◦ Eliminate differences between code and deployed resources ◦ Difference detection and safe deployment with IaC tool ◦ Automate package creation and make the version up easier
13. Terraform Implementation Tips Example Case
14. Example Case : Lambda
15. Example Case : Terraform Directory Structure tree ├── config │ ├── lambda.yaml │ ├── stepfunctions.yaml │ ├── .... ├── dev │ ├── .... ├── prod │ ├── backend.tf │ ├── lambda.tf -> ../lambda.tf │ ├── lambda_locals.tf -> ../lambda_locals.tf │ ├── import_lambda.tf │ ├── local.tf │ ├── .... ├── lambda.tf ├── lambda_locals.tf ├── .... Resource configs Create env directories, symlink common files, and manage env specific params AWS now offers best practices https://docs.aws.amazon.com/prescriptive-guidance/latest/terr aform-aws-provider-best-practices/structure.html
16. Example Case : Lambda Config (1/2) tree ├── config │ ├── lambda.yaml │ ├── stepfunctions.yaml │ ├── .... ├── dev │ ├── .... ├── prod │ ├── backend.tf │ ├── lambda.tf -> ../lambda.tf │ ├── lambda_locals.tf -> ../lambda_locals.tf │ ├── import_lambda.tf │ ├── local.tf │ ├── .... ├── lambda.tf ├── lambda_locals.tf ├── ....
17. Example Case : Lambda Config (2/2) LambdaFunctions: Sample1: Properties: Code: lambda/sample_1/sample.py FunctionName: sample_1 Handler: sample.main Version: 0.0.1 Sample2: Properties: Code: lambda/sample_2/package.zip OverwriteBuildCommand: "bash build.sh" FunctionName: sample_2 Handler: sample_2.main Timeout: 60 Runtime: python3.12 Version: 0.0.2 Parameters can be overridden; if unspecified, apply default values Lambda builds and deploys occur only on version tag changes
18. Example Case : Lambda Config Handler (1/2) tree ├── config │ ├── lambda.yaml │ ├── stepfunctions.yaml │ ├── .... ├── dev │ ├── .... ├── prod │ ├── backend.tf │ ├── lambda.tf -> ../lambda.tf │ ├── lambda_locals.tf -> ../lambda_locals.tf │ ├── import_lambda.tf │ ├── local.tf │ ├── .... ├── lambda.tf ├── lambda_locals.tf ├── ....
19. Example Case : Lambda Config Handler (2/2) locals { lambda_config = yamldecode( templatefile("../config/lambda.yaml", { Env = local.env AccountId = local.account_id })) lambda-params = flatten([ for lambda in local.lambda_config.LambdaFunctions : { key = lambda.Properties.FunctionName value = { "function_name" = lambda.Properties.FunctionName "handler" = lambda.Properties.Handler "lambda_version" = lambda.Version "lambda_runtime" = contains(keys(lambda.Properties), "Runtime") ? lambda.Properties.Runtime : local.lambda_runtime "build_command" = contains(keys(lambda.Properties), "OverwriteBuildCommand") ? lambda.Properties.OverwriteBuildCommand : element(split(".", basename(lambda.Properties.Code)), 1) == "zip" ? "zip -r ${element(split(".", basename(lambda.Properties.Code)), 0)}.zip ." : "zip ${element(split(".", basename(lambda.Properties.Code)), 0)}.zip ${element(split(".", basename(lambda.Properties.Code)), 0)}.py" ....... } } ]) lambda-inputs = { for item in local.lambda-params : item.key => item.value } Handle config parameters and generate output resource parameters
20. Example Case : Lambda Build & Deploy (1/5) tree ├── config │ ├── lambda.yaml │ ├── stepfunctions.yaml │ ├── .... ├── dev │ ├── .... ├── prod │ ├── backend.tf │ ├── lambda.tf -> ../lambda.tf │ ├── lambda_locals.tf -> ../lambda_locals.tf │ ├── import_lambda.tf │ ├── local.tf │ ├── .... ├── lambda.tf ├── lambda_locals.tf ├── ....
21. Example Case : Lambda Build & Deploy (2/5) resource "terraform_data" "build_lambda_package" { for_each = local.lambda-inputs # build only if lambda_version is updated triggers_replace [ each.value.lambda_version ] provisioner "local-exec" { # Default build command is zip operation.This can be overwrite by BuildCommand in lambda.yaml. BuildCommand will be used for requirements.txt installation.. lambda layer creation etc. command = <<-EOF cd ../../${each.value.code_dir_path} && ${each.value.build_command} EOF on_failure = fail } } Triggers execute only on config version tag changes Create Lambda packages(zip) Processing input with a loop
22. Example Case : Lambda Build & Deploy (3/5) !/bin/bash set -eux docker build --platform linux/x86_64 --no-cache -t lambda-img . docker rm -f lambda-container docker run -itd --name lambda-container lambda-img docker cp lambda-container:/app/Pipfile.lock ./Pipfile.lock docker cp lambda-container:/app/requirements.txt ./requirements.txt docker cp lambda-container:/app/python ./python zip -r package.zip ./python rm -r ./python Lambda layer creation is also automated, making version upgrades easier
23. Example Case : Lambda Build & Deploy (4/5) resource "terraform_data" "upload_lambda_package" { for_each = local.lambda-inputs depends_on = [ null_resource.build_lambda_package, ] # upload only if lambda_version is updated triggers_replace [ each.value.lambda_version ] provisioner "local-exec" { command = <<-EOF aws s3 cp ../../${each.value.code_dir_path}/${each.value.package_name}.zip s3://${local.bucket_name}/lambda_build_packages/${each.value.function_name}/${each.value.package_name}_${each.value.lambda _version}.zip --content-type \"text/plain\" EOF on_failure = fail } } Upload the created package (zip) to S3
24. Example Case : Lambda Build & Deploy (5/5) resource "aws_lambda_function" "lambda_function" { for_each = local.lambda-inputs depends_on = [ null_resource.upload_lambda_package, aws_lambda_layer_version.lambda_layer ] function_name = each.value.function_name s3_key = "lambda_build_packages/${each.value.function_name}/${each.value.package_name}_${each.value.lambda_version}.zip" dynamic "vpc_config" { for_each = { for key, val in each.value : key => val if key == "vpc_config" && val != null } content { security_group_ids = vpc_config.value.SecurityGroupIds subnet_ids = vpc_config.value.SubnetIds } } lifecycle { ignore_changes = [ last_modified, tags, tags_all ] } Specify the version tag in the S3_key, deploy only when it changes Exclude tags managed by the audit team Dynamically generate blocks based on the configuration file parameters
25. Terraform Implementation Tips Terraform feature
26. Terraform Feature : Drift Detection # aws_lambda_function.lambda_function["sample_1"] will be updated in-place ~ resource "aws_lambda_function" "lambda_function" { id = "sample_1" tags = {} ~ timeout = 900 -> 3 # (28 unchanged attributes hidden) # (3 unchanged blocks hidden) } Plan: 0 to add, 1 to change, 0 to destroy. If resources have changed manually, differences can be detected with terraform plan including default value
27. Terraform Feature : Resource Generation # __generated__ by Terraform # Please review these resources and move them into your main configuration files. # __generated__ by Terraform resource "aws_lambda_function" "lambda_function" { architectures = ["x86_64"] code_signing_config_arn = null description = null filename = null function_name = "sample_1" handler = "lambda_function.lambda_handler" image_uri = null kms_key_arn = null layers = [] memory_size = 128 package_type = "Zip" - terraform plan -generate-config-out=<filename>.tf Output the terraform definition of existing resources
28. Terraform Feature : Resource Import locals { lambda_names = { "sample_1" = "" "sample_2" = "" } } import { for_each = local.lambda_names id = each.key to = aws_lambda_function.lambda_function[each.key] } Efficiently import resources using for_each and safely verify differences
29. Terraform Feature : S3 Backend Native Lock terraform { backend "s3" { bucket = "terraform-state" key = "terraform.tfstate" use_lockfile = true dynamodb_table = "state-lock-table" } } DynamoDB is no longer need from v1.10 and this aligns with our organizations policy - terraform force-unlock <LOCK_ID> Manually unlock the state file State locking prevents conflicts by allowing only one operation to modify the state file at a time
30. Terraform Feature : Ephemeral Values ephemeral "random_password" "db_password" { length = 16 override_special = "!#$%&*()-_=+[]{}<>:?" } resource "aws_secretsmanager_secret" "db_password" { name = "db-password" } resource "aws_secretsmanager_secret_version" "db_password" { secret_id = aws_secretsmanager_secret.db_password.id secret_string_wo = ephemeral.random_password.db_password.result secret_string_wo_version = 1 } ephemeral "aws_secretsmanager_secret_version" "db_password" { secret_id = aws_secretsmanager_secret_version.db_password.secret_id } resource "aws_db_instance" "example" { instance_class = "db.t3.micro" engine = "postgres" username = "example" password_wo = ephemeral.aws_secretsmanager_secret_version.db_password.secret_string password_wo_version = aws_secretsmanager_secret_version.db_password.secret_string_wo_version } Secrets will no longer remain in the Plans output or State by using the ephemeral resource https://www.hashicorp.com/ja/blog/ephemeral-values-in-terraform
31. Terraform Contribute Opportunity • terraform-provider-aws ◦ Add an option to forcibly delete Event Bridge Rules ▪ https://github.com/hashicorp/terraform-provider-aws/pull/34905 • terraform-provider-databricks ◦ Sort based on the Task Key specified in the 'Depends On' field ▪ https://github.com/databricks/terraform-provider-databricks/pull/3000 ▪ https://github.com/databricks/terraform-provider-databricks/pull/3183 ◦ Fix sql table column type upper and lower mismatch error ▪ https://github.com/databricks/terraform-provider-databricks/pull/3501 ◦ Added an item to check if the ticket opener wants to do a bug fix ▪ https://github.com/databricks/terraform-provider-databricks/pull/3020
32. Summary • AXA manages AWS and Databricks with Terraform • Terraform provided the following benefits: ◦ Improved development productivity through automation and standardization of IaC tools ◦ Resolve operational issues by automating processes and using Terraform features ◦ Terraform now enables us to quickly and safely manage manually created resources • AXA uses Terraform to keep delivering value and contribute to the Terraform community
33. Thank you