AWS Community Day Thailand 2026: From Bedrock to EKS: A GenAI Maturity Path

Transcript

1. THAILAND
2. From Bedrock to EKS: A GenAI Maturity Path V-ris Jaijongrak | 3 Oct 2026
3. About Me Senior Platform Engineer at Pi Securities PCL All AWS Certified (13 x) CNCF Certified Golden Kubestronaut (12 x) Main Responsibilities: - CI/CD (ArgoCD, Github Actions) - EKS - Cloud Platform Engineer (AWS) https://www.linkedin.com/in/v-ris-jaijongrak/
4. Agenda - LLM Concepts Managed GenAI with AWS Self-hosting solutions Demo Conclusion
5. Key Takeaways Start managed. Go self-hosted when the requirements make the trade worthwhile. - Managed first - Measure - Define the trigger - Operate the self-hosted
6. LLM Concepts
7. From prediction to generation Machine Learning Transformers - Pattern Recognition using ML techniques (e.g. NN, SVM, PCA) NN-based that transforms the input sequence into output sequence then uses context and relationship in the model to process - Generic Pre-Trained transformers (GPT) are then commonly used for GenAI tasks Generative Model
8. Generative ≠ random (≠ deterministic) Prompt • • • • LLM Response Generative is still following a process that uses context and prompt mapping over probability distribution to generate appropriate response Sampling parameters can reduce or increase output variability. Even with low randomness, outputs are not equivalent to a traditional deterministic function. Production systems therefore need evaluation, guardrails, and observability.
9. Managed GenAI with AWS
10. Bedrock or SageMaker Bedrock (API-first) – if the requirements do not stray away from mainstream models made available from Bedrock (i.e., no need for customization and calling inference API is acceptable) Sagemaker (Code-first MLOps platform) – develop your own model and deploy on AWS
11. Bedrock: Choose before you build - Confirm the model, availability suiting your requirements first (also in another region if you need to have multi-region to elevate throttles and HA for region black-out)
12. Bedrock: Demo (skipped) Client API Gateway Lambda Bedrock Converse API Terraform code: https://github.com/guxkung/bedrock-chat-demo
13. Bedrock: IaC make it simpler to adjust (skipped) variable "model_id" { default = "..." # choose a low-cost model } variable "system_prompt" { default = "You are a helpful assistant..." } resource "aws_lambda_function" "inference" { ... } # Lambda → bedrock-runtime:Converse
14. “with great power there must also come—great responsibility!” – Spiderman When Managed stops fitting Unfortunately, Managed service also comes with restrictions. Some not suitable for real-world use-cases. For example, - Security compliance mandate - Cost-specific limitations - Integration with existing systems limitation Control Compliance The trade: more control Scaling Economics more platform responsibility. … etc.
15. Self-hosting solutions
16. Self-hosting: ECS or EKS? ECS EKS - - - “Simpler” for orchestration AWS-native Might be cheaper (depending on scale/size) Operate via AWS-CLI or Console - “Kubernetes” ecosystem Cloud-native Control-plane and worker nodes require version upgrade regarding K8s lifecycle Operate with kubectl (k8s-api)
17. LiteLLM (AI Gateway) A very critical component to control access towards our provisioned LLM. With respective to: - Runtime security (Guardrails): ensure that the context respect the restrictive topics, secrets, etc. - Authentication: control who has access to the provisioned LLM (optionally, which models) - Intelligent routing: able to optimize the AI model servicing the request - Observability: service quality metrics, request tracing to ensure compliance - Rate limit: throttle the request rate for user for cost/request hogging - Cost control: throttle the service based on the cost spent
18. vLLM: the inference serving engine “Easy, fast, and cost-efficient LLM serving for everyone” - Opensource (Apache-2.0 license) - Also support Kubernetes/Docker - Various open models can be deployed and served via vLLM
19. Scale the workload — and the compute KEDA – scale the vLLM pods to satisfy the request workloads Karpenter– scale the nodes to satisfy the compute workload request by vLLM
20. EKS Auto Mode: another operating point Scaling solution can be “outsourced” to AWS EKS with Auto Mode. (EKS will manage the scaling of node provisioning.
21. Demo — self-hosted inference on EKS Demo — self-hosted inference on EKS Terraform code: https://github.com/guxkung/eks-vllm-demo
22. Demo — self-hosted inference on EKS Demo — self-hosted inference on EKS
23. Demo — self-hosted inference on EKS
24. Conclusion - - Managed first: tailored to suit requirements, if possible. Experiment around if required Measure: all the metrics (cost, latency, quality, governance, operational constraints) Trigger: would the managed service satisfy the requirements? Will the selfhosted option better suited in any “measurable” objective? Migration: beware, it is more of a platform instead of one container