A conversation about your code to build features in a fraction of the time

CloudConf, Turin, May 29th, 2024

Generative artificial intelligence (AI) can accelerate all phases of your software development lifecycle (SDLC) and streamline tasks such as research, design, coding, testing, debugging, and troubleshooting. This approach can extend to modernization, including upgrades to the latest language version. Through practical demos, this session will show how to incorporate cutting-edge technologies into your daily workflow, unlocking new levels of productivity and efficiency.

Transcript

1. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. A conversation about your code to build features in a fraction of the time Danilo Poccia (he/him) Chief Evangelist (EMEA) @danilop 1
2. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. A brief conversation with about generative AI 2 The capital of France is…
3. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI – Base models 3 The capital of France is Paris. User AI
4. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI – Instruction tuning 4 What is the capital of France? The capital of France is Paris. User Assistant
5. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI – Hallucinations 5 What are the two capitals of France? The two capitals of France are Paris and Strasbourg. Paris is the capital of France and the seat of the French national government, while Strasbourg is the capital of the Grand Est region and the seat of the European Parliament and other European institutions. O lder m odels 🫠
6. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Generative AI – Guardrails and checks 6 What are the two capitals of France? France has only one capital city, which is Paris. There are no other cities considered capital cities of France. Paris has been the capital and seat of government for centuries. N ew er m odels 👍
7. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do large language models (LLMs) work? 7 Text Model Text The model tries to continue a sentence (base model) or follow instructions (instruction tuning) What is the capital of France? The capital of France is Paris.
8. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do large language models (LLMs) work? 8 Text Model Text What is the capital of France? Oh, Paris, such a beautiful capital for France! Temperature (Randomness) Temperature controls how randomly next tokens (words) are chosen
9. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How do large language models (LLMs) work? 9 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers
10. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 10 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers Embeddings take something else and convert it into “numbers” (text, image, sound) → numbers + meaning
11. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 11 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk
12. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 12 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Similar concepts → Near
13. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 13 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Something in between
14. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 14 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors
15. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 15 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors
16. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 16 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x 2D Embeddings Vectors → Numbers
17. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 17 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x (3, 3.5) (4, 3) (5, 2) 2D Embeddings
18. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Embeddings 18 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) (5, 2, 1.6, 4, …) N-Dimensional Embeddings 2D are not enough
19. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. (5, 2, 1.6, 4, …) (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) How do large language models (LLMs) work? 19 Text Model Input Context (Embeddings) Output (Embeddings) Text What is the capital of France? The capital of France is Paris.
20. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input How do multimodal models work? 20 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Multimodal models are trained with a mix of different kind of inputs, such as text and images, including relations connecting different types, such as textual descriptions of images.
21. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input Conversations? 21 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Models have no memory
22. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input Conversations 22 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Models have no memory
23. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Conversations – Input context 23 What is the capital of France? The capital of France is Paris. What about Sweden? The capital of Sweden is Stockholm. User Assistant User Assistant Input Output
24. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Conversations – Lack of context 24 What about Sweden? Sweden is a nice country. User Assistant Input Output
25. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input Conversations and agents 25 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat)
26. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input 26 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Retrieval Augmented Generation Additional data Vector Index
27. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input 27 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append + Plan (Chat / Agent) Retrieval Augmented Generation Additional data Vector Index Agent Plan / Chain of Thoughts 1. First, I need to do this 2. Then, I can do that 3. Finally, I can get the result
28. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input 28 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Read / Write APIs Web Search Content Repositories (CRM, Emails, …) Indexed Content (Wikipedia, …) Access Databases (SQL, NoSQL)
29. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Output Input 29 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Agent Plan + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result
30. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Guardrails Output Input 30 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) M ental m odel Agent Plan + RAG + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result
31. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Guardrails Output Input 31 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Context size
32. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 32 Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in- the-haystack eval. …this sentence seems very out of place and unrelated to the rest of the content in the documents… I suspect this pizza topping “fact” may have been inserted as a joke or to test if I was paying attention. Context size
33. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. How can generative AI help software developers? 33
34. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 34 From: boss@example.com To: me@example.com Subject: Important news
35. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 35 Can you look after the customer feedback app? It’s the oldest app in production in our company! Don’t worry, we don’t expect many changes. It’ll mostly be maintenance…
36. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. One week later… 36
37. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 37 From: boss@example.com To: me@example.com Subject: More important news
38. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 38 The new Trouble Ticket platform is strategic for our company! Can you connect the old customer feedback app with the new Trouble Ticket SaaS? It’d be great to have the integration ready by the end of the quarter…
39. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. And our adventure begins… 39
40. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON)
41. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 41
42. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 42
43. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 43
44. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 44
45. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON)
46. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java 17 application External API TT API Feedback message (JSON) Docker “feedback”
47. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 47
48. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration
49. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 49
50. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration
51. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback”
52. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 52
53. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 53
54. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 54
55. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 55 Trouble Ticket SaaS REST API HTTP POST/PUT/GET/DELETE { "id": "123", "priority": "low, medium, high", "title": "Ticket Title", "message": "Long message or email" } { "ticket_id": "123", "priority": 1 to 5, "status": "open", "title": "Ticket Title", "body": "Max 200 characters" } Customer Feedback REST API HTTP POST/PUT/GET/DELETE
56. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 56
57. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” CDK “Integration”
58. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) Lambda function New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API Python Lambda function TT API Feedback message (JSON) Docker “feedback” CDK “Integration” Amazon Bedrock Summarization and categorization
59. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization
60. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 60
61. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 61
62. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 62
63. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization
64. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization DynamoDB Table Admin
65. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 65
66. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Done • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 66
67. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Which technologies did we use? 67
68. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 68 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI
69. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 69 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat
70. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 70 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev
71. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 71 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform
72. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 72 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform Code agent + repository Split a complex issue into smaller tasks Agent + RAG + Tools Amazon Q Developer in Amazon CodeCatalyst Issues + Pull Request (PR)
73. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. C A N L A N G U A G E M O D E L S R E S O L V E R E A L - W O R L D G I T H U B I S S U E S ? 73 SWE-bench Leaderboard https://www.swebench.com/
74. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 74 Start with Amazon Q Developer at no cost • Code faster with code suggestions in the IDE § JetBrains § Visual Studio Code § Visual Studio (preview) § CLI (macOS) • Review code licenses with reference tracking • Limited monthly access for advanced features
75. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. 75 You can disable sharing content with AWS
76. © 2024, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! Danilo Poccia @danilop Please give me your feedback on X/Twitter or LinkedIn 😊