Small Pieces, Big Picture: Breaking Down Complexity

AWS Community Day, Rome, October 18th, 2023

Opening keynote, from complexity to generative AI.

Transcript

1. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Small Pieces, Big Picture: Breaking Down Complexity Danilo Poccia (he/him) Chief Evangelist (EMEA) AWS
2. Construction of the “Great Eastern” ship
3. Concept Construction starts 1852 1854 First trip 1860 Suez Canal opens 1869 Scrapped 1888
4. How big is too big? How little is too little?
5. Photo by Kalen Emsley on Unsplash
6. https://github.com/SebLague/Ant-Simulation https://www.youtube.com/@SebastianLague
7. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Complexity arises when the dependencies among the elements become important.” Scott E. Page, John H. Miller Complex Adaptive Systems
8. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Monolithic Application Services Microservices
9. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. How Amazon SQS works Front End Back End Metadata Amazon DynamoDB Load Manager
10. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “A complex system that works is invariably found to have evolved from a simple system that worked.” Gall’s Law
11. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”
12. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Amazon S3 is intentionally built with a minimal feature set. The focus is on simplicity and robustness.” – Amazon S3 Press Release, March 14, 2006
13. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon S3 8 → more than 200 microservices Mai-Lan Tomsen Bukovec AWS Vice President (Storage, Streaming, Messaging, and Monitoring/Observability)
14. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
15. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Serverless Serverful
16. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved.
18. 19 Science Museum Group Collection © The Board of Trustees of the Science Museum Analytical Engine
20. The engine “…might act upon other things besides number” “…the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent” Ada Lovelace, Note A (1843)
21. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Machine Learning (ML) models Model Input Output Numbers Numbers Embeddings Text, Images, …
22. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI Word Embeddings dog cat car bus
23. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI 24 Word Embeddings dog cat car bus person driver animal vehicle walk drive traffic tea
24. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI Word Embeddings Language Models
25. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ person. Next word prediction with simple word embedding. ? ? ? ? ? ? ? Sentence:
26. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ rabbit. Next word prediction with ‘memory’ such as Recurrent Neural Networks (RNN). ? ? ? Sentence:
27. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Transformers - Attention Is All You Need - 2017 28 “Attention is all you need” Vaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. (2017)
28. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Transformers - Attention Is All You Need - 2017 29 “Attention is all you need” Vaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. (2017) The Transformer model architecture. • “Self-attention” enables models to scale the understanding of relationships between words. • Efficiently use parallel computing.
29. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ dog.✓ Next word prediction with transformer based networks. Sentence:
30. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy dog. Is an English-language pangram — a sentence that contains all the letters of the alphabet. The phrase is commonly used for touch-typing practice. Next word prediction with transformer based networks. Typing Tips Large Language Models (LLM) ✓✓✓
31. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rise of the machines Subjective Performance Parameter Count (Model Size) GPT-2 1.5 billion GPT-3 175 billion BLO O M 176 billion BERT 345 m illion ?
32. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rise of the machines Subjective Performance Parameter Count (Model Size) ? Foundation Models GPT-2 1.5 billion GPT-3 175 billion BLO O M 176 billion BERT 345 m illion
33. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Applications for Generative AI • Text generation (many types) • Game design • Industrial design • Drug design research • Software development…
34. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Write a Blog How to win at Tic Tac Toe Title: Game theory Topic: 10 Length (paragraphs): OK
35. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Prompt engineering Write a Blog How to win at Tic Tac Toe Title: Game theory Topic: 10 Length (paragraphs): OK “You are a journalist for amazing-acme- blogs.com, an online publication for sophisticated game players aged 18 and up. Articles are written to inform and entertain and represent a unique and non- intuitive perspective. Write a blog post titled “How to win at Tic Tac Toe” on the topic of game theory. The article should be no longer than 10 paragraphs long. At the end list 3 prompts that can be used to generate images for the article.”
36. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. chatbot = Chatbot( default_reply_length = 20, bot_persona = pmodel[‘friendly’], users_name =“User”, bots_name = “Chatbot”, topic = “tech support” ) prompt = """This is a friendly and safe chat session between a user and a computer called Chatbot.## User:I am a real person with a question to ask. Who are you?## Chatbot:I am a chatbot, and I am here to help.## User:What kind of questions will you answer?## Chatbot:I will answer questions about tech support.## User:My computer has crashed. What should I do?## Chatbot:""" A new way to program…
37. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Demo
38. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Where to from here? • Models will keep getting bigger, and more sophisticated. • Foundation models and prompt engineering are software development tools. • Agents can execute complex business tasks and invoke APIs. • Inference costs and sustainability come into focus.
39. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you! © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Danilo Poccia @danilop