Small Pieces, Big Picture: Breaking Down Complexity

Slide 1

Slide 1 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. © 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Small Pieces, Big Picture: Breaking Down Complexity Danilo Poccia (he/him) Chief Evangelist (EMEA) AWS

Slide 2

Slide 2 text

Construction of the “Great Eastern” ship

Slide 3

Slide 3 text

Concept Construction starts 1852 1854 First trip 1860 Suez Canal opens 1869 Scrapped 1888

Slide 4

Slide 4 text

How big is too big? How little is too little?

Slide 5

Slide 5 text

Photo by Kalen Emsley on Unsplash

Slide 6

Slide 6 text

https://github.com/SebLague/Ant-Simulation https://www.youtube.com/@SebastianLague

Slide 7

Slide 7 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Complexity arises when the dependencies among the elements become important.” Scott E. Page, John H. Miller Complex Adaptive Systems

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”

Slide 12

Slide 12 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. “Amazon S3 is intentionally built with a minimal feature set. The focus is on simplicity and robustness.” – Amazon S3 Press Release, March 14, 2006

Slide 13

Slide 13 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon S3 8 → more than 200 microservices Mai-Lan Tomsen Bukovec AWS Vice President (Storage, Streaming, Messaging, and Monitoring/Observability)

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

19 Science Museum Group Collection © The Board of Trustees of the Science Museum Analytical Engine

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

The engine “…might act upon other things besides number” “…the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent” Ada Lovelace, Note A (1843)

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ person. Next word prediction with simple word embedding. ? ? ? ? ? ? ? Sentence:

Slide 26

Slide 26 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ rabbit. Next word prediction with ‘memory’ such as Recurrent Neural Networks (RNN). ? ? ? Sentence:

Slide 27

Slide 27 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Transformers - Attention Is All You Need - 2017 28 “Attention is all you need” Vaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. (2017)

Slide 28

Slide 28 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Transformers - Attention Is All You Need - 2017 29 “Attention is all you need” Vaswani, Ashish & Shazeer, Noam & Parmar, Niki & Uszkoreit, Jakob & Jones, Llion & Gomez, Aidan & Kaiser, Lukasz & Polosukhin, Illia. (2017) The Transformer model architecture. • “Self-attention” enables models to scale the understanding of relationships between words. • Efficiently use parallel computing.

Slide 29

Slide 29 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy _ dog.✓ Next word prediction with transformer based networks. Sentence:

Slide 30

Slide 30 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. The road to generative AI The quick brown fox jumps over the lazy dog. Is an English-language pangram — a sentence that contains all the letters of the alphabet. The phrase is commonly used for touch-typing practice. Next word prediction with transformer based networks. Typing Tips Large Language Models (LLM) ✓✓✓

Slide 31

Slide 31 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rise of the machines Subjective Performance Parameter Count (Model Size) GPT-2 1.5 billion GPT-3 175 billion BLO O M 176 billion BERT 345 m illion ?

Slide 32

Slide 32 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Rise of the machines Subjective Performance Parameter Count (Model Size) ? Foundation Models GPT-2 1.5 billion GPT-3 175 billion BLO O M 176 billion BERT 345 m illion

Slide 33

Slide 33 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Applications for Generative AI • Text generation (many types) • Game design • Industrial design • Drug design research • Software development…

Slide 34

Slide 34 text

Slide 35

Slide 35 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Prompt engineering Write a Blog How to win at Tic Tac Toe Title: Game theory Topic: 10 Length (paragraphs): OK “You are a journalist for amazing-acme- blogs.com, an online publication for sophisticated game players aged 18 and up. Articles are written to inform and entertain and represent a unique and non- intuitive perspective. Write a blog post titled “How to win at Tic Tac Toe” on the topic of game theory. The article should be no longer than 10 paragraphs long. At the end list 3 prompts that can be used to generate images for the article.”

Slide 36

Slide 36 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. chatbot = Chatbot( default_reply_length = 20, bot_persona = pmodel[‘friendly’], users_name =“User”, bots_name = “Chatbot”, topic = “tech support” ) prompt = """This is a friendly and safe chat session between a user and a computer called Chatbot.## User:I am a real person with a question to ask. Who are you?## Chatbot:I am a chatbot, and I am here to help.## User:What kind of questions will you answer?## Chatbot:I will answer questions about tech support.## User:My computer has crashed. What should I do?## Chatbot:""" A new way to program…

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© 2023, Amazon Web Services, Inc. or its affiliates. All rights reserved. Where to from here? • Models will keep getting bigger, and more sophisticated. • Foundation models and prompt engineering are software development tools. • Agents can execute complex business tasks and invoke APIs. • Inference costs and sustainability come into focus.