AI in Software Development

Slide 1

Slide 1 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 AI in Software Development Vortex's Waterhole - June 14, 2023 Midjourney - "a waterhole in the jungle, but instead of animals there are only drones and robots. Movie style" Ivo Jansch [email protected] @ijansch

Slide 2

Slide 2 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 DISCLAIMER By the time we ﬁnish this talk it's probably already outdated Midjourney - "a time machine, dr who style, landed alongside a waterhole in the jungle"

Slide 3

Slide 3 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 Chapter 1: How does Generative AI work? Midjourney - "a brain of a robot, full of neurons. Science fiction style"

Slide 4

Slide 4 text

2005 - 2015: Rise of Deep Learning Introducing layers into the ﬁeld of machine learning: pixels grouped pixels edges green hues … a cat a cat staring a cat staring at you a cat at a waterhole in the jungle

Slide 5

Slide 5 text

2017: Invention of the Transformer Architecture Has nothing to do with The Transformers Has everything to do with a new, scalable architecture that created the ability to process large quantities of data and most importantly: - Self supervised learning - Self attention

Slide 6

Slide 6 text

Understanding attention The cat drank from the waterhole until it was full The cat drank from the waterhole until it was empty

Slide 7

Slide 7 text

Understanding attention The cat drank from the waterhole until it was full The cat drank from the waterhole until it was empty

Slide 8

Slide 8 text

2018: GPT Generative Predict the next word(s) from a sequence of words Pre-trained Trained on a large corpus of text Transformer Built on the transformer architecture

Slide 9

Slide 9 text

Ok but how does it work? The cat drank from the waterhole until it was full Tokenizer creates tokens per layer: T, h, e, c, a, t, d, r, a, n, k, f, r, o, m, t, h, e, … The, cat, drank, from, the, water, hole, … The, cat, drank, from, the, waterhole, … The cat drank, from the waterhole, until it was full, ... All tokens get converted to numbers.

Slide 10

Slide 10 text

The model learns relationships between tokens Cat Drink Pet Water Fat Rat C A T Jungle Rhyme Vast

Slide 11

Slide 11 text

The model learns relationships between tokens Cat Drink Pet Water Fat Rat C A T Jungle Rhyme Vast These are the 'parameters'

Slide 12

Slide 12 text

Then it starts predicting Given a prompt, what is the most likely word that comes next What is a cat? Fat A pet

Slide 13

Slide 13 text

Token by token Given a prompt, what is the most likely word that comes next Input: Why does a cat drink water? Prediction: A cat

Slide 14

Slide 14 text

Token by token by token Given a prompt, what is the most likely word that comes next Input: Why does a cat drink water? A cat Prediction: needs

Slide 15

Slide 15 text

Token by token by token by token Given a prompt, what is the most likely word that comes next Input: Why does a cat drink water? A cat needs Prediction: nutrition

Slide 16

Slide 16 text

Token by token by token by token Given a prompt, what is the most likely word that comes next Done: Why does a cat drink water? A cat needs nutrition

Slide 17

Slide 17 text

Disclaimer That was by no means a scientiﬁcally correct explanation. But explains in a very simpliﬁed way how it works.

Slide 18

Slide 18 text

GPT's are "stochastic parrots" - They "understand" - Yet, they don't truly understand what they are talking about

Slide 19

Slide 19 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 Chapter 2: Application areas in development Midjourney - "a development team sitting around a table, everyone wearing AR glasses. Science fiction style"

Slide 20

Slide 20 text

Increase Coding Productivity

Slide 21

Slide 21 text

Coaching Junior Developers

Slide 22

Slide 22 text

Coaching Junior Developers

Slide 23

Slide 23 text

Designing Data Models LLMs can help with creating data models and normalization.

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

LLMs as tool in a chain SQL Query Document Database Factual Results Summarize LLM Present LLM Translate to query Request in natural language

Slide 26

Slide 26 text

Good old "Rubber Duck Debugging" "In software engineering, rubber duck debugging is a method of debugging code by articulating a problem in spoken or written natural language. The name is a reference to a story in the book The Pragmatic Programmer in which a programmer would carry around a rubber duck and debug their code by forcing themselves to explain it, line by line, to the duck.

Slide 27

Slide 27 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 Chapter 3: The dark side Midjourney - "a postapocalyptic city run by AI, full of drones. Humans have disappeared or are enslaved. Photorealistic image"

Slide 28

Slide 28 text

Hallucinations

Slide 29

Slide 29 text

Hallucinations

Slide 30

Slide 30 text

Hallucinations

Slide 31

Slide 31 text

Hallucinations

Slide 32

Slide 32 text

Hallucinations

Slide 33

Slide 33 text

What probably happened Tokens 5+ real methods starting with SecKeyCopy

Slide 34

Slide 34 text

What probably happened Tokens Attention!

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Security concerns Paper: https://arxiv.org/abs/2211.03622

Slide 37

Slide 37 text

Environmental concerns

Slide 38

Slide 38 text

Environmental concerns Source: https://blogs.nvidia.com/blog/2022/03/25/what-is-a-transformer-model

Slide 39

Slide 39 text

Existential concerns FAKE NEWS Asimov's first law of robotics: "a robot shall not harm a human, or by inaction allow a human to come to harm"

Slide 40

Slide 40 text

Back to reality! AI tools are useful, but: ● Ensure human supervision -> Code reviews ● Be transparent about the use of AI ● Pay attention to Terms & Conditions of the tools you use ● Keep an eye on copyright legislation ● Choose the right tool for the job ○ ChatGPT is not the only player in town ○ Consider open source alternatives such as LLAMA ■ https://github.com/eugeneyan/open-llms

Slide 41

Slide 41 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 Chapter 4: The future Midjourney - "A robot and a human holding hands, watching the sunset in the distance. Science fiction style"

Slide 42

Slide 42 text

Will AI replace developers? I gave it a try:

Slide 43

Slide 43 text

Let's see how far we can take this…

Slide 44

Slide 44 text

Let's see how far we can take this…

Slide 45

Slide 45 text

At this point, ChatGPT is lying through the teeth

Slide 46

Slide 46 text

Yay!

Slide 47

Slide 47 text

Now the tricky part…

Slide 48

Slide 48 text

But ChatGPT is easily convinced…

Slide 49

Slide 49 text

Chugging along…

Slide 50

Slide 50 text

And we're done! Little apple, little egg

Slide 51

Slide 51 text

Ouch…

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

AI PROOF!

Slide 54

Slide 54 text

Will AI replace developers? Visual Basic didn't make developers obsolete Outsourcing didn't make developers obsolete No-code systems didn't make developers obsolete AI won't make developers obsolete

Slide 55

Slide 55 text

Will AI replace developers? Development is so much more than producing code. AI will make programming more productive, but we will still require software engineering.

Slide 56

Slide 56 text

Vertrouwelijk Aangepast voor naam van het bedrijf Versie 1.0 THANK YOU!!! Vortex's Waterhole - June 14, 2023 Midjourney - "a waterhole in the jungle, but instead of animals there are only drones and robots. Movie style" Ivo Jansch [email protected] @ijansch