Google I/O extended 2024 - Gemini: Exploration of Multimodality and Improvement through Code

Slide 1

Slide 1 text

Gemini: Exploration of multimodality and improvement through code Women Techmakers Montreal Team

Slide 2

Slide 2 text

Women Techmakers is a program created by Google to celebrate International Women’s Day by highlighting the talent of women in technology. Our mission at Women Techmakers Montreal? To provide inspiration to start or continue a career in technology and help establishing a solid networking to realize your professional project beyond the day of the event. Hello! Become a volunteer!

Slide 3

Slide 3 text

Dr. Stefania Pecore From a university degree in foreign languages to a Ph.D. in CS, programming, and Natural Language Processing. With a passion for leveraging technology to enhance communication, she played a pivotal role in assisting numerous enterprises across diverse sectors—education, manufacturing, biotech, and even venturing into video games—by integrating NLP solutions into their products. Eager to share her insights, she became a mentor for women, holding positions as a Google Women Techmakers, Women in AI, and Women in Games Ambassador. In 2024, she became lead of Google Women Techmakers Montréal Chapter and she was also appointed as an AI Expert for UNESCO's Women4Ethical AI allowing her to further apply her expertise in fostering responsible AI and supporting Ethical approaches in AI systems creation. Speakers

Slide 4

Slide 4 text

With a degree in Civil Engineering and 3x Google Cloud Certified, she recently pursued an Artificial Intelligence course in Montreal and is currently pursuing a postgraduate degree in MLOps. Driven by a passion for solving problems with technology, she has contributed to diverse projects in education, media, investments funds and startups using GCP. She runs her own company where she serves as a Data Scientist Consultant and ML Instructor. She believes strongly in the power of education, which is why she is committed to teaching and empowering others through her journey. Beyond algorithms and code, she is equally passionate about community and mentorship. As an Ambassador for Women Techmakers, Alura Stars Ambassador and a mentor for Mulheres em Dados, she is dedicated to empowering more women in tech and fostering a diverse, inclusive community. Leticia Pires Speakers

Slide 5

Slide 5 text

A software engineering student, she joined Women Techmakers two years ago, demonstrating her commitment to building a more inclusive tech space. As the founder lead of Google Developer Student Clubs (GDSC) at Concordia, Hanine commits to fostering a vibrant tech community on campus through workshops and events around various areas of technology. Hanine also established a mentorship program, pairing students with experienced mentors to provide them with support, guidance, tech skills and career advice. Her experience spans across backend development and cloud integration to secure connections between servers and applications. She also dabbles in frontend development, developing user interfaces that are both functional and visually appealing. Hanine's curiosity extends to the fields of robotics and system controls. Hanine Tydrini Speakers Speakers

Slide 6

Slide 6 text

Agenda 01 02 03 Gemini 1.0 versus Gemini 1.5 (theory) Gemini 1.0 versus Gemini 1.5 (practice) The multimodal world of Gemini Google I/O Extended - Speaker slides

Slide 7

Slide 7 text

Gemini 1.0 versus Gemini 1.5 (theory) Speaker: Hanine Tydrini

Slide 8

Slide 8 text

Transformer-based architectures Relies on a single, large neural network architecture with multiple layers (encoders and decoders) for processing sequential data like text. Sparse Mixture-of-Experts (MoE) Decomposes the traditional Transformer architecture into a collection of smaller, specialized "expert" models. A gating network then determines which expert(s) handle specific parts of the input sequence. Gemini 1.5 Gemini 1.0 Architecture Caption Imagine a large team of researchers working on a complex project. Each researcher has expertise in different areas, but they all need to communicate and collaborate to complete the project. Caption This would be like dividing the project into smaller sub-projects and assigning each sub-project to a team of specialized researchers. A coordinator then ensures each team gets the relevant information and collaborates effectively.

Slide 9

Slide 9 text

32,000 tokens Public: 128 000 tokens Private & devs: 1 million tokens Available for waitlist: 2 million Private & devs: 1 million tokens Available for waitlist: 2 million Tested for up to 10 million tokens 1.0 Pro 1.5 Pro 1.5 Flash Larger Context Window

Slide 10

Slide 10 text

Text Images Audio Code Multimodal Capacities Input Pipeline Remains the primary input format Convolutional Neural Networks (CNNs): expert image analysts who extract high-level features like shapes, edges, and objects Mel-Frequency Cepstral Coefficients (MFCCs) transform the raw audio into a numerical representation that captures its characteristics Syntax parsing and code embeddings

Slide 11

Slide 11 text

Enhanced Domain-Specific Knowledge with Gemini 1.5 The model has been trained with a more extensive and diverse dataset, allowing it to excel in specialized areas such as medical, legal, and technical domains.

Slide 12

Slide 12 text

87% When tested on a panel of text, code, image, audio and video evaluations 1.5 Pro outperforms 1.0 Pro on of the benchmarks

Slide 13

Slide 13 text

Lorem ipsum dolor sit amet consectetur Mar 14 Mar 14 Mar 14 July 15 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 14 Apr 14 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Mar 31 Mar 31 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 7 Apr 7 May 31 Lorem ipsum dolor sit amet consectetur May 24 June 21 June 28 Sept 6-7 US EMEA IN CN Key Milestones TABLE ELEMENTS

Slide 14

Slide 14 text

Lorem ipsum dolor sit amet consectetur hfujii@ Lorem ipsum dolor sit amet consectetur sodhia@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur lhflores@ Lorem ipsum dolor sit amet consectetur lhflores@ Label Owner TABLE ELEMENTS

Slide 15

Slide 15 text

Label 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. ● Lorem ipsum dolor sit amet ● Consectetur adipiscing elit ● Sed do eiusmod tempor incididunt ut labore. Label 2 Label 3 Label 4 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. ● Lorem ipsum dolor sit amet ● Consectetur adipiscing elit ● Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. ● Lorem ipsum dolor sit amet ● Consectetur adipiscing elit ● Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. ● Lorem ipsum dolor sit amet ● Consectetur adipiscing elit ● Sed do eiusmod tempor incididunt ut labore. TABLE ELEMENTS

Slide 16

Slide 16 text

Gemini 1.0 versus Gemini 1.5 (practice) Speaker: Dr. Stefania Pecòre

Slide 17

Slide 17 text

Aimed at individual developers and teams, it’s a web-based tool that “enables you to quickly develop prompts and then get an API key to use in your app development.” The “Get code” button lets you transfer work to your IDE in cURL, JavaScript, Python, Android (Kotlin), and Swift. Google AI Studio - what’s this? Montréal

Slide 18

Slide 18 text

I have 3 apples today, yesterday I ate an apple. How many apples do I have now? Quiz!

Slide 19

Slide 19 text

Gemini 1.5 Gemini 1.0 Test: how many apples?

Slide 20

Slide 20 text

What’s heavier: a kilo of feathers or a pound of steel? Quiz!

Slide 21

Slide 21 text

Gemini 1.5 Flash Gemini 1.0 Test: What’s heavier? Gemini 1.5 pro

Slide 22

Slide 22 text

Gemini 1.0 Gemini 1.5 Test: Apples… again!

Slide 23

Slide 23 text

Gemini 1.5 Flash Test: Needle In a Haystack

Slide 24

Slide 24 text

Gemini 1.5 Pro Test: Needle In a Haystack

Slide 25

Slide 25 text

The multimodal world of Gemini Speakers: Dr. Stefania Pecòre & Leticia Pires

Slide 26

Slide 26 text

Scan here for the code

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Thank You Hanine Tydrini Software Student Dr. Stefania Pecòre AI Director Leticia Pires Data Scientist

Slide 29

Slide 29 text

Our palette uses the Google primaries mixed as a gradient. A Background Grey should be used as to not overwhelm an audience with pure RGB white. Outlines for UI elements are set in 2.5pt in Black (Grey 900). All type should be Black (Grey 900) and highlights/color pops should be gradients. Primaries Blue 500 Hex #4285F4 RGB 66, 133, 244 CMYK 88. 40, 0, 0 PMS 2173 Coated PMS 2193 Uncoated Blend Only Colors Orange Hex #F46831 Purple Hex #9F6CD4 Yellow Hex #FFCB32 Neutrals Background Grey Hex #EEEEEE RGB 238, 238, 238 CMYK 5,4,5,0 PMS Cool Grey 1 Coated PMS Cool Grey 1 Uncoated Grey 900 Hex #202124 RGB 32, 33, 36 CMYK 5,0,0,90 PMS 426 Coated PMS 426 Uncoated White Hex #FFFFFF RGB 255, 255, 255 Green 500 Hex #34A853 RGB 52, 168, 83 CMYK 85, 0, 92, 0 PMS 2257 Coated PMS 2271 Uncoated Yellow 500 Hex #FBBC04 RGB 251, 188, 4 CMYK 0. 37, 100, 0 PMS 7549 Coated PMS 7549 Uncoated Red 500 Hex #EA4335 RGB 234, 67, 53 CMYK 0, 87, 89, 0 PMS 179 Coated PMS 2347 Uncoated Google I/O Extended - Colors

Slide 30

Slide 30 text

Section title slides

Slide 31

Slide 31 text

Short Title Google Sans Medium, 120px Long Title Google Sans Medium, 90px Section Title Google Sans Medium, 75px Headline Google Sans Medium, 60px Subhead Google Sans Medium, 36px Body Copy Google Sans Normal, 20px Typography General guidelines for Font styling and size. Code Google Sans Mono Normal, 29px Google I/O Extended - Typography

Slide 32

Slide 32 text

Google I/O Extended - Logos

Slide 33

Slide 33 text

Google I/O Extended - Editable Logos {Change Location} {Change Location}

Slide 34

Slide 34 text

Google I/O Extended - Google Logos

Slide 35

Slide 35 text

Title slides

Slide 36

Slide 36 text

Optional tag here Here is a short title for a talk.

Slide 37

Slide 37 text

Optional tag here Here is a longer title. A title length should not exceed three lines.

Slide 38

Slide 38 text

Section title slides

Slide 39

Slide 39 text

Supporting text if needed A section title should not exceed two lines Section 1

Slide 40

Slide 40 text

Supporting text if needed A section title should not exceed two lines Section 2

Slide 41

Slide 41 text

Supporting text if needed A section title should not exceed two lines Section 3

Slide 42

Slide 42 text

Content slides

Slide 43

Slide 43 text

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Headline goes here Right-click and select ‘Replace image’ Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Section 3

Slide 46

Slide 46 text

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Duis aute irure dolor in sunt reprehenderit in voluptate ea commodo consequat deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Right-click and select ‘Replace image’ Headline goes here Section #

Slide 47

Slide 47 text

Headline goes here Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Right-click and select ‘Replace image’ Section #

Slide 48

Slide 48 text

This is body copy. Connect with fellow developers and startups in your area, hear from industry experts, share your skills, and learn new ones. This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events. This is a subhead This is a subhead Headline goes here Section #

Slide 49

Slide 49 text

Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Headline goes here Section #

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

“Good design is as little design as possible.” Dieter Rams

Slide 53

Slide 53 text

This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. This is body copy. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects for others to learn from. Recommendation Learnings Section #

Slide 54

Slide 54 text

This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Learnings 200 Statistic Caption Section #

Slide 55

Slide 55 text

1.5B Statistic caption this is body copy and it goes a little like this

Slide 56

Slide 56 text

1.5B Statistic caption this is body copy and it goes a little like this

Slide 57

Slide 57 text

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consectetur velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consectetur velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Right-click and select ‘Replace image’ Lorem Ipsum

Slide 58

Slide 58 text

Right-click and select ‘Replace image’

Slide 59

Slide 59 text

Right-click and select ‘Replace image’ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consectetur velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Section #

Slide 60

Slide 60 text

Slide 61

Slide 61 text

86% Statistic Caption 14% Statistic Caption This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Learnings Section #

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Connect with fellow developers and startups in your area, hear from industry experts, share your skills, and learn new ones. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. 1 billion 2 billion Bringing developers together in-person and online. Stay in the know about upcoming events. 100,000 Bringing developers together in-person and online. Stay in the know about upcoming events. Connect with fellow developers and startups in your area, hear from industry experts, share your skills, and learn new ones. 20 million 2,000 Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. 10 Section #

Slide 64

Slide 64 text

This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Tablet Slide Right-click and select ‘Replace image’ Section #

Slide 65

Slide 65 text

Image in tablet caption Right-click and select ‘Replace image’ Section #

Slide 66

Slide 66 text

This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Headline Right-click and select ‘Replace image’ Section #

Slide 67

Slide 67 text

Right-click and select ‘Replace image’ This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Headline Section #

Slide 68

Slide 68 text

Embed a video 1. Select video under the insert dropdown. 2. Paste in a video URL. You will have to resize the thumbnail once it is on this slide. 3. Or Select a video that has been uploaded to Google Drive. Make sure video is shared in the settings. 4. Ensure captions are active when playing 5. Add a copy of the original video file in the Google Drive folder for this session

Slide 69

Slide 69 text

Displaying code

Slide 70

Slide 70 text

code_slides.txt // A couple more notes on code slides: // *Use the Google Sans Mono font // *Set line spacing to 1.2 // *Don’t use font sizes below 25 Code is set in Google Sans Mono

Slide 71

Slide 71 text

code_slides.txt protected void onTryUpdate(int reason) throws RetryException { // Do some awesome stuff int foo = 15; publishArtwork(new Artwork.Builder() .title(photo.name) .imageUri(Uri.parse(photo.image_url)) .viewIntent(new Intent(Intent.ACTION_VIEW, Uri.parse("http://500px.com/photo/" + photo.id))) .build()); scheduleUpdate(System.currentTimeMillis() + ROTATE_TIME_MILLIS); } Use this template for code snippets longer than 6 lines Copy / paste your code using this tool for formatting using the ‘Dark Alternate’ theme

Slide 72

Slide 72 text

Use this style to highlight code code_slides.txt protected void onTryUpdate(int reason) throws RetryException { // Do some awesome stuff int foo = 15; publishArtwork(new Artwork.Builder() .title(photo.name) .imageUri(Uri.parse(photo.image_url)) .viewIntent(new Intent(Intent.ACTION_VIEW, Uri.parse("http://500px.com/photo/" + photo.id))) .build()); scheduleUpdate(System.currentTimeMillis() + ROTATE_TIME_MILLIS); }

Slide 73

Slide 73 text

Image with code snippet * Use Google Sans Mono * Set line spacing to 1.2 * Don’t use font sizes below 25

Slide 74

Slide 74 text

Image with code snippet * Use Google Sans Mono * Set line spacing to 1.2 * Don’t use font sizes below 25

Slide 75

Slide 75 text

Thank you slides

Slide 76

Slide 76 text

Thank You Speaker Name pronouns Job Title

Slide 77

Slide 77 text

Speaker Name pronouns Job Title Speaker Name pronouns Job Title Speaker Name pronouns Job Title Speaker Name pronouns Job Title Thank You

Slide 78

Slide 78 text

Resources Charts, tables, graphs

Slide 79

Slide 79 text

Lorem Ipsum May 24, 2023 2,000 Lorem Ipsum June 21, 2023 1,000 Lorem Ipsum June 28, 2023 1,200 Lorem Ipsum Sept 6-7, 2023 2,000 TOTAL 6,200 Lorem Lorem Lorem TABLE ELEMENTS 2px Border No border on outside of table

Slide 80

Slide 80 text

Slide 81

Slide 81 text

Slide 82

Slide 82 text

Slide 83

Slide 83 text

Caption Large Box Caption 30px Caption 4px Object Border when stroke only Medium Box Caption Caption Caption Small Box Caption Use for nested items only 23px Caption Label s Label Label Arrows Simple Charts: 4px 19px Caption 23px Caption CHART ELEMENTS Caption Caption Caption Caption Caption

Slide 84

Slide 84 text

Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label CHART ELEMENTS

Slide 85

Slide 85 text

Step B Short Label Step D Step E Step A Step C CHART ELEMENTS

Slide 86

Slide 86 text

Column Label Caption 10 Two lines Caption 9 Two lines Caption 8 Two lines Caption 7 Two lines Caption 6 Two lines Caption 5 Two lines Caption 4 Two lines Caption 3 Two lines Caption 2 Two lines Caption 1 Two lines Group Label Column Label Column Label Column Label CHART ELEMENTS

Slide 87

Slide 87 text

Short Label Short Label Short Label Short Label Short Label Short Label Short Label CHART ELEMENTS

Slide 88

Slide 88 text

Label One Short Label Short Label Short Label Label Two Label Three Label Four Label Five Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label CHART ELEMENTS

Slide 89

Slide 89 text

2012 2014 2016 2018 2020 Category Label One Caption 1 Two lines Caption 3 Two lines Caption 2 Two lines Caption 5 Two lines Caption 4 Two lines Caption 6 Two lines Caption 7 Two lines Caption 9 Two lines Caption 8 Two lines Category Label Two CHART ELEMENTS