Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Google I/O extended 2024 - Gemini: Exploration...

GDG Montreal
June 28, 2024
11

Google I/O extended 2024 - Gemini: Exploration of Multimodality and Improvement through Code

GDG Montreal

June 28, 2024
Tweet

More Decks by GDG Montreal

Transcript

  1. Women Techmakers is a program created by Google to celebrate

    International Women’s Day by highlighting the talent of women in technology. Our mission at Women Techmakers Montreal? To provide inspiration to start or continue a career in technology and help establishing a solid networking to realize your professional project beyond the day of the event. Hello! Become a volunteer!
  2. Dr. Stefania Pecore From a university degree in foreign languages

    to a Ph.D. in CS, programming, and Natural Language Processing. With a passion for leveraging technology to enhance communication, she played a pivotal role in assisting numerous enterprises across diverse sectors—education, manufacturing, biotech, and even venturing into video games—by integrating NLP solutions into their products. Eager to share her insights, she became a mentor for women, holding positions as a Google Women Techmakers, Women in AI, and Women in Games Ambassador. In 2024, she became lead of Google Women Techmakers Montréal Chapter and she was also appointed as an AI Expert for UNESCO's Women4Ethical AI allowing her to further apply her expertise in fostering responsible AI and supporting Ethical approaches in AI systems creation. Speakers
  3. With a degree in Civil Engineering and 3x Google Cloud

    Certified, she recently pursued an Artificial Intelligence course in Montreal and is currently pursuing a postgraduate degree in MLOps. Driven by a passion for solving problems with technology, she has contributed to diverse projects in education, media, investments funds and startups using GCP. She runs her own company where she serves as a Data Scientist Consultant and ML Instructor. She believes strongly in the power of education, which is why she is committed to teaching and empowering others through her journey. Beyond algorithms and code, she is equally passionate about community and mentorship. As an Ambassador for Women Techmakers, Alura Stars Ambassador and a mentor for Mulheres em Dados, she is dedicated to empowering more women in tech and fostering a diverse, inclusive community. Leticia Pires Speakers
  4. A software engineering student, she joined Women Techmakers two years

    ago, demonstrating her commitment to building a more inclusive tech space. As the founder lead of Google Developer Student Clubs (GDSC) at Concordia, Hanine commits to fostering a vibrant tech community on campus through workshops and events around various areas of technology. Hanine also established a mentorship program, pairing students with experienced mentors to provide them with support, guidance, tech skills and career advice. Her experience spans across backend development and cloud integration to secure connections between servers and applications. She also dabbles in frontend development, developing user interfaces that are both functional and visually appealing. Hanine's curiosity extends to the fields of robotics and system controls. Hanine Tydrini Speakers Speakers
  5. Agenda 01 02 03 Gemini 1.0 versus Gemini 1.5 (theory)

    Gemini 1.0 versus Gemini 1.5 (practice) The multimodal world of Gemini Google I/O Extended - Speaker slides
  6. Transformer-based architectures Relies on a single, large neural network architecture

    with multiple layers (encoders and decoders) for processing sequential data like text. Sparse Mixture-of-Experts (MoE) Decomposes the traditional Transformer architecture into a collection of smaller, specialized "expert" models. A gating network then determines which expert(s) handle specific parts of the input sequence. Gemini 1.5 Gemini 1.0 Architecture Caption Imagine a large team of researchers working on a complex project. Each researcher has expertise in different areas, but they all need to communicate and collaborate to complete the project. Caption This would be like dividing the project into smaller sub-projects and assigning each sub-project to a team of specialized researchers. A coordinator then ensures each team gets the relevant information and collaborates effectively.
  7. 32,000 tokens Public: 128 000 tokens Private & devs: 1

    million tokens Available for waitlist: 2 million Private & devs: 1 million tokens Available for waitlist: 2 million Tested for up to 10 million tokens 1.0 Pro 1.5 Pro 1.5 Flash Larger Context Window
  8. Text Images Audio Code Multimodal Capacities Input Pipeline Remains the

    primary input format Convolutional Neural Networks (CNNs): expert image analysts who extract high-level features like shapes, edges, and objects Mel-Frequency Cepstral Coefficients (MFCCs) transform the raw audio into a numerical representation that captures its characteristics Syntax parsing and code embeddings
  9. Enhanced Domain-Specific Knowledge with Gemini 1.5 The model has been

    trained with a more extensive and diverse dataset, allowing it to excel in specialized areas such as medical, legal, and technical domains.
  10. 87% When tested on a panel of text, code, image,

    audio and video evaluations 1.5 Pro outperforms 1.0 Pro on of the benchmarks
  11. Lorem ipsum dolor sit amet consectetur Mar 14 Mar 14

    Mar 14 July 15 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 14 Apr 14 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Mar 31 Mar 31 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 7 Apr 7 May 31 Lorem ipsum dolor sit amet consectetur May 24 June 21 June 28 Sept 6-7 US EMEA IN CN Key Milestones TABLE ELEMENTS
  12. Lorem ipsum dolor sit amet consectetur hfujii@ Lorem ipsum dolor

    sit amet consectetur sodhia@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur lhflores@ Lorem ipsum dolor sit amet consectetur lhflores@ Label Owner TABLE ELEMENTS
  13. Label 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit,

    sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Label 2 Label 3 Label 4 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. TABLE ELEMENTS
  14. Aimed at individual developers and teams, it’s a web-based tool

    that “enables you to quickly develop prompts and then get an API key to use in your app development.” The “Get code” button lets you transfer work to your IDE in cURL, JavaScript, Python, Android (Kotlin), and Swift. Google AI Studio - what’s this? Montréal
  15. I have 3 apples today, yesterday I ate an apple.

    How many apples do I have now? Quiz!
  16. Our palette uses the Google primaries mixed as a gradient.

    A Background Grey should be used as to not overwhelm an audience with pure RGB white. Outlines for UI elements are set in 2.5pt in Black (Grey 900). All type should be Black (Grey 900) and highlights/color pops should be gradients. Primaries Blue 500 Hex #4285F4 RGB 66, 133, 244 CMYK 88. 40, 0, 0 PMS 2173 Coated PMS 2193 Uncoated Blend Only Colors Orange Hex #F46831 Purple Hex #9F6CD4 Yellow Hex #FFCB32 Neutrals Background Grey Hex #EEEEEE RGB 238, 238, 238 CMYK 5,4,5,0 PMS Cool Grey 1 Coated PMS Cool Grey 1 Uncoated Grey 900 Hex #202124 RGB 32, 33, 36 CMYK 5,0,0,90 PMS 426 Coated PMS 426 Uncoated White Hex #FFFFFF RGB 255, 255, 255 Green 500 Hex #34A853 RGB 52, 168, 83 CMYK 85, 0, 92, 0 PMS 2257 Coated PMS 2271 Uncoated Yellow 500 Hex #FBBC04 RGB 251, 188, 4 CMYK 0. 37, 100, 0 PMS 7549 Coated PMS 7549 Uncoated Red 500 Hex #EA4335 RGB 234, 67, 53 CMYK 0, 87, 89, 0 PMS 179 Coated PMS 2347 Uncoated Google I/O Extended - Colors
  17. Short Title Google Sans Medium, 120px Long Title Google Sans

    Medium, 90px Section Title Google Sans Medium, 75px Headline Google Sans Medium, 60px Subhead Google Sans Medium, 36px Body Copy Google Sans Normal, 20px Typography General guidelines for Font styling and size. Code Google Sans Mono Normal, 29px Google I/O Extended - Typography
  18. Optional tag here Here is a longer title. A title

    length should not exceed three lines.
  19. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do

    eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident. Headline goes here Section 1
  20. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do

    eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Headline goes here Section 2 Right-click and select ‘Replace image’
  21. Headline goes here Right-click and select ‘Replace image’ Lorem ipsum

    dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Section 3
  22. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do

    eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. Duis aute irure dolor in sunt reprehenderit in voluptate ea commodo consequat deserunt mollit anim id est laborum. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Right-click and select ‘Replace image’ Headline goes here Section #
  23. Headline goes here Lorem ipsum dolor sit amet, consectetur adipiscing

    elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Right-click and select ‘Replace image’ Section #
  24. This is body copy. Connect with fellow developers and startups

    in your area, hear from industry experts, share your skills, and learn new ones. This is body copy. Bringing developers together in-person and online. Stay in the know about upcoming events. This is a subhead This is a subhead Headline goes here Section #
  25. Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit,

    sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Lorem ipsum Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua Headline goes here Section #
  26. Connect with fellow developers and startups in your area, hear

    from industry experts, share your skills, and learn new ones. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. This is a subhead This is a subhead Bringing developers together in-person and online. Stay in the know about upcoming events. This is a subhead Headline goes here Section #
  27. Connect with fellow developers and startups in your area, hear

    from industry experts, share your skills, and learn new ones. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. Tile Goes Here Tile Goes Here Bringing developers together in-person and online. Stay in the know about upcoming events. Tile Goes Here Section # 01 02 03
  28. This is body copy. Bringing developers together in-person and online.

    Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. This is body copy. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects for others to learn from. Recommendation Learnings Section #
  29. This is body copy. Bringing developers together in-person and online.

    Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Learnings 200 Statistic Caption Section #
  30. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consectetur

    velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris consectetur velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Right-click and select ‘Replace image’ Lorem Ipsum
  31. Right-click and select ‘Replace image’ Lorem ipsum dolor sit amet,

    consectetur adipiscing elit. Mauris consectetur velit massa, eget dapibus justo lobortis tempus. Nullam sit amet pulvinar quam, et auctor libero. Proin id sem aliquet, scelerisque purus nec, vehicula odio. Aliquam orci nisi, placerat blandit massa vitae, laoreet lobortis ipsum. Section #
  32. This is body copy. Bringing developers together in-person and online.

    Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Learnings 97% Statistic Caption Section #
  33. 86% Statistic Caption 14% Statistic Caption This is body copy.

    Bringing developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Learnings Section #
  34. Connect with fellow developers and startups in your area, hear

    from industry experts, share your skills, and learn new ones. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. 1 billion 2 billion Bringing developers together in-person and online. Stay in the know about upcoming events. 100,000 Section #
  35. Connect with fellow developers and startups in your area, hear

    from industry experts, share your skills, and learn new ones. Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. 1 billion 2 billion Bringing developers together in-person and online. Stay in the know about upcoming events. 100,000 Bringing developers together in-person and online. Stay in the know about upcoming events. Connect with fellow developers and startups in your area, hear from industry experts, share your skills, and learn new ones. 20 million 2,000 Get inspired to use Google technologies. Search for open source projects, learn from other developers, and submit your latest projects. 10 Section #
  36. This is body copy. Bringing developers together in-person and online.

    Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Tablet Slide Right-click and select ‘Replace image’ Section #
  37. This is body copy. Bringing developers together in-person and online.

    Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Headline Right-click and select ‘Replace image’ Section #
  38. Right-click and select ‘Replace image’ This is body copy. Bringing

    developers together in-person and online. Stay in the know about upcoming events, catch up on content you missed, and connect with Google experts. Headline Section #
  39. Embed a video 1. Select video under the insert dropdown.

    2. Paste in a video URL. You will have to resize the thumbnail once it is on this slide. 3. Or Select a video that has been uploaded to Google Drive. Make sure video is shared in the settings. 4. Ensure captions are active when playing 5. Add a copy of the original video file in the Google Drive folder for this session
  40. code_slides.txt // A couple more notes on code slides: //

    *Use the Google Sans Mono font // *Set line spacing to 1.2 // *Don’t use font sizes below 25 Code is set in Google Sans Mono
  41. code_slides.txt protected void onTryUpdate(int reason) throws RetryException { // Do

    some awesome stuff int foo = 15; publishArtwork(new Artwork.Builder() .title(photo.name) .imageUri(Uri.parse(photo.image_url)) .viewIntent(new Intent(Intent.ACTION_VIEW, Uri.parse("http://500px.com/photo/" + photo.id))) .build()); scheduleUpdate(System.currentTimeMillis() + ROTATE_TIME_MILLIS); } Use this template for code snippets longer than 6 lines Copy / paste your code using this tool for formatting using the ‘Dark Alternate’ theme
  42. Use this style to highlight code code_slides.txt protected void onTryUpdate(int

    reason) throws RetryException { // Do some awesome stuff int foo = 15; publishArtwork(new Artwork.Builder() .title(photo.name) .imageUri(Uri.parse(photo.image_url)) .viewIntent(new Intent(Intent.ACTION_VIEW, Uri.parse("http://500px.com/photo/" + photo.id))) .build()); scheduleUpdate(System.currentTimeMillis() + ROTATE_TIME_MILLIS); }
  43. Image with code snippet * Use Google Sans Mono *

    Set line spacing to 1.2 * Don’t use font sizes below 25
  44. Image with code snippet * Use Google Sans Mono *

    Set line spacing to 1.2 * Don’t use font sizes below 25
  45. Speaker Name pronouns Job Title Speaker Name pronouns Job Title

    Speaker Name pronouns Job Title Speaker Name pronouns Job Title Thank You
  46. Lorem Ipsum May 24, 2023 2,000 Lorem Ipsum June 21,

    2023 1,000 Lorem Ipsum June 28, 2023 1,200 Lorem Ipsum Sept 6-7, 2023 2,000 TOTAL 6,200 Lorem Lorem Lorem TABLE ELEMENTS 2px Border No border on outside of table
  47. Lorem ipsum dolor sit amet consectetur Mar 14 Mar 14

    Mar 14 July 15 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 14 Apr 14 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Mar 31 Mar 31 May 31 Lorem ipsum dolor sit amet consectetur Mar 31 Apr 7 Apr 7 May 31 Lorem ipsum dolor sit amet consectetur May 24 June 21 June 28 Sept 6-7 US EMEA IN CN Key Milestones TABLE ELEMENTS
  48. Lorem ipsum dolor sit amet consectetur hfujii@ Lorem ipsum dolor

    sit amet consectetur sodhia@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur mariareis@ Lorem ipsum dolor sit amet consectetur lhflores@ Lorem ipsum dolor sit amet consectetur lhflores@ Label Owner TABLE ELEMENTS
  49. Label 1 Lorem ipsum dolor sit amet, consectetur adipiscing elit,

    sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Label 2 Label 3 Label 4 Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. Veniam et dolore magna aliqua. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore. • Lorem ipsum dolor sit amet • Consectetur adipiscing elit • Sed do eiusmod tempor incididunt ut labore. TABLE ELEMENTS
  50. Caption Large Box Caption 30px Caption 4px Object Border when

    stroke only Medium Box Caption Caption Caption Small Box Caption Use for nested items only 23px Caption Label s Label Label Arrows Simple Charts: 4px 19px Caption 23px Caption CHART ELEMENTS Caption Caption Caption Caption Caption
  51. Short Label Short Label Short Label Short Label Short Label

    Short Label Short Label Short Label CHART ELEMENTS
  52. Column Label Caption 10 Two lines Caption 9 Two lines

    Caption 8 Two lines Caption 7 Two lines Caption 6 Two lines Caption 5 Two lines Caption 4 Two lines Caption 3 Two lines Caption 2 Two lines Caption 1 Two lines Group Label Column Label Column Label Column Label CHART ELEMENTS
  53. Short Label Short Label Short Label Short Label Short Label

    Short Label Short Label CHART ELEMENTS
  54. Label One Short Label Short Label Short Label Label Two

    Label Three Label Four Label Five Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label Short Label CHART ELEMENTS
  55. 2012 2014 2016 2018 2020 Category Label One Caption 1

    Two lines Caption 3 Two lines Caption 2 Two lines Caption 5 Two lines Caption 4 Two lines Caption 6 Two lines Caption 7 Two lines Caption 9 Two lines Caption 8 Two lines Category Label Two CHART ELEMENTS