All Things Open 2023: Imagined Dragons

Presented at All Things Open 2023:

The Artificial Intelligence landscape is a wild, rapidly developing frontier with some unique challenges. The most sophisticated and capable models are controlled by a handful of companies, gated behind waitlists, and the costs of training and running your own large language models (LLMs) are prohibitive.

But this doesn’t mean we can’t have a little fun with them right now and develop open source tooling to help the rest of the community explore their capabilities. We’re already starting to see how more tools, and more education, lead to more innovation and a future where open, cheap LLMs will be available to everyone.

In this talk, we’ll walk through the process of harnessing an LLM’s imagination and empowering it with knowledge and tools from the real world to perform one of the most delicate human tasks: creating mixtapes.

We’ll discover and discuss the subtle nuances of Prompt Engineering, learn how to manage and manipulate a chatbot’s memory, and create and hone open source tools that we can leverage with existing and future LLMs as we explore this new frontier.

Transcript

1. Imagined Dragons
2. “Music gives a soul to the universe, wings to the mind, flight to the imagination and life to everything.” - Plato
3. “Music gives a soul to the universe, wings to the mind, flight to the imagination and life to everything.” - Plato
4. Building an Imagination-Powered Music Recommendation Platform
11. “Music gives a soul to the universe, wings to the mind, flight to the imagination and life to everything.” - Plato
14. 120,000+ songs every day
15. How do we find new music?
22. 120,000+ songs every day
23. “Music gives a soul to the universe, wings to the mind, flight to the imagination and life to everything.” - Plato (Probably)
24. Disclaimer:
25. Disclaimer: I am not an AI expert.
26. Disclaimer: I will be simplifying things.
27. Disclaimer: I might anthropomorphize.
28. QR Codes Incoming
31. interweb.wtf /chatgpt-vm
39. interweb.wtf /beatbrain-pasta-party
42. interweb.wtf /beatbrain-hipster-coffee-shop
44. interweb.wtf /craiyon
53. “If the only tool you have is a hammer, it is tempting to treat everything as if it were a nail.” - Maslow’s Hammer
54. “If the only tool you have is a chatbot, it is tempting to treat everything as if it were a conversation.”
55. “If the only tool you have is a chatbot, it is tempting to treat everything as if it were a conversation.” - ChatGPT’s Hammer
56. Conversations are bad interfaces
57. Conversations are complicated
58. Conversations are bad interfaces
59. Conversations are bad user interfaces
60. Conversations are bad user interfaces
61. Conversations are bad application interfaces
63. Lesson:
64. Lesson: You may not need a conversation.
76. Does ChatGPT have an imagination?
77. What is a prompt?
78. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
79. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
80. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
81. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
82. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
83. You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
84. Analyze the current Spotify <Track> based on attributes like genre, tempo, instrumentation, lyrics, and mood in order to provide the link to a similar <Track> from Spotify.
85. Analyze the current Spotify <Track> based on attributes like genre, tempo, instrumentation, lyrics, and mood in order to provide the link to a similar <Track> from Spotify.
86. <Track>
87. <Track> <Song>Get Better John</Song> <Artist>Mux Mool</Artist> </Track>
88. <Track> <Song>Get Better John</Song> <Artist>Mux Mool</Artist> </Track>
89. <Track> <Song>Get Better John</Song> <Artist>Mux Mool</Artist> </Track>
90. <Track> <Song>Get Better John</Song> <Artist>Mux Mool</Artist> </Track>
91. <Track>
92. • LQML
93. • LQML • TypeChat
94. • LQML • TypeChat • Output Parsers
95. • LQML • TypeChat • Output Parsers • SudoLang
96. • LQML • TypeChat • Output Parsers • SudoLang • Outlines
97. Analyze the current Spotify <Track> based on attributes like genre, tempo, instrumentation, lyrics, and mood in order to provide the link to a similar <Track> from Spotify.
98. Are language models creative?
100. Are language models adaptive?
101. Are language models imaginative?
102. Lesson:
103. Caveat: Your data will be outdated.
105. What the heck are tokens anyway?
106. What the heck are tokens anyway?
107. What is tokenization?
108. What is tokenization?
109. 3923374 4037206530
110. Why do tokens matter?
111. [System]
112. [System] You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
113. [System]\n You are BeatBrain, the advanced AI Assistant for the Music Genome Project working in partnership with the Echo Nest music intelligence platform.
114. gpt-tokenizer.dev
115. [User] Hey there, BeatBrain!
116. [Assistant] Hey, welcome to BeatBrain. You want to talk music? Let's make a Top 5 list of your new favorites. Cool? Cool.
117. [User] <Track> <Song>Gypsy</Song> <Artist>Suzanne Vega</Artist> </Track>
118. [BeatBrain] ```json{ "tracks": [ "song": "Landslide", "artist": "Fleetwood Mac", "notes": "Gypsy is about young love and self-discovery, while Landslide tackles change and self-doubt…” ] }```
119. [BeatBrain] If I were to make a mixtape about reminiscing and wrestling with the past, these two would be back-to-back. Yeah, they'd be right there.
120. Lesson:
121. Lesson: Your bot has a voice.
123. [System] Your commentary and notes are written in the voice of Rob Gordon, John Cusack's character from the 2000 film High Fidelity.
124. [System] Your commentary and notes are written in the voice of Rob Gordon, John Cusack's character from the 2000 film High Fidelity.
125. [BeatBrain] These tracks were chosen based on their similar acoustic and vocal instrumentation and slow, melancholic moods that fit the theme of the playlist.
126. [BeatBrain] They're like two sides of the same emotional coin, talking about growing up, relationships, life.
129. Why do tokens matter?
130. Tokens are expensive
131. Tokens are context
132. Tokens are additive
133. Lesson:
134. Lesson: Be concise.
135. Lesson: Be concise with prompts.
136. Lesson: “Be concise” in prompts.
140. There are tools for this
141. • ChatGPT Functions
142. • ChatGPT Functions • LangChain Tools
143. • ChatGPT Functions • LangChain Tools • AutoGPT
144. • ChatGPT Functions • LangChain Tools • AutoGPT • NexusRaven / Gorilla
145. Tools are hammers
146. Tools are abstractions
147. Tools are distractions
148. Lesson:
149. Lesson: Build your own tools.
152. • AI SDK
153. • AI SDK • AI JSX
154. • AI SDK • AI JSX • LangUI
157. Memory is malleable
159. Memory is important
160. Lesson:
161. Lesson: Manage the conversation.
163. interweb.wtf /beatbrain-beta
164. interweb.wtf /github
165. interweb.wtf /ato-2023