AI Engineer World's Fair '24 - Cooking with Fire without

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Sustaining Innovation Innovation based on current customer needs and feedback typically based on your existing product and market. Disruptive Innovation Two types of innovation as deﬁned by Clayton Christensen in "The Innovator's Dilemma" Lower quality / performance for key market. Meets needs of neglected niche. Evolves to meet future needs of customers.

Slide 10

Slide 10 text

©2023 TWILIO INC. ALL RIGHTS RESERVED ● AI features in core product workﬂows ● "human in the loop" experiences Sustaining AI Innovation Disruptive AI Innovation ● "GPTs" ● Agents AI innovation is happening on both ends RAG-based chatbots

Slide 11

Slide 11 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Agents are not ready for enterprise prime time yet Uncertain reliability & quality Adoption & rollout often managed by legal Cost seen as too high → looking for AI as cost savings

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

©2023 TWILIO INC. ALL RIGHTS RESERVED It's natural for businesses to focus on sustaining innovation ● You have an established customer base ● Understand your target customer ● Problems are well deﬁned ● You are making money ● Market is more predictable

Slide 15

Slide 15 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Ignoring disruptive innovation right now can be dangerous ● Biggest diﬀerence between sustaining and disruptive innovation is quality ● Capabilities of AI are changing faster than ever before ● A company/product that wasn't a concern for you yesterday might well be tomorrow

Slide 16

Slide 16 text

Slide 17

Slide 17 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Origin story of ETI Explore new features and products that would help customers implement our customer engagement ﬂywheel vision. Focus on rapid prototyping of customer engagement features Started as 3-person "Special Projects" team Operate independently of our two business units and their OKRs, reporting directly to the Oﬃce of the CEO

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

©2024 TWILIO INC. ALL RIGHTS RESERVED Lessons learned Being customer obsessed is key ● Knew we were on the right track because we were focused on talking to customers and understanding their challenges and their future visions ● What wasn't working: Had to build ideas out further and get them into the hands of people Ship early, ship often ● Don't wait too long to get you work into people's hands ● Gather regularly feedback and iterate ● Set the right expectations with customers ● Find as many internal opportunities to dogfood your work Share internally & externally ● Built enormous set of learnings in short time within the team that other teams could beneﬁt from ● We can help our customers be thought leaders in their space and establish ourselves as a thought leader along the way Lesson #1 Lesson #2 Lesson #3

Slide 23

Slide 23 text

Slide 24

Slide 24 text

©2024 TWILIO INC. ALL RIGHTS RESERVED ● Leverage internal hackathons and 1-day onsite customer hackathons ● Provide access to prototype rough edges and all What we did diﬀerently this time Find internal use cases to start dogfooding Begin to build an engineering organization Get it into hands of customers don't just show it ● Started with low-risk use cases such as our IT helpdesk ● Acknowledged it wasn't our target customer but helped us to learn and gather data ● Create an engineering organization that's focused on solidifying our foundation ● Prepare to give actual customers access to AI Assistants as fast as possible

Slide 25

Slide 25 text

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Disruptive technology dilemma How do we launch something that doesn't ﬁt our normal quality bar without upsetting our remaining customer base? Quality is not good enough yet Need to ship to make it better Need to keep up with fast AI pace and ever changing expectations

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

©2023 TWILIO INC. ALL RIGHTS RESERVED Flexibility is key System Flexibility ● Expect that the model you use today can be obsolete tomorrow ● Don't chase every model and or every new paper ● Know challenges & limitations of your current model ● Understand how you can quickly evaluate a model without overhead Roadmap Flexibility ● Embrace no commitments to serve existing customers ● Be judicial with your roadmap to retain ﬂexibility for unexpected AI advancements ● Understand what expectations are constant vs shaped by AI advancements → Trust & safety vs. multi-modality ● Constantly iterate on your roadmap ● Don't be afraid of failures

Slide 38

Slide 38 text

Slide 39

Slide 39 text

©2023 TWILIO INC. ALL RIGHTS RESERVED What didn't work Developed quietly away from others Sharing work on a need-to-know basis internally & externally Tried to not be a distraction Worked with what we had (public APIs) and operated more as an ISV than collaborate with others Unintended results ● Most teams didn't know we existed ● It was on us to ﬁnd potential conﬂicting work & missed opportunities ● Could only talk to customers who were ○ already existing customers ○ we knew would be good target customers

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

©2024 TWILIO INC. ALL RIGHTS RESERVED ● Talk to customers early & often ● Don't sell them today but understand their business, challenges and vision ● Anticipate and build for their future needs ● Set yourself up to be able to ship fast and change course if need be ● Create a team that's curious & owns problems together ● Set the right expectations with your customers The foundational principles of our team Ship early, ship often Share as we go Customer and developer obsession ● Share your learnings both internally & externally ● Find internally ways to help others with your work ● Externally help your customers be thought leaders in their space

Slide 43

Slide 43 text

Slide 44

Slide 44 text