Benchmarking Brand Discoverability: LLMs vs. Traditional Search

Slide 1

Slide 1 text

Benchmarking Brand Discoverability LLMs vs. Traditional Search

Slide 2

Slide 2 text

JR Oakes JR Oakes is the VP of Strategy for LOCOMOTIVE Agency. He has been an SEO since 2011 and was formerly an architectural glass artist. His focus areas are in SEO, machine learning, language, and user experience. jroakes locomotive.agency

Slide 3

Slide 3 text

We OPENSOURCE Link Link Link

Slide 4

Slide 4 text

Key G2 About the data LLM Comparison Geography Modifiers Takeaways

Slide 5

Slide 5 text

G2 Section 01

Slide 6

Slide 6 text

CONFIDENTIAL G2 Historical Performance

Slide 7

Slide 7 text

From Industry Terms to Intent-Rich Keyword Expansion Extracted key base terms by category and expanded with relevant modifiers.

Slide 8

Slide 8 text

CONFIDENTIAL Company Data Organic Data Pipeline Model Data

Slide 9

Slide 9 text

G2 Industries 1.9k Search Modifiers 63 The Final Data Set G2.com gave us a structured way to extract a cohesive set of B2B SaaS industries, with search data (Ahrefs), and data on search modifiers. Brand Results 1.9M Companies Found 57k

Slide 10

Slide 10 text

About the Data Section 02

Slide 11

Slide 11 text

11 Context Tools / MCP RL RAG Memory There are a number of ways that model output can be transformed to contain additional information. These are not measured here.

Slide 12

Slide 12 text

Top Brand Mentions Top 30 Domains Mentioned in LLM Models Top 30 Domains Mentioned in Organic Models

Slide 13

Slide 13 text

Business Type Distribution Distribution of Business Types

Slide 14

Slide 14 text

Company Headquarters by Country Company Headquarters by Country (Top 9+ Other)

Slide 15

Slide 15 text

Social Media Presence: All Companies

Slide 16

Slide 16 text

Company Distribution by Revenue

Slide 17

Slide 17 text

LLM Comparison Section 03

Slide 18

Slide 18 text

Top Industries Since the time of claude-2.0 and gpt-3.5, LLMs have increased their mentions of brands in commercial query responses.

Slide 19

Slide 19 text

Which Models Share the Love Traditional Google search is still on top.

Slide 20

Slide 20 text

Top Domains Mentioned in Under 10k Visitor Bucket Many niche brands with low awareness, but LLMs find them.

Slide 21

Slide 21 text

LLMs Tracking Very Close to Traditional Organic

Slide 22

Slide 22 text

Metrics for SEO Matter for LLMs Having a better performing site for traditional search, potentially leads to better visibility outcomes for LLMs as well.

Slide 23

Slide 23 text

Model Overlap How similar are brand recommendations between models?

Slide 24

Slide 24 text

B2B Blues Many brands in niche industries have to pay to to get tailored messages to a small audience. The bigger brands don’t need it.

Slide 25

Slide 25 text

Links Still Matter…

Slide 26

Slide 26 text

We Wanted to Show This Again

Slide 27

Slide 27 text

Geography and Time Section 04

Slide 28

Slide 28 text

Google Has the Edge on New Company Knowledge

Slide 29

Slide 29 text

Models Getting Better at New Information

Slide 30

Slide 30 text

LLMs a Bit More US -Centric

Slide 31

Slide 31 text

LLMs Getting a Bit More Worldly

Slide 32

Slide 32 text

Damn California… Leave Some for the Rest of Us.

Slide 33

Slide 33 text

Do Models Care about Social Media? Google has a slightly stronger preference for companies with social media. Or, companies with social media tend to have more visibility in Google.

Slide 34

Slide 34 text

Modifiers Section 05

Slide 35

Slide 35 text

Modifiers We have a frequent modifier set (63) for B2B SaaS that we aligned to tested search terms. We can look at the data through this facet.

Slide 36

Slide 36 text

LLMs and Search Are Very Different

Slide 37

Slide 37 text

But, it Depends…

Slide 38

Slide 38 text

Unique Brands per Response by Modifier Category

Slide 39

Slide 39 text

LLMs Are More “Giving”...

Slide 40

Slide 40 text

Other AI ≠ Google AI Other LLMs display wildly different results than Google search and AI Overviews for many industries

Slide 41

Slide 41 text

Using Modifiers as Fingerprints

Slide 42

Slide 42 text

Average Unique Brands per Response by Provider Google Organic has the most Unique Brands returned per search term, beating out all 3 model providers. Anthropic and OpenAI are still behind Google Gemini.

Slide 43

Slide 43 text

Takeaways Section 06

Slide 44

Slide 44 text

Diversity There is a lot of diversity in brand recall from model to model so opportunities to test and optimize by platform.

Slide 45

Slide 45 text

Transferability Focusing on what works for search seems to help for LLMs as well.

Slide 46

Slide 46 text

Global Understand how models react to global audiences

Slide 47

Slide 47 text

Location, Location, Location Where you are located may impact your discoverability.

Slide 48

Slide 48 text

Lag LLMs lag QDF

Slide 49

Slide 49 text

Aggregator Sites F**45ed? Agents don’t care about your affiliate commission.

Slide 50

Slide 50 text

CONFIDENTIAL Interact with the Data Special Thanks locomotive.agency/seoweek Patrick Stox Kari Sugarman

Slide 51

Slide 51 text

Thanks That’s All