Slide 1
Slide 1 text
Mining the SERPs for SEO,
Content & Customer
Insights
Rory Truesdale // Conductor
http://cndr.co/brighton
@RoryT11
Slide 2
Slide 2 text
About Me
Rory Truesdale
•SEO Strategist at Conductor
•EMEA SEO lead for WeWork
Get In Touch
[email protected]
@RoryT11
@RoryT11
Slide 3
Slide 3 text
Get The Slides
@RoryT11
http://cndr.co/brighton
Slide 4
Slide 4 text
•SERPs are a great resource to learn what Google
‘thinks’ our customers want
•Workflows that will help you understand the intent of
the people you want to reach
•How to use these insights to improve the quality of your
on-page optimisation
What To Expect
@RoryT1
Slide 5
Slide 5 text
That’s how often Google rewrites the SERP
displayed meta description
Slide 6
Slide 6 text
WHY?
Slide 7
Slide 7 text
To make
SEOs sad?
Slide 8
Slide 8 text
Just for a
laugh?
Slide 9
Slide 9 text
Nope…
Slide 10
Slide 10 text
It’s because Google thinks
it is smarter than us
Slide 11
Slide 11 text
Intriguing…
Can we use that to our advantage?
Slide 12
Slide 12 text
Yes, we
can!
(sorry, that was
the last puppy pic)
Slide 13
Slide 13 text
How?
@RoryT11
Slide 14
Slide 14 text
We can deconstruct &
analyse the language in
SERP displayed content
to learn what Google
thinks our customers
are interested in
@RoryT1
Slide 15
Slide 15 text
Curious?
This is important
because we are
in the age of
semantic search
@RoryT1
Slide 16
Slide 16 text
Google isn’t ranking a page based on how
it uses a keyword.
Google provides accurate results based on
intent, query context & word relationships.
On-page Optimisation
@RoryT1
Slide 17
Slide 17 text
• User intent
• Query context
•Topical relevance
• Word relationships
Target the keyword, but optimise for this.
On-page Optimisation
@RoryT1
Slide 18
Slide 18 text
Understand
customer intent
& desire to better
tailor your
messaging
@RoryT1
Slide 19
Slide 19 text
Structure landing
pages to help
Google understand
context & how it
meets the needs
of the searcher
@RoryT1
Slide 20
Slide 20 text
Build more
meaningful
online
experiences that
better convert
website visitors
@RoryT1
Slide 21
Slide 21 text
Your
Toolkit
@RoryT11
Slide 22
Slide 22 text
You need
SERP content
There are
three ways
you can get
this.
@RoryT1
Slide 23
Slide 23 text
Scrape at scale with
Screaming Frog
Follow these instructions
@RoryT1
Option
A
Slide 24
Slide 24 text
Option
B
Get SERP content via an
API
Slide 25
Slide 25 text
Option
C
Get SERP content using the
Scraper Chrome extension
Get
Scraper
Slide 26
Slide 26 text
There are four
ways you can
get this.
You need
Jupyter Notebook
What is
that?
Slide 27
Slide 27 text
The Jupyter Notebook is an open-source web
application that allows you to create and share
documents that contain live code, equations,
visualizations and narrative text. Uses include: data
cleaning and transformation, numerical simulation,
statistical modeling, data visualization, machine
learning, and much more.
Jupyter.org
@RoryT1
Slide 28
Slide 28 text
Stumped?
Me too…
Here’s my definition
Slide 29
Slide 29 text
Jupyter Notebook is an environment on my laptop
where I can learn Python by copying scripts created
by people significantly smarter than me and
breaking them or making them do something
slightly different.
Rory Truesdale
Python Charlatan
@RoryT1
Slide 30
Slide 30 text
Resources to get started…
Jupyter
Notebook –
Getting
Started Guide
Robin Lord
Find
scripts
Paul Shapiro JR Oakes Hamlet
Batista
Find
scripts
Find
scripts
Slide 31
Slide 31 text
You’ll end up
with…
@RoryT11
Slide 32
Slide 32 text
Your SERP content in a CSV
@RoryT1
Slide 33
Slide 33 text
Imported into Jupyter Notebook
@RoryT1
Slide 34
Slide 34 text
You’re
ready to
use Python
to analyse
the SERPs!
@RoryT1
Slide 35
Slide 35 text
There’s a
treat for you.
Slide 36
Slide 36 text
I’ll share a link to a Dropbox
with everything you need to get
you started
@RoryT11
Slide 37
Slide 37 text
Before we
dive in…
Slide 38
Slide 38 text
Start by
cleaning
your SERP
content
@RoryT1
Slide 39
Slide 39 text
Lower case avoids
duplication &
punctuation adds
no value to this
analysis
Lower Case &
Remove
Punctuation
@RoryT1
Slide 40
Slide 40 text
Stop words are
commonly
occurring words
that don’t improve
our analysis
Remove
Stop
Words
@RoryT1
Slide 41
Slide 41 text
The process of
chopping up a
sentence into
individual pieces,
called ‘tokens’
Tokenization
@RoryT1
Slide 42
Slide 42 text
The process of
converting a word
to its root (i.e.
“playing” becomes
“play”)
Lemmatization
(optional)
@RoryT1
Slide 43
Slide 43 text
@RoryT11
Slide 44
Slide 44 text
@RoryT11
Slide 45
Slide 45 text
How many times
does a word or
combination of
words appear in
your SERP
content?
Co-occurrence
@RoryT1
Slide 46
Slide 46 text
Co-
occurrence
Snapshot of
phrases
frequently
occurring in
the SERPs
@RoryT1
Slide 47
Slide 47 text
Co-
occurrence
Demonstrates
the topics
competitors
cover on landing
pages
@RoryT1
Slide 48
Slide 48 text
Co-
occurrence
Understand the
types of phrases
that Google sees
as semantically
relevant to a target
keyword set
Slide 49
Slide 49 text
•Additional source of data for keyword research
•Identify topical content gaps on landing pages
•Optimise landing page content by incorporating
semantically relevant phrases
HOW CAN WE APPLY THIS?
@RoryT1
Slide 50
Slide 50 text
Cost:
Range:
Time to Charge:
Battery Size/Capacity:
All Wheel Drive:
Towing Capacity:
Semi-Conductor SERP XLT: Product Page
£
44,360 MSV
9,620 MSV
7,470 MSV
380 MSV
3,040 MSV
180 MSV
Slide 51
Slide 51 text
What are the most
frequently
occurring nouns,
verbs &
adjectives in a
SERP?
Part of Speech
Tagging
@RoryT1
Slide 52
Slide 52 text
PoS
Tagging
Uncover the
phrases or topics
you should include
in your landing
pages to rank for a
term
Nouns (people, place, thing)
@RoryT1
Slide 53
Slide 53 text
PoS
Tagging
Get clues around
how Google is
interpreting the
context and intent
of a search
Verbs (action or state)
@RoryT1
Slide 54
Slide 54 text
PoS
Tagging
Understand the
language and
tone that might
resonate with a
searcher
Adjectives (descriptive word)
@RoryT1
Slide 55
Slide 55 text
PoS
Tagging
Credit Card Example – P1 Verbs
Intent Clues: What is the specific motivation
our searcher has?
Slide 56
Slide 56 text
PoS
Tagging
Credit Card Example - P1 Nouns
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1
Slide 57
Slide 57 text
PoS
Tagging
Credit Card Example - P1 Adjectives
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1
Slide 58
Slide 58 text
•Create landing pages that are aligned with the
intent of a searcher
•Help copywriters understand the language and
desires of a target audience
•Tactically incorporate more semantically relevant
phrases into landing pages
HOW CAN WE APPLY THIS?
@RoryT1
Slide 59
Slide 59 text
Can we use NLP
to uncover topical
trends in the
SERPs to help us
with content
ideation?
Topic
Modelling
@RoryT1
Slide 60
Slide 60 text
Topic
Modelling
Topic modelling is an NLP method that assumes a
corpus contains a mixture of topics. It looks at how
words and phrases co-occur in a corpus and attempts
to group them in coherent themes or topics.
@RoryT1
Slide 61
Slide 61 text
Topic
Modelling
OK, computer. Here’s some words. Group them.
@RoryT1
Rory Truesdale
Cheapening machine
learning since 2019
Slide 62
Slide 62 text
Topic
Modelling
Each bubble
represents a
topic
@RoryT1
Slide 63
Slide 63 text
Topic
Modelling
The bigger
the bubble
the more
prominent
the topic
@RoryT1
Slide 64
Slide 64 text
Topic
Modelling
The further
away the
bubbles are,
the more
distinct those
topic are
Slide 65
Slide 65 text
Topic
Modelling
Get a
breakdown of
the terms our
topics consist
of
@RoryT1
Slide 66
Slide 66 text
Topic
Modelling
The output is an
interactive visual
on topical trends
that can be easily
shared with other
teams
@RoryT1
Slide 67
Slide 67 text
Topic
Modelling
Use Google’s
algorithm to help
us identify areas
of interest for our
audience
Slide 68
Slide 68 text
Topic
Modelling
Uncover topical
trends hidden in
the language of
the SERPs that
can inform
content ideation
@RoryT1
Slide 69
Slide 69 text
•Valuable data point to reference for content
ideation
•Inform internal linking and content
recommendations across a website
•Incorporate topically relevant phrases into existing
pages to improve semantic relevance
HOW CAN WE APPLY THIS?
@RoryT1
Slide 70
Slide 70 text
How can we make
our scripts work
across other data
sources to
understand our
customers?
Other
Useful
Applications
@RoryT1
Slide 71
Slide 71 text
Product
Reviews
@RoryT11
Slide 72
Slide 72 text
Product Reviews
@RoryT11
Slide 73
Slide 73 text
GMB
Reviews
@RoryT11
Slide 74
Slide 74 text
GMB Reviews
@RoryT11
Slide 75
Slide 75 text
Reddit
@RoryT11
Slide 76
Slide 76 text
Reddit
@RoryT11
Slide 77
Slide 77 text
YouTube
Captions
@RoryT11
Slide 78
Slide 78 text
YouTube Captions
@RoryT11
Slide 79
Slide 79 text
Competitors & Top
Ranking Pages
@RoryT11
Slide 80
Slide 80 text
Competitors & Top Ranking Pages
@RoryT11
Slide 81
Slide 81 text
With some minor
tweaks we can
make our scripts
work across a huge
corpus of user-
centric content
Pretty cool, right?
@RoryT1
Slide 82
Slide 82 text
Potential to ramp up and apply sentiment analysis
to these sources for useful visualisations
@RoryT11
Slide 83
Slide 83 text
Deconstruct product reviews to find out what really
matters to customers
•Simple
•Easy to use
•Intuitive
•Buggy
•Slow
@RoryT1
Slide 84
Slide 84 text
A lot to
take
in…what
does it all
mean?
@RoryT11
Slide 85
Slide 85 text
SERPs give us
amazing insight
into what
customers want
@RoryT1
Slide 86
Slide 86 text
Python makes
getting these
insights at scale
accessible
@RoryT1
Slide 87
Slide 87 text
Use these insights
to align landing
pages with intent
and semantic
relevance
@RoryT1
Slide 88
Slide 88 text
Scripts we create
allow us to get these
insights from lots of
other user-centric
sources beyond the
SERPs
@RoryT1
Slide 89
Slide 89 text
http://cndr.co/jupyter
Python Dropbox Link
@RoryT1
Slide 90
Slide 90 text
Get The Slides
@RoryT11
http://cndr.co/brighton
Slide 91
Slide 91 text
• https://www.searchenginejournal.com/scrape-google-serp-custom-extractions/267211/
• https://www.searchenginejournal.com/mine-serps-seo-content-customer-insights/311137/
• https://www.seerinteractive.com/blog/user-testing-serps-an-audience-first-approach-to-seo/
• https://www.dropbox.com/sh/vl5miyt6sgbvmkl/AAC5365YcWTun_EzkQLtixe1a?dl=0 (Jupyter
Notebook tutorial)
• http://www.blindfiveyearold.com/algorithm-analysis-in-the-age-of-embeddings
• https://www.searchenginejournal.com/semantic-search-seo/264037/#close
• https://www.slideshare.net/DawnFitton/natural-language-processing-and-search-intent-
understanding-c3-conductor-2019-dawn-anderson
• https://moz.com/blog/what-is-semantic-search
• https://www.slideshare.net/paulshapiro/redefining-technical-seo-mozcon-2019-by-paul-shapiro
Useful Resources
@RoryT1