$30 off During Our Annual Pro Sale. View Details »

Measure the whole SERP at any scale #brightonSEO

Measure the whole SERP at any scale #brightonSEO

Slides from my brightonSEO talk on November 11, 2023 in San Diego, CA.

The era of traditional rank tracking is over.

In this era of SEO, we need to understand the value that can be derived from, instead, monitoring and measuring the entire SERP.

The shape of the modern SERP is evolving quickly.

It is critical to understand both the ""the how and the why"" of this data, which can be leveraged not only by SEOs but also by Paid, Content, and Product teams.

There are numerous challenges involved in collecting accurate data at scale and even more challenges in turning this data into signals that are relevant for business teams.

Scale is not only about technical capture but also ensuring maximum impact and we discuss how to ensure that both criteria are met.

We also introduce some key KPIs and metrics that can be used to measure the whole SERP, including visual ranking, pixel depth, pixel height, fold visibility, and more.

Ray Grieselhuber
PRO

November 11, 2023
Tweet

Other Decks in Marketing & SEO

Transcript

  1. Measuring the whole
    SERP at any scale
    Ray Grieselhuber
    DemandSphere
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber

    View Slide

  2. The era of traditional rank tracking is over
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Up Down
    23 12

    View Slide

  3. But the shape of the SERP is changing faster than ever
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber

    View Slide

  4. Impacts:
    ● Fold visibility
    ● CTR
    ● Clicks / Sessions
    ● Conversions
    ● Revenue
    600+ distinct elements across the SERPs
    Paid Content
    ● Pixel depth: 120px
    ● Pixel height: 320px
    ● Visual position: 1
    ● Nested position: 2
    Organic Result
    ● Pixel depth: 440px
    ● Pixel height: 330px
    ● Visual position: 3
    ● Title changed

    View Slide

  5. Daily layout changes on Google SERPs
    Paid Content
    ● Position (order of items)
    ● Product Name
    ● Price
    ● URL
    ● Pixel Data
    Organic Results
    ● Organic Position
    ● Visual Position
    ● Title
    ● Meta Description
    ● Product URL
    ● Pixel Data
    Popular Products
    ● Position (order of items)
    ● Product Name
    ● Price
    ● URL
    ● Pixel Data

    View Slide

  6. View Slide

  7. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Scale driver #1: AI + content marketing is driving
    explosive growth in content on the web

    View Slide

  8. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Scale driver #2: The cost of money is affecting
    investment in ads
    Money supply SPIKED but now is decreasing Money is not as cheap anymore

    View Slide

  9. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Typical clients: 75K-300K+ SERPs
    Larger clients: 1M+ SERPs
    Let’s define scale (volume) for our purposes

    View Slide

  10. What is the right volume to monitor?
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    1%-10% of your
    indexed pages is a
    good place to start

    View Slide

  11. The bigger question is:
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Who uses this data and for what?

    View Slide

  12. Old answer:
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    SEOs

    View Slide

  13. Better answer:
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Good SEO is good
    product management,
    and vice-versa.

    View Slide

  14. Even better answer: exploding use cases
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Complexity increases
    Developer
    ● APIs
    ● Data lakes
    SEO Analyst
    ● SERP data
    ● Crawl data
    ● GSC
    ● GA
    ● Exploration
    Content Team
    ● Content &
    topics
    ● Content
    performance
    ● Visibility trends
    Product
    ● Technical
    ● SERP rankings
    ● Indexation
    ● PAA +
    Knowledge
    Graph
    Executive /
    Reporting
    ● Permissions
    ● Dashboards
    ● Speed of
    answers
    ● Forecasting
    ● Predictive
    Performance
    Team
    ● Ad data
    ● Hotel Ads
    ● Shopping
    ● Google for Jobs
    ● etc.

    View Slide

  15. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    How to meet the needs of all of these
    groups?

    View Slide

  16. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Step 1: understand the data pipeline

    View Slide

  17. Typical data pipelines
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Fetching
    (HTML)
    ETL
    (JSON &
    DB)
    Analytics
    (DB)
    Dashboards,
    etc.
    SERPs
    (HTML)
    GSC
    GA4
    Build? Buy?
    Build? Buy?
    Keywords

    View Slide

  18. Where do development bottlenecks occur?
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Fetching
    (HTML)
    ETL
    (JSON &
    DB)
    Analytics
    (DB)
    Dashboards,
    etc.
    SERPs
    (HTML)
    GSC
    GA4
    Build? Buy?
    Build? Buy?
    Keywords

    View Slide

  19. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Step 2: examine parallels from other
    problem domains

    View Slide

  20. Similar use cases
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Data
    Operations
    Observability
    (log files + search
    index / database)
    Problem domains Asset / Technology Parallels to SEO
    ● Log files: SERPs
    ● Parsed log files: Parsed SERPS
    (JSON, etc.)
    ● Index: ???
    Data
    Science
    BI
    Datasets
    ML Models
    Dashboards &
    Visualizations
    ● SERPs
    ● GSC
    ● GA4
    ● Keyword sets, etc.
    ● Looker Studio
    ● Superset
    ● DOMO
    ● etc.

    View Slide

  21. Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Step 3: use tools with a track record of
    acceleration

    View Slide

  22. Tools that can help accelerate and support many use cases
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Data Lakes
    BI /
    Dashboards /
    Data apps
    Cloud Data
    Warehouse
    ● S3-compatible
    object storage
    ● BigQuery
    ● Snowflake
    ● Redshift
    ● Looker Studio
    ● Superset
    ● Jupyter & other notebooks
    ● Python data apps (Dash,
    Streamlit)

    View Slide

  23. SQL is a decent lingua franca for turning SERP data
    into BI
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    select
    search_domain,
    url,
    title,
    desc,
    position,
    people_also_asked_snippets
    from results.organic_results
    where fetch_date = “2023-11-08”
    limit 100000

    View Slide

  24. We mapped every single element on the SERP and
    put it into a data warehouse
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber

    View Slide

  25. As data science matures, we’re seeing many
    Python-based tools become available
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    ● Pandas / Dask / Pola.rs
    ● Data Science notebooks
    ● Numpy
    ● PyTorch
    ● Scrapy
    ● Advertools
    ● Plotly
    ● spaCy
    ● NLTK

    View Slide

  26. Simplified access to SERP data
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    Developers
    SEO Team Content Team
    Product
    Team
    Executive /
    Reporting
    Performance
    Team
    APIs Data
    Warehouse
    Data Lakes
    Data
    Warehouse
    Data
    Warehouse
    SEO &
    Content
    Tools
    BI

    View Slide

  27. Let’s talk!
    Speakerdeck.com/raygrieselhuber
    @raygrieselhuber
    ● Scan the QR code to interact with
    these slides on Slido
    ● Follow us at x.com/demandsphere
    ● Connect with me at
    linkedin.com/in/raygrieselhuber

    View Slide