Upgrade to Pro — share decks privately, control downloads, hide ads and more …

News SEO is the cutting edge of all SEO

News SEO is the cutting edge of all SEO

Slides from my talk at YoastCon 2023 where I showed some of the lessons I've learned about Google's inner workings from years of working with news publishers.

Barry Adams

May 11, 2023
Tweet

More Decks by Barry Adams

Other Decks in Marketing & SEO

Transcript

  1. #YoastCon
    #YoastCon
    News is at the cutting edge of SEO
    Barry Adams
    May 2023

    View Slide

  2. #YoastCon
    Advance Warning
    “The whole problem with the world is
    that fools and fanatics are always so
    certain of themselves and wiser
    people so full of doubts.”
    - Bertrand Russell

    View Slide

  3. #YoastCon
    #YoastCon

    View Slide

  4. #YoastCon
    #YoastCon
    How does Google work?

    View Slide

  5. #YoastCon
    #YoastCon
    Web Search Engines
    Crawling Indexing Ranking

    View Slide

  6. #YoastCon

    View Slide

  7. #YoastCon
    Google’s model
    Crawl Queue Crawling Processing Index
    Render
    Queue
    Rendering
    Index
    Index

    View Slide

  8. #YoastCon
    #YoastCon
    1. Crawling
    Ranking
    Crawling Indexing

    View Slide

  9. #YoastCon

    View Slide

  10. #YoastCon
    Three ‘layers’ of Googlebot?
    Crawling Processing
    Render
    Queue
    Rendering
    Crawling
    Crawling Index
    Crawl Queue
    Crawl Queue
    Crawl Queue

    View Slide

  11. #YoastCon
    Three ‘layers’ of Googlebot
    1. Realtime crawler
    2. Regular crawler
    3. Legacy content crawler

    View Slide

  12. #YoastCon
    Realtime Crawler
    • Crawls VIPs
    ➢ Very Important Pages;
    Webpages that have a high change frequency and/or are
    seen as highly authoritative
    News website homepages & key section pages
    • Main purpose = discovery of valuable new content
    ➢ i.e., news articles
    • Rarely re-crawls newly discovered URLs
    ➢ Unless they’re new VIPs

    View Slide

  13. #YoastCon
    Regular Crawler
    • Google’s main crawler;
    ➢ Does most of the hard work
    ➢ Probably the crawler that
    fetches page resources

    View Slide

  14. #YoastCon
    Legacy Content Crawler
    • Crawls VUPs
    ➢ Very Unimportant Pages;
    URLs that have very little link value and/or are
    very rarely updated
    ➢ Re-crawls URLs that serve 4XX errors

    View Slide

  15. #YoastCon
    Robots.txt = Crawl Management … or is it?
    User-agent: Googlebot-News
    Disallow: /

    View Slide

  16. #YoastCon
    User-agent: Googlebot-News
    Disallow: /
    Robots.txt = Crawl Management … or is it?
    Robots.txt disallow for
    index management?!

    View Slide

  17. #YoastCon
    What can non-news sites learn from this?
    1. Turn key pages into VIPs;
    Make them more valuable by;
    - Improving link value
    - Increasing change frequency
    2. Use robots.txt disallow rules to manage indexing & ranking;
    For example, block Googlebot-Image to prevent product
    images from showing in Image search

    View Slide

  18. #YoastCon
    #YoastCon
    2. Indexing
    Crawling Indexing Ranking

    View Slide

  19. #YoastCon
    Indexing and Rendering
    Crawl Queue Crawling Processing Index
    Render
    Queue
    Rendering
    Index
    Index

    View Slide

  20. #YoastCon
    Indexing and Rendering
    Render Queue Rendering
    Crawl Queue Crawling Processing Index
    Index
    Index

    View Slide

  21. #YoastCon
    Indexing and Rendering
    Rendering takes time, and news doesn’t have time.
    Indexing is initially with raw HTML only.
    Crawl Queue Crawling Processing Index
    Render Queue Rendering
    Index
    Index

    View Slide

  22. #YoastCon
    Rendering isn’t the only shortcut…
    Google wants publishers to noindex syndicated content.
    Because Google sucks at identifying duplicate content.
    At least, it can’t de-duplicate quickly.

    View Slide

  23. #YoastCon
    Indexing is a multi-layered set of processes
    Render Queue Rendering
    Crawl Queue Crawling Processing
    Index
    Processing
    Processing
    Processing

    View Slide

  24. #YoastCon
    What about the Index itself?
    Render Queue Rendering
    Crawl Queue Crawling Processing Index
    Index
    Index

    View Slide

  25. #YoastCon
    Three Crawlers… Three Indices?
    Realtime crawler
    Regular crawler
    Legacy content crawler
    RAM storage
    SSD storage
    HDD storage

    View Slide

  26. #YoastCon
    Three Layers of Index Storage
    1. RAM storage
    ➢ Pages that need to be served quickly and frequently
    Includes news articles but also popular content
    2. SSD storage
    ➢ Pages that are regularly served in SERPs but aren’t super popular
    3. HDD storage
    ➢ Pages that are rarely (if ever) served in SERPs

    View Slide

  27. #YoastCon
    It’s probably more complicated
    Realtime crawler
    Regular crawler
    Legacy content crawler
    RAM storage
    SSD storage
    HDD storage

    View Slide

  28. #YoastCon
    What can non-news sites learn from this?
    1. Make indexing easy for Googlebot;
    Put all your critical content in the HTML source
    Don’t rely on rendering to load valuable content
    2. There’s no such thing as a duplicate content penalty;
    However, duplicate content on a single site means the site
    is competing with itself… and that’s stupid.

    View Slide

  29. #YoastCon
    #YoastCon
    3. Ranking
    Crawling Indexing Ranking

    View Slide

  30. #YoastCon
    Search Intent – first BERT, then MUM

    View Slide

  31. #YoastCon

    View Slide

  32. #YoastCon
    However…

    View Slide

  33. #YoastCon
    Most tools are out of date

    View Slide

  34. #YoastCon
    Google Trends is better, but lacks numbers

    View Slide

  35. #YoastCon
    Very few tools are (near) real-time

    View Slide

  36. #YoastCon
    And even fewer accurately report on
    SERP features

    View Slide

  37. #YoastCon

    View Slide

  38. #YoastCon
    SERP Features
    • Many SERP features are volatile;
    ➢ None more than Top Stories & other news boxes
    • Top Stories are triggered when two conditions are met;
    ➢ Sudden increase in search volume
    ➢ Sudden increase in publishing volume

    View Slide

  39. #YoastCon
    SERP Features and CTR

    View Slide

  40. #YoastCon
    Who gets the Top Stories Top Spot?
    • Topical Authority
    • Authorship
    • E-E-A-T
    Expressions of the
    Knowledge Graph

    View Slide

  41. #YoastCon
    Knowledge Graph
    Arnold
    Schwarzenegger
    Bodybuilding
    Predator
    (1987)
    Governor of
    California
    Maria Shriver
    Ronnie
    Coleman
    JFK

    View Slide

  42. #YoastCon
    Knowledge Graph
    Arnold
    Schwarzenegger
    Bodybuilding
    Independent.co.uk
    … …

    Predator
    (1987)

    View Slide

  43. #YoastCon
    Internal Linking to Topic Hubs

    View Slide

  44. #YoastCon
    Knowledge Graph
    Arnold
    Schwarzenegger
    Predator
    (1987)


    Independent.co.uk

    Bodybuilding

    View Slide

  45. #YoastCon

    View Slide

  46. #YoastCon
    What can non-news sites learn from this?
    1. Understand the intent behind the keywords
    you’re targeting;
    Don’t try to rank content that doesn’t match the intent
    If there are SERP features, try to get into those
    2. Improve your Knowledge Graph presence;
    Category pages = topic hubs
    Use internal linking to your advantage
    Schema.org markup helps Google connect the dots

    View Slide

  47. #YoastCon
    #YoastCon
    So why is news at the
    cutting edge of SEO?

    View Slide

  48. #YoastCon
    News websites…
    … are crawled the most
    … are crawled the fastest
    … are indexed the quickest
    … are ranked according to the latest signals
    … are ranked based on the best interpretation of intent

    View Slide

  49. #YoastCon
    From News SEO you can learn…
    … how Google crawls websites
    … how Google indexes content
    … how Google evaluates quality and authority
    … how SERP features impact on ranking and traffic
    … and much, much more.

    View Slide

  50. #YoastCon
    www.SEOforGoogleNews.com

    View Slide

  51. #YoastCon
    #YoastCon
    Thank You
    [email protected]
    @badams
    /in/barryadams/

    View Slide