Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Crawl Capacity Management - SEontheBeach 2022

Crawl Capacity Management - SEontheBeach 2022

C01a4b76bb1d14618ac8f51ecc7a415d?s=128

Gaston Riera

June 18, 2022
Tweet

Other Decks in Marketing & SEO

Transcript

  1. SOB 2022 Crawl capacity management on Envato Elements Gastón Riera

    - @gastonriera
  2. Heads up.. Slides in English 󰎉 Speako en Español 󰎆

    Gastón Riera - @gastonriera
  3. At the end I'll share a 90% discount for Elements!

    ☺ Gastón Riera - @gastonriera
  4. That's how I used to look, all well dressed and

    all. Gastón Riera - @gastonriera Gastón Riera
  5. Everything you need to get your creative projects done. Gastón

    Riera - @gastonriera The big names: Other very cool products:
  6. Gastón Riera - @gastonriera The two things I like the

    most about working at envato: - Being sustainable and caring about the community - Fully remote (ANZ/MX) and working from abroad
  7. Let's get into SEO 🔎 Gastón Riera - @gastonriera

  8. The problem: A big part of the site was not

    being indexed 👎 Gastón Riera - @gastonriera
  9. ☹ Gastón Riera - @gastonriera

  10. Gastón Riera - @gastonriera It looked like Google was not

    recrawling a large amount of pages!
  11. Gastón Riera - @gastonriera And taking very long to recrawl

    some pages
  12. Was Google actually crawling the site?🤔 Gastón Riera - @gastonriera

  13. Gastón Riera - @gastonriera Yes, it was👍

  14. But, it wasn't getting to crawl the entire site 😕

    Gastón Riera - @gastonriera
  15. And, why was that?󰤈 Gastón Riera - @gastonriera

  16. No idea Gastón Riera - @gastonriera

  17. Just kidding 😆 Gastón Riera - @gastonriera

  18. We came up with two theories to work on 😎

    Gastón Riera - @gastonriera
  19. - Content quality - Internal linking Gastón Riera - @gastonriera

    We needed to work on
  20. Gastón Riera - @gastonriera We needed to work on I'll

    get to them in a bit. - Content quality - Internal linking
  21. What did we do? 🤔 Gastón Riera - @gastonriera

  22. The basics! Gastón Riera - @gastonriera

  23. The basics! - Noindex Gastón Riera - @gastonriera

  24. The basics! - Noindex - Redirects Gastón Riera - @gastonriera

  25. The basics! - Noindex - Redirects - Nofollow Gastón Riera

    - @gastonriera
  26. The basics! - Noindex - Redirects - Nofollow - Crawl

    paths (more/less) Gastón Riera - @gastonriera
  27. Gastón, that's nothing new! 😡 Gastón Riera - @gastonriera

  28. I know! 😉 Gastón Riera - @gastonriera

  29. How are we using them? Let's get to it Gastón

    Riera - @gastonriera
  30. Battle_1: Content quality Gastón Riera - @gastonriera

  31. Content is not just text on the page, but everything

    on it. Every page is content. Gastón Riera - @gastonriera
  32. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1.

    Add content focussing on quality over quantity. 2. Remove content from Google's index. We already had +9M items!
  33. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1.

    Add content focussing on quality over quantity. ❌ 2. Remove content from Google's index. We already had +9M items!
  34. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1.

    Add content focussing on quality over quantity. ❌ 2. Remove content from Google's index. ✅ We already had +9M items!
  35. Do you know what reduces the content quality of any

    site? Gastón Riera - @gastonriera
  36. Do you know what reduces the content quality of any

    site? DUPLICATE CONTENT! Gastón Riera - @gastonriera
  37. Noindex and remove duplicates, RUTHLESSLY Gastón Riera - @gastonriera Noindex

    a good part of the items library. -> Several million less discoverable pages! Why we decided to noindex instead of a fancier solution? Ask me later 😉
  38. A few tips on how to get what to noindex?

    - Use google's crawled not indexed as a proxy - Check duplicate titles/urls/content description - Just a different image doesn't make it a different page to the eyes of Google! Gastón Riera - @gastonriera Battle_1: Content quality
  39. Noindex and remove duplicates, RUTHLESSLY Gastón Riera - @gastonriera Why

    the redirected path had 15% of site's traffic and 20x the destination. Ask me later 😉 Merged two translations that ended up being way more similar that intended -> A few millions pages removed from Google.
  40. Other big things we did • Turned Tag pages into

    Search pages • Search pages are noindex by default The overall result? Decreased the index size to a half without impacting organic traffic. Gastón Riera - @gastonriera 50% Battle_1: Content quality
  41. Battle_2 Internal linking Gastón Riera - @gastonriera Reference

  42. Battle_2: Internal linking Gastón Riera - @gastonriera Out of many

    tactics: 1. Reduce the number of crawl paths 2. Nofollow on links to low-value pages
  43. Basically, Be intentional and smart about crawl paths. Gastón Riera

    - @gastonriera
  44. Gastón Riera - @gastonriera Link to only valuable pages Added

    links between related search pages 10% Organic traffic! If it's a useful search page, it will not have a noindex. Note that
  45. Gastón Riera - @gastonriera Link to only valuable pages Remove

    hreflang when you're uncertain of the quality on other languages 15% size of index! hreflang are bidirectional, remove them on every language. Remember 😉
  46. What are valuable pages? Gastón Riera - @gastonriera In short,

    pages we want Google to index.
  47. Gastón Riera - @gastonriera Link to only valuable pages As

    per nofollow: • Nofollow on links to noindex pages • Filters and facets, all nofollow The overall result? Google re-crawled more pages. 60%
  48. So, why crawl capacity management? Gastón Riera - @gastonriera

  49. Crawl budget stayed the same. Gastón Riera - @gastonriera *On

    average, over the last 2yrs.
  50. BONUS TRACK and unpopular opinion. Gastón Riera - @gastonriera

  51. BONUS TRACK and unpopular opinion. Gastón Riera - @gastonriera

  52. BONUS TRACK and unpopular opinion. We learnt that • Sitemaps

    didn't help indexing AT ALL 󰤃 • Helpful only for debugging 🤓 Gastón Riera - @gastonriera
  53. 1st month 1 USD🥳 Go to SOB22.com Gastón Riera -

    @gastonriera Yeah no kidding, I did register that domain to share the discount ☺
  54. Gastón Riera - @gastonriera Gracias! - Thank You!