Crawl Capacity Management - SEontheBeach 2022

Transcript

1. SOB 2022 Crawl capacity management on Envato Elements Gastón Riera - @gastonriera
2. Heads up.. Slides in English 󰎉 Speako en Español 󰎆 Gastón Riera - @gastonriera
3. At the end I'll share a 90% discount for Elements! ☺ Gastón Riera - @gastonriera
4. That's how I used to look, all well dressed and all. Gastón Riera - @gastonriera Gastón Riera
5. Everything you need to get your creative projects done. Gastón Riera - @gastonriera The big names: Other very cool products:
6. Gastón Riera - @gastonriera The two things I like the most about working at envato: - Being sustainable and caring about the community - Fully remote (ANZ/MX) and working from abroad
7. Let's get into SEO 🔎 Gastón Riera - @gastonriera
8. The problem: A big part of the site was not being indexed 👎 Gastón Riera - @gastonriera
9. ☹ Gastón Riera - @gastonriera
10. Gastón Riera - @gastonriera It looked like Google was not recrawling a large amount of pages!
11. Gastón Riera - @gastonriera And taking very long to recrawl some pages
12. Was Google actually crawling the site?🤔 Gastón Riera - @gastonriera
13. Gastón Riera - @gastonriera Yes, it was👍
14. But, it wasn't getting to crawl the entire site 😕 Gastón Riera - @gastonriera
15. And, why was that?󰤈 Gastón Riera - @gastonriera
16. No idea Gastón Riera - @gastonriera
17. Just kidding 😆 Gastón Riera - @gastonriera
18. We came up with two theories to work on 😎 Gastón Riera - @gastonriera
19. - Content quality - Internal linking Gastón Riera - @gastonriera We needed to work on
20. Gastón Riera - @gastonriera We needed to work on I'll get to them in a bit. - Content quality - Internal linking
21. What did we do? 🤔 Gastón Riera - @gastonriera
22. The basics! Gastón Riera - @gastonriera
23. The basics! - Noindex Gastón Riera - @gastonriera
24. The basics! - Noindex - Redirects Gastón Riera - @gastonriera
25. The basics! - Noindex - Redirects - Nofollow Gastón Riera - @gastonriera
26. The basics! - Noindex - Redirects - Nofollow - Crawl paths (more/less) Gastón Riera - @gastonriera
27. Gastón, that's nothing new! 😡 Gastón Riera - @gastonriera
28. I know! 😉 Gastón Riera - @gastonriera
29. How are we using them? Let's get to it Gastón Riera - @gastonriera
30. Battle_1: Content quality Gastón Riera - @gastonriera
31. Content is not just text on the page, but everything on it. Every page is content. Gastón Riera - @gastonriera
32. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1. Add content focussing on quality over quantity. 2. Remove content from Google's index. We already had +9M items!
33. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1. Add content focussing on quality over quantity. ❌ 2. Remove content from Google's index. We already had +9M items!
34. Battle_1: Content quality Gastón Riera - @gastonriera Two options: 1. Add content focussing on quality over quantity. ❌ 2. Remove content from Google's index. ✅ We already had +9M items!
35. Do you know what reduces the content quality of any site? Gastón Riera - @gastonriera
36. Do you know what reduces the content quality of any site? DUPLICATE CONTENT! Gastón Riera - @gastonriera
37. Noindex and remove duplicates, RUTHLESSLY Gastón Riera - @gastonriera Noindex a good part of the items library. -> Several million less discoverable pages! Why we decided to noindex instead of a fancier solution? Ask me later 😉
38. A few tips on how to get what to noindex? - Use google's crawled not indexed as a proxy - Check duplicate titles/urls/content description - Just a different image doesn't make it a different page to the eyes of Google! Gastón Riera - @gastonriera Battle_1: Content quality
39. Noindex and remove duplicates, RUTHLESSLY Gastón Riera - @gastonriera Why the redirected path had 15% of site's traffic and 20x the destination. Ask me later 😉 Merged two translations that ended up being way more similar that intended -> A few millions pages removed from Google.
40. Other big things we did • Turned Tag pages into Search pages • Search pages are noindex by default The overall result? Decreased the index size to a half without impacting organic traffic. Gastón Riera - @gastonriera 50% Battle_1: Content quality
41. Battle_2 Internal linking Gastón Riera - @gastonriera Reference
42. Battle_2: Internal linking Gastón Riera - @gastonriera Out of many tactics: 1. Reduce the number of crawl paths 2. Nofollow on links to low-value pages
43. Basically, Be intentional and smart about crawl paths. Gastón Riera - @gastonriera
44. Gastón Riera - @gastonriera Link to only valuable pages Added links between related search pages 10% Organic traffic! If it's a useful search page, it will not have a noindex. Note that
45. Gastón Riera - @gastonriera Link to only valuable pages Remove hreflang when you're uncertain of the quality on other languages 15% size of index! hreflang are bidirectional, remove them on every language. Remember 😉
46. What are valuable pages? Gastón Riera - @gastonriera In short, pages we want Google to index.
47. Gastón Riera - @gastonriera Link to only valuable pages As per nofollow: • Nofollow on links to noindex pages • Filters and facets, all nofollow The overall result? Google re-crawled more pages. 60%
48. So, why crawl capacity management? Gastón Riera - @gastonriera
49. Crawl budget stayed the same. Gastón Riera - @gastonriera *On average, over the last 2yrs.
50. BONUS TRACK and unpopular opinion. Gastón Riera - @gastonriera
51. BONUS TRACK and unpopular opinion. Gastón Riera - @gastonriera
52. BONUS TRACK and unpopular opinion. We learnt that • Sitemaps didn't help indexing AT ALL 󰤃 • Helpful only for debugging 🤓 Gastón Riera - @gastonriera
53. 1st month 1 USD🥳 Go to SOB22.com Gastón Riera - @gastonriera Yeah no kidding, I did register that domain to share the discount ☺
54. Gastón Riera - @gastonriera Gracias! - Thank You!