Stop Guessing, Start Testing – The SEO Experimentation Playbook

In this 30-minute webinar, Ryan Jones (Marketing Manager at SEOTesting) walks through the core challenges most SEOs face — lack of time, uncertainty about what works, and difficulty proving value — and shows how SEO testing can solve them.

The presentation introduces three key frameworks — the SEO Hamster Wheel, the SEO Guesswork Tax, and the Prove-It Gap — that explain why traditional SEO work often leads to motion without learning. Ryan then demonstrates how data-driven experimentation helps teams escape the cycle, prove impact, and scale successful tactics.

You’ll learn:

✅ How to build SEO tests that isolate cause and effect
✅ The difference between time-based and split-URL tests
✅ How to form strong hypotheses and select control/test pages
✅ Which metrics to track for both SEO and ecommerce outcomes
✅ How to communicate test results to earn trust and resources

Packed with real examples, frameworks, and actionable steps, this playbook helps SEOs move from guesswork to evidence-based growth — proving what truly drives performance.

Transcript

2. Ryan Jones | SEOTesting | @RyanJonesSEO Three Problems
3. Ryan Jones | SEOTesting | @RyanJonesSEO Your Problems: - You’re lacking time. - You don’t know what works. - You don’t always feel valued by managers, stakeholders, and clients.
4. Ryan Jones | SEOTesting | @RyanJonesSEO Stick with me for 25-30 minutes and I’ll take you through why you (probably) have these problems, and what you can do about it.
5. Ryan Jones | SEOTesting | @RyanJonesSEO We’ll Cover: 1) The Three Main Problems SEOs Face 2) Frameworks to Solve Those Problems 3) Why SEO Testing is the Link 4) What SEO Testing Is 5) Types of SEO Tests Available to You 6) How to Run Successful SEO Tests
6. Ryan Jones | SEOTesting | @RyanJonesSEO Hi, I’m Ryan! 󰗝 Marketing Manager @ SEOTesting Over a Decade in SEO & Digital Marketing International Conference Speaker Lover of All Things Sports 🏈⚽⚾🏀
7. Ryan Jones | SEOTesting | @RyanJonesSEO Why Listen? - I work for an SEO testing tool. - I’ve run hundreds of SEO tests across different sites & niches. - I work closely with other SEOs to understand what works. - I regularly speak and write about SEO testing topics.
8. Ryan Jones | SEOTesting | @RyanJonesSEO You Lack Time
10. Ryan Jones | SEOTesting | @RyanJonesSEO The SEO Hamster Wheel - We make dozens of changes weekly, hoping something works. - When traffic moves, we sometimes can’t explain why. - So… We keep tweaking, guessing, and repeating the cycle. That’s the hamster wheel! 😉 - More work → more uncertainty → more work again!
11. Ryan Jones | SEOTesting | @RyanJonesSEO “All of that, while we could be testing, finding what works, doubling down on that, and saving a whole bunch of time!” - Ryan Jones, November 2025.
12. Ryan Jones | SEOTesting | @RyanJonesSEO Finding What Works
14. Ryan Jones | SEOTesting | @RyanJonesSEO The SEO Guesswork Tax - We make changes, metrics improve, but we don’t know why. - Every unmeasured change is a small “guesswork payment.” - These payments compound over time into wasted time and lost focus. That’s the SEO Guesswork Tax! 😉 - We can’t double down on what works if we don’t know what did!
15. Ryan Jones | SEOTesting | @RyanJonesSEO “When you make changes without testing the impact, you lack the ability to understand what’s actually working for your site.” - Ryan Jones, November 2025.
16. Ryan Jones | SEOTesting | @RyanJonesSEO Not Feeling Valued
18. Ryan Jones | SEOTesting | @RyanJonesSEO The SEO “Prove-It” Gap - We do the work, but we struggle to show the results. - Wins feel almost anecdotal, not analytical. - Stakeholders ask “What caused this?” And we can’t always answer! - Without proof, SEO looks like guesswork instead of a strategy.
19. Ryan Jones | SEOTesting | @RyanJonesSEO “When you can’t prove your impact, it’s so difficult for stakeholders to trust you enough to give you budget for other work. Leaving you feeling undervalued!” - Ryan Jones, November 2025.
20. Ryan Jones | SEOTesting | @RyanJonesSEO
21. Ryan Jones | SEOTesting | @RyanJonesSEO Okay, cool, now I have your attention! 😉 So what’s the thing that solves these problems?
23. Ryan Jones | SEOTesting | @RyanJonesSEO Come on, Ryan… We’ve been doing that the entire time! 😩
24. Ryan Jones | SEOTesting | @RyanJonesSEO Time-Based Testing (Old)
25. Ryan Jones | SEOTesting | @RyanJonesSEO Checking a rank tracker (such as Ahrefs or Semrush) shows that this page has lost prime position for a crucial keyword: Keyword: Performance Polo Shirts Search Volume: 1,450 Previous Position: 2nd Organic Result New Position: 8th Organic Result Lost Rankings
26. Ryan Jones | SEOTesting | @RyanJonesSEO - Improve Page Speed - Re-Work Wording on PLP - Improve Internal Linking - Remove Intrusive Interstitials Potential Actions
27. Ryan Jones | SEOTesting | @RyanJonesSEO Following some improvements to the page, we can check our rank tracker and see that the performance of this page for our keyword has improved again. Keyword: Performance Polo Shirts Search Volume: 1,450 Previous Position: 8th Organic Result New Position: 3rd Organic Result Ranking Recovery
29. Ryan Jones | SEOTesting | @RyanJonesSEO We now have more/better data (and tools) to do SEO tests the right way.
30. Ryan Jones | SEOTesting | @RyanJonesSEO MatchWornShirts
31. Work Done (Test) 1) Added Top-of-the-Fold Text 2) Changed Meta Title (Keyword | Brand | Emotional Language) 3) Improved Filter/Sorting 4) Optimized for On-Page Metrics Ryan Jones | SEOTesting | @RyanJonesSEO
32. The Results Clicks Per Day - 524.14% Increase Site Clicks Decreased Impressions Up 42% CTR Up To 1.32% Ryan Jones | SEOTesting | @RyanJonesSEO
33. The Results Ryan Jones | SEOTesting | @RyanJonesSEO
34. Ryan Jones | SEOTesting | @RyanJonesSEO Why SEO Testing?
35. Ryan Jones | SEOTesting | @RyanJonesSEO Building a Business Case - Prove what works for your niche/site. - Avoid making costly mistakes. - Prove hypotheses and get larger project buy-in. - Show the impact of your SEO team. - Add team accountability by tracking the results of changes. - Use past tests to build a backlog of what’s happened on the site.
37. Ryan Jones | SEOTesting | @RyanJonesSEO Breaking the Hamster Wheel - SEO testing gives every change a clear purpose and outcome. - You run smart, controlled experiments. Not endless tweaks. - You find what works, and you double down on that. The SEO Hamster Wheel is motion (work) without learning.
39. Ryan Jones | SEOTesting | @RyanJonesSEO Stop Paying the Guesswork Tax - Testing gives us measurable cause and effect. No guessing required! - Every test acts as a data point that leads to learning, not confusion. - You can replace “it depends” with “I can show.” The SEO Guesswork Tax is paid every time we act without evidence.
41. Ryan Jones | SEOTesting | @RyanJonesSEO Closing the Prove-It Gap - Testing bridges that gap with data that we can show off. - We can demonstrate the exact impact of the work we do. - Testing builds trust with clients, managers, and executives. The “Prove-It” Gap is the space between effort and evidence.
42. Ryan Jones | SEOTesting | @RyanJonesSEO Types of SEO Tests
43. Ryan Jones | SEOTesting | @RyanJonesSEO Two Main Test Types 1) Time-Based Tests 2) Split Tests
44. Ryan Jones | SEOTesting | @RyanJonesSEO “Split tests are your ecommerce SEO testing gold standard due to the fact sites are often high-traffic, and have lots of similar page templates (PDPs, PLPs) to act as control and test variants.” - Ryan Jones, November 2025.
45. Ryan Jones | SEOTesting | @RyanJonesSEO Time-Based Tests Before After TIME
46. Ryan Jones | SEOTesting | @RyanJonesSEO Results
47. Ryan Jones | SEOTesting | @RyanJonesSEO Split Tests
49. Ryan Jones | SEOTesting | @RyanJonesSEO Results
50. Ryan Jones | SEOTesting | @RyanJonesSEO Split Test Examples
51. New Page Design Ryan Jones | SEOTesting | @RyanJonesSEO
52. Simplifying Page Titles Ryan Jones | SEOTesting | @RyanJonesSEO
53. Adding Action Words Ryan Jones | SEOTesting | @RyanJonesSEO
54. More Data on PDPs Ryan Jones | SEOTesting | @RyanJonesSEO
55. How to Run SEO Tests Ryan Jones | SEOTesting | @RyanJonesSEO
56. 1) Form Your Hypothesis 2) Select Your Control and Variant Pages 3) Make Your Change 4) Monitor Over Time 5) Next Steps (Rollout, Rollback, Iterate) Five Steps Ryan Jones | SEOTesting | @RyanJonesSEO
57. Your hypothesis should be statement-based, including something you’re not happy with, something you can change, and something that you can measure to determine whether you were correct or not. Hypothesis Ryan Jones | SEOTesting | @RyanJonesSEO
58. Bad Hypothesis: What will happen if we remove the product carousel from our polo shirt PDP? Will organic traffic / conversions increase or decrease? Hypothesis Ryan Jones | SEOTesting | @RyanJonesSEO
59. Good Hypothesis: Our category page for “running shoes” has a below average CTR of 2.1% in the SERPs. By adding structured FAQ schema markup answering common questions, we can improve our CTR to 3.5% or above within six weeks. Hypothesis Ryan Jones | SEOTesting | @RyanJonesSEO
60. Something You’re Unhappy With: Current CTR Something to Change: Adding FAQ Schema Markup Something to Measure: The Page’s CTR (After The Test) Hypothesis Ryan Jones | SEOTesting | @RyanJonesSEO
61. Rules for Selecting Control and Test Pages: - Match Page Templates (PDPs, PLPs) - Select High-Traffic Pages If Possible - Remove Outliers (Performance Outliers) - Match Seasonality (Tennis Rackets and Football Boots) Selecting Pages Ryan Jones | SEOTesting | @RyanJonesSEO
62. Step 1) Make the change to the test page/s. Step 2) Note the start date of the test. Step 3) Inform ANYONE that could impact your test data. Ryan Jones | SEOTesting | @RyanJonesSEO Making Changes
63. Ryan Jones | SEOTesting | @RyanJonesSEO Enterprise Solutions (seoClarity & SearchPilot) Hands-On Solutions (GSC/SEOTesting) Yourself/your team will be the one making the changes to your website. These types of tools will automatically make the changes on your site for you. ** Reflected in the cost…
64. Metrics to Track (SEO): - Clicks - Click-Through Rate - Impressions - Average Position - On-Page Metrics Ryan Jones | SEOTesting | @RyanJonesSEO Monitoring
65. Metrics to Track (Ecommerce): - Revenue / Total Sales - Purchases / Transactions - Product Views - Checkout Starts Ryan Jones | SEOTesting | @RyanJonesSEO Monitoring
66. Ryan Jones | SEOTesting | @RyanJonesSEO “An SEO test can technically be successful and increase traffic, but if it tanks conversion-based metrics, then was it really a successful test?” - Ryan Jones, November 2025.
67. ✅ Test Worked → Roll Out Sitewide ❌ Test Failed → Iterate Further 🤔 Unclear → Try a New Variation Ryan Jones | SEOTesting | @RyanJonesSEO ❌ Test Failed → Revert & Learn Next Steps
68. If you take one thing from this presentation… Let it be this: Use your site’s data to tell stories. Ryan Jones | SEOTesting | @RyanJonesSEO
69. 1 2 3 4 Problem Solution Test Results We realised that product images on our PDPs were slow to load, causing higher exit rates and lost conversions. We propose spending $2,500 in developer time to fix this page speed issue on our PDPs. To improve conversions. We fixed this page speed issue on 1,000 PDPs, leaving another 1,000 PDPs unchanged as a control group. The “fixed” PDPs increased in performance almost immediately. Leading to $14,000 more revenue. Ryan Jones | SEOTesting | @RyanJonesSEO
70. Ryan Jones | SEOTesting | @RyanJonesSEO Tools
71. Ryan Jones | SEOTesting | @RyanJonesSEO 1) Google Search Console 2) Google Analytics 3) Any Other Free Analytics Software Remember: Takes time, and you’ll need to pair this with Google Sheets or Microsoft Excel. Free Tools
72. Ryan Jones | SEOTesting | @RyanJonesSEO Paid Tools
73. Ryan Jones | SEOTesting | @RyanJonesSEO Resources
74. Ryan Jones | SEOTesting | @RyanJonesSEO Go Here 1) SEOTesting’s Blog (https://seotesting.com/blog/) 2) SEOTesting’s SEO Test Ideas (https://seotesting.com/blog/seo-testing-guide/) 3) Giulia Panozzo’s SEO Testing Workshop (https://www.youtube.com/watch?v=dzTwC77WW5g) 4) Adam Gent’s SEO Case Study Database (Paid $125) (https://adamgent.gumroad.com/l/seo-case-studies-database)
75. Thank You! 󰚥
76. Connect With Me! 󰛈 LinkedIn - linkedin.com/in/ryan-g-jones/ X - x.com/ryanjonesseo Bluesky - bsky.app/profile/ryanjonesseo.bsky.social