Tips and Tricks for Understanding User Search Intent

10 tips and tricks that we've learnt over the years on how to how start understanding your user search intent

These can be applied to your Google Search Console data, your internal site search data and data pulled from keyword data tools such as Ahrefs, SEMRush and Sistrix

First presented at London Measurecamp 2023

Transcript

1. Author: Charles Meaden Tips and Tricks To Really Understand Charles Meaden Digital Nation
2. Author: Charles Meaden No More Than 30 Seconds On Me • Started Digital Nation 26 years ago • Specialise in search (paid and organic) and analytics • Reside in the strangely named The Mumbles (Swansea, Wales) • If you’re looking to move out from the city, you get views like thse
3. Author: Charles Meaden What’s The User Intent? • The phrases users type into search boxes are a goldmine of useful data • The combination of words lets us know precisely what the user was looking for • Also, where they are on the journey – Best lightweight hiking boots • Informational query – Roclite 345 price • Transactional query
4. Author: Charles Meaden Quickly Establish What People Are Looking For • The most searched term on UK councils sites is… • Recycling – People need to know what weeks and what recycling to put out
5. Author: Charles Meaden Top Level Search Intent • The most common are – Navigational – Informational – Commercial – Transactional • Google has a slightly different model – Know” queries: – “Do” queries: – “Website” queries: – “Visit-in-person” queries • My challenge with them is that they are way too broad
6. Author: Charles Meaden I hate word clouds… • Single words that is
7. Author: Charles Meaden 2 Or 3 Words Provide Context • In this case adding – Gender – Brand – Colour • Tells us far more about what someone is looking for
8. Author: Charles Meaden There Is A Lot Of Data To Be Crunched • For two major UK retailers we’ve extracted over 750,000 unique search terms via the Google Search Console API • For our GoSimpleTax client with a lower search volume per month, you’re still looking at – 82,000 unique search terms – 7,800 different words – 26,000 question phrases • What, where, how
9. Author: Charles Meaden Google Search Console to Big Query Export • Google slipped this out in May 2023 • Does a daily export to Big Query • Tested it across 3 different sized clients • You can do some really clever stuff like generate ngrams • If you are at involved with search you should enable this
10. Author: Charles Meaden Tip 1: Every Data Set Is Slightly Different • We’ve worked on search query intent projects for a wide range of customers • The phrases people use to find them and enter in their internal search engines are slightly different • Take the time to build an adaptable model which you can use time and time again
11. Author: Charles Meaden Tip 2: Eyeball The Data • Before running any automated process over the top, use your eyes • You’ll get a feel for the data • You spot patterns and anomalies in the data straight away
12. Author: Charles Meaden Tip 3: Find The Most Used Words • This will allow you to spot quickly any potential issues • In this example we’ve got – Plurals – Apostrophes – Synonyms • You need to decide for each project which ones to change • The list was generated using the Hermetic Word (and Phrase) Frequency Counter Advanced Version
13. Author: Charles Meaden Tip 4: Clean Up Your Data • Assume your original data isn’t perfect • Lowercase all your search terms – Thank you GA4 for not lowercasing search terms • Remove unnecessary spaces • Remove apostrophes and any unnecessary special characters • Correct common spelling mistakes • My favourite tool for doing this is Analytics Edge – Excel plugin for Windows – Mac and standalone versions in Beta • Allows me to create macros that automate common text cleaning tasks
14. Author: Charles Meaden Tip 5: Depluarlise Your Words – Part 1 • What we are looking for is the intent • Tracksuit and tracksuit is the same word • Women, womens and women’s is the same word • This regular expression will do the trick • \b(\w+)(s)\b • However there is a gotcha…
15. Author: Charles Meaden Tip 5: Depulararise Your Words – Part 2 • Taking off the s works for most words • But not if they are a name – Brands – Citys • Eyeballing the data will help you spot these – Reusable queries are even better • Have a routine that adds the S back to these
16. Author: Charles Meaden Tip 6 – Make A Decision On Synonyms • Some phrases are treated the same by search engines – Does your internal search do the same? • Kids and Childrens • Ladies, Ladys and women • There is no hard and fast rule here, apart from common sense
17. Author: Charles Meaden Tip 7: Two Word Combinations • Some words were meant to go together • Splitting these up makes analysing them harder – HMS Daring – Ralph Lauren – San Francisco – Trailfly 270 • Add a hyphen between these words – hms-daring – ralph-lauren – san-Francisco • These will now be treated a single words
18. Author: Charles Meaden Tip 8: Word Order Doesn’t Always Matter – Part 1 • What’s the difference in intent between these queries – womens french-connection jeans – french-connection womens jeans – jeans womens french-connection • None what so ever - they’re all looking for the same thing • Sorting the phrases into alphabetical order and deduping can massively reduce the number of phrases you need to work with • On a recent project, we cut the number of phrases we needed to analyse by 50% • This is how we do it – We do it in Analytics Edge using a macro, but the process could be easily adapted
19. Author: Charles Meaden Tip 8: Word Order Doesn’t Always Matter – Part 2 1. Load your queries into a table 2. Take each query in turn such as ‘womens french- connection jeans’ 3. Split this into separate words each on a separate row 4. Sort alphabetically 5. Merge the rows into one row and separate by a space 6. You’ll now have a query that looks like this ‘french- connection jeans womens’ 7. Repeat across all queries 8. Deduplicate the data and total any numerical columns
20. Author: Charles Meaden Tip 9: Look For Common Word Combinations (Ngrams) • It’s easy to spot patterns if it’s just 100 phrases • Harder at 1000, • Impossible at 10,000 or more • Couple of handy tools • Hermetic Word Frequency Counter – Best £30 you’ll spend I promise • Analytics Edge • Big Query ML.Ngrams function – Calculate 2,3 and 4 word combinations over millions of rows in seconds
21. Author: Charles Meaden Tip 10: Build Up Libraries of Common Phrases • Every project will be similar, but different • We build up libraries of common terms – A general level – Client and Industry specific • Some examples – Question words such as what, why, how – Colours – Industry specific terms
22. Author: Charles Meaden What Next – A Couple of Ideas • Clustering – Look for common patterns using machine learning – https://www.keywordinsights.ai/ – Follow Lee Foot on Twitter @LeeFoot as he has done some amazing stuff using Python • Missing Content on Your Sites – Combine the data with data from website crawls and SERP scraping tools such as ValueSERP to find missing content gaps • Product Discovery – What products and services are people looking for that you don’t have
23. Author: Charles Meaden Thank You • Find me on Twitter and Linkedin • https://twitter.com/charlesmeaden • https://www.linkedin.com/in/charlesmeaden/