TW-Bert Good for Users, Good for SEOs (Brighton SEO April 2024)

Term Weighting BERT helps Google fix issues with showing the right results based on users intent which BERT alone didn't quite achieve. The step forward involves a clever but simple statistical model: find out why Google need it, how it works and how as SEOs we can begin to optimise for it.

Transcript

1. TW-BERT: Good for Users, Good for SEOs Dave Cousin Oban International @davetheseo https://bit.ly/tw-bert https://www.linkedin.com/in/davecousin/
2. obaninternational.com What we’re going to cover Why BERT needs improving? What’s Term Weighting? What is BERT? What’s TW-BERT? Where this fits in Google’s System? How to start optimising obaninternational.com
3. TW-BERT: the basics
4. obaninternational.com obaninternational.com Humans understand context intuitively Nike Running Shoes Nike Running Shoes Running Shoes obaninternational.com obaninternational.com
5. obaninternational.com obaninternational.com What TW-BERT is basically •A model •Developed by Google Researchers •Explained in a paper called ‘End to End Query Term Weighting’* * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815
6. obaninternational.com obaninternational.com What TW-BERT is basically •TW-BERT helps(/will help) Google think like a human • Deciding what results to show • What order to show them •Massively improves search intent understanding * https://dl.acm.org/doi/pdf/10.1145/3580305.3599815 It understand the importance of words and sequences of words Whereas BERT currently deals with wordpieces Because…
7. The problem to be solved
8. obaninternational.com obaninternational.com The problem for users •BERT was meant to understand intent better than ever •It helps Google… “grasp the subtle nuances of language that computers don’t quite understand the way humans do”
9. obaninternational.com obaninternational.com The problem for users • Users have over the past couple of years been maybe more critical of Google’s results than ever:
10. obaninternational.com obaninternational.com The problem for SEOs •We’ve all had SERPs where we used to be top and we see who took over and think WTF
11. obaninternational.com obaninternational.com Real or imagined? • There’s even a 2024 research paper saying Google’s results are worse: https://downloads.webis.de/publications/papers/bevendorff_2024a.pdf • Ok it’s not due to BERT • But BERT hasn’t fixed it • There IS more Spam
12. The Solution
13. obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words are important? •Non Weighted: • Match MORE terms Where can I get great Korean BBQ in Brighton
14. obaninternational.com obaninternational.com What is Term Weighting? •Which sequences of words are important? Weighted: • Match IMPORTANT terms Where can I get great Korean BBQ in Brighton
15. obaninternational.com obaninternational.com Words, n-grams and wordpieces Wordpieces (19): I|want|##ed|a|new|##er|day|##ly|##s|##paper|from|the|##agent|to| ##day|get|##est|one|##morrow I wanted a newer daily newspaper from the newsagent today, I want to get the newest one tomorrow Unigrams (16): |I|wanted|a|newer|daily|newspaper|from|the|newsagent|today|want| to|get|newest|one|tomorrow| Bigrams (17): |I wanted|wanted a|a newer|newer daily|daily newspaper|newspaper from|from the|the newsagent|newsagent today|today, i|i want|want to|to get|get the|the newest|newest one|one tomorrow|
16. obaninternational.com obaninternational.com Importance of n-grams 30,000 Wordpieces 35,000 Words
17. obaninternational.com obaninternational.com Importance of n-grams •Humans, understand other human’s without consciously thinking about it 30,000 Wordpieces 10 million+ N-grams
18. obaninternational.com obaninternational.com Importance of n-grams •We identify phrases or terms we recognise 30,000 Wordpieces 10 million+ N-grams
19. obaninternational.com obaninternational.com Importance of n-grams •Challenge to replicate in NLP 30,000 Wordpieces 10 million+ N-grams
20. obaninternational.com obaninternational.com Importance of n-grams •Needs massive high dimensional neural networks and training corpora 30,000 Wordpieces 10 million+ N-grams
21. obaninternational.com obaninternational.com What is BERT? Bidirectional Encoder Representations from Transformers
22. obaninternational.com obaninternational.com What is BERT? •Deep Learning based Lexical Retriever for NLP Bidirectional Encoder Representations from Transformers
23. obaninternational.com obaninternational.com What is BERT? •Based on a (relatively) huge corpus or training material Bidirectional Encoder Representations from Transformers
24. obaninternational.com obaninternational.com What is BERT? •This allows the creation of a neural network (right) Bidirectional Encoder Representations from Transformers
25. obaninternational.com obaninternational.com What is BERT? •Each wordpiece is mapped to others with direction & magnitude Bidirectional Encoder Representations from Transformers
26. obaninternational.com obaninternational.com •Weights all wordpieces •Dynamic and considers: •Context •Meaning •Semantics •Sentiment What BERT does to search queries? •Expands queries: not just synonyms also semantic & contextual terms e.g. +‘restaurant’ to ‘Korean bbq’ bose noise cancel ##ing head with micro and blue tooth ##phone
27. obaninternational.com obaninternational.com What BERT does to search queries? bose noise cancel ##ing head with micro and blue tooth ##phone
28. obaninternational.com obaninternational.com What BERT does to search queries? bose noise cancel ##ing head with micro and blue tooth ##phone
29. obaninternational.com 29 The basics of how search engines work Crawl (+Render) Index Ranking Algorithm Search / SERP Send Fetch Results Query
30. obaninternational.com 30 IR and where BERT fits in Index Match Score Ranking Algorithm BERT Search Candidate Docs Fetch Results Query Weighted expanded 3 : 2 Scored Pairs
31. obaninternational.com obaninternational.com TW-BERT’s bridging solution BERT’s Wordpiece Island TW-BERT Statistical Bridge Opened 2023 Fast Efficient Can Misunderstand Deeper Understanding of Language Slow Expensive Isle of Deep Learning
32. obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise cancel ##ing head with micro and blue tooth ##phone
33. obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise cancel ##ing head with micro and blue tooth ##phone
34. obaninternational.com obaninternational.com What TW-BERT does to search queries? bose noise cancel ##ing head with micro and blue tooth ##phone
35. obaninternational.com 35 And where TW-BERT fits Index Match Score Ranking Algorithm TW- BERT BERT Search Candidate Docs Fetch Results Query Weighted, expanded 3 : 2 Scored Pairs Feedback Loop
36. obaninternational.com obaninternational.com The paper: End to End Query Term Weighting •Findings Vs BERT & Deep learning:
37. obaninternational.com obaninternational.com The paper: End to End Query Term Weighting •Findings Vs BERT & Deep learning: Better overall precision Particularly good for top 10, top 100
38. obaninternational.com obaninternational.com The paper: End to End Query Term Weighting •Findings Vs BERT & Deep learning: Performs very well for 1-10 word terms
39. obaninternational.com obaninternational.com Is Google using TW-BERT? •Probably….. a bit •Why wouldn’t they? •It helps fix a major flaw •Some put July/August fluctuations down to TW- BERT: Source: Algoroo
40. obaninternational.com obaninternational.com If Google are, then is this it? •Can tweak this and significance •More feedback loop data = more certainty and accuracy •Easy to copy for other search engines
41. obaninternational.com obaninternational.com Are N-Grams the new keywords? •No KWs aren’t dead •Just a bit limp – another step in their diminishing importance
42. obaninternational.com obaninternational.com Significance of n-grams in SEO •We have to optimise for intent first •KW stuffing will do nothing if the content doesn’t deliver
43. obaninternational.com obaninternational.com Significance of improved query expansion •More competition from relevant content without the same KWs •Users will use more long tail & natural language
44. obaninternational.com obaninternational.com Are you ready for TW-BERT? •Less competition from irrelevant results •But what if you have the irrelevant result? •Put yourself in the shoes of searchers?
45. obaninternational.com obaninternational.com How to start optimising now: Match intent •Understand intent •What is ranking? •Who has gained? •What are they doing? •Look at SGE too for intent
46. obaninternational.com obaninternational.com How to start optimising now: n-gram optimisation •Identify important n- grams in: • Related hero search terms • Questions or 5+ words 1. How to make a beef goulash? 2. How to cook a beef goulash? 3. Cook a beef goulash 4. Beef goulash recipe 5. Step by step beef goulash 6. Beef goulash how to
47. obaninternational.com obaninternational.com How to stay optimised •Check every 3-6 months for key terms: • How’s language changing? • Is knowledge changing? • Consider intent in different countries • What is it? • Is it a scam? • How To Invest? • Focus on BitCoin • News • Prices • Latest scams 2012 2017 2024 ‘cryptocurrency’ intent over time
48. obaninternational.com obaninternational.com So Good for Users, Google and SEOs? • Improved intent matching helps Users find what they want faster • And see less answers full of brand mentions Users • More likely to keep using Google as a search engine, • Brand can’t cheat their way to the top, some will buy ads instead Google • SEOs get more relevant, qualified traffic • Don’t have to go after irrelevant terms SEOs @davetheseo https://bit.ly/tw-bert linkedin.com/in/davecousin