Navigating the AI Revolution: Elevating Your SEO with a Touch of Magic

Are you overwhelmed with all the new developments, possibilities, and challenges that seem to present themselves at an unprecedented speed? We certainly are. But instead of getting lost in the noise, let’s focus on what really matters to keep you ahead in the ever-evolving world of SEO. In this session, Bastian will share practical workflows, hands-on tips, and innovative ideas on how to harness AI for your SEO work in ways that almost feel like magic. With plenty of actionable insights, he will show you how to elevate your SEO and thrive in this AI-driven era.

Transcript

1. Navigating the AI Revolution Elevating Your SEO with a Touch of Magic Bastian Grimm | Peak Ace AG | @basgr 25th October 2024
2. We are on the brink of the most profound tech-transition in human history.
3. From healthcare to education and jobs – hardly any aspect of our everyday lives will remain unaffected by AI. AI is changing everything
4. The innovations of today are built on the technological breakthroughs made in recent years
5. 5 peakace.agency Vast amounts of data have been published online for decades – all of which are available as training material for LLMs. #1
6. Machines are now able to take over various human activities and make precise predictions. Progress in Deep Learning #2
7. Significant advances in cloud, infrastructure and consumer technology are making it easier and cheaper than ever to develop and deploy AI technology. Resource Availability #3
8. Those advancements will not only have an impact on AI development, but also on our work life
9. 9 peakace.agency With generative AI, 30% of the hours worked today could be automated by 2030. Source: https://pa.ag/3FbhX8J
10. 10 peakace.agency How people are using GenAI Six top-level themes give an immediate sense of what generative AI is currently being used for: Source: https://pa.ag/4brddtI • Technical Assistance & Troubleshooting (23%) • Content Creation & Editing (22%) • Personal & Professional Support (17%) • Learning & Education (15%) • Creativity & Recreation (13%) • Research, Analysis & Decision Making (10%)
11. For SEO, this is already the case.
12. I appreciate that you‘re all extremely busy…
13. This you?
14. 50,000+
15. 70,000+
16. My goal for today’s session: Give you back some of your extremely valuable time!
17. 17 peakace.agency What have I brought for you today? SEO Automation Internal Linking Redirects Custom GPTs
18. Before we dive in, let’s have a look at the key ingredients
19. What are Large Language Models (LLMs)?
20. Large Language Models (LLMs) are AI systems trained on vast data sets (thus “large”) to understand, predict and generate data using transformer-based neural networks. Simply put:
21. What are LLMs good at?
22. 22 peakace.agency Information Retrieval and Analysis LLMs can sift through large volumes of text data to extract relevant information, summarise key points, and answer questions, making them valuable for research, data analysis, and decision-making support. Personalised Recommendations LLMs can analyse user preferences and behaviour to provide personalised recommendations, such as articles or products, thus enhancing UX and engagement. Natural Language Processing LLMs excel in understanding language, making them ideal for applications such as chat bots, language translation, sentiment analysis, and text summarisation. What are LLMs good at?
23. What are LLMs NOT good at?
24. 24 peakace.agency Understanding Context Beyond Training Data LLMs may not perform well in situations requiring an understanding of context or knowledge beyond their original training data set. Making Ethical or Moral Judgments LLMs lack the ability to make ethical or moral judgments and should not be used in situations where such considerations are crucial. Most LLMs’ decisions are also biased. Limited Understanding and Reasoning LLMs can't form a chain of logical conclusions, instead they’re following probability rules; even if the most common answer to a question is irrational or outright wrong, it will still provide said answer. What are LLMs NOT good at?
25. 25 peakace.agency Keep in mind: There are risks that need to be managed (Obviously, this is true for both commercial and open-source models) Source: https://pa.ag/3Td5ucz Consent Ensuring training data is gathered responsibly, in compliance with AI governance and regulations. Security Security risks include data leaks or malicious use of LLMs by criminals. Bias Happens when the data source is not diverse or representative enough. Hallucinations Ensuring training data complies with AI governance and regulations.
26. 26 peakace.agency Will hallucinations ever disappear? "It’s inherent in the mismatch between the technology and the proposed use cases," says Emily Bender, professor in the Department of Linguistics and director of the Computational Linguistics Laboratory at the University of Washington. Source: https://pa.ag/3PqP0Mh LLMs are designed to predict the next word – of course there will be cases where the model is wrong.
27. 27 peakace.agency LLMs are not good at creating original content LLMs don’t “write” anything. They generate text based on probabilities and the number of parameters used in their training, using content they've encountered before.
28. In all the following cases, AI/ML and related technologies will be combined with other ingredients to elevate the final product's flavor and complexity Enough theory, now let’s get cooking
29. No-Code SEO Automation #1 Core Ingredients:
30. 30 peakace.agency Source: https://pa.ag/3Usq4oI Have you tried Make.com? Boost productivity across every area or team. Use Make to design powerful workflows without having to rely on developer resources.
31. 31 peakace.agency Tons of pre-built modules Scraping, parsing, reading/writing/storing data in different formats & sources, any-2-any API connections, etc.
32. 32 peakace.agency More than "just" no-code workflow automation Simply drag and drop apps to automate existing workflows or build new complex processes to save time: e.g., create short form social media copy based on your WP posts using ChatGPT, then send to LinkedIn, FB, etc. Source: https://pa.ag/3Usq4oI
33. 33 peakace.agency WordPress is dominating the CMS market Source: https://pa.ag/3BQXCr1 43.5% of all websites are using WP (based on W3Tech‘s data)
34. 34 peakace.agency Source: https://pa.ag/3UgCdxO Lots of pre-built functions for WP out of the box Before you can use any of these, you need to install the WP plug-in and specify an auth-key
35. "Make an API Call" The one that excites me the most?
36. 36 peakace.agency Query (your) WordPress through its built-in API e.g. fetching (pre-selected/filtered) posts from your website, or literally anything else you can think of: Source: https://pa.ag/4dLEO9L
37. 37 peakace.agency Run Make.com‘s WordPress module (later, auto-schedule) You'll get all the details for a single post, from the title and content to the metadata and more:
38. You‘ve got the data. Great, but now what?
39. How about Google Search Console, just for fun? Pass it to any other module you can possibly imagine
40. 40 peakace.agency Before being able to query GSC, you need to transform WordPress’ API response:
41. 41 peakace.agency The built-in JSON parser is here to help! The WP API call returns a JSON response by default, which means we need to access the response body and get the link-attribute value (as string/text):
42. 42 peakace.agency This took me almost no time to build: Live indexing check using GSC API
43. Generating alternative copywriting titles (per URL) based on suggestions from Google Search Console to improve CTR Endless possibilities
44. Creating/suggesting FAQ sections for your existing content including structured data Endless possibilities
45. Sentiment analysis using Google Cloud Natural Language, with insights from Gemini (Vertex AI) on how to improve content Endless possibilities
46. You get the idea…
47. Internal Linking #2 Core Ingredients:
48. Start small, validate initial ideas & concepts, then strategically scale up where feasible. For any of the cases
49. There’s often more than one solution to achieve the same goal
50. How well does your internal linking reflect your website’s key topics, and how can you find out?
51. Analysing and understanding topical clusters is crucial for assessing relevance and thematic context of internal links. For background:
52. 52 peakace.agency Collect internal URL inventory with a crawling tool Export to Excel (or Sheets) and filter out irrelevant links (e.g., to 404s, noindex, etc.). Once done, feed the data into Gephi and watch the magic unfold: Gephi is a powerful tool that transforms complex data into dynamic visual networks, making hidden connections instantly visible.
53. 53 peakace.agency Gephi turns data into knowledge Gephi delivers data-driven insights by visualising your website's internal linking, hierarchy, and topical structure, automatically calculating PageRank and Modularity metrics: Source: Gephi Gephi in a nutshell: ▪ Turn raw data into actionable insights with just a few clicks ▪ Uncover internal linking patterns and relationships with Gephi’s visual maps ▪ Pinpoint key nodes and clusters using PageRank and Modularity classes Structure of most "regular" websites:
54. 54 peakace.agency PageRank & Modularity illustrate authority and clusters A brief overview of key concepts and their significance While PageRank focuses on the importance of individual nodes (pages), Modularity helps identify groups of interconnected nodes (communities). PageRank ▪ PageRank (in this case) is calculated within a single website ▪ The calculation is based on how pages are linked to each other ▪ Pages with many incoming links from other authoritative pages have a higher PageRank Modularity ▪ Modularity measures how well a network decomposes into modular communities ▪ In the context of website analysis, communities represent groups of closely related pages within a site ▪ The objective is to minimise clusters, while ensuring that each cluster contains only thematically related pages
55. Using Python to calculate existing PageRank and simulate potential future changes from internal linking adjustments Another solution
56. 56 peakace.agency PageRank recalculation prior to implementation Johan von Hülsen's script lets you test optimisations by calculating PageRank for different scenarios, showing URL relevance changes before implementation: Script: https://pa.ag/4e28PTG URL Internal PageRank New Internal PageRank index pages only Change in % /kontakt/ 0.01653 0.00076 - 95.4% /presse/ 0.00779 - - 100% /auszeichnungen/ 0.00582 - - 100% /einlagensicherung/ 0.00552 0.00797 + 44.3% /depot/ 0.00531 0.00797 + 50.0% /fondsuebersicht/ 0.00528 0.00795 + 50.5% /etf/ 0.00517 0.00797 + 54.1% /aktien/ 0.00511 0.00795 + 55.5% /steuern/ 0.00501 0.00915 + 82.6% The main difference is that the script focuses on relevance and PageRank, while Gephi also accounts for thematic relationships through Modularity.
57. Manually optimising for better Modularity and PageRank distribution is time-consuming and error-prone.
58. 58 peakace.agency Optimise PageRank and Modularity with ChatGPT Use PageRank and Modularity data, provide ChatGPT with insights on whicyh landing pages need better linking, and let AI recalculate for optimised results, streamlining the process for efficiency and speed. Export data from Gephi (PageRank, Modularity) Visualise the new data ChatGPT optimises the data based on your requirements
59. 59 peakace.agency AI-powered internal linking: ChatGPT ‘knows’ Gephi data Leverage ChatGPT to analyse and optimise internal linking by uploading PageRank and Modularity data directly from Gephi.
60. 60 peakace.agency AI-powered internal linking: provide focus areas For the next step, provide corresponding instructions to ChatGPT - which pages or areas should be focused on? Which should be removed? Where to focus for better interlinking? In our case: remove certain sections from linking entirely, include specific pages we consider important, and provide focus areas or core topics for more prominent overall linking.
61. 61 peakace.agency AI-powered internal linking: download new dataset Based your instructions, ChatGPT optimises the existing data and completely recalculates the PageRank and Modularity for the URLs and topics
62. 62 peakace.agency Compare old vs new (utilising Gephi again) Watch as new PageRank and Modularity calculations reveal the optimised internal linking structure and significantly improved topical clustering between pages:
63. 63 peakace.agency The ’AI Revolution’ in internal linking: faster, better results By leveraging AI (ChatGPT) and smart automation this process becomes faster, more efficient, and more accurate Key-benefits include: ▪ Incredibly easy to use, suitable for junior- and mid-level SEOs ▪ Data-driven and far more accurate than “traditional guesswork” ▪ Significantly faster than manual methods, freeing up valuable time for other tasks ▪ Manually evaluating topical clusters in internal linking is nearly impossible, especially on large-scale websites Sistrix visibility development after deployment:
64. Redirects #3 Core Ingredients:
65. AI-powered redirect mapping is much faster and more efficient than using spreadsheets
66. 66 peakace.agency Embeddings and vector database = redirect win Necessary steps for better automated redirects (and an improved customer journey): Extract main content of every (old) site/URL Generate embeddings Save together with metadata in vector database Semantic search in vector DB based on embeddings of old URLs
67. Embeddings are numerical vectors representing words, capturing their meanings and relationships in a multidimensional space. What are embeddings?
68. You can convert any word into a vector and start calculating with them: "king" minus "man" plus "woman" equals "queen". Synonyms and more can also be found this way. What are embeddings?
69. A vector DB utilises data embeddings as index, facilitating fast and scalable searches among unstructured data points, enhancing efficiency in retrieving similar items or information. What about vector databases?
70. A vector DB allows you to find matches between anything and anything (e.g., use an image as a query to find similar pieces of text, video, other images, etc.). Simply put:
71. A quick, step-by-step overview: Putting it all together
72. 72 peakace.agency Extracting the main content of every old URL <title> tag <h1>s each first & last sentence <p> <h2>s <h2>s Combine everything Content = Title + h1 + h2s + … ▪ Extract: <title> + main content ▪ Combine: <title>, <h1>, <h2>s and first & last sentence of each paragraph
73. 73 peakace.agency Generate embeddings and store in vector database For each website URL: ▪ Transfer previously generated content to vector DB ▪ Generate embeddings (BERT, GloVe, FastText) ▪ Save embeddings in a vector DB incl. metadata (URL, title, etc.) Content Content Content 0.03 … 0.19 -0.21 … 0.03 0.08 … -0.15
74. 74 peakace.agency Search the vector database for the best semantic match For every outdated page: ▪ Vectoric semantic search for KNN (k-nearest neighbour) ▪ Set 301 to NN URL ▪ No more weak redirects ▪ Play with certainty/ temperature settings 0.31 … -0.41 {Get { Article ( nearVector: { limit: 1, content: { vector:[embedding], certainty: 0.8 } } ) { url } }} Future 404
75. I know… efficiency and all! Don‘t want to do this all by hand?
76. 76 peakace.agency ScreamingFrog has a killer feature to do this out of the box In my defence, I still believe it’s crucial to understand what's happening “behind the scenes”… Source: https://pa.ag/40fdlu2 Turn on JS Rendering, then head to Configuration > Custom > Custom JavaScript: Select Add from Library > (ChatGPT) Extract embeddings […] > Click on “JS” to open the code and add your OpenAI key:
77. Down the rabbit hole…
78. 78 peakace.agency State-of-the-art sentence transformers are the gold standard The Levenshtein distance (basic fuzzy matching) provides an alternative, as we’re mainly dealing with small text snippets and minimal deviations between URL versions: Source: https://pa.ag/49RHG3y The more substantial the changes between two versions, the higher the likelihood that you’ll reap significant benefits from leveraging sentence transformers. h/t Will Nye for the data set
79. Calculating similarity scores across multiple elements and selecting the best matches always works best. Rule of thumb
80. … you need solid QA afterwards! Whatever you choose…
81. Garbage in = garbage out! Don‘t forget about input quality
82. Analyse page contents and automatically create redirect maps based on two (old vs new) SF crawls. Facebook AI Similarity Search (FAISS)
83. 83 peakace.agency Automated redirect matchmaker for site migrations Fantastic script by Daniel Emery utilising two SF crawls (origin + destination.csv with titles, metas, URLs and headings) to perform a fast semantic search (using sentence transformers) and create a redirect map: Sources: https://pa.ag/4bWAgxy & https://pa.ag/3USteUJ FAISS is an outstanding library designed for the fast retrieval of nearest neighbours in high- dimensional spaces. It enables quick semantic nearest neighbour searches even on a large scale.
84. Not 100% perfect, but ~90% accurate/sensible matches are perfectly realistic. Significant time savings
85. As with most things, it can boost efficiency, but it isn't a complete replacement for a human.
86. Be smart with your redirects: put them on the edge
87. 87 peakace.agency Cloudflare Workers to execute redirects on CDN/edge level I already spoke about using CF Workers for a variety of technical SEO tasks including redirects at the SMX Advanced in Berlin back in 2021. Looking to dive deeper? Make sure to grab a copy of the deck: Source: https://pa.ag/4bSxauE Pro tip: this rarely requires dev resources; either you can do it yourself, or use sys ops (less busy)
88. Core Ingredients: Custom GPTs for ChatGPT #4
89. Custom GPTs are a way to create tailored, custom versions of ChatGPT that combine instructions, extra knowledge, and any combination of skills. What are Custom GPTs (for ChatGPT)?
90. 90 peakace.agency A Custom GPT in its simplest form: Using Peak Ace’s Structured Data GPT to debug and fix errors in JSON-LD mark-up
91. 91 peakace.agency Noticed how I provided no instructions to fix the JSON? You need significantly fewer instructions (per prompt), such as a specific context, as you already provided these details when you created/trained/set up the Custom GPT:
92. Greatly increase your teams‘ productivity with Custom GPTs (and pre-defined workflows)
93. 93 peakace.agency Making GPTs smarter with external data A Custom GPT can also be used to fetch additional information from a third-party data-source via API:
94. Here‘s a quick three-step guide on how to DIY it. So, how can you build this yourself?
95. 95 peakace.agency #1 Provide basic info to get started (name, description, …) Log in to ChatGPT > choose Explore GPTs > Create (you need ChatGPT Plus) Well defined instructions are key, think prompting.
96. 96 peakace.agency #2 Create an ‘Action’ to call a 3rd party API Head to your API provider and grab your credentials. In our case this was the API Dashboard at DataForSEO.com: Get the OpenAPI Schema for DataForSEO: https://pa.ag/3Pa7oZ3 To use with an action, you need to generate a base64- encoded version of your login credentials: btoa(‘APIemail:APIpass’) The annoying part: you need a Schema according to the OpenAPI spec. But no one reads docs anymore – we just leverage ChatGPT to do this:
97. Remember: APIs usually aren‘t free, so make sure you only publish your new Custom GPT for yourself! #3 Test and publish your GPT
98. Just reauthenticate (base64-encoded version of your login). You also need a new schema (again based on OAS spec). Customisation for other APIs is easy (e.g., Sistrix, etc.)
99. 99 peakace.agency Did you know you can link using pre-filled prompts? You can also link directly to pre-filled prompts and execute them. This works for both Custom GPTs and GPT-4o models. Simply add the query string (using “q=xxx“) to the end of your ChatGPT URL. Source: https://pa.ag/crsum 𝗙𝗼𝗿 any C𝘂𝘀𝘁𝗼𝗺 𝗚𝗣𝗧 𝗮𝗱𝗱: ?q=your+prompt+goes+here 𝗙𝗼𝗿 the 𝗚𝗣𝗧-𝟰o 𝗺𝗼𝗱𝗲𝗹: ?model=gpt-4o&q=your+prompt Use directly in your Chrome browser
100. 100 peakace.agency When to use a Custom GPT? Long-term context Custom GPTs are a powerful tool to ensure that instructions remain contextualised over long periods of time. In addition to seamless third-party data integration, here are my top three reasons why building and using Custom GPTs is highly beneficial. Building workflows Custom GPTs are ideal for creating workflows for individuals who may not know how to effectively design contextual prompt sequences. Sharing instructions For sharing the same instructions across teams, without having to worry about specifying them (or how) at the prompt level.
101. How about we get rid of the annoying "copy and paste" when using Chat GPT? Remember my promise to give you some of your time back?
102. 102 peakace.agency Let‘s set up Make.com to listen to ChatGPT traffic For this to work, we‘ll need a make.com Custom Webhook, OpenAPI spec for said Webhook and a Custom GPT which acts the frontend to forward your data: ActionsGPT: https://pa.ag/4eS196T - or copy the spec: https://pa.ag/3NBGp7D New Scenario > Custom Webhook > Create Use OpenAI’s ActionsGPT + the prompt below:
103. 103 peakace.agency You all know the drill: setting up a new Custom GPT
104. 104 peakace.agency Before we run it, let’s decide where to send the data. Let’s store it in Google Sheets and add a new row for each response that ChatGPT produces:
105. 105 peakace.agency Let‘s give this a try, shall we? Ask your Custom GPT anything, which will then send the data to your sheet automatically:
106. 106 peakace.agency If you need to dive deeper, check the data flow in Make.com Each module provides its own specific output including operational details and the actual data going through:
107. From now on: no more copy-pasting needed!
108. 108 peakace.agency At Peak Ace, we use Custom GPTs everywhere For individual tasks and client teams working on specific projects – all aimed at driving efficiency and streamlining processes. A few examples are listed below, though there are many more...
109. Thank you! Ciao, London. =