How to audit for AI Accessibility on your Front & Back End

An introduction to technical AI SEO from Dave the SEO, presented at the Barcelona International Search Summit on the 13th November 2025. Covering current standards, what's coming up right now and in the near future as AI Search and LLM Chatbots continue to evolve.

Transcript

1. How to audit AI accessibility of your site's front end & back end davetheseo.com linkedin.com/in/davecousin/ @davetheseo.bsky.social
2. OR davetheseo.com linkedin.com/in/davecousin/ @davetheseo.bsky.social
3. An intro to Technical AI SEO davetheseo.com linkedin.com/in/davecousin/ @davetheseo.bsky.social
4. Intro This stuff wasn’t a thing 24 months ago Most of it wasn’t 12 months ago Apologies if this time next year it’s out of date Or even next week
5. Intro LLM Chatbots aren’t search engines But they are used for Search
6. Intro We need to optimise for LLM Chatbots & AI Search While remembering they aren’t built like what we’ve known to date
7. LLM Chatbots aren’t search engines: • Designed to extract information • Not recommend pages Want to access information: • Easily as possible • Machine friendly way, • For themselves
8. Machine Friendly AI is trying to understand a web built for Humans Our job is to make it accessible to Machines (LLM Chatbots & other AI Tools)
9. Front End & Back End Your Front End is your website, The interface humans use to: •Interact with your Business •Access Information •Carry out Actions Google spent decades understanding the web ‘like a human’
10. What is the Backend? Endpoints allow connection directly to where your site’s data exists.
11. Timeline LLMs Search (for RAG) Fetch Text from Pages Yesterday LLMs using browser based tools LLMs take actions on site Today Tomorrow Agentic Commerce ACP & More Feeds & Discovery MCP / Back End Access N
12. Timeline LLMs using browser based tools LLMs take actions on site Today Tomorrow Agentic Commerce ACP & More Feeds & Discovery MCP / Back End Access ? Next Week
13. Yesterday
14. AIOs Uses Search Engine(s), then check live page Uses an Index to find Candidate Documents * *Gemini searches Google and only retrieves pages from an index
15. How & Why LLM Chatbots Search? RAG for: (Retrieval Augmented Generation) • More complex queries • Recent information Use a Search Engine
16. AI Assistant Search Engine Gemini Google ChatGPT Google Bing Claude BraveSearch Copilot Bing Meta Google Bing
17. What AI Tools can (and can’t see) Visible Text Anything Else
18. What AI Tools can (and can’t see) Visible Text if in source code & maybe marked up Anything Else What AI Tools can (and can’t see)
19. Basic Accessibility Standards ChatGPT can see: • All visible text content • Status Codes (if none provided then this generally indicates a 200) Claude can see: • Visible Main Content Text • Where there would be images & Alt Text • Errors e.g. ‘Unable to fetch the URL’ All can see: • Headings • Links • Lists (bullet points and numbered) • Tables Perplexity can see: • Relevant sections of pages, based on a retrieval algorithm • Probably uses H Tags, Schema and Semantic HTML as a minimum to understand and chunk up page Gemini can see: • Everything, Rendered page stored in Index
20. Assuming you don’t block Them
21. Assuming you don’t block Them Robots.txt
22. Assuming you don’t block Them Server Errors 5xx, 0, 4xx (user gets 200)
23. Assuming you don’t block Them Server Errors: Blocked User Agents
24. Assuming you don’t block Them Server Errors: Not Explicitly Allowed User Agents
25. Assuming you don’t block Them Server Errors: Out of Date Published IPs or WAF
26. Assuming you don’t block Them Dynamic Redirects e.g. to Human Verification
27. What do you Want them to See Probably Allow ClaudeBot GPTBot PerplexityBot Google-Extended meta-externalagent ccBot Applebot-Extended FacebookBot Allow ChatGPT_User Claude-User Claude-SearchBot Perplexity-User Googlebot GoogleAgent-Mariner NovaAct Gemini-Deep-Research
28. Auditing Stage 1: Screaming Frog
29. Auditing Stage 1: Screaming Frog
30. Auditing Stage 1: Screaming Frog
31. Auditing Stage 1: Screaming Frog
32. Auditing Stage 1: Screaming Frog
33. Auditing Stage 1: Screaming Frog
34. Auditing Stage 1: Screaming Frog
35. Page Accessible in Crawler = The LLM Chatbot can see it Page Not Accessible in Crawler = It might be your IP Address
36. Prompting AI to check: Without Hallucinations ChatGPT: “Try to open this URL [URL] and tell me exactly what happens including any error codes or partial results.” All: “On this URL [URL] can you see the text [text]? What text comes immediately afterwards? Let me know of any error messages you get fetching the page”
37. Auditing What LLM chatbots can see Stage 2: Ask the Bots
38. Log Files make it much easier
39. Fixing your Security Settings: WAF Use WAF (Web Application Firewall) explicit settings and Update Published IP Ranges ChatGPT: https://help.openai.com/en/articles/11845367-chatgpt-agent- allowlisting Claude: https://docs.claude.com/en/api/ip-addresses Perplexity: https://docs.perplexity.ai/guides/bots
40. Fixing your Security Settings: Robots.txt More Specific Rules take Precedent…. …Then Order Explicitly allow known bots Test in Screaming Frog
41. Fixing your Security Settings: Robots.txt More Specific Rules take Precedent Explicitly allow all known bots Test in Screaming Frog
42. Fixing ‘Invisibility’ You could ask your developers to help
43. Fixing ‘Invisibility’ Or take matters into your own hands e.g. Prerender & CDN
44. Today
45. From Project Mariner to Atlas & Comet Rendering pages for users takes a lot of resource and energy So let’s use someone else’s resources & Energy
46. AI Assistant Browsing Ability Gemini Project Mariner Future Version of Chrome? ChatGPT Atlas Browser (MacOS, windows to come) Claude Puppeteer & Other Add-Ons Copilot CoPilot in Edge Perplexity Comet Browser
47. What an AI Browser Session Can See Snap Shots The Rendered Code incl. •Markup & Tags
48. Standards for Browser Based AI Accessibility Make ‘human’ pages ‘Machine Friendly’ with: • Semantic HTML • Structured Data – any available • Headings • ARIA Roles, Labels, States & Properties • Static Links Semantic HTML Tags: • <article> • <details> • <footer> • <header> • <main> • <nav> • <section> • <summary> Common useful ARIA Roles include: • role="main" • role="navigation" • role="button" • role="banner" • role="contentinfo" https://www.w3schools.com/html/html5_semantic_elements.asp
49. Standards for Browser Based AI Accessibility •Are all elements easy to interact with using: • Tick Boxes • Radio Buttons • Text Boxes https://www.w3schools.com/html/html5_semantic_elements.asp
50. Standards for Browser Based AI Accessibility Avoid Interactions requiring Human Input Avoid Human Verification https://www.w3schools.com/html/html5_semantic_elements.asp
51. Tomorrow
52. MCP, ACP More Protocols to Come This could become a standards war So far everyone is sharing though Could change, especially with Agentic Commerce & Payments
53. AI Assistant Commerce Ability Google: Gemini / AI Mode Project Mariner Shop with AI Mode - Beta ChatGPT Instant Checkout Meta Claude CoPilot Nope Amazon Alexa+, ‘Buy for Me’ (in Beta) Perplexity Comet Browser
54. Open AI are getting a head start in ChatGPT In partnership with Shopify & Stripe & now PayPal https://developers.openai.com/commerce/
55. ChatGPT can choose & purchase for you through compatible sites Etsy & Shopify live now on ‘Instant Checkout’ https://developers.openai.com/commerce/
56. Create a feed to Specs Expand beyond other channels, more: • Fields • Detail & USPs • Product Specs • Use Cases https://developers.openai.com/commerce/ Optimise your Feed to win
57. Apply to OpenAI’s Instant Checkout now Build ACP Checkout API requiring: a) REST API Endpoint b) Using Webhooks (push notifications) c) Ability to return Checkout Status Use Stripe or PayPal (more to come) https://developers.openai.com/commerce/ There’s a delay processing Applications so:
58. MCP Server Access MCP Server can be remote or local Tells AI Agent what Endpoint does Translates between the two
59. MCP Server Access Whatever the Endpoint: AI Agent gets an easy to understand JSON file
60. Where is MCP Now? ‘Productivity’ & SaaS e.g. Google Docs Limited Support from B2C Sites & Services Limited ‘Out of the Box’ Integration
61. Where is MCP Now? Needs Discovery Solution Major Security Concerns Mainly Local Servers based on Open Source
62. MCP or it’s successor, shows us what’s possible
63. What MCP could be used for in future SaaS Working with Smart Devices Audio & Video control Automation Appointment Bookings: Restaurant, Hotels, JSON-LD versions of pages & Content
64. What else can we do now to Futureproof? • MCP • AI / Machine Friendly / Machine Ready Tag • Machine friendly (e.g JSON-LD), alternate URL • AI Usable Actions Tags • Additional ‘best practice’ / recommended Schema Types &/or properties • Extended Semantic HTML Tags • Extended ARIA • Meta License Tags Ensure site/platform has flexibility e.g. injecting tags Ensure you have Endpoint that can work with External
65. Thank You davetheseo.com/ daves-decks/ linkedin.com/in/davecousin/ @davetheseo.bsky.social