Built for browsers, invisible to agents: Bridging the rendering gap for AI and search

Talk given at SMX Advanced Boston in June 2026.

Your rendering strategy was probably decided by your engineering team for your users. But it's also one of the most consequential SEO decisions on your site, and for most teams, nobody in SEO is actively managing it. That gap is getting harder to ignore as AI crawlers and agents become a real part of how your content gets found and shared.

Come discover how every rendering approach — SSR, SSG, ISR, and CSR — maps to its real-world impact on discoverability, structured data accessibility, and agent compatibility. You'll walk away with a practical audit framework, so you know exactly where your site stands and what to do about it.

After this session, you'll be able to:
- Explain the rendering spectrum, from SSR to CSR, and what each one actually delivers to Googlebot, AI crawlers, and agents.
- Identify which rendering approaches make your content invisible to LLM crawlers like GPTBot and ClaudeBot (and why it's already happening).
- Convince engineering to make rendering changes using a concrete framework tied to discoverability, structured data, and agent readiness.

Transcript

1. Built for Browsers, Invisible to Agents Bridging the Rendering Gap for AI and Search Sam Torres SEO Mermaid / Pipedrive
2. What I nerd out on: • JavaScript SEO • Headless CMS site architecture • Translating SEO for engineering teams The SEO Mermaid 🧜 • 15 years experience, both agency & in-house • Based in Atlanta • Former frontend developer Aloha y’all, I’m Sam.
3. your rendering strategy — likely chosen by engineering, based on: • Team experience/talent • Costs • User experience Nobody in SEO is actively managing it. The Decision You Didn’t Make
4. Crawling ≠ Rendering ≠ Indexing CRAWL Fetch the URL. The server returns its initial HTML response. RENDER Execute the page. A browser (or rendering service) runs JS, builds the DOM, and creates the final, visible HTML. INDEX Store the result. The crawler decides what gets remembered, and what gets cited. Three distinct, separate steps. Each platform handles them differently.
5. Response HTML Rendered HTML The raw HTML your server sends back. • What you see with: View Source • What you see with: cURL This is what AI crawlers see. The DOM after JS executes. • What you see with: Inspect Element • What you see with: View Rendered Source This is what users see, and what Googlebot eventually catches up to. Key Distinction
6. Rendering Spectrum SSR Server-Side Rendering HTML built: Per-request SSG Static Site Generation HTML built: Scheduled build time ISR Incremental Static Regeneration HTML built: Periodic, as needed CSR Client-Side Rendering HTML built: In the user’s browser ← Content in response HTML Content built by JavaScript →
7. Not All SEO Content is Created Equal CRITICAL Must-have in response. • Title tag • Meta robots • Canonical • Structured data IMPORTANT Should be in response. • Navigation • Headings (H1, H2, etc.) • Hreflang • Pagination NICE TO HAVE Ok to render CSR…usually. • Most body copy • In-content hyperlinks • Images & alt text • Media (i.e. video)
8. The Split Visibility Problem 69% Of AI crawlers can’t execute JS - Vercel + MERJ, 2024 Your JS-based can rank #1 on Google and be completely blank to Claude, ChatGPT, & Perplexity.
9. Who Renders JS, and Who Doesn’t CRAWLER RENDERS JS? Googlebot ✓ Yes (with delays) Bingbot ~ Limited Applebot ✓ Yes Google-Extended (Gemini training) ✓ Yes — inherits Google infra GPTBot · OAI-SearchBot · ChatGPT-User ✗ No ClaudeBot · Claude-SearchBot · Claude-User ✗ No PerplexityBot ✗ No CCBot (Common Crawl) ✗ No Sources: Vercel/MERJ research (2024) · Public crawler documentation from OpenAI, Anthropic, Google, Perplexity
10. Response HTML Rendered HTML What’s in the raw server response. Text, links, structured data, metadata. ✓ Content added by JavaScript. Anything that needs the DOM to exist. ✗ So What Do LLM Crawlers Actually See?
11. If your content needs JavaScript to appear, AI search doesn't know it exists. THE RULE TO REMEMBER …mostly. LLMs sometimes pull from Google’s cached, post-render index. So it’s not 100% - just unreliable enough that you can’t depend on it. And agents browsing/interacting live? They’re on their own.
12. Agents DO render JavaScript. But they have their own problems. They drive real browsers. ü They see your full rendered page. ü They can click, scroll, and fill forms. ü They follow real navigation flows. ü They can transact on your site. Slow sites cost extra tokens and patience – which they don’t have. ✗ They bail on long page loads. ✗ They struggle with SPA hash modes (no distinct URLs). ✗ Modals, popups, & cookies are automatically blocked. ✗ Anti-bot site security may not realize they’re “good” actors. And then there are agents…
13. Rendering Approach × What Bots Actually Get Response HTML has content? Googlebot sees it? AI crawlers see it? Agents see it? SSR ✓ Full ✓ Yes ✓ Yes ✓ Yes SSG ✓ Full ✓ Yes ✓ Yes ✓ Yes ISR ✓ Full (may be stale) ✓ Yes ✓ Yes ✓ Yes CSR ✗ Empty shell* ~ After render queue ✗ No ~ If patient
14. The Pages Where This Hurts the Most 4.4× AI-referred traffic converts vs. standard organic search High-citation value pages • Product pages 20.1% of ChatGPT citations • Comparison & vs. pages Heavy AI traffic for purchase research • Pricing & plan pages First click after a model recommends you • Documentation & FAQ The pages models love to quote • Category & collection pages Discovery for e-commerce Sources: Lantern (Feb 2026 citation analysis); Superlines / Mersel aggregated conversion data
15. Common Way This Goes Wrong Critical SEO not in response Title, meta robots, canonical injected by JS instead of served in the HTML. Links that aren't links <span onclick=…> instead of <a href=…>. Bots can't follow them. Metadata that changes after render Server says one canonical. JavaScript rewrites it. Bots see one, users see another. Soft 404s on real content Page loads with status 200 but no real content until JS runs. Looks empty to crawlers. Junk URLs in the index Orphaned build artifacts, framework debug routes, preview URLs that escaped. Broken <head> Invalid HTML before </head> can stop Googlebot mid-render.
16. The 5-Minute Audit You Can Run on Monday 01 View Source. Ctrl+U on any critical page. Is your content actually there? 02 Disable JavaScript. Reload. If the page goes blank, that's what AI crawlers see. 03 cURL the URL. curl -A "GPTBot" yoursite.com — what does the bot actually receive? 04 Search a unique phrase. Wrap it in quotes on Google. If indexed, it's in someone's training data. 05 Check server logs. Are GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot getting 200s on content?
17. How to Talk to Engineering USE THEIR LANGUAGE Say “client-side rendering”, not “the SEO stuff”. • Rendering types • Response HTML vs. rendered DOM • Hydration / time to first byte USE THEIR PROCESS File a defect ticket, not a Slack message. • Clear title & affected URLs • Expected vs. actual behavior • Steps to reproduce • Acceptance criteria FRAME THE PRIORITIES SSR is a performance win AND an AI win. • Faster TTFB and LCP • Visibility in AI search • Resilience against agents Reframe “this is an SEO problem” as “this is a discoverability problem that performance and AI search both solve”.
18. The Numbers that Can Help Get Engineering’s Attention 569M Monthly GPTBot requests across Vercel’s network. Vercel/MERJ 370M Monthly ClaudeBot requests across Vercel’s network. Vercel/MERJ 4.4× Higher conversion rate from AI-referred traffic vs. organic traffic. Lantern -25% Projected drop in traditional search volume as AI absorbs informational queries. Gartner
19. What to Walk Out With 01 Rendering is an SEO decision now. It used to be an engineering preference. AI crawlers — and agents — made it a discoverability lever you can't ignore. 02 Response HTML is your AI surface area. If it's not in the raw server response, GPTBot, ClaudeBot, and PerplexityBot can't see it. Audit accordingly. 03 You don't need engineering's permission to start. Run the 5-minute audit this week. Build the case with their language, their process, their priorities.
20. Thank you! Questions? Sam Torres · The SEO Mermaid Sr. Manager, Technical SEO @ Pipedrive LinkedIn: /in/samantha-torres-seo Slides + extras: theseomermaid.com/smx-boston-2026