AI SEO Foundations

Module 1 — How Search Changed

Lesson 3: How AI Agents Read a Website

Last updated: 2026-10-09

An AI agent does not see your hero image, your colour palette, or your clever tagline. It sees code, structure, and signals. To optimise a site for agents, you first need a clear picture of what they are actually scanning.

Lesson 3: How AI Agents Read a Website

Blackboard

Five signals, three interaction modes

12 min
Concept 1

Agents evaluate websites along five signals

Across published guides from Vercel, Microsoft, Tealium, and the major analyst houses, the same five signal categories appear. Structured data: schema markup that explicitly labels what the page is and what it contains. Content clarity: plain language that answers concrete questions without burying them in storytelling. API compatibility: machine-readable endpoints for inventory, pricing, availability, and booking. Reputation: cross-references on other sites — reviews, citations, brand mentions — that the agent can verify independently. Freshness: visible last-updated dates and recently refreshed pages. A site that is strong on all five becomes a default recommendation; one that is weak on two or more is filtered out before the human ever hears about it.

Concept 2

There are three agent interaction modes

McKinsey's report on agentic commerce divides agent traffic into three modes. Agent-to-site: an agent visits merchant websites directly, scans them, and reports back. This is the closest analogue to a human visit and the easiest to optimise for. Agent-to-agent: the consumer's agent talks to the merchant's agent, exchanging structured queries and offers. This is where the Agent-to-Agent (A2A) protocol matters. Brokered: an intermediary agent — for example, a booking platform's agent or a payment-network agent — sits between the two and applies extra rules such as loyalty discounts or fraud checks. Each mode places different demands on the merchant site.

Concept 3

Most agents do not run JavaScript

A frequent surprise for developers is that most AI crawlers — including OpenAI's GPTBot, Anthropic's ClaudeBot, Common Crawl's CCBot, and Google's research crawlers when collecting training data — do not execute client-side JavaScript. They read the raw HTML the server sends. A few, including Google's Gemini and Apple's Applebot, do execute scripts, but they are the exception. If a page's product information, pricing, or core copy only appears after a JavaScript bundle has loaded and rendered, most agents see a near-empty page and move on. Server-rendered HTML, or a static fallback for non-JS clients, is therefore not a nice-to-have.

Concept 4

Accessibility for humans is visibility for AI

The same engineering choices that make a site usable for someone using a screen reader — semantic HTML, ARIA labels, clear alt text, predictable navigation, descriptive button text — make it readable for an agent. Most retrofits that improve agent visibility are accessibility retrofits. This is good news for two reasons. First, accessibility work has independent legal and ethical grounding, so it is rarely wasted. Second, sites that already meet WCAG (Web Content Accessibility Guidelines) standards usually need only modest additions — schema markup, clean canonical tags, freshness signals — to score well on the AI-readability frameworks published by Vercel, Microsoft, and others.

Concept 5

How agents actually read a page: three combined channels

Google's web.dev guidance for agent-friendly design describes how the reading actually happens. An agent usually combines three channels. The first is a screenshot, processed by a vision model. It is slow and token-expensive, but it captures layout and visual hierarchy. The second is the raw HTML and DOM, which is fast and exposes nesting, attributes, and the relationships between elements. The third is the browser's accessibility tree, a native semantic summary that lists roles, names, and states of every interactive element with the CSS-driven noise stripped out. Agents cross-reference the three. A div styled to look like a button shows up in the DOM, but only the accessibility tree confirms whether the developer wired up a real role and behaviour for it. So clean HTML, valid semantic elements, and an honest accessibility tree together give an agent a more reliable signal than any one of them alone.

Pause and Think

If a literal-minded researcher visited your website with a stopwatch and a list of yes-or-no questions about your products, how many would they answer correctly without scrolling, expanding, or guessing?

Flashcards

Key Terms

Click or press Enter to flip a card. Review these before the practice question.

Additional Resources
Practice Question

A developer says: 'Our site is built as a single-page app. All product data is loaded dynamically by JavaScript after the user lands on the page.' What does this lesson predict will happen with most AI agents?

Does not count for certificate
Choose one answer, use the hint if needed, then check your answer.
Use the hint before checking if you need a nudge.