What Is Crawled But Not Cited?

Crawled but not cited means an AI system can access a page, but the page does not appear to be reused in answers, citations, referrals, or recommendations.

It is one of the most important AI visibility failure modes because it looks healthy from the outside.

The page exists.

It is indexable.

It is technically crawlable.

AI systems may even fetch it repeatedly.

But the page still fails the practical test: it does not help your brand show up when buyers ask questions.

Access is not reuse

Most teams start with access checks:

• Is the page live?
• Is it blocked in robots.txt?
• Does llms.txt exist?
• Can the HTML be fetched?
• Is the page fast enough to load?

Those checks matter. But they only answer whether the front door is open.

They do not answer whether AI systems can confidently use the page.

Reuse requires more than access. A page needs enough structure, clarity, specificity, and evidence for an AI system to quote, summarize, compare, or recommend it without inventing the missing pieces.

A simple example

Imagine a product comparison page.

AI systems fetch it often. That tells you the page is accessible and potentially relevant.

But the page has three problems:

1. It opens with a broad brand story instead of the comparison.
2. It uses generic claims like "powerful insights" and "all-in-one platform."
3. It never names the tradeoffs between the products being compared.

That page can be crawled without being useful.

Now imagine a better version:

1. It states the category and alternatives clearly.
2. It explains who each option is best for.
3. It includes decision criteria, pricing context, limitations, and evidence.
4. It has concise sections that can stand alone in an AI answer.

That page is easier to reuse because it reduces ambiguity.

Common causes

Pages often become crawled but not cited when:

• the answer is buried too far down the page
• the page has weak entity definitions
• the commercial angle is unclear
• the comparison framing is vague
• claims are unsupported
• important facts are trapped in visual elements
• the page tries to serve too many audiences at once
• the content is crawlable but not extractable

This is why crawled but not cited is usually a content architecture problem, not just a technical SEO problem.

How to diagnose it

Start with important pages, not the whole site.

Look at pages tied to pipeline, evaluation, or revenue:

• pricing pages
• comparison pages
• category pages
• product pages
• documentation
• high-intent editorial posts

For each page, ask:

1. Are AI systems fetching it?
2. Which systems fetch it?
3. Is the page later cited, referred to, or reused?
4. Did the state change after an edit?
5. What would make the page easier to quote, compare, or defend?

If fetches are present but reuse is absent, the next step is not "publish more content." The next step is to improve the page that already matters.

What to fix first

The most useful fixes are usually practical:

• move the direct answer higher
• add a short summary block
• clarify the category definition
• name the alternatives and tradeoffs
• add original evidence or specific examples
• make product positioning more concrete
• tighten decision-stage copy
• remove vague claims that could apply to any competitor

AI systems reward clarity because clarity is easier to quote, summarize, compare, and defend.

Where SeeLLM fits

SeeLLM helps teams find the gap between access and reuse.

Start with the free AI Visibility Score to check whether an important page is readable. Then use page-level monitoring when you need to see which pages AI systems fetch, revisit, skip, or leave crawled but not cited.

For the broader strategy, read The New SEO Problem: Crawled, But Not Cited.