AI Discovery Index: how fast AI finds new pages

6 min read Original article ↗

Live data · Updated 24 Oct 2026

We publish a page. How long until AI finds it?

Every new page on mojodojo.io starts a clock. We time the first verified visit from each search engine and AI crawler, and the first time an assistant reads the page live for a person. These are aggregates across every new page published in the period.

New pages published

23

10 blog posts

First crawl, any bot

1.3h

median. ClaudeBot was first on 39% of pages

First AI crawler

1.8h

median. ClaudeBot was first on 43%

AI found within 24h

100%

of new pages had a visit from an AI crawler within a day

By crawler

Hours from publish to first visit

The dot is the median across the new pages the crawler has reached (log scale), and the bar covers the middle half of those pages. Diamonds are live fetches for a person. Hollow dots mean fewer than 3 pages reached, so treat them as anecdotes.

Median hours from publish to first visit, per crawler, for the last 7 days

  • Search engine
  • AI crawler
  • Live user fetch

Coverage

Share of new pages found, over time since publishing

Of the new pages published in the period, the share each crawler had visited by that point after publishing. Pages published near the end of the period have had less time.

Share of new pages visited by each crawler over time since publishing, for the last 7 days

  • PerplexityBot
  • Bingbot
  • ClaudeBot
  • Googlebot
  • GoogleOther
  • OAI-SearchBot

Trend

Daily median time to discovery

Median time from publish for the pages published in each bucket (log scale). The number under each bucket is how many new pages it contains. Gaps are buckets with no new pages.

Median time to discovery by publish day, for the last 7 days

  • First crawl, any bot
  • First AI crawler
  • First live user fetch

About this data. 350 pages were published since tracking began on 6 Jun 2026. 224 are excluded because their URL was already live before they were published. Visits since 12 Oct 2026 carry exact timestamps (100% of the pages in this view have at least one). Before that only the day is known, so first visits are estimated at the middle of the remaining day.

New pages published

104

30 blog posts, 15 city pages, 45 service pages

First crawl, any bot

14h

median. Bingbot was first on 44% of pages

First AI crawler

14h

median. ClaudeBot was first on 42%

AI found within 24h

79%

of new pages had a visit from an AI crawler within a day

By crawler

Hours from publish to first visit

The dot is the median across the new pages the crawler has reached (log scale), and the bar covers the middle half of those pages. Diamonds are live fetches for a person. Hollow dots mean fewer than 3 pages reached, so treat them as anecdotes.

Median hours from publish to first visit, per crawler, for the last 30 days

  • Search engine
  • AI crawler
  • Live user fetch

Coverage

Share of new pages found, over time since publishing

Of the new pages published in the period, the share each crawler had visited by that point after publishing. Pages published near the end of the period have had less time.

Share of new pages visited by each crawler over time since publishing, for the last 30 days

  • Bingbot
  • ClaudeBot
  • PerplexityBot
  • GoogleOther
  • OAI-SearchBot
  • GPTBot

Trend

Weekly median time to discovery

Median time from publish for the pages published in each bucket (log scale). The number under each bucket is how many new pages it contains. Gaps are buckets with no new pages.

Median time to discovery by publish week, for the last 30 days

  • First crawl, any bot
  • First AI crawler
  • First live user fetch

About this data. 350 pages were published since tracking began on 6 Jun 2026. 224 are excluded because their URL was already live before they were published. Visits since 12 Oct 2026 carry exact timestamps (92% of the pages in this view have at least one). Before that only the day is known, so first visits are estimated at the middle of the remaining day.

New pages published

123

38 blog posts, 15 city pages, 50 service pages, 6 playbooks

First crawl, any bot

14h

median. Bingbot was first on 44% of pages

First AI crawler

14h

median. ClaudeBot was first on 36%

AI found within 24h

77%

of new pages had a visit from an AI crawler within a day

By crawler

Hours from publish to first visit

The dot is the median across the new pages the crawler has reached (log scale), and the bar covers the middle half of those pages. Diamonds are live fetches for a person. Hollow dots mean fewer than 3 pages reached, so treat them as anecdotes.

Median hours from publish to first visit, per crawler, for the last 90 days

  • Search engine
  • AI crawler
  • Live user fetch

Coverage

Share of new pages found, over time since publishing

Of the new pages published in the period, the share each crawler had visited by that point after publishing. Pages published near the end of the period have had less time.

Share of new pages visited by each crawler over time since publishing, for the last 90 days

  • Bingbot
  • ClaudeBot
  • PerplexityBot
  • GoogleOther
  • OAI-SearchBot
  • GPTBot

Trend

Weekly median time to discovery

Median time from publish for the pages published in each bucket (log scale). The number under each bucket is how many new pages it contains. Gaps are buckets with no new pages.

Median time to discovery by publish week, for the last 90 days

  • First crawl, any bot
  • First AI crawler
  • First live user fetch

About this data. 350 pages were published since tracking began on 6 Jun 2026. 224 are excluded because their URL was already live before they were published. Visits since 12 Oct 2026 carry exact timestamps (87% of the pages in this view have at least one). Before that only the day is known, so first visits are estimated at the middle of the remaining day.

New pages published

126

40 blog posts, 15 city pages, 50 service pages, 7 playbooks

First crawl, any bot

14h

median. Bingbot was first on 43% of pages

First AI crawler

14h

median. ClaudeBot was first on 35%

AI found within 24h

78%

of new pages had a visit from an AI crawler within a day

By crawler

Hours from publish to first visit

The dot is the median across the new pages the crawler has reached (log scale), and the bar covers the middle half of those pages. Diamonds are live fetches for a person. Hollow dots mean fewer than 3 pages reached, so treat them as anecdotes.

Median hours from publish to first visit, per crawler, for all time

  • Search engine
  • AI crawler
  • Live user fetch

Coverage

Share of new pages found, over time since publishing

Of the new pages published in the period, the share each crawler had visited by that point after publishing. Pages published near the end of the period have had less time.

Share of new pages visited by each crawler over time since publishing, for all time

  • Bingbot
  • ClaudeBot
  • PerplexityBot
  • GoogleOther
  • OAI-SearchBot
  • GPTBot

Trend

Monthly median time to discovery

Median time from publish for the pages published in each bucket (log scale). The number under each bucket is how many new pages it contains. Gaps are buckets with no new pages.

Median time to discovery by publish month, for all time

  • First crawl, any bot
  • First AI crawler
  • First live user fetch

About this data. 350 pages were published since tracking began on 6 Jun 2026. 224 are excluded because their URL was already live before they were published. Visits since 12 Oct 2026 carry exact timestamps (86% of the pages in this view have at least one). Before that only the day is known, so first visits are estimated at the middle of the remaining day.

Method. Publish times come from the CMS (published_at). First visits come from our own server log and leave out any hit that fails verification against the operator's published IP ranges or reverse DNS, so fake bots never count. Pages whose URL was already being crawled before their publish date (republished or migrated pages) are excluded, so the clock only starts for pages that were genuinely new. Medians use only the pages a crawler has reached; "still waiting" counts the rest.

Cite this. Mojo Dojo, “AI Discovery Index”, mojodojo.io/ai-discovery-index, accessed 24 Oct 2026. Free to reuse under CC BY 4.0 with a link back.