GitHub - Page-Replica/nextjs-vercel-page-replica-middleware: Next.js Vercel middleware for serving prerendered HTML to bots using Page Replica

3 min read Original article ↗

Next.js Bot Prerender Middleware (Page Replica + Vercel)

Serve prerendered HTML to search engine bots and social crawlers using Page Replica and Vercel Edge Middleware.

This middleware improves SEO for client-rendered (SPA) Next.js applications by serving cached HTML to bots while keeping the normal experience for users.

🚀 What This Does

When a bot (Google, Bing, Facebook, etc.) visits your site:

  1. The middleware detects the bot via User-Agent.
  2. It fetches a prerendered version from Page Replica.
  3. It returns that HTML instead of your client-rendered React app.

Normal visitors receive your standard Next.js app.

📦 Installation

Create a file in the root of your Next.js project:

middleware.js

Paste the following code:

const BOT_AGENTS = [
  // Search engines
  'googlebot',
  'google-inspectiontool',
  'bingbot',
  'slurp',
  'duckduckbot',
  'baiduspider',
  'yandexbot',

  // SEO crawlers
  'ahrefsbot',
  'ahrefssiteaudit',
  'semrushbot',
  'rogerbot',

  // Social previews
  'facebookexternalhit',
  'facebot',
  'twitterbot',
  'linkedinbot',
  'slackbot',
  'telegrambot',
  'whatsapp',

  // AI / LLM crawlers
  'gptbot',
  'chatgpt-user',
  'perplexitybot',
  'claudebot',
  'anthropic-ai',
  'ccbot',
  'applebot',
  'bytespider',
  'diffbot',
  'meta-externalagent',
  'amazonbot',
  'cohere-ai'
];

const STATIC_EXT =
  /\.(js|css|png|jpg|jpeg|gif|ico|svg|woff|woff2|ttf|eot|webp|avif|map|json|xml|txt|pdf)$/i;

export default async function middleware(request) {
  try {
    const url = new URL(request.url);
    const pathname = url.pathname;
    const search = url.search;

    // Skip static assets immediately
    if (STATIC_EXT.test(pathname)) {
      return fetch(request);
    }

    // Detect bots safely
    const ua = (request.headers.get('user-agent') || '').toLowerCase();
    const isBot = BOT_AGENTS.some((bot) => ua.includes(bot));

    if (!isBot) {
      return fetch(request);
    }

    const host = request.headers.get('host');
    if (!host) {
      return fetch(request);
    }

    const siteId = process.env.PAGE_REPLICA_SITE_ID;
    if (!siteId) {
      console.error('Missing PAGE_REPLICA_SITE_ID');
      return fetch(request);
    }

    // Preserve full path + query
    const path = pathname + search;

    const cacheUrl = `https://cache.page-replica.com/${siteId}/${host}${path}`;

    // Fetch prerendered HTML snapshot
    const cached = await fetch(cacheUrl, {
      headers: {
        'User-Agent': 'Vercel Middleware'
      }
    });

    if (!cached.ok) {
      return fetch(request);
    }

    const html = await cached.text();

    // Return prerendered response safely
    return new Response(html, {
      status: 200,
      headers: {
        'Content-Type': 'text/html; charset=utf-8',
        'X-Prerendered': 'page-replica',
        'Cache-Control': 'public, max-age=600',
        'Vary': 'User-Agent'
      }
    });

  } catch (error) {
    console.error('Middleware error:', error);
    return fetch(request);
  }
}

export const config = {
  matcher: ['/((?!api|_next/static|_next/image|.*\\..*).*)']
};

🔐 Vercel Setup (Required)

This middleware requires your Page Replica Site ID.

Step 1 --- Add Environment Variable

In your Vercel dashboard:

  1. Open your project
  2. Go to Settings
  3. Click Environment Variables
  4. Add:

Key: PAGE_REPLICA_SITE_ID
Value: your-site-id-here
Environment: Production (and Preview if needed)

Step 2 --- Redeploy

After adding the variable, trigger a redeploy.

Environment variables are only available after deployment.

🧪 How To Test

Test as Googlebot

curl -A "googlebot" -I https://yourdomain.com

If working correctly, you will see:

x-prerendered: page-replica

Test as Normal User

curl -I https://yourdomain.com

You should NOT see the x-prerendered header.

📋 Requirements

  • Deployment on Vercel
  • Page Replica account and site ID

📄 License

MIT License