Stop AI agents from scraping your website without hurting your Google rankings: have a coding agent audit how your site answers bots, verify real Googlebot by IP, and serve scrapers a polite no instead of an error they retry: AI agent prompt

Help me protect my own website from AI agents and scripts that scrape it, without hurting search indexing. My site: [domain, what's valuable on it (e.g. listings, prices, a dataset), stack (e.g. Next.js on Cloudflare, WordPress, Express)]. Where I want scrapers sent instead: [signup page, API page or product URL]. 1) Audit first, change nothing: list which pages, sitemaps and /api/ routes expose the valuable data or trigger expensive work, and how each currently responds to a normal browser, curl, a fake 'Googlebot' user agent and a headless browser. 2) Propose a layered plan: a robots.txt and llms.txt policy (which crawlers to allow and which to disallow), verifying real Googlebot by IP and network rather than user agent, a polite HTTP 200 notice page with no data rows and noindex headers for blocked requests, browser-header checks, per-IP rate limits, and locking public JSON endpoints. 3) For each layer, explain the risk of blocking real visitors, accessibility tools or search engines, and how to test that Google can still crawl my pages. 4) Write the code changes for my stack as a reviewable diff, behind a setting I can switch off. 5) Give me a test checklist I can run only against my own site. Don't deploy anything or touch any site I don't own; show me every change first.