User-agent: Mediapartners-Google Disallow: User-agent: * Allow: / Disallow: /search Allow: /search/label # --- Block AI training crawlers --- User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Google-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: Bytespider Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: Amazonbot Disallow: / User-agent: Diffbot Disallow: / User-agent: cohere-ai Disallow: / User-agent: DeepSeekBot Disallow: / User-agent: Applebot-Extended Disallow: / Sitemap: https://sergedaney.blogspot.com/sitemap.xml Sitemap: https://sergedaney.blogspot.com/sitemap-pages.xml