# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file User-agent: * Disallow: /admin/ User-agent: GPTBot User-agent: ClaudeBot User-agent: anthropic-ai User-agent: CCBot User-agent: Google-Extended User-agent: Bytespider User-agent: Amazonbot User-agent: Applebot-Extended User-agent: meta-externalagent User-agent: Diffbot User-agent: ImagesiftBot User-agent: DataForSeoBot Disallow: / # Main application sitemap Sitemap: https://medtribe.com/sitemap.xml.gz # Blog sitemap (Astro) Sitemap: https://medtribe.com/blog/sitemap-index.xml # Documentation sitemap (Astro) Sitemap: https://medtribe.com/docs/sitemap-index.xml