# Adbloop robots.txt # Everything is public and crawlable. AI crawlers are explicitly welcomed # below (they are already covered by the wildcard, but listing them removes # any ambiguity for bots that look for their own name first). User-agent: * Allow: / # --- Search + AI answer engines --------------------------------------------- # OpenAI (ChatGPT) User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / # Anthropic (Claude) User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: anthropic-ai Allow: / # Perplexity User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / # Google (AI Overviews / Gemini training opt-in signal) User-agent: Google-Extended Allow: / # Apple Intelligence User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # Common Crawl (feeds many open models) User-agent: CCBot Allow: / # Amazon User-agent: Amazonbot Allow: / # Cohere User-agent: cohere-ai Allow: / # Meta AI User-agent: Meta-ExternalAgent Allow: / # DuckDuckGo AI assist User-agent: DuckAssistBot Allow: / # --- Sitemap ---------------------------------------------------------------- Sitemap: https://adbloop.com/sitemap.xml