User-agent: * Allow: / # Major search crawlers — explicit allow User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: DuckDuckBot Allow: / # International search crawlers — explicit allow so crawl-budget for # non-English markets gets prioritized (Baidu, Yandex, Naver, Seznam, # Sogou, Mail.ru). We're not yet shipping translated content for these # locales but the structure is ready and we want to be discoverable # the moment translations land. User-agent: Baiduspider Allow: / User-agent: Baiduspider-image Allow: / User-agent: Yandex Allow: / User-agent: YandexBot Allow: / User-agent: YandexImages Allow: / User-agent: Yeti Allow: / User-agent: NaverBot Allow: / User-agent: SeznamBot Allow: / User-agent: Sogou web spider Allow: / User-agent: Mail.RU_Bot Allow: / # AI crawlers — full discoverability User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: CCBot Allow: / User-agent: Google-Extended Allow: / User-agent: anthropic-ai Allow: / User-agent: applebot-extended Allow: / # Administrative paths User-agent: * Disallow: /admin/ Disallow: /uploads/ Sitemap: https://zypto.com/sitemap-index.xml