User-Agent: Googlebot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: bingbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: Applebot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: Slurp Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: DuckDuckBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: Google-Extended Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: BingPreview Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: GPTBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: ChatGPT-User Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: ClaudeBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: Claude-Web Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: PerplexityBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 1 User-Agent: Applebot-Extended Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: YouBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: anthropic-ai Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: CCBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 3 User-Agent: Meta-ExternalAgent Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Diffbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 3 User-Agent: facebookexternalhit Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Slackbot-LinkExpanding Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: LinkedInBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Twitterbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: TelegramBot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: WhatsApp Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Discordbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Snoobot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: redditbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Pinterestbot Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: Mastodon Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Crawl-delay: 2 User-Agent: SemrushBot Disallow: / User-Agent: DataForSeoBot Disallow: / User-Agent: PetalBot Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: DotBot Disallow: / User-Agent: BLEXBot Disallow: / User-Agent: MegaIndex Disallow: / User-Agent: Scrapy Disallow: / User-Agent: YandexBot Disallow: / User-Agent: Baiduspider Disallow: / User-Agent: Sogou Disallow: / User-Agent: 360Spider Disallow: / User-Agent: * Allow: / Disallow: /internal Disallow: /tools Disallow: /preview Disallow: /tags Disallow: /facets Disallow: /legal Disallow: /drafts Disallow: /_next Disallow: /pages Disallow: /api/ Crawl-delay: 5 Sitemap: https://metalhatscats.com/sitemap.xml Sitemap: https://metalhatscats.com/ai/site-graph.json Sitemap: https://metalhatscats.com/ai/services.json Sitemap: https://metalhatscats.com/ai/rnd-log.json Sitemap: https://metalhatscats.com/life-os/datasets/manifest Sitemap: https://metalhatscats.com/life-os/datasets/zones Sitemap: https://metalhatscats.com/life-os/datasets/hacks