IP = 13.249.74.121
robots.txt
User-agent: Googlebot
Disallow:
User-agent: Bingbot
Disallow:
User-agent: DuckDuckBot
Disallow:
User-agent: Applebot
Disallow:
User-agent: Yandex
Disallow: /
User-agent: Baiduspider
Disallow: /
User-agent: *
Crawl-delay: 5
User-agent: SemrushBot
Disallow: /
User-agent: AhrefsBot
Disallow: /
User-agent: MJ12bot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: spbot
Disallow: /
User-agent: rogerbot
Disallow: /
User-agent: SiteExplorer
Disallow: /
User-agent: archive.org_bot
Disallow: /
User-agent: wget
Disallow: /
User-agent: curl
Disallow: /
User-agent: python-requests
Disallow: /
User-agent: libwww-perl
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: Scrapy
Disallow: /
User-agent: CCBot
Disallow: /
# ----------------------------
# Block known AI crawlers / data scrapers (2025 list)
# ----------------------------
User-agent: GPTBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: ChatGPT-Scraper
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: OpenAI
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: gpt-bot
Disallow: /
User-agent: DeepSeekBot
Disallow: /
User-agent: GeminiBot
Disallow: /
User-agent: PerplexityBot
Disallow: /
Sitemap: https://codingnomads.com/sitemap.xml
Look up this url in the url tool
https://codingnomads.com/.well-known/acme-challenge: 200 text/html;charset=UTF-8
https://codingnomads.com/.well-known/csvm: 200 text/html;charset=UTF-8
https://codingnomads.com/.well-known/nostr.json: 200 text/html;charset=UTF-8
https://codingnomads.com/.well-known/security.txt: 200 text/html;charset=UTF-8
https://codingnomads.com/.well-known/traffic-advice: 200 text/html;charset=UTF-8