Search

Matched domain: codepoints.net

`robots.txt`

Sitemap: https://codepoints.net/sitemap.xml

User-agent: *
Disallow: /search
Disallow: /random

# sorry, but this bot created too much traffic
User-agent: search.marginalia.nu
Disallow: /

# same
User-agent: SemrushBot
Disallow: /

# creating ~50% of all requests is not nice, Amazon. Sorry.
User-agent: Amazonbot
Disallow: /

# Yep, .... nope.
User-agent: GPTBot
Disallow: /

# try to speed down this one a bit
User-agent: AhrefsBot
Crawl-Delay: 20

Look up this url in the url tool https://codepoints.net/.well-known/acme-challenge: 404 text/html; charset=UTF-8
https://codepoints.net/.well-known/csvm: 404 text/html; charset=UTF-8
https://codepoints.net/.well-known/nostr.json: 404 text/html; charset=UTF-8
https://codepoints.net/.well-known/security.txt: 403 text/html; charset=iso-8859-1
https://codepoints.net/.well-known/traffic-advice: 404 text/html; charset=UTF-8