Search

Matched domain: metacpan.org

`robots.txt`

# Stop anything going into these locations
User-agent: *
Disallow: /login/
Disallow: */diff/
Disallow: */source/
Disallow: */source
Disallow: */raw/
Disallow: /account/
Disallow: /search

# Do not allow changing the default per page as is not useful
Disallow: /*?*size=*

Sitemap: https://metacpan.org/sitemap-authors.xml.gz
Sitemap: https://metacpan.org/sitemap-releases.xml.gz

# Stop the bots - using signalsciences to block

Look up this url in the url tool https://metacpan.org/.well-known/acme-challenge: 200 text/html; charset=utf-8
https://metacpan.org/.well-known/csvm: 200 text/html; charset=utf-8
https://metacpan.org/.well-known/nostr.json: 200 text/html; charset=utf-8
https://metacpan.org/.well-known/security.txt: 200 text/html; charset=utf-8
https://metacpan.org/.well-known/traffic-advice: 200 text/html; charset=utf-8