IP = 209.126.35.79
robots.txtUser-agent: *
Disallow: /packages/search/
Disallow: /packages/?
Disallow: /packages/?*
Sitemap: https://www.archlinux.org/sitemap.xml
Crawl-delay: 2
Look up this url in the url tool
https://archlinux.org/.well-known/acme-challenge: 404 text/html; charset=utf-8
https://archlinux.org/.well-known/csvm: 404 text/html; charset=utf-8
https://archlinux.org/.well-known/nostr.json: 404 text/html; charset=utf-8
https://archlinux.org/.well-known/security.txt: 404 text/html; charset=utf-8
https://archlinux.org/.well-known/traffic-advice: 404 text/html; charset=utf-8