IP = 38.145.34.32
robots.txt# contact sourcemaster@sourceware.org for questions.
# see http://www.robotstxt.org/
# for information about the file format.
User-agent: Scooter
Disallow: /ml/
User-agent: Test Crawler
Disallow: /
User-agent: DigExt
Disallow: /
User-agent: CW Crawler
Disallow: /
User-agent: Sosospider
Disallow: /
User-agent: Googlebot
Disallow: donations.html
User-agent: *
Disallow: /bugzilla/
Disallow: /cgi-bin/
Disallow: /cgi/
Disallow: /cgi2-bin/
Disallow: /ml/overseers/
Disallow: /packages/
Disallow: /snapshots/
Disallow: /viewcvs/
Disallow: /viewvc/
Disallow: /git/
Disallow: /git-cygwin-packages/
Disallow: /cgit/
Disallow: setup-x86.exe
Disallow: setup-x86_64.exe
Disallow: mirrors.lst
Crawl-Delay: 60
Look up this url in the url tool
https://cygwin.com/.well-known/acme-challenge: 403 text/html
https://cygwin.com/.well-known/csvm: 404 text/html; charset=iso-8859-1
https://cygwin.com/.well-known/nostr.json: 404 text/html; charset=iso-8859-1
https://cygwin.com/.well-known/security.txt: 404 text/html; charset=iso-8859-1
https://cygwin.com/.well-known/traffic-advice: 404 text/html; charset=iso-8859-1