Search

Matched domain: web.mit.edu

`robots.txt`

User-agent: *
Disallow: /afs/
Disallow: /cgi-bin/
Disallow: /user/
Disallow: /org/
Disallow: /activity/
Disallow: /contrib/
Disallow: /dept/
Disallow: /software/
Disallow: /bin/
Disallow: */Public/*

Look up this url in the url tool https://web.mit.edu/.well-known/acme-challenge: 404 text/html
https://web.mit.edu/.well-known/csvm: 404 text/html
https://web.mit.edu/.well-known/nostr.json: 404 text/html
https://web.mit.edu/.well-known/security.txt: 404 text/html
https://web.mit.edu/.well-known/traffic-advice: 404 text/html