Search

Matched domain: www.cs.columbia.edu

`robots.txt`

User-agent: *
Disallow: /crf/phone/3com/
Disallow: /CAVE/private
Disallow: /CAVE/exclude
Disallow: /CAVE/phpadmin
Disallow: /nlp/newsblaster/archives/
Disallow: /?plugin=attach&pcmd=open&file=*
Disallow: /mice
Disallow: /crftest
Disallow: /crfnew
Disallow: /crf/support_charge.shtml
Disallow: /webtest
Disallow: /~nayar/akash

User-agent: Slurp
Crawl-delay: 10

Look up this url in the url tool https://www.cs.columbia.edu/.well-known/acme-challenge: 200 text/html;charset=UTF-8
https://www.cs.columbia.edu/.well-known/csvm: 404 text/html; charset=UTF-8
https://www.cs.columbia.edu/.well-known/nostr.json: 404 text/html; charset=UTF-8
https://www.cs.columbia.edu/.well-known/security.txt: 404 text/html; charset=UTF-8
https://www.cs.columbia.edu/.well-known/traffic-advice: 404 text/html; charset=UTF-8