IP = 151.101.130.217
robots.txtUser-agent: *
Disallow: /*/actions_panel*
Disallow: /users/auth/twitter*
Disallow: /users/auth/github*
Disallow: /report-abuse?url=*
Disallow: /connect/@*
Disallow: /search?q=*
Disallow: /search/?q=*
Disallow: /search/feed_content?*
Disallow: /listings*?q=*
Disallow: /mod/*
Disallow: /mod?*
Disallow: /admin/*
Disallow: /reactions?*
Disallow: /async_info/base_data
Disallow: /ahoy/*
Disallow: /auth_pass/*
Sitemap: https://dev.to/sitemap-index.xml
Look up this url in the url tool
https://dev.to/.well-known/acme-challenge: 404 text/html; charset=UTF-8
https://dev.to/.well-known/csvm: 404 text/html; charset=UTF-8
https://dev.to/.well-known/nostr.json: 404 text/html; charset=UTF-8
https://dev.to/.well-known/security.txt: 404 text/html; charset=UTF-8
https://dev.to/.well-known/traffic-advice: 404 text/html; charset=UTF-8