IP = 45.132.246.106
robots.txtUser-agent: AhrefsBot
User-agent: JobboerseBot
User-agent: seoscanners
User-agent: dotbot
User-agent: SemrushBot
User-agent: MJ12bot
Disallow: /
User-agent: *
Disallow: /admin/
Disallow: /pages/
Disallow: /lucee/
Allow: /pages/cms/getmedbyid.cfm
Allow: /pages/img/
Allow: /pages/css/
Allow: .css
Allow: .js
Allow: /style/*.css*
Allow: /pages/*.css*
Allow: /admin/*.css*
Allow: /pages/*.js*
Allow: /admin/*.js*
Crawl-delay: 15
Sitemap: https://www.arbor-ag.ch/sitemap.xml
Sitemap: https://www.arbor-ag.ch/image_sitemap.xml
Look up this url in the url tool
https://arbor-ag.ch/.well-known/acme-challenge: 404 text/html;charset=UTF-8
https://arbor-ag.ch/.well-known/csvm: 404 text/html;charset=UTF-8
https://arbor-ag.ch/.well-known/nostr.json: 404 text/html;charset=UTF-8
https://arbor-ag.ch/.well-known/security.txt: 404 text/html;charset=UTF-8
https://arbor-ag.ch/.well-known/traffic-advice: 404 text/html;charset=UTF-8