Search

Host: libraries.io
User-agent: *
Disallow: /admin/
Disallow: /github/
Disallow: /gitlab/
Disallow: /bitbucket/
Disallow: /explore/
Disallow: /search
Disallow: /login
Disallow: */usage$
Disallow: */dependents$
Disallow: */dependencies$
Disallow: */top_dependent_repos$
Disallow: */sourcerank$
Disallow: */versions$
Disallow: /auth/
Disallow: */suggestions$
Disallow: */tree$
Disallow: */*.about$
Disallow: */tags$
Sitemap: https://libraries.io/sitemaps/sitemap.xml.gz

# stop yandex from crawling usage pages
User-agent: Yandex
Clean-param: kind&requirements&page