Top 9.9% on proxy.golang.org
proxy.golang.org : github.com/PerformLine/robotstxt
Package robotstxt implements the robots.txt Exclusion Protocol as specified in http://www.robotstxt.org/wc/robots.html with various extensions.
Registry
-
Source
- Documentation
- JSON
purl: pkg:golang/github.com/%21perform%21line/robotstxt
License: MIT
Latest release: over 2 years ago
First release: over 2 years ago
Namespace: github.com/PerformLine
Last synced: 6 days ago
Loading...
Readme
Loading...