crates.io : robotstxt-with-cache
A native Rust port of Google's robots.txt parser and matcher C++ library.
Registry
-
Source
- Documentation
- JSON
purl: pkg:cargo/robotstxt-with-cache
Keywords:
parser
, robots
, robotstxt
, google-robots-parser
, rust
License: Apache-2.0
Latest release: about 4 years ago
First release: about 4 years ago
Dependent packages: 1
Dependent repositories: 2
Downloads: 4,396 total
Stars: 97 on GitHub
Forks: 15 on GitHub
See more repository details: repos.ecosyste.ms
Funding links: https://github.com/sponsors/Folyd
Last synced: 21 days ago
Loading...
Readme
Loading...