Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

Top 6.2% downloads on rubygems.org
Top 7.4% forks on rubygems.org

rubygems.org : robotstxt-parser

Robotstxt-Parser allows you to the check the accessibility of URLs and get other data. Full support for the robots.txt RFC, wildcards and Sitemap: rules.

Registry - Source - Documentation - JSON
purl: pkg:gem/robotstxt-parser
License: MIT
Latest release: over 9 years ago
First release: over 9 years ago
Dependent repositories: 1
Downloads: 163,799 total
Stars: 8 on GitHub
Forks: 17 on GitHub
See more repository details: repos.ecosyste.ms
Funding links: https://github.com/sponsors/gjtorikian
Last synced: 7 days ago

    Loading...
    Readme
    Loading...