Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
Top 6.2% downloads on packagist.org
Top 4.7% dependent packages on packagist.org
Top 4.4% dependent repos on packagist.org
Top 8.2% forks on packagist.org
packagist.org : tomverran/robots-txt-checker
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
Registry
-
Source
- Homepage
- JSON
purl: pkg:composer/tomverran/robots-txt-checker
Keywords: crawler, robots.txt
License: MIT
Latest release: almost 8 years ago
First release: about 10 years ago
Namespace: tomverran
Dependent packages: 3
Dependent repositories: 9
Downloads: 30,877 total
Stars: 10 on GitHub
Forks: 4 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 15 days ago