Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

Top 9.7% dependent packages on packagist.org

packagist.org : crwlr/robots-txt

Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

Registry - Source - Homepage - JSON
purl: pkg:composer/crwlr/robots-txt
Keywords: parser, scraper, spider, crawler, bot, robots.txt, robots, protocol, standard, scraping, bots, exclusion, crwlr, hacktoberfest, robots-exclusion-protocol, robots-exclusion-standard, robots-txt, robots-txt-parser, web-crawling, web-scraping
License: MIT
Latest release: over 1 year ago
First release: over 2 years ago
Namespace: crwlr
Dependent packages: 1
Dependent repositories: 1
Downloads: 6,045 total
Stars: 8 on GitHub
Forks: 2 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 6 days ago

    Loading...
    Readme
    Loading...