Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

packagist.org "robots-txt-parser" keyword

Top 4.9% on packagist.org

bopoda/robots-txt-parser v2.4.1

PHP Class for parsing robots.txt files according to Google, Yandex specifications.
3 versions - Latest release: about 3 years ago - 1 dependent package - 7 dependent repositories - 146 thousand downloads total - 41 stars on GitHub - 1 maintainer

crwlr/robots-txt v1.1.1

Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
6 versions - Latest release: over 1 year ago - 1 dependent package - 1 dependent repositories - 5.36 thousand downloads total - 8 stars on GitHub - 1 maintainer

Related Keywords

parser 2 robots.txt 2 web-scraping 1 web-crawling 1 robots-txt 1 robots-exclusion-standard 1 robots-exclusion-protocol 1 hacktoberfest 1 crwlr 1 exclusion 1 bots 1 scraping 1 standard 1 protocol 1 robots 1 bot 1 crawler 1 spider 1 scraper 1 php 1 yandex 1 google 1