Data

Tools

Indexes

Applications

Experiments

Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

packagist.org "robots-txt-parser" keyword

View the packages on the packagist.org package registry that are tagged with the "robots-txt-parser" keyword.

crwlr/robots-txt v1.1.2 💰
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
7 versions - Latest release: 6 months ago - 1 dependent package - 1 dependent repositories - 16.4 thousand downloads total - 8 stars on GitHub - 1 maintainer
Top 4.9% on packagist.org
bopoda/robots-txt-parser v2.5.0
PHP Class for parsing robots.txt files according to Google, Yandex specifications.
4 versions - Latest release: 3 months ago - 1 dependent package - 7 dependent repositories - 203 thousand downloads total - 45 stars on GitHub - 1 maintainer
Related Keywords
parser 2 robots.txt 2 php 1 yandex 1 google 1 web-scraping 1 web-crawling 1 robots-txt 1 robots-exclusion-standard 1 robots-exclusion-protocol 1 hacktoberfest 1 crwlr 1 exclusion 1 bots 1 scraping 1 standard 1 protocol 1 robots 1 bot 1 crawler 1 spider 1 scraper 1