packagist.org "Robots Exclusion Protocol" keyword
View the packages on the packagist.org package registry that are tagged with the "Robots Exclusion Protocol" keyword.
Top 3.7% on packagist.org
12 versions - Latest release: over 4 years ago - 5 dependent packages - 16 dependent repositories - 686 thousand downloads total - 26 stars on GitHub - 1 maintainer
vipnytt/robotstxtparser v2.1.0
Robots.txt parsing library, with full support for every directive and specification.12 versions - Latest release: over 4 years ago - 5 dependent packages - 16 dependent repositories - 686 thousand downloads total - 26 stars on GitHub - 1 maintainer
Top 9.1% on packagist.org
11 versions - Latest release: 5 months ago - 3 dependent packages - 25 dependent repositories - 858 thousand downloads total - 1 stars on GitHub - 1 maintainer
vipnytt/useragentparser v1.0.7
User-Agent parser for robot rule sets11 versions - Latest release: 5 months ago - 3 dependent packages - 25 dependent repositories - 858 thousand downloads total - 1 stars on GitHub - 1 maintainer
Related Keywords
spider
2
robots.txt
2
robot
2
robots-txt
2
REP
2
user-agent-parser
1
robots-tag
1
robots-meta-tag
1
robots-exclusion-protocol
1
rep
1
Robots meta tag
1
x-robots-tag
1
useragent
1
user-agent
1
crawler
1
robots-parser
1
Martijn Koster
1
Sean Conner
1
Robot exclusion standard
1
sitemaps.org
1
The Web Robots Pages
1
yandex
1
web-crawler
1
robots
1
W3C
1
google
1
parser
1