Data

Tools

Indexes

Applications

Experiments

Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "robots-parser" keyword

View the packages on the pypi.org package registry that are tagged with the "robots-parser" keyword.

Top 3.7% on pypi.org
protego 0.4.0
Pure-Python robots.txt parser with support for modern conventions
11 versions - Latest release: 3 months ago - 6 dependent packages - 799 dependent repositories - 1.52 million downloads last month - 64 stars on GitHub - 4 maintainers
jwm.robotstxt 1.0.8
Provides python access to Googles parser for robot.txt files as used by their GoogleBot webscraper.
4 versions - Latest release: 9 months ago - 1.8 thousand downloads last month - 0 stars on GitHub - 1 maintainer
robotspy 0.12.0
Robots Exclusion Protocol File Parser
15 versions - Latest release: 6 months ago - 1 dependent repositories - 1.01 thousand downloads last month - 14 stars on GitHub - 1 maintainer
Related Keywords
parser 3 python 3 robots 2 robots-txt 2 robotstxt 2 robots.txt 1 rep 1 hacktoberfest 1 wrapper 1 google 1 python-types 1 python3 1 robots-txt-parser 1 robots-exclusion-protocol 1 robots-exclusion-standard 1