Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "crawlers" keyword

drissionpage-0503 4.0.4.21
Python based web automation tool. It can control the browser and send and receive data packets.
1 version - Latest release: 17 days ago - 200 downloads last month - 5,376 stars on GitHub - 1 maintainer
drissionpageplus 4.0.4.22.1
Add some functions to DrissionPage
2 versions - Latest release: 9 days ago - 190 downloads last month - 5,376 stars on GitHub - 1 maintainer
robotsparse 1.0
A python package that enhances speed and simplicity of parsing robots files.
2 versions - Latest release: 13 days ago - 264 downloads last month - 0 stars on GitHub - 1 maintainer
sneakpeek-py 0.2.2
Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It's the best c...
11 versions - Latest release: 10 months ago - 51 downloads last month - 34 stars on GitHub - 1 maintainer
wikilink 0.3.0
A multiprocessing web-scraping application to scrape wiki pages and find minimum number of links ...
2 versions - Latest release: about 5 years ago - 1 dependent repositories - 25 downloads last month - 10 stars on GitHub - 1 maintainer
shaman 0.0.2a1
Multiprocessing application to download and analyze a content of an html pages.
7 versions - Latest release: 9 months ago - 1 dependent repositories - 33 downloads last month - 1 maintainer
proxy-list-scrapper 0.2.2
Proxy list scrapper from various websites. They gives the free proxies for temporary use.
9 versions - Latest release: about 3 years ago - 1 dependent repositories - 582 downloads last month - 94 stars on GitHub - 1 maintainer
hunters 2.1.1
A spider automation framework, provide uniform API to use chrome Dev Protocol and requests
37 versions - Latest release: about 6 years ago - 1 dependent repositories - 155 downloads last month - 1 maintainer
is-bot 0.2.4
Python package to detect bots/crawlers/spiders via user-agent
10 versions - Latest release: 9 months ago - 2.22 thousand downloads last month - 8 stars on GitHub - 1 maintainer
smoothcrawler-cluster 0.2.0
Develop and build web spider cluster humanly.
2 versions - Latest release: about 1 year ago - 19 downloads last month - 1 stars on GitHub - 1 maintainer