Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "crawler" keyword

Top 0.4% on pypi.org
scrapy 2.11.1
A high-level Web Crawling and Web Scraping framework
98 versions - Latest release: 3 months ago - 111 dependent packages - 2,753 dependent repositories - 1.42 million downloads last month - 51,036 stars on GitHub - 4 maintainers
pubopinion 0.1.0a3
public opinion module
1 version - Latest release: over 4 years ago - 1 dependent repositories - 13 downloads last month - 2 maintainers
botasaurus 4.0.14 💰
The All in One Web Scraping Framework
74 versions - Latest release: about 1 month ago - 1 dependent repositories - 3.53 thousand downloads last month - 947 stars on GitHub - 2 maintainers
talospider 0.0.6
A simple,lightweight scraping micro-framework
6 versions - Latest release: about 6 years ago - 5 dependent repositories - 91 downloads last month - 54 stars on GitHub - 2 maintainers
jmcomic 2.5.11
Python API For JMComic (禁漫天堂)
112 versions - Latest release: 14 days ago - 15.2 thousand downloads last month - 435 stars on GitHub - 1 maintainer
pubchem-api-crawler 1.0.3
PubChem REST API crawler to retrieve compound properties using a molecular formula search
1 version - Latest release: 3 months ago - 15 downloads last month - 0 stars on GitHub - 2 maintainers
langcrawler 0.0.4
Language Crawler
4 versions - Latest release: over 3 years ago - 1 dependent repositories - 39 downloads last month - 0 stars on GitHub - 2 maintainers
Top 3.6% on pypi.org
ruia 0.8.5
Async Python 3.6+ web scraping micro-framework based on asyncio.
55 versions - Latest release: over 1 year ago - 2 dependent packages - 21 dependent repositories - 699 downloads last month - 1,733 stars on GitHub - 2 maintainers
article-crawler 0.0.4
A package for crawling markdown formatted articles from certain webpage and storing them locally.
4 versions - Latest release: 9 months ago - 45 downloads last month - 25 stars on GitHub - 2 maintainers
xehentai 2.2
xeHentai Downloader
1 version - Latest release: over 4 years ago - 1 dependent repositories - 12 downloads last month - 717 stars on GitHub - 2 maintainers
digs 0.1.7
Making easier the text crawling tasks over websites with depth levels.
8 versions - Latest release: over 8 years ago - 2 dependent repositories - 30 downloads last month - 2 maintainers
proxyhub 0.0.1a5
An advanced [Finder | Checker | Server] tool for proxy servers, supporting both HTTP(S) and SOCKS...
4 versions - Latest release: 6 months ago - 103 downloads last month - 172 stars on GitHub - 2 maintainers
Top 9.8% on pypi.org
httpclient 0.0.2
A headless HTTP browser.
2 versions - Latest release: over 12 years ago - 8 dependent repositories - 191 downloads last month - 2 maintainers
pygooglenewsscraper 0.1.2
Scrape news content from the Google News website
3 versions - Latest release: over 2 years ago - 1 dependent repositories - 15 downloads last month - 3 stars on GitHub - 2 maintainers
datasetrising 1.0.4
Toolchain for creating and training Stable Diffusion models with custom datasets
86 versions - Latest release: 5 months ago - 802 downloads last month - 11 stars on GitHub - 2 maintainers
scrapqd 1.0.1b0
Scrape query definition intends to eliminate backend process of crawling and focus on xpath neede...
2 versions - Latest release: about 2 years ago - 1 dependent repositories - 18 downloads last month - 1 maintainer
movie-crawl 0.1
Movie Crawler Library
1 version - Latest release: about 8 years ago - 2 dependent repositories - 12 downloads last month - 2 maintainers
maxxn 1.1.0
simple crawler
2 versions - Latest release: over 9 years ago - 2 dependent repositories - 8 downloads last month - 2 maintainers
Top 7.4% on pypi.org
lightnovel-crawler 3.7.1 💰
An app to download novels from online sources and generate e-books.
180 versions - Latest release: 2 days ago - 1 dependent package - 1 dependent repositories - 3.13 thousand downloads last month - 1,164 stars on GitHub - 1 maintainer
proxy-list-scrapper 0.2.2
Proxy list scrapper from various websites. They gives the free proxies for temporary use.
9 versions - Latest release: about 3 years ago - 1 dependent repositories - 582 downloads last month - 94 stars on GitHub - 2 maintainers
mangacrawler 1.0.0a3
Crawler for finding manga to read.
3 versions - Latest release: about 7 years ago - 1 dependent repositories - 32 downloads last month - 3 stars on GitHub - 2 maintainers
pydude-pyto 0.22.0 💰
dude uncomplicated data extraction (For Pyto on iOS)
13 versions - Latest release: over 1 year ago - 1 dependent repositories - 63 downloads last month - 413 stars on GitHub - 1 maintainer
nightcrawler 0.1.6
Website crawling bot
6 versions - Latest release: about 5 years ago - 1 dependent repositories - 54 downloads last month - 0 stars on GitHub - 2 maintainers
mocy 0.2.3
A lightweight web crawling framework.
3 versions - Latest release: almost 2 years ago - 1 dependent repositories - 37 downloads last month - 3 stars on GitHub - 2 maintainers
Top 8.9% on pypi.org
dosage 2.9
a comic strip downloader and archiver
28 versions - Latest release: 9 months ago - 5 dependent repositories - 99 downloads last month - 121 stars on GitHub - 4 maintainers
wscrap 0.1.0
Command line web scrapping tool
1 version - Latest release: over 3 years ago - 1 dependent repositories - 11 downloads last month - 0 stars on GitHub - 2 maintainers
products-crawler 0.1.9
A scrapy project for crawl product pictures and information.
10 versions - Latest release: almost 2 years ago - 1 dependent repositories - 78 downloads last month - 10 stars on GitHub - 1 maintainer
pixivhack 0.1.5
Pixiv Hack is a tool to automatically crawl illustrations filtered by ratings on www.pixiv.net
1 version - Latest release: over 8 years ago - 2 dependent repositories - 12 downloads last month - 15 stars on GitHub - 2 maintainers
multiprocessingspider 1.1.2
A multiprocessing web crawling and web scraping framework.
4 versions - Latest release: about 4 years ago - 1 dependent repositories - 59 downloads last month - 1 stars on GitHub - 1 maintainer
Top 2.8% on pypi.org
news-please 1.5.44 💰
news-please is an open source easy-to-use news extractor that just works.
125 versions - Latest release: 5 months ago - 2 dependent packages - 64 dependent repositories - 5.09 thousand downloads last month - 1,935 stars on GitHub - 2 maintainers
Top 6.3% on pypi.org
magic-google 0.2.9
A google search results crawler
2 versions - Latest release: almost 4 years ago - 1 dependent package - 2 dependent repositories - 344 downloads last month - 374 stars on GitHub - 2 maintainers
sosse 1.9.0
Selenium Open Source Search Engine
21 versions - Latest release: 2 months ago - 114 downloads last month - 1 stars on GitLab.com - 2 maintainers
mlscraper 0.1.2
Scrape HTML automatically with machine learning.
4 versions - Latest release: over 3 years ago - 1 dependent repositories - 95 downloads last month - 1,221 stars on GitHub - 1 maintainer
Top 7.9% on pypi.org
scylla 1.1.7
Intelligent proxy pool for Humans™
30 versions - Latest release: over 4 years ago - 13 dependent repositories - 262 downloads last month - 3,867 stars on GitHub - 1 maintainer
Top 7.2% on pypi.org
bilix 0.18.8
⚡️Lightning-fast asynchronous download tool for bilibili and more
85 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 1.33 thousand downloads last month - 1,501 stars on GitHub - 1 maintainer
Top 4.6% on pypi.org
emailfinder 0.3.0b0
EmailFinder - Emails search through Search Engines
5 versions - Latest release: over 2 years ago - 86 dependent repositories - 5.2 thousand downloads last month - 276 stars on GitHub - 2 maintainers
Top 6.9% on pypi.org
crawlerdetect 0.1.7
CrawlerDetect is a Python class for detecting bots/crawlers/spiders via the user agent.
8 versions - Latest release: 9 months ago - 1 dependent package - 5 dependent repositories - 19.6 thousand downloads last month - 38 stars on GitHub - 2 maintainers
Top 2.8% on pypi.org
pyspider 0.3.10
A Powerful Spider System in Python
17 versions - Latest release: about 6 years ago - 1 dependent package - 98 dependent repositories - 1.16 thousand downloads last month - 16,276 stars on GitHub - 1 maintainer
scrapfly-sdk 0.8.16
Scrapfly SDK for Scrapfly
37 versions - Latest release: 24 days ago - 2 dependent repositories - 15.6 thousand downloads last month - 17 stars on GitHub - 1 maintainer
Top 4.2% on pypi.org
scrapyrt 0.16.0
Put Scrapy spiders behind an HTTP API
8 versions - Latest release: 3 months ago - 75 dependent repositories - 2.1 thousand downloads last month - 815 stars on GitHub - 4 maintainers
Top 9.2% on pypi.org
moodle-dl 2.3.9 💰
Moodle-DL downloads course content fast from Moodle (eg. lecture pdfs)
89 versions - Latest release: 12 days ago - 1 dependent repositories - 1.25 thousand downloads last month - 384 stars on GitHub - 2 maintainers
Top 1.8% on pypi.org
gallery-dl 1.26.9 💰
Command-line program to download image galleries and collections from several image hosting sites
143 versions - Latest release: about 2 months ago - 5 dependent packages - 31 dependent repositories - 39 thousand downloads last month - 9,174 stars on GitHub - 1 maintainer
pinscrape 3.0.3
Pinterest data scraper
10 versions - Latest release: 11 months ago - 1 dependent repositories - 260 downloads last month - 44 stars on GitHub - 1 maintainer
Top 4.5% on pypi.org
igramscraper 0.3.5 💰
scrapes medias, likes, followers, tags and all metadata
9 versions - Latest release: almost 4 years ago - 35 dependent repositories - 522 downloads last month - 2,495 stars on GitHub - 2 maintainers
salted 0.7.2
Smart, Asynchronous Link Tester with Database backend
10 versions - Latest release: almost 3 years ago - 1 dependent repositories - 80 downloads last month - 3 stars on GitHub - 1 maintainer
Top 7.0% on pypi.org
bose 2.0.22 💰
The Ultimate Web Scraping Framework
31 versions - Latest release: 5 months ago - 10 dependent repositories - 1.19 thousand downloads last month - 928 stars on GitHub - 2 maintainers
Top 4.4% on pypi.org
scrapy-crawlera 1.7.2
Crawlera middleware for Scrapy
16 versions - Latest release: over 3 years ago - 52 dependent repositories - 46.3 thousand downloads last month - 348 stars on GitHub - 8 maintainers
Top 9.3% on pypi.org
page_finder 0.1.9
Find which links on a web page are pagination links
10 versions - Latest release: over 7 years ago - 14 dependent repositories - 447 downloads last month - 30 stars on GitHub - 4 maintainers
newspaper4k 0.9.3
Simplified python article discovery & extraction.
5 versions - Latest release: about 2 months ago - 9.33 thousand downloads last month - 303 stars on GitHub - 2 maintainers
finxos 1.0.0
Open source data tools.
5 versions - Latest release: over 4 years ago - 1 dependent repositories - 52 downloads last month - 3 stars on GitHub - 2 maintainers
scrapingant-client 2.0.1
Official python client for the ScrapingAnt API.
16 versions - Latest release: 9 months ago - 3 dependent repositories - 1.96 thousand downloads last month - 31 stars on GitHub - 2 maintainers
firecrawl-py 0.0.8
Python SDK for Firecrawl API
8 versions - Latest release: 4 days ago - 6.81 thousand downloads last month - 2,728 stars on GitHub - 2 maintainers
mindfactory-crawling 1.0.4
A crawler for mindfactory.de
2 versions - Latest release: about 5 years ago - 1 dependent repositories - 20 downloads last month - 3 stars on GitHub - 2 maintainers
kocrawl-cna 1.0.1
Korean web crawler collections
2 versions - Latest release: about 2 years ago - 1 dependent repositories - 11 downloads last month - 0 stars on GitHub - 1 maintainer
trophyfetcher 0.1.0
A package used to fetch public information about trophies from PSN.
1 version - Latest release: almost 8 years ago - 2 dependent repositories - 21 downloads last month - 2 maintainers
Top 1.7% on pypi.org
trafilatura 1.9.0 💰
Python package and command-line tool designed to gather text on the Web, includes all necessary d...
44 versions - Latest release: 11 days ago - 62 dependent packages - 63 dependent repositories - 476 thousand downloads last month - 2,688 stars on GitHub - 1 maintainer
bookerepubtool 2023.7.9.1
iBooker/ApacheCN 知识库抓取工具
2 versions - Latest release: 10 months ago - 35 downloads last month - 2 maintainers
python-darc 1.0.2 💰
Darkweb crawler & search engine.
49 versions - Latest release: 4 months ago - 1 dependent repositories - 373 downloads last month - 105 stars on GitHub - 2 maintainers
scrapy-hls 0.1
scrapy integration for m3u8 files
1 version - Latest release: about 3 years ago - 1 dependent repositories - 18 downloads last month - 51,036 stars on GitHub - 1 maintainer
Top 9.5% on pypi.org
pylab-utils 0.5
python utility tools
2 versions - Latest release: over 2 years ago - 1 dependent repositories - 156 downloads last month - 51,036 stars on GitHub - 1 maintainer
aminer-scrapy 2.11.1
A high-level Web Crawling and Web Scraping framework
3 versions - Latest release: 3 months ago - 47 downloads last month - 51,036 stars on GitHub - 1 maintainer
cyberplant-scrapy 1.2.0.dev2
A high-level Web Crawling and Web Scraping framework
1 version - Latest release: almost 8 years ago - 1 dependent repositories - 29 downloads last month - 51,015 stars on GitHub - 2 maintainers
scrapy-scraper 1.7 💰
Web crawler and scraper based on Scrapy and Playwright's headless browser.
8 versions - Latest release: 2 days ago - 39 downloads last month - 4 stars on GitHub - 2 maintainers
Top 7.6% on pypi.org
comiccrawler 2024.4.11
An image crawler, including multiple modules and GUI.
181 versions - Latest release: about 1 month ago - 2 dependent repositories - 1.97 thousand downloads last month - 258 stars on GitHub - 1 maintainer
shopee-api-wrapper 0.2.1
A simple library that communicates with the Shopee website API.
3 versions - Latest release: 3 months ago - 18 downloads last month - 2 maintainers
spidey.py 0.4.5
Web spiders are usually disliked by websites, but useful for recursive API/page downloads for off...
3 versions - Latest release: over 7 years ago - 32 downloads last month - 1 stars on GitHub - 2 maintainers
greenflare 0.98.1
SEO Web Crawler and Analysis Tool
6 versions - Latest release: about 3 years ago - 1 dependent repositories - 95 downloads last month - 140 stars on GitHub - 2 maintainers
dtadmin 0.0.3
DTAdmin - a crawler and data visualization management system based on flash framework.
1 version - Latest release: over 3 years ago - 1 dependent repositories - 20 downloads last month - 2 maintainers
simple-site-crawler 0.1.1
Simple website crawler that asynchronously crawls a website and all subpages that it can find, al...
2 versions - Latest release: over 7 years ago - 1 dependent repositories - 33 downloads last month - 3 stars on GitHub - 2 maintainers
aws_crawler 1.2.4
Crawl through AWS accounts in an organization using master assumed role.
14 versions - Latest release: 3 months ago - 90 downloads last month - 0 stars on GitHub - 1 maintainer
bolsa 2.1.0
Biblioteca feita em python com o objetivo de facilitar o acesso a dados de seus investimentos na ...
11 versions - Latest release: about 3 years ago - 1 dependent repositories - 82 downloads last month - 58 stars on GitHub - 1 maintainer
smoothcrawler 0.2.0
Build crawler humanly as different roles which be combined with different components.
2 versions - Latest release: almost 2 years ago - 1 dependent package - 2 dependent repositories - 140 downloads last month - 2 stars on GitHub - 2 maintainers
fulmar 0.0.2
A Distributed Web Crawler System in Python
2 versions - Latest release: over 7 years ago - 1 dependent repositories - 10 downloads last month - 4 stars on GitHub - 2 maintainers
crawlee 0.0.2
Crawlee for Python
13 versions - Latest release: about 1 month ago - 700 downloads last month - 2 maintainers
easy-twitter-crawler 1.0.4
简易、强大的推特（Twitter）采集程序,支持元搜索,用户,粉丝,关注,发文,回复,评论等采集
5 versions - Latest release: 8 months ago - 82 downloads last month - 16 stars on GitHub - 2 maintainers
gmap-scrabbler 0.2.2
The Google Map Reviews Web Scrabbler is a specialized tool designed to extract Google Maps review...
1 version - Latest release: 6 months ago - 11 downloads last month - 0 stars on GitHub - 2 maintainers
mbapy 0.7.4
MyBA in Python
35 versions - Latest release: 3 days ago - 234 downloads last month - 0 stars on GitHub - 1 maintainer
python-facebook-bot 0.1.14
Using API to get Facebook Events by location, etc... with Python
14 versions - Latest release: over 6 years ago - 2 dependent repositories - 55 downloads last month - 40 stars on GitHub - 2 maintainers
xalpha 0.12.0
all about fund investment
55 versions - Latest release: about 1 month ago - 1 dependent repositories - 798 downloads last month - 1,878 stars on GitHub - 2 maintainers
caqui 2.1.3
Run asynchronous commands in WebDrivers
27 versions - Latest release: 9 months ago - 157 downloads last month - 10 stars on GitHub - 2 maintainers
google-seo-analyzer 0.1.1
This simple script parses Google Web Master Tools report and analyzes results.
2 versions - Latest release: over 10 years ago - 2 dependent repositories - 11 downloads last month - 12 stars on GitHub - 2 maintainers
wronnay-search-lib 1.0.1
A library of classes which can be used to build a search engine.
1 version - Latest release: over 4 years ago - 1 dependent repositories - 18 downloads last month - 1 stars on GitHub - 2 maintainers
evtol-crawler 1.0.0
Evtol Crawler
5 versions - Latest release: almost 2 years ago - 7 downloads last month - 1 stars on GitHub - 2 maintainers
gain 0.1.4
Web crawling framework for everyone.
5 versions - Latest release: almost 7 years ago - 4 dependent repositories - 70 downloads last month - 2,027 stars on GitHub - 4 maintainers
geckordp 0.4.53
A client implementation of Firefox DevTools over remote debug protocol
22 versions - Latest release: about 1 year ago - 1 dependent repositories - 1.84 thousand downloads last month - 13 stars on GitHub - 2 maintainers
Top 8.3% on pypi.org
proxycrawl 3.2.2
A Python class that acts as wrapper for ProxyCrawl scraping and crawling API
12 versions - Latest release: 10 months ago - 3 dependent repositories - 4.12 thousand downloads last month - 59 stars on GitHub - 1 maintainer
underdata 0.1.2
Scraping data package for www.understat.com
1 version - Latest release: over 2 years ago - 1 dependent repositories - 9 downloads last month - 3 stars on GitHub - 2 maintainers
crawler4py 0.5
A distributed crawler framework based on Python
5 versions - Latest release: over 3 years ago - 1 dependent repositories - 42 downloads last month - 6 stars on GitHub - 2 maintainers
tradingview-scraper 0.1.6
Tradingview scraper tool
16 versions - Latest release: almost 2 years ago - 1 dependent repositories - 66 downloads last month - 49 stars on GitHub - 2 maintainers
gzspidertools 0.0.18
魔改使用工具库
18 versions - Latest release: about 1 month ago - 277 downloads last month - 1 maintainer
persession 0.1.4
A wrapper on requests session with persistence and login functionalities
6 versions - Latest release: over 4 years ago - 1 dependent repositories - 56 downloads last month - 3 stars on GitHub - 2 maintainers
bookermarkdowntool 2023.7.9.1
iBooker/ApacheCN 知识库抓取工具
4 versions - Latest release: 10 months ago - 42 downloads last month - 2 maintainers
Top 5.3% on pypi.org
scrapy-zyte-smartproxy 2.3.4
Scrapy middleware for Zyte Smart Proxy Manager
8 versions - Latest release: 4 days ago - 7 dependent repositories - 14.9 thousand downloads last month - 348 stars on GitHub - 2 maintainers
scrapy-proxycrawl-middleware 1.2.0
Scrapy ProxyCrawl Proxy Middleware: ProxyCrawl interfacing middleware for Scrapy
5 versions - Latest release: 10 months ago - 2 dependent repositories - 201 downloads last month - 10 stars on GitHub - 2 maintainers
socialregexes 0.1
Identify social network user account from url
1 version - Latest release: about 7 years ago - 1 dependent repositories - 54 downloads last month - 4 stars on GitHub - 2 maintainers
proxy-master 2.0.1
My first package to scrap free proxies from open resources
17 versions - Latest release: about 1 year ago - 181 downloads last month - 1 stars on GitHub - 1 maintainer
psstore4-ru 1.0.0
Play Station Store Russian Python Interface
3 versions - Latest release: about 3 years ago - 1 dependent repositories - 32 downloads last month - 1 stars on GitHub - 2 maintainers
psstore-ru 2.0.0
Play Station Store Russian Python Interface
2 versions - Latest release: about 3 years ago - 2 dependent repositories - 22 downloads last month - 1 stars on GitHub - 2 maintainers
reecapi 0.0.2
Library to access to the data offered by the 'Registro Español de Estudios Clínicos'
1 version - Latest release: over 3 years ago - 1 dependent repositories - 15 downloads last month - 2 stars on GitHub - 2 maintainers
ptt.py 1.0.0a2
a ptt crawler
2 versions - Latest release: about 6 years ago - 11 downloads last month - 3 stars on GitHub - 2 maintainers
Related Keywords
python 203 scraper 124 spider 109 scraping 87 scrapy 67 crawling 50 python3 49 web 45 web-scraping 31 api 27 selenium 24 requests 24 asyncio 24 proxy 22 data 22 webscraping 21 downloader 21 parser 20 framework 19 html 19 news 18 crawl 16 hacktoberfest 15 twitter 15 distributed 14 aiohttp 14 web-crawler 14 bot 13 redis 13 cli 12 async 12 search 12 automation 11 crawler-python 11 beautifulsoup 10 proxies 10 http 10 web-scraping-python 10 scrape 10 beautifulsoup4 9 pypi 9 bs4 9 nlp 8 news-aggregator 8 webscraper 8 website 8 proxypool 8 security 7 lxml 7 machine-learning 7 scrapper 7 mongodb 7 webcrawler 7 爬虫 7 search engine 7 google 7 osint 7 webcrawling 6 extractor 6 pdf 6 search-engine 6 pip 6 web-scraper 6 crawling-framework 6 webdriver 6 data-science 6 scrapy-redis 6 download 5 proxy-server 5 asynchronous 5 proxy-list 5 bilibili 5 elasticsearch 5 video 5 news-crawler 5 sitemap 5 scraping-framework 5 document 5 wiki 5 image 5 selenium-python 5 zhihu 5 instagram 5 frontier 5 data-mining 5 web-crawling 5 web-spider 5 ptt 5 testing 4 pixiv 4 images 4 csv 4 fastapi 4 novel 4 douyin 4 security-tools 4 sync 4 python-3 4 xpath 4 comic 4