An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "spider" keyword

scrapy-cffi 0.4.5
A lightweight asyncio worker kernel with a specialized curl_cffi crawler runtime.
25 versions - Latest release: 17 days ago - 248 downloads last month - 8 stars on GitHub - 1 maintainer
spider-mcp-client 0.1.8
Official Python client for Spider MCP web scraping API
9 versions - Latest release: about 1 year ago - 19 downloads last month - 1 maintainer
waifuboard 2.0.0
Asynchronous API for downloading images, tags, and metadata from image board sites (e.g., Danboor...
33 versions - Latest release: about 2 months ago - 194 downloads last month - 2 stars on GitHub - 1 maintainer
dimtown-spider 0.1.2
A spider
9 versions - Latest release: almost 2 years ago - 40 downloads last month - 2 stars on GitHub - 1 maintainer
Top 3.7% on pypi.org
grab 1.2.0
Web scraping framework
124 versions - Latest release: about 1 year ago - 1 dependent package - 123 dependent repositories - 1.02 thousand downloads last month - 2,464 stars on GitHub - 1 maintainer
pab-cli 0.2.4
A command-line tool for deploying Scrapy spiders to APCloudy
6 versions - Latest release: 9 months ago - 27 downloads last month - 0 stars on GitHub - 1 maintainer
Top 4.4% on pypi.org
feapder 1.9.3
feapder是一款支持分布式、批次采集、数据防丢、报警丰富的python爬虫框架
179 versions - Latest release: 10 months ago - 4 dependent repositories - 2.5 thousand downloads last month - 3,738 stars on GitHub - 1 maintainer
haipproxy2 0.1.4
High aviariable proxy pool client for crawlers.
5 versions - Latest release: over 5 years ago - 1 dependent repositories - 38 downloads last month - 5,522 stars on GitHub - 1 maintainer
rankcomp 0.0.2
high fided extraction of differential expression genes without considering the batch effects
1 version - Latest release: over 6 years ago - 1 dependent repositories - 3 downloads last month - 5,522 stars on GitHub - 1 maintainer
haipproxy 0.11.6
High aviariable proxy pool client for crawlers.
6 versions - Latest release: over 8 years ago - 1 dependent repositories - 47 downloads last month - 5,522 stars on GitHub - 1 maintainer
rapid-crawl 0.2.0 💰
A powerful, local-first Python SDK for web scraping, crawling, and data extraction
2 versions - Latest release: 4 months ago - 189 downloads last month - 0 stars on GitHub - 1 maintainer
lxmlhtml 0.0.2
a html parser based lxml
2 versions - Latest release: over 6 years ago - 1 dependent repositories - 114 downloads last month - 0 stars on GitHub - 1 maintainer
leon-mediacrawler 0.1.4
A social media crawler project, support Xiaohongshu, Weibo, Zhihu, Bilibili, Douyin, BaiduTieBa etc.
2 versions - Latest release: over 1 year ago - 30 downloads last month - 0 stars on GitHub - 1 maintainer
pianyuan 1.4.0
a spider from pianyuan.la
33 versions - Latest release: almost 7 years ago - 1 dependent repositories - 10 downloads last month - 10 stars on GitHub - 1 maintainer
Top 2.8% on pypi.org
pyspider 0.3.10
A Powerful Spider System in Python
17 versions - Latest release: over 8 years ago - 1 dependent package - 98 dependent repositories - 367 downloads last month - 16,767 stars on GitHub - 1 maintainer
Top 4.8% on pypi.org
gerapy 0.9.13
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, D...
47 versions - Latest release: about 3 years ago - 34 dependent repositories - 329 downloads last month - 3,510 stars on GitHub - 1 maintainer
Top 8.8% on pypi.org
decryptlogin 0.3.9 💰
DecryptLogin: APIs for loginning some websites by using requests.
46 versions - Latest release: about 4 years ago - 1 dependent repositories - 812 downloads last month - 2,853 stars on GitHub - 1 maintainer
decryptloginexamples 0.1.1 💰
DecryptLogin-examples: Some crawler examples for DecryptLogin.
2 versions - Latest release: over 4 years ago - 1 dependent repositories - 22 downloads last month - 2,853 stars on GitHub - 1 maintainer
Top 6.7% on pypi.org
django-dynamic-scraper 0.13.3
Creating Scrapy scrapers via the Django admin interface
65 versions - Latest release: over 5 years ago - 34 dependent repositories - 248 downloads last month - 1,154 stars on GitHub - 1 maintainer
simplejsonspider 0.2.2
A simple package to crawl JSON APIs and save response to local files.
3 versions - Latest release: over 1 year ago - 16 downloads last month - 1 maintainer
Top 2.7% on pypi.org
icrawler 0.6.10
A multi-thread crawler framework with many builtin image crawlers provided.
45 versions - Latest release: over 1 year ago - 2 dependent packages - 86 dependent repositories - 50.1 thousand downloads last month - 935 stars on GitHub - 2 maintainers
xhs-post-capture-sdk 1.0.0
小红书数据采集 & 运营 SDK - 支持笔记采集、用户画像、创作者中心、蒲公英、千帆平台
1 version - Latest release: 4 months ago - 9 downloads last month - 7,976 stars on GitHub - 1 maintainer
Top 3.6% on pypi.org
ruia 0.8.5
Async Python 3.6+ web scraping micro-framework based on asyncio.
55 versions - Latest release: about 4 years ago - 2 dependent packages - 21 dependent repositories - 303 downloads last month - 1,738 stars on GitHub - 1 maintainer
gain 1.0.1
Async web crawling framework for everyone.
7 versions - Latest release: 5 months ago - 4 dependent repositories - 140 downloads last month - 2,019 stars on GitHub - 2 maintainers
spidercreator 0.0.1
Automated web scraping spider generation using Browser Use and LLMs.
1 version - Latest release: over 1 year ago - 11 downloads last month - 227 stars on GitHub - 1 maintainer
Top 8.4% on pypi.org
xsrfprobe 3.0.0
The Prime Cross Site Request Forgery (CSRF) Audit & Exploitation Toolkit
6 versions - Latest release: 3 months ago - 1 dependent repositories - 408 downloads last month - 1,309 stars on GitHub - 1 maintainer
douyin-tiktok-scraper250605 0.0.4 💰
A fast and asynchronous Douyin/TikTok data scraper and API
4 versions - Latest release: over 1 year ago - 24 downloads last month - 20,450 stars on GitHub - 1 maintainer
Top 9.9% on pypi.org
douyin-tiktok-scraper 1.2.9 💰
Douyin/TikTok async data scraper.
21 versions - Latest release: about 3 years ago - 1.19 thousand downloads last month - 20,450 stars on GitHub - 1 maintainer
dt-scraper 2.0.2 💰
Douyin/TikTok crawler and no watermark video download.
7 versions - Latest release: almost 4 years ago - 52 downloads last month - 20,450 stars on GitHub - 1 maintainer
scrapy-drissionpage 1.0.3
将Scrapy爬虫框架与DrissionPage网页自动化工具进行无缝集成
2 versions - Latest release: over 1 year ago - 49 downloads last month - 1 maintainer
pyfreeproxy 0.5.2 💰
FreeProxy: Collecting free proxies from internet.
32 versions - Latest release: 20 days ago - 2 dependent repositories - 3.21 thousand downloads last month - 556 stars on GitHub - 1 maintainer
Top 3.1% on pypi.org
bilibili-api-python 17.4.2
The fork of module bilibili-api. 哔哩哔哩的各种 API 调用便捷整合(视频、动态、直播等),另外附加一些常用的功能。
105 versions - Latest release: 4 months ago - 9 dependent packages - 7 dependent repositories - 102 thousand downloads last month - 4,172 stars on GitHub - 1 maintainer
bilibili-api-dev 0.0.8
The fork of module bilibili-api. 哔哩哔哩的各种 API 调用便捷整合(视频、动态、直播等),另外附加一些常用的功能。
11 versions - Latest release: about 3 years ago - 7.93 thousand downloads last month - 4,172 stars on GitHub - 1 maintainer
streamget 4.0.10
A Multi-Platform Live Stream Parser Library.
9 versions - Latest release: 5 months ago - 1.54 thousand downloads last month - 273 stars on GitHub - 1 maintainer
Top 5.5% on pypi.org
photon 1.1.9 💰
Incredibly fast crawler designed for OSINT
11 versions - Latest release: almost 8 years ago - 12 dependent repositories - 1.08 thousand downloads last month - 13,248 stars on GitHub - 1 maintainer
markdown-spider 0.1.1
A configurable web spider that converts online documentation into well-structured Markdown files ...
1 version - Latest release: over 1 year ago - 49 downloads last month - 2 stars on GitHub - 1 maintainer
spiderclient-py 0.0.1
Python SDK for Spider Cloud API
1 version - Latest release: over 2 years ago - 1 downloads last month - 26 stars on GitHub - 1 maintainer
spidercloud-py 0.0.1
Python SDK for Spider Cloud API
1 version - Latest release: over 2 years ago - 10 downloads last month - 26 stars on GitHub - 1 maintainer
spider-client 0.1.93
Python SDK for Spider Cloud API
115 versions - Latest release: 3 months ago - 1 dependent package - 396 thousand downloads last month - 26 stars on GitHub - 1 maintainer
xhs-crawl 0.1.5
一个异步的小红书爬虫工具,支持笔记内容和图片的批量下载
6 versions - Latest release: over 1 year ago - 54 downloads last month - 0 stars on GitHub - 1 maintainer
spider-admin-pro 3.0.9
a spider admin based vue, scrapyd api and APScheduler
41 versions - Latest release: almost 2 years ago - 1 dependent repositories - 163 downloads last month - 614 stars on GitHub - 1 maintainer
spideradmin 0.0.32
a spider admin based scrapyd api and APScheduler
19 versions - Latest release: over 6 years ago - 1 dependent repositories - 97 downloads last month - 95 stars on GitHub - 1 maintainer
easy-spider-tool 1.0.17
简易、好用的爬虫工具,减少重复代码与文件冗余
17 versions - Latest release: over 2 years ago - 4 dependent packages - 246 downloads last month - 6 stars on GitHub - 1 maintainer
Top 7.4% on pypi.org
aktools 0.0.91
AKTools is a tool for AKShare HTTP API!
86 versions - Latest release: 11 months ago - 1 dependent package - 1 dependent repositories - 1.71 thousand downloads last month - 1,495 stars on GitHub - 1 maintainer
Top 7.4% on pypi.org
bilili 1.4.15 💰
🍻 bilibili video and danmaku downloader | B站视频、弹幕下载器
67 versions - Latest release: about 3 years ago - 2 dependent repositories - 293 downloads last month - 1,176 stars on GitHub - 1 maintainer
crawlberg 1.8.0
High-performance web crawling engine
35 versions - Latest release: 9 days ago - 2.84 thousand downloads last month - 180 stars on GitHub - 1 maintainer
talospider 0.0.6
A simple,lightweight scraping micro-framework
6 versions - Latest release: over 8 years ago - 5 dependent repositories - 40 downloads last month - 57 stars on GitHub - 1 maintainer
comicguispider 2.12.1 💰
GUI Comic Downloader
54 versions - Latest release: 22 days ago - 514 downloads last month - 925 stars on GitHub - 1 maintainer
baidu-image-crawling 0.0.1 💰
Baidu Image Spider
1 version - Latest release: over 1 year ago - 139 downloads last month - 0 stars on GitHub - 1 maintainer
mdbq 4.4.6
世界上最庄严的问题:我能做什么好事?
372 versions - Latest release: 4 months ago - 53 downloads last month - 1 maintainer
spkcspider 0.30
A secure, decentral platform for maintaining your online identity
78 versions - Latest release: over 6 years ago - 1 dependent repositories - 295 downloads last month - 7 stars on GitHub - 2 maintainers
spider.py 0.44
FTP and Web spiders and mirroring utilities
2 versions - Latest release: about 3 years ago - 2 dependent repositories - 1 maintainer
mr.migrator 1.2
Drive-by transmogrifiction made easy!
15 versions - Latest release: over 9 years ago - 5 dependent repositories - 31 downloads last month - 12 stars on GitHub - 4 maintainers
fetcher 0.9
simple python fetcher, support head, get, post action, file upload, etc
11 versions - Latest release: almost 16 years ago - 8 dependent repositories - 213 downloads last month - 1 maintainer
digslash 1.7
A site mapping and enumeration tool for Web applications analysis
10 versions - Latest release: almost 4 years ago - 1 dependent repositories - 11 downloads last month - 1 stars on GitHub - 1 maintainer
bdindex 0.1.1
A spider on baidu and toutiao
1 version - Latest release: almost 5 years ago - 1 dependent repositories - 18 downloads last month - 0 stars on GitHub - 1 maintainer
smallder 0.0.1
An out-of-the-box lightweight asynchronous crawler framework
1 version - Latest release: almost 2 years ago - 5 downloads last month - 8 stars on GitHub - 1 maintainer
pokeycrawl 0.1.1a5
A python webcrawler
2 versions - Latest release: over 10 years ago - 2 dependent repositories - 52 downloads last month - 0 stars on GitHub - 1 maintainer
youdaodict 6.1.2
通过有道爬虫查询单词
29 versions - Latest release: 5 months ago - 1 dependent repositories - 69 downloads last month - 36 stars on GitHub - 1 maintainer
byteapp-ak 1.15.7
AKShare is an elegant and simple financial data interface library for Python, built for human bei...
2 versions - Latest release: almost 2 years ago - 35 downloads last month - 1 maintainer
byteapp-akt 0.0.87
AKTools is a tool for AKShare HTTP API!
2 versions - Latest release: almost 2 years ago - 27 downloads last month - 1 maintainer
spider-rs 0.0.56
The fastest web crawler and indexer.
53 versions - Latest release: over 1 year ago - 2.33 thousand downloads last month - 109 stars on GitHub - 1 maintainer
scrapit 1.0.3
FastAPI wrapper for Scrapy that lets you to test Scrapy spiders via the API.
6 versions - Latest release: 10 months ago - 29 downloads last month - 0 stars on GitHub - 1 maintainer
silkworm-rs 0.14.2
Async-first web scraping framework built on wreq and scraper-rs
85 versions - Latest release: 6 days ago - 1.32 thousand downloads last month - 76 stars on GitHub - 1 maintainer
spider-orm 0.1.5
Spider-ORM is a lightweight and flexible ORM (Object-Relational Mapping) library for Python.
6 versions - Latest release: about 2 years ago - 58 downloads last month - 4 stars on GitHub - 1 maintainer
font-obfuscator 0.1.2
字体反爬、字体混淆工具是一个用于混淆字体文件的工具,可以将字体文件中的字形进行混淆,从而防止字体文件被直接提取出来。Font Obfuscator is an open-source Pytho...
3 versions - Latest release: about 2 years ago - 115 downloads last month - 14 stars on GitHub - 1 maintainer
tsdk 0.1.2
taobao login
3 versions - Latest release: about 2 years ago - 14 downloads last month - 738 stars on GitHub - 1 maintainer
curl-reap 0.2.2
Reap the web: browser-grade TLS impersonation, self-healing selectors, structured extraction, and...
6 versions - Latest release: 3 months ago - 239 downloads last month - 3 stars on GitHub - 1 maintainer
pycewl 0.1.0 💰
Python async re-implementation of CeWL (Custom Word List Generator)
1 version - Latest release: 8 months ago - 35 downloads last month - 2,844 stars on GitHub - 1 maintainer
scrapy-stealth 1.0.1
Stealthy Crawling. Maximum Results. A pluggable anti-bot and stealth framework for Scrapy.
56 versions - Latest release: 11 days ago - 81.8 thousand downloads last month - 9 stars on GitHub - 1 maintainer
wechatarticles 0.7.0
wechat articles scrapy
25 versions - Latest release: almost 5 years ago - 1 dependent repositories - 207 downloads last month - 3,479 stars on GitHub - 1 maintainer
cloudmusic 0.2.0
A small Python client for NetEase Cloud Music
3 versions - Latest release: 7 days ago - 1 dependent repositories - 23 downloads last month - 114 stars on GitHub - 1 maintainer
aksharepro 1.18.68
AKShare is an elegant and simple financial data interface library for Python, built for human bei...
2 versions - Latest release: 4 months ago - 66 downloads last month - 22,780 stars on GitHub - 1 maintainer
zakshare 1.12.8
AKShare is an elegant and simple financial data interface library for Python, built for human bei...
1 version - Latest release: almost 3 years ago - 22,780 stars on GitHub - 1 maintainer
Top 1.4% on pypi.org
akshare 1.18.97
AKShare is an elegant and simple financial data interface library for Python, built for human bei...
2,644 versions - Latest release: 16 days ago - 28 dependent packages - 54 dependent repositories - 2.11 million downloads last month - 22,780 stars on GitHub - 1 maintainer
akshare-proxy 1.17.88
AKShare is an elegant and simple financial data interface library for Python, built for human bei...
5 versions - Latest release: 11 months ago - 91 downloads last month - 22,780 stars on GitHub - 1 maintainer
scrapy-helper 1.5.3
scrapy helper
2 versions - Latest release: about 3 years ago - 1 dependent repositories - 11 downloads last month - 1 maintainer
spider-brew-kit 0.1.6
A library for scrapy tools, including but not limited to the usual pipelines, middlewares, etc.
6 versions - Latest release: over 2 years ago - 25 downloads last month - 0 stars on GitHub - 1 maintainer
bilili-eco 0.0.5
魔改BILILI
4 versions - Latest release: over 4 years ago - 69 downloads last month - 1 maintainer
spider-nest 0.2.0
Tools to pull text data from github, slack, website, stackoverflow, etc.
1 version - Latest release: over 3 years ago - 8 downloads last month - 1 stars on GitHub - 1 maintainer
gerapy-team 0.1.3
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, D...
1 version - Latest release: over 4 years ago - 6 downloads last month - 3,510 stars on GitHub - 1 maintainer
wauo 0.9.7
A comprehensive Python utility library featuring web scraping, decorators, thread pool management...
40 versions - Latest release: 11 months ago - 284 downloads last month - 3 stars on GitHub - 1 maintainer
autospider 1.1.11
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
2 versions - Latest release: over 2 years ago - 24 downloads last month - 0 stars on GitHub - 1 maintainer
trip 0.0.10
Async HTTP for humans, coroutine Requests.
11 versions - Latest release: over 8 years ago - 1 dependent repositories - 52 downloads last month - 208 stars on GitHub - 1 maintainer
spiderwebai-py 0.1.4
Python SDK for SpiderWebAI API
10 versions - Latest release: over 2 years ago - 18 downloads last month - 26 stars on GitHub - 1 maintainer
pyspider3 0.0.2
由于pyspider模块年久失修,pyspider3基于原作者github源码制作而成
1 version - Latest release: about 5 years ago - 1 dependent repositories - 6 downloads last month - 16,767 stars on GitHub - 1 maintainer
waizao 1.1.3
Simple financial data interface library for Python!
3 versions - Latest release: over 2 years ago - 11 downloads last month - 84 stars on GitHub - 1 maintainer
webcrawler 0.3
Web Crawler
1 version - Latest release: almost 13 years ago - 3 dependent repositories - 117 downloads last month - 0 stars on GitHub - 1 maintainer
aio-scrapy 2.1.9
A high-level Web Crawling and Web Scraping framework based on Asyncio
46 versions - Latest release: 8 months ago - 1 dependent repositories - 436 downloads last month - 71 stars on GitHub - 1 maintainer
cewler 1.4.1
Custom Word List generator Redefined
16 versions - Latest release: 11 months ago - 561 downloads last month - 164 stars on GitHub - 1 maintainer
boost-spider 1.5
横冲直闯 自由奔放 无回调 无继承写法的高速爬虫框架
5 versions - Latest release: 12 months ago - 48 downloads last month - 26 stars on GitHub - 1 maintainer
scrapy-scraper 4.0 💰
Web crawler and scraper based on Scrapy and Playwright's headless browser.
21 versions - Latest release: 7 months ago - 21 downloads last month - 17 stars on GitHub - 1 maintainer
captcha-identify 0.0.3
A package to solve captcha_identify
1 version - Latest release: about 3 years ago - 23 downloads last month - 193 stars on GitHub - 1 maintainer
chaser-crawler 1.1.1
A fast, reliable web crawling framework built on modern Python asyncio
1 version - Latest release: 18 days ago - 0 stars on GitHub - 1 maintainer
graphinder 1.11.6
Escape Graphinder
36 versions - Latest release: almost 4 years ago - 61 downloads last month - 229 stars on GitHub - 1 maintainer
domo 0.52
Domo is a web crawler & archiver
1 version - Latest release: about 3 years ago - 1 dependent repositories - 1 maintainer
xjtlu 0.1.0 💰
This is how to get all the network resources of XJTLU
1 version - Latest release: almost 3 years ago - 9 downloads last month - 4 stars on GitHub - 1 maintainer
Top 5.0% on pypi.org
scrapydweb 1.6.0
Web app for Scrapyd cluster management, with support for Scrapy log analysis & visualization.
20 versions - Latest release: over 1 year ago - 23 dependent repositories - 1.53 thousand downloads last month - 3,412 stars on GitHub - 1 maintainer
gbv 0.1.2
A package to get videos from Bilibili
3 versions - Latest release: about 2 years ago - 22 downloads last month - 1 stars on GitHub - 1 maintainer
animecrawler 0.2.1
一个可免费下载动漫的爬虫
6 versions - Latest release: over 3 years ago - 35 downloads last month - 1 stars on GitHub - 1 maintainer