Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

Top 0.4% on pypi.org
Top 0.3% downloads on pypi.org
Top 0.2% dependent packages on pypi.org
Top 0.2% dependent repos on pypi.org
Top 0.1% forks on pypi.org
Top 1.4% docker downloads on pypi.org

pypi.org : scrapy

A high-level Web Crawling and Web Scraping framework

Registry - Source - Homepage - Documentation - JSON
purl: pkg:pypi/scrapy
Keywords: crawler, crawling, framework, hacktoberfest, python, scraping, web-scraping, web-scraping-python
License: BSD-3-Clause
Latest release: 21 days ago
First release: over 14 years ago
Dependent packages: 136
Dependent repositories: 2,753
Downloads: 1,546,462 last month
Stars: 51,316 on GitHub
Forks: 10,371 on GitHub
Docker dependents: 64
Docker downloads: 588,527
See more repository details: repos.ecosyste.ms
Last synced: about 4 hours ago

surveyeval 0.1.12
A toolkit for survey evaluation
16 versions - Latest release: 18 days ago - 1 thousand downloads last month - 1 stars on GitHub - 1 maintainer
modis-crawler-utils 0.3.16
Scrapy utils for Modis crawlers projects.
29 versions - Latest release: 19 days ago - 217 downloads last month - 1 maintainer
langroid 0.1.245
Harness LLMs with Multi-Agent Programming
232 versions - Latest release: 20 days ago - 1 dependent repositories - 7.63 thousand downloads last month - 1,779 stars on GitHub - 1 maintainer
e-models 1.8.10
Tools for helping build of extraction models with scrapy spiders.
67 versions - Latest release: 20 days ago - 1.26 thousand downloads last month - 0 stars on GitHub - 1 maintainer
clappscrapers 0.2.21
Clappform Python scraper
36 versions - Latest release: 21 days ago - 503 downloads last month - 1 maintainer
samge-fork-scrapyd 1.4.3
A service for running Scrapy spiders, with an HTTP API
1 version - Latest release: 22 days ago - 0 stars on GitHub - 1 maintainer
Top 3.0% on pypi.org
fontbakery 0.12.6
A font quality assurance tool for everyone
133 versions - Latest release: 23 days ago - 1 dependent package - 222 dependent repositories - 12.4 thousand downloads last month - 526 stars on GitHub - 2 maintainers
samge-scrapyd 1.4.3 removed
A service for running Scrapy spiders, with an HTTP API
1 version - Latest release: 23 days ago - 1 maintainer
scrapy-scraper 1.7 💰
Web crawler and scraper based on Scrapy and Playwright's headless browser.
8 versions - Latest release: 24 days ago - 39 downloads last month - 4 stars on GitHub - 1 maintainer
docrawl 1.3.1
Do automated crawling of pages using scrapy
56 versions - Latest release: 25 days ago - 1 dependent repositories - 708 downloads last month - 3 stars on GitHub - 1 maintainer
Top 5.3% on pypi.org
scrapy-zyte-smartproxy 2.3.4
Scrapy middleware for Zyte Smart Proxy Manager
8 versions - Latest release: 26 days ago - 7 dependent repositories - 14.9 thousand downloads last month - 348 stars on GitHub - 1 maintainer
scrapy-playwright-full 0.0.3404
Playwright integration for Scrapy
7 versions - Latest release: 27 days ago - 181 downloads last month - 862 stars on GitHub - 1 maintainer
sparkpipelineframework 2.0.30
Framework for simpler Spark Pipelines
285 versions - Latest release: 27 days ago - 1 dependent package - 3 dependent repositories - 3.5 thousand downloads last month - 9 stars on GitHub - 1 maintainer
Top 4.6% on pypi.org
spidermon 1.22.0
Spidermon is a framework to build monitors for Scrapy spiders.
20 versions - Latest release: 27 days ago - 35 dependent repositories - 11.3 thousand downloads last month - 516 stars on GitHub - 2 maintainers
webcomix 3.10.0
Webcomic downloader
38 versions - Latest release: 28 days ago - 1 dependent repositories - 425 downloads last month - 26 stars on GitHub - 1 maintainer
zyte-spider-templates 0.7.2
Spider templates for automatic crawlers.
10 versions - Latest release: 28 days ago - 1 dependent repositories - 2.85 thousand downloads last month - 11 stars on GitHub - 1 maintainer
python-bvk 0.3.1
Python library for tracking water consumption from BVK (Brnenske vodarny a kanalizace, bvk.cz)
9 versions - Latest release: about 1 month ago - 1 dependent repositories - 162 downloads last month - 1 stars on GitHub - 1 maintainer
pyfeeds 2024.5.1
DIY Atom feeds in times of social media and paywalls
4 versions - Latest release: about 1 month ago - 1 dependent repositories - 149 downloads last month - 79 stars on GitHub - 2 maintainers
shub-workflow 1.13.8
Workflow manager for Zyte ScrapyCloud tasks.
232 versions - Latest release: about 1 month ago - 2 dependent packages - 1 dependent repositories - 3.29 thousand downloads last month - 12 stars on GitHub - 1 maintainer
matricula-online-scraper 0.4.1
Command Line Interface tool for scraping Matricula Online https://data.matricula-online.eu.
5 versions - Latest release: about 1 month ago - 83 downloads last month - 0 stars on GitHub - 1 maintainer
Top 7.1% on pypi.org
cyberdrop-dl 5.2.51 💰
Bulk downloader for multiple file hosts
871 versions - Latest release: about 1 month ago - 1 dependent repositories - 123 thousand downloads last month - 1,471 stars on GitHub - 2 maintainers
Top 10.0% on pypi.org
scrapy-zyte-api 0.18.2
Client library to process URLs through Zyte API
36 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 31.6 thousand downloads last month - 33 stars on GitHub - 1 maintainer
Top 6.0% on pypi.org
scrapy-poet 0.22.3
Page Object pattern for Scrapy
33 versions - Latest release: about 1 month ago - 2 dependent packages - 3 dependent repositories - 9.22 thousand downloads last month - 112 stars on GitHub - 3 maintainers
scrapy-impersonate 1.3.0 💰
Scrapy download handler that can impersonate browser fingerprints
8 versions - Latest release: about 1 month ago - 10.2 thousand downloads last month - 61 stars on GitHub - 1 maintainer
Top 2.1% on pypi.org
datalad 1.0.2
data distribution geared toward scientific datasets
114 versions - Latest release: about 2 months ago - 43 dependent packages - 78 dependent repositories - 20 thousand downloads last month - 493 stars on GitHub - 5 maintainers
scrapfly-sdk 0.8.16
Scrapfly SDK for Scrapfly
38 versions - Latest release: about 2 months ago - 2 dependent repositories - 13.1 thousand downloads last month - 18 stars on GitHub - 1 maintainer
Top 6.8% on pypi.org
pyobo 0.10.11
Handling and writing OBO
70 versions - Latest release: about 2 months ago - 4 dependent packages - 5 dependent repositories - 1.95 thousand downloads last month - 57 stars on GitHub - 1 maintainer
scrapy-colorlog 0.1.1
Color log output support for Scrapy
2 versions - Latest release: about 2 months ago - 42 downloads last month - 0 stars on GitHub - 1 maintainer
aio-scrapy 2.1.0
A high-level Web Crawling and Web Scraping framework based on Asyncio
39 versions - Latest release: about 2 months ago - 1 dependent repositories - 304 downloads last month - 51 stars on GitHub - 1 maintainer
jgdv 0.1.2
4 versions - Latest release: about 2 months ago - 1 dependent package - 42 downloads last month - 1 maintainer
scrapy-util 1.0.3
scrapy util
12 versions - Latest release: about 2 months ago - 1 dependent repositories - 163 downloads last month - 4 stars on GitHub - 1 maintainer
chronos_ai 0.1.0
1 version - Latest release: about 2 months ago - 231 downloads last month - 1 maintainer
dbservice 2.1.3
a Database Wrapper for Redis and MySQL
24 versions - Latest release: about 2 months ago - 1 dependent repositories - 170 downloads last month - 2 maintainers
scrapy-aiohttp-downloader 1.0.0b1
Scrapy download handler that integrates aiohttp
1 version - Latest release: about 2 months ago - 179 downloads last month - 0 stars on GitHub - 1 maintainer
scrachy 0.7.0
Enhanced functionality for Scrapy.
10 versions - Latest release: about 2 months ago - 1 dependent repositories - 129 downloads last month - 1 maintainer
scrapy-influxdb-exporter 1.0.6
A simple package to export Scrapy spider stats to InfluxDB
8 versions - Latest release: about 2 months ago - 79 downloads last month - 2 stars on GitHub - 1 maintainer
scrapy-settings-log 1.3.4
An extension that allows a user to display all or some of their scrapy spider settings at runtime.
10 versions - Latest release: 2 months ago - 2.27 thousand downloads last month - 0 stars on GitHub - 3 maintainers
edx-repo-tools 0.9.0
This repo contains a number of tools Open edX uses for working with GitHub repositories.
32 versions - Latest release: 2 months ago - 1 dependent repositories - 166 downloads last month - 30 stars on GitHub - 1 maintainer
scrapy-azure 0.2.0
A Scrapy extension to integrate with Microsoft Azure services
2 versions - Latest release: 2 months ago - 13 downloads last month - 1 maintainer
miscellaneous-utils 0.3.10
Random collection of utilities that I've found useful.
15 versions - Latest release: 2 months ago - 1 dependent package - 60 downloads last month - 0 stars on GitHub - 1 maintainer
llmstack 0.0.31
Low-code platform to build generative AI apps, chatbots and agents with your data
22 versions - Latest release: 2 months ago - 1.08 thousand downloads last month - 1,161 stars on GitHub - 1 maintainer
ecoindex_cli 2.27.1
`ecoindex-cli` is a CLI tool that let you make ecoindex tests on given pages
50 versions - Latest release: 2 months ago - 338 downloads last month - 46 stars on GitHub - 1 maintainer
Top 8.8% on pypi.org
apify 1.7.0
Apify SDK for Python
158 versions - Latest release: 3 months ago - 4 dependent repositories - 22.2 thousand downloads last month - 108 stars on GitHub - 3 maintainers
scrape-nhs-conditions 1.0.4
Scrapes the text from NHS conditions into txt files - one for each condition.
5 versions - Latest release: 3 months ago - 36 downloads last month - 0 stars on GitHub - 1 maintainer
jarvis-conversationalist 0.5.0
A voice assistant for the command line
25 versions - Latest release: 3 months ago - 187 downloads last month - 1 stars on GitHub - 1 maintainer
scrapy-selenium2 0.0.8
Scrapy with selenium
1 version - Latest release: 3 months ago - 39 downloads last month - 887 stars on GitHub - 1 maintainer
ayugespidertools 3.9.7
scrapy 扩展库：用于扩展 Scrapy 功能来解放双手。
89 versions - Latest release: 3 months ago - 1 dependent repositories - 266 downloads last month - 60 stars on GitHub - 1 maintainer
kafka-scrapy-connect 2.5.0
Integrating Scrapy with kafka using the confluent-kafka python client
8 versions - Latest release: 3 months ago - 87 downloads last month - 6 stars on GitHub - 1 maintainer
scrapy-requests-manipulate 0.0.2
requests downloader middleware for scrapy, send request by requests.
2 versions - Latest release: 3 months ago - 15 downloads last month - 1 maintainer
ecobp 0.1.2 removed
Refactoring of ecoindex in one monorepo using polylith pattern
2 versions - Latest release: 3 months ago - 280 downloads last month - 5 stars on GitHub - 1 maintainer
latest-scrapy-redis 0.7.3
Redis-based components for Scrapy.
1 version - Latest release: 3 months ago - 9 downloads last month - 0 stars on GitHub - 1 maintainer
Top 2.4% on pypi.org
advertools 0.14.2
Productivity and analysis tools for online marketing
57 versions - Latest release: 3 months ago - 4 dependent packages - 30 dependent repositories - 101 thousand downloads last month - 1,073 stars on GitHub - 1 maintainer
yangke 1.15.10 💰
个人工具综合平台，包含常用工具，网络爬虫，知识图谱，神经网络预测等工具
37 versions - Latest release: 3 months ago - 1 dependent repositories - 173 downloads last month - 1 maintainer
spider-brew-kit 0.1.6
A library for scrapy tools, including but not limited to the usual pipelines, middlewares, etc.
6 versions - Latest release: 4 months ago - 42 downloads last month - 0 stars on GitHub - 1 maintainer
board-game-scraper 2.22.0 💰
Board games data scraping and processing from BoardGameGeek and more!
58 versions - Latest release: 4 months ago - 5 dependent repositories - 467 downloads last month - 21 stars on GitLab.com - 1 maintainer
camcops-server 2.4.18
CamCOPS server
22 versions - Latest release: 4 months ago - 167 downloads last month - 3 maintainers
comment-recommendation-framework 0.16.6
The open-source repo for docs.github.com
5 versions - Latest release: 4 months ago - 41 downloads last month - 15,614 stars on GitHub - 1 maintainer
volleystats 0.8.1
Command-line tool to scrape volleyball statistics from Data Project Web Competition websites
8 versions - Latest release: 4 months ago - 43 downloads last month - 8 stars on GitHub - 1 maintainer
cli-yams 1.1
Yet Another Media Scraper
2 versions - Latest release: 4 months ago - 11 downloads last month - 3 stars on GitHub - 1 maintainer
scrapy-sentry-errors 1.0.0
Scrapy extension that logs errors to Sentry
2 versions - Latest release: 4 months ago - 620 downloads last month - 1 maintainer
favorites-crawler 0.2.0
Crawl your personal favorite images, photo albums, comics from website. Support pixiv, yande.re f...
26 versions - Latest release: 5 months ago - 1 dependent repositories - 144 downloads last month - 16 stars on GitHub - 1 maintainer
city-scrapers-sentry 1.0.0b1
Scrapy extension that logs errors to Sentry
4 versions - Latest release: 5 months ago - 24 downloads last month - 1 maintainer
scrapy-puppeteer-client 0.1.5
A library to use Puppeteer-managed browser in Scrapy spiders
11 versions - Latest release: 5 months ago - 1 dependent package - 1 dependent repositories - 205 downloads last month - 48 stars on GitHub - 2 maintainers
habra-favorites 2.0.0
Sort your favorites posts from Habrahabr.ru
16 versions - Latest release: 5 months ago - 2 dependent repositories - 63 downloads last month - 10 stars on GitHub - 1 maintainer
scrapy-nimble 0.0.2 💰
Scrapy Downloader Middleware that helps to integrate Scrapy with Nimble Web API.
2 versions - Latest release: 5 months ago - 16 downloads last month - 0 stars on GitHub - 1 maintainer
scrapy-qos 0.0.2
implement QOS(TokenBucket) in scrapy download middleware
2 versions - Latest release: 5 months ago - 17 downloads last month - 1 maintainer
easyspider 1.9.1
an easy way to use Scrapy
23 versions - Latest release: 5 months ago - 1 dependent repositories - 79 downloads last month - 2 maintainers
Top 2.8% on pypi.org
scrapy-playwright 0.0.34
Playwright integration for Scrapy
35 versions - Latest release: 5 months ago - 4 dependent packages - 22 dependent repositories - 36.6 thousand downloads last month - 862 stars on GitHub - 1 maintainer
Top 4.0% on pypi.org
python-office 0.4.23
www.python-office.com
145 versions - Latest release: 5 months ago - 3 dependent packages - 4 dependent repositories - 3.39 thousand downloads last month - 826 stars on GitHub - 1 maintainer
scrapy-aiohttp 0.1.2
Scrapy middleware for sending requests with aiohttp.
3 versions - Latest release: 6 months ago - 19 downloads last month - 1 stars on GitHub - 1 maintainer
Top 9.5% on pypi.org
hepcrawl 13.0.72
Scrapy project for feeds into INSPIRE-HEP (http://inspirehep.net).
176 versions - Latest release: 7 months ago - 6 dependent repositories - 970 downloads last month - 16 stars on GitHub - 2 maintainers
scrapy-kit 0.1.12
A library for scrapy tools, including but not limited to the usual pipelines, middlewares, etc.
9 versions - Latest release: 8 months ago - 71 downloads last month - 0 stars on GitHub - 1 maintainer
scrapy-selenium-mm 0.1.1
Scrapy with selenium
4 versions - Latest release: 8 months ago - 35 downloads last month - 887 stars on GitHub - 1 maintainer
gggspider 0.0.1
通用采集框架。
1 version - Latest release: 8 months ago - 17 downloads last month - 0 stars on GitHub - 1 maintainer
scrapy_huo_utilities 0.0.3
scrapy_huo_utilities是自用的一些scrapy工具代码合集。
3 versions - Latest release: 8 months ago - 31 downloads last month - 1 maintainer
ensembl-rest 0.3.4
An interface to the Ensembl REST APIs, biological data at your fingertips.
23 versions - Latest release: 8 months ago - 1 dependent repositories - 1.08 thousand downloads last month - 11 stars on GitHub - 1 maintainer
scrapy-spider-metadata 0.1.2
Utilities to extend Scrapy spiders with usable metadata.
3 versions - Latest release: 8 months ago - 1 dependent package - 4.74 thousand downloads last month - 4 stars on GitHub - 1 maintainer
angeltools 0.3.7
personal python small tools collection
25 versions - Latest release: 8 months ago - 1 dependent repositories - 196 downloads last month - 0 stars on GitHub - 1 maintainer
wbparser 0.2.0
WB parser (async)
4 versions - Latest release: 8 months ago - 44 downloads last month - 0 stars on GitHub - 1 maintainer
alx-tool 2.0.1 removed
A Python package for automating ALX School tasks.
2 versions - Latest release: 8 months ago - 216 downloads last month - 0 stars on GitHub - 1 maintainer
Top 1.5% on pypi.org
scrapyd 1.4.3
A service for running Scrapy spiders, with an HTTP API
11 versions - Latest release: 8 months ago - 4 dependent packages - 525 dependent repositories - 31.2 thousand downloads last month - 2,862 stars on GitHub - 5 maintainers
gerapy-item-pipeline 0.1.3
Item Pipeline Components for Scrapy & Gerapy
5 versions - Latest release: 9 months ago - 1 dependent repositories - 30 downloads last month - 1 stars on GitHub - 1 maintainer
yzwspider 0.1.6
A web spider for Chinese graduate student examination catalogue.
13 versions - Latest release: 9 months ago - 1 dependent repositories - 126 downloads last month - 64 stars on GitHub - 1 maintainer
scrapy-tls-client 0.0.5
tls client downloader middleware for scrapy, send request by tls client.
4 versions - Latest release: 9 months ago - 99 downloads last month - 10 stars on GitHub - 1 maintainer
finscraper 0.2.5
Web scraping API for Finnish websites
32 versions - Latest release: 9 months ago - 1 dependent repositories - 118 downloads last month - 8 stars on GitHub - 1 maintainer
neuralpit 3.5.6
NeuralPit SDK
48 versions - Latest release: 9 months ago - 79 downloads last month - 1 maintainer
scrapy-custom-proxy-pool 0.1.0
Scrapy proxy pool that allows custom proxy provider
1 version - Latest release: 10 months ago - 15 downloads last month - 1 maintainer
crawler-studio 1.5.7
crawler_studio
9 versions - Latest release: 10 months ago - 29 downloads last month - 1 maintainer
newsarticlesscraper 0.2.7
Scraping news articles
21 versions - Latest release: 11 months ago - 135 downloads last month - 1 maintainer
scraid 0.12
A package for advanced Scrapy functionality and utilities.
3 versions - Latest release: 11 months ago - 25 downloads last month - 1 stars on GitHub - 1 maintainer
Top 4.8% on pypi.org
gerapy 0.9.13
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, D...
47 versions - Latest release: 11 months ago - 34 dependent repositories - 1.01 thousand downloads last month - 3,206 stars on GitHub - 1 maintainer
Top 6.5% on pypi.org
crawlab-sdk 0.6.2
Python SDK for Crawlab
41 versions - Latest release: 11 months ago - 2 dependent packages - 4 dependent repositories - 450 downloads last month - 52 stars on GitHub - 1 maintainer
chady 0.2
A package for ML libraries
1 version - Latest release: 11 months ago - 16 downloads last month - 4 stars on GitHub - 1 maintainer
chadi 0.1 removed
A package for ML libraries
1 version - Latest release: 11 months ago
scrapy-db 0.0.5
Similar to [scrapy-redis](https://github.com/rmax/scrapy-redis), using the database as a queue, d...
5 versions - Latest release: 11 months ago - 49 downloads last month - 1 stars on GitHub - 1 maintainer
bocfx 0.8.1
Easy API to get foreign exchange rate from Bank of China.
17 versions - Latest release: 12 months ago - 1 dependent repositories - 3.06 thousand downloads last month - 52 stars on GitHub - 1 maintainer
scrapy-manipulate-request 0.0.2
An async scrapy request downloader middleware, support random request and response manipulation.
2 versions - Latest release: 12 months ago - 40 downloads last month - 10 stars on GitHub - 1 maintainer
scrapy-s3logstorage 0.1.1
Upload scrapy logs to S3
2 versions - Latest release: almost 1 year ago - 8 downloads last month - 1 maintainer
matchscraper 0.2.1.1 removed
CLI tool to get volleyball match statistics from the Web Competition by Data Project websites
3 versions - Latest release: about 1 year ago - 17 downloads last month - 2 stars on GitHub - 1 maintainer
datalad-crawler 1.0.2
DataLad extension package for crawling external web resources into an automated data distribution
27 versions - Latest release: about 1 year ago - 1 dependent repositories - 550 downloads last month - 5 stars on GitHub - 3 maintainers