Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "crawler" keyword

bigwing 1.4.3
bingwing project
16 versions - Latest release: almost 5 years ago - 1 dependent repositories - 37 downloads last month - 1 maintainer
scrapfly-sdk 0.8.16
Scrapfly SDK for Scrapfly
37 versions - Latest release: about 1 month ago - 2 dependent repositories - 13.1 thousand downloads last month - 18 stars on GitHub - 1 maintainer
page-parser 0.0.4
web crawler or spider parse page
4 versions - Latest release: about 5 years ago - 1 dependent repositories - 62 downloads last month - 45 stars on GitHub - 1 maintainer
python-darc 1.0.2 💰
Darkweb crawler & search engine.
49 versions - Latest release: 4 months ago - 1 dependent repositories - 428 downloads last month - 106 stars on GitHub - 1 maintainer
Top 7.4% on pypi.org
lightnovel-crawler 3.7.1 💰
An app to download novels from online sources and generate e-books.
180 versions - Latest release: 8 days ago - 1 dependent package - 1 dependent repositories - 3.45 thousand downloads last month - 1,164 stars on GitHub - 1 maintainer
pydude-pyto 0.22.0 💰
dude uncomplicated data extraction (For Pyto on iOS)
13 versions - Latest release: over 1 year ago - 1 dependent repositories - 68 downloads last month - 413 stars on GitHub - 1 maintainer
googlegroupexporter 1.0
GoogleGroup Exporter - Unlock your mailing list
2 versions - Latest release: about 8 years ago - 2 dependent repositories - 13 downloads last month - 13 stars on GitHub - 1 maintainer
origins 0.1.0a
Data introspection, indexer, and semantic analyzer
1 version - Latest release: 9 months ago - 4 dependent repositories - 39 stars on GitHub - 1 maintainer
pinscrape 3.0.5
Pinterest | a simple data scraper for pinterest
12 versions - Latest release: 5 days ago - 1 dependent repositories - 460 downloads last month - 61 stars on GitHub - 1 maintainer
Top 0.4% on pypi.org
scrapy 2.11.2
A high-level Web Crawling and Web Scraping framework
99 versions - Latest release: 5 days ago - 136 dependent packages - 2,753 dependent repositories - 1.48 million downloads last month - 51,148 stars on GitHub - 4 maintainers
wsearch 0.0.1
a search engine crwaler based on python, can be easily configed
2 versions - Latest release: over 5 years ago - 1 dependent repositories - 11 downloads last month - 1 stars on GitHub - 1 maintainer
spider.py 0.5
FTP and Web spiders and mirroring utilities
2 versions - Latest release: 9 months ago - 2 dependent repositories - 1 maintainer
pytse 1.6.2
A small web crawler for tsetmc.com
8 versions - Latest release: almost 3 years ago - 2 dependent repositories - 91 downloads last month - 41 stars on GitHub - 1 maintainer
wutong-search 0.1.4
a search engine crwaler based on python, can be easily configed
3 versions - Latest release: over 5 years ago - 1 dependent repositories - 18 downloads last month - 1 stars on GitHub - 1 maintainer
Top 1.8% on pypi.org
gallery-dl 1.26.9 💰
Command-line program to download image galleries and collections from several image hosting sites
143 versions - Latest release: about 2 months ago - 6 dependent packages - 31 dependent repositories - 38.2 thousand downloads last month - 9,174 stars on GitHub - 1 maintainer
filecrawler 0.1.8
File Crawler index files and search hard-coded credentials.
8 versions - Latest release: about 1 year ago - 42 downloads last month - 22 stars on GitHub - 1 maintainer
nasty 0.2.7
NASTY Advanced Search Tweet Yielder
9 versions - Latest release: over 3 years ago - 3 dependent repositories - 201 downloads last month - 49 stars on GitHub - 1 maintainer
spidy-web-crawler 1.6.5
Spidy is the simple, easy to use command line web crawler.
12 versions - Latest release: over 6 years ago - 1 dependent repositories - 327 downloads last month - 323 stars on GitHub - 1 maintainer
firecrawl-py 0.0.8
Python SDK for Firecrawl API
8 versions - Latest release: 10 days ago - 1 dependent package - 8.58 thousand downloads last month - 2,987 stars on GitHub - 1 maintainer
seleniumlogin 0.0.4
Login some website using selenium.
2 versions - Latest release: almost 4 years ago - 1 dependent repositories - 483 downloads last month - 39 stars on GitHub - 1 maintainer
digslash 1.0.0
A site mapping and enumeration tool for Web applications analysis
10 versions - Latest release: over 1 year ago - 1 dependent repositories - 4 downloads last month - 1 stars on GitHub - 1 maintainer
cyberplant-scrapy 1.2.0.dev2
A high-level Web Crawling and Web Scraping framework
1 version - Latest release: almost 8 years ago - 1 dependent repositories - 24 downloads last month - 51,148 stars on GitHub - 1 maintainer
aminer-scrapy 2.11.1
A high-level Web Crawling and Web Scraping framework
3 versions - Latest release: 4 months ago - 42 downloads last month - 51,148 stars on GitHub - 1 maintainer
Top 9.5% on pypi.org
pylab-utils 0.5
python utility tools
2 versions - Latest release: over 2 years ago - 1 dependent repositories - 143 downloads last month - 51,148 stars on GitHub - 1 maintainer
scrapy-hls 0.1
scrapy integration for m3u8 files
1 version - Latest release: about 3 years ago - 1 dependent repositories - 20 downloads last month - 51,097 stars on GitHub - 1 maintainer
geckordp 0.5.0
A client implementation of Firefox DevTools over remote debug protocol
23 versions - Latest release: 3 days ago - 2 dependent packages - 1 dependent repositories - 1.8 thousand downloads last month - 13 stars on GitHub - 1 maintainer
botasaurus 4.0.20 💰
The All in One Web Scraping Framework
76 versions - Latest release: 3 days ago - 1 dependent package - 1 dependent repositories - 3.98 thousand downloads last month - 962 stars on GitHub - 1 maintainer
simpyder 0.1.12 💰
Distributed multithreading universal crawler
30 versions - Latest release: over 3 years ago - 1 dependent repositories - 131 downloads last month - 76 stars on GitHub - 1 maintainer
cocrawler 0.1.14
A modern web crawler framework for Python
8 versions - Latest release: about 3 years ago - 1 dependent repositories - 41 downloads last month - 176 stars on GitHub - 1 maintainer
aio-scrapy 2.1.0
A high-level Web Crawling and Web Scraping framework based on Asyncio
39 versions - Latest release: about 1 month ago - 1 dependent repositories - 304 downloads last month - 51 stars on GitHub - 1 maintainer
ayugespidertools 3.9.7
scrapy 扩展库：用于扩展 Scrapy 功能来解放双手。
88 versions - Latest release: 2 months ago - 1 dependent repositories - 224 downloads last month - 58 stars on GitHub - 1 maintainer
dalsa 0.0.8
This package does Aspect Level Sentiment Analysis (ALSA) on user comments about a given product
8 versions - Latest release: about 1 month ago - 870 downloads last month - 1 maintainer
gplaycrawler 0.2.1
Crawl the Google PlayStore
2 versions - Latest release: almost 3 years ago - 1 dependent repositories - 34 downloads last month - 0 stars on GitLab.com - 1 maintainer
lwd-utils 1.0.0
rename-version of PaperCrawlerUtil
1 version - Latest release: about 1 year ago - 16 downloads last month - 14 stars on GitHub - 1 maintainer
spider-rs 0.0.34
The fastest web crawler and indexer.
33 versions - Latest release: about 1 month ago - 411 downloads last month - 14 stars on GitHub - 1 maintainer
papercrawlerutil 0.1.39
a collection of utils
135 versions - Latest release: 3 months ago - 1 dependent repositories - 429 downloads last month - 14 stars on GitHub - 1 maintainer
twitton 1.0.0
A simple Data Scraping library for Twitter API
1 version - Latest release: about 6 years ago - 1 dependent repositories - 17 downloads last month - 0 stars on GitHub - 1 maintainer
doc_crawler 1.2
Explore a website recursively and download all the wanted documents (PDF, ODT…)
3 versions - Latest release: about 6 years ago - 30 downloads last month - 20 stars on GitHub - 1 maintainer
crawler-telefone-pt-br-version 1.0
Crawler para pegar telefones em sites. Totalmente amador e apenas de TESTE.
1 version - Latest release: 10 months ago - 10 downloads last month - 1 maintainer
Top 1.6% on pypi.org
scrapy-redis 0.7.3
Redis-based components for Scrapy.
18 versions - Latest release: almost 2 years ago - 6 dependent packages - 392 dependent repositories - 7.15 thousand downloads last month - 5,448 stars on GitHub - 3 maintainers
ig-scraper 0.1.0 💰
Instagram hashtag scraper
8 versions - Latest release: over 4 years ago - 1 dependent repositories - 68 downloads last month - 21 stars on GitHub - 1 maintainer
Top 6.1% on pypi.org
dirhunt 1.0.0
Find web directories without bruteforce
17 versions - Latest release: 9 months ago - 3 dependent repositories - 1.99 thousand downloads last month - 1,674 stars on GitHub - 1 maintainer
hsc 1.2.5
Hackerrank Solution Crawler
11 versions - Latest release: over 2 years ago - 1 dependent repositories - 107 downloads last month - 18 stars on GitHub - 3 maintainers
Top 8.4% on pypi.org
xsrfprobe 2.3.1
The Prime Cross Site Request Forgery (CSRF) Audit & Exploitation Toolkit
5 versions - Latest release: over 4 years ago - 1 dependent repositories - 709 downloads last month - 915 stars on GitHub - 1 maintainer
github-email-scraper 1.0.0
Scrape GitHub to get user emails
1 version - Latest release: over 2 years ago - 1 dependent repositories - 26 downloads last month - 1 stars on GitHub - 1 maintainer
Top 7.9% on pypi.org
scylla 1.1.7
Intelligent proxy pool for Humans™
30 versions - Latest release: over 4 years ago - 13 dependent repositories - 286 downloads last month - 3,894 stars on GitHub - 1 maintainer
gd-scylla 10.0.0
Intelligent proxy pool for Humans™
1 version - Latest release: about 4 years ago - 1 dependent repositories - 12 downloads last month - 3,894 stars on GitHub - 1 maintainer
scrab 0.0.6
Fast and easy to use scraper for the content-centered web pages, e.g. blog posts, news, etc.
6 versions - Latest release: almost 4 years ago - 1 dependent repositories - 89 downloads last month - 0 stars on GitHub - 1 maintainer
Top 6.9% on pypi.org
crawlerdetect 0.1.7
CrawlerDetect is a Python class for detecting bots/crawlers/spiders via the user agent.
8 versions - Latest release: 9 months ago - 2 dependent packages - 5 dependent repositories - 20.4 thousand downloads last month - 38 stars on GitHub - 1 maintainer
inspire-crawler 3.0.4
Crawler integration with INSPIRE-HEP.
33 versions - Latest release: almost 5 years ago - 6 dependent repositories - 157 downloads last month - 4 stars on GitHub - 2 maintainers
uniparser 3.0.2
Provide a universal solution for crawler platforms. Read more: https://github.com/ClericPy/unipar...
74 versions - Latest release: almost 2 years ago - 1 dependent package - 2 dependent repositories - 575 downloads last month - 8 stars on GitHub - 1 maintainer
Top 8.9% on pypi.org
dosage 2.9
a comic strip downloader and archiver
28 versions - Latest release: 9 months ago - 5 dependent repositories - 99 downloads last month - 121 stars on GitHub - 2 maintainers
livingbio-newspaper 1514473007.65
Simplified python article discovery & extraction.
2 versions - Latest release: over 6 years ago - 1 dependent repositories - 19 downloads last month - 13,780 stars on GitHub - 1 maintainer
newspaper3k-nop0x 0.2.8
Simplified python article discovery & extraction.
1 version - Latest release: over 4 years ago - 1 dependent repositories - 13 downloads last month - 13,780 stars on GitHub - 1 maintainer
enlivensystems-newspaper 0.3.11
Simplified python article discovery & extraction.
12 versions - Latest release: about 2 years ago - 1 dependent repositories - 160 downloads last month - 13,780 stars on GitHub - 1 maintainer
elon-newspaper1 0.3.1
ffffffffffffffffff
1 version - Latest release: 3 months ago - 20 downloads last month - 13,780 stars on GitHub - 1 maintainer
Top 2.3% on pypi.org
newspaper 0.0.9
Simplified python article discovery & extraction.
22 versions - Latest release: over 9 years ago - 264 dependent repositories - 8.79 thousand downloads last month - 13,780 stars on GitHub - 1 maintainer
newspaper3k-no-image 0.2.9
Simplified python article discovery & extraction. (With image processing removed)
1 version - Latest release: over 5 years ago - 1 dependent repositories - 125 downloads last month - 13,780 stars on GitHub - 1 maintainer
newspaper4k 0.9.3
Simplified python article discovery & extraction.
5 versions - Latest release: 2 months ago - 9.48 thousand downloads last month - 306 stars on GitHub - 1 maintainer
feedsearch-crawler 1.0.3
Search sites for RSS, Atom, and JSON feeds
37 versions - Latest release: almost 2 years ago - 1 dependent repositories - 364 downloads last month - 56 stars on GitHub - 1 maintainer
Top 9.8% on pypi.org
scrapple 0.3.0
A framework for creating web content extractors
10 versions - Latest release: over 7 years ago - 2 dependent repositories - 246 downloads last month - 494 stars on GitHub - 1 maintainer
crawlee 0.0.3
Crawlee for Python
16 versions - Latest release: 4 days ago - 856 downloads last month - 1 maintainer
Top 0.7% on pypi.org
newspaper3k 0.2.8
Simplified python article discovery & extraction.
18 versions - Latest release: over 5 years ago - 75 dependent packages - 1,068 dependent repositories - 631 thousand downloads last month - 13,727 stars on GitHub - 1 maintainer
guang-toolkit 3.0.13
python toolkit
4 versions - Latest release: about 4 years ago - 1 dependent repositories - 58 downloads last month - 6 stars on GitHub - 1 maintainer
crawlist 0.1.0
A universal solution for web crawling lists
10 versions - Latest release: 4 days ago - 613 downloads last month - 23 stars on GitHub - 1 maintainer
zcbot-scrapy-redis 0.7.3.2110.2
Redis-based components for Scrapy 2.11.0+.
3 versions - Latest release: 6 months ago - 24 downloads last month - 5,448 stars on GitHub - 1 maintainer
Top 2.7% on pypi.org
icrawler 0.6.8
A multi-thread crawler framework with many builtin image crawlers provided.
43 versions - Latest release: 4 days ago - 2 dependent packages - 86 dependent repositories - 99.9 thousand downloads last month - 813 stars on GitHub - 2 maintainers
frontoxy 1.0.3
Distributed URLs frontier for Scrapy with RabbitMQ
4 versions - Latest release: over 7 years ago - 1 dependent repositories - 23 downloads last month - 7 stars on GitHub - 1 maintainer
Top 8.2% on pypi.org
lulu 0.5.3
A simple and clean video/music/image downloader that supports many websites 👾
32 versions - Latest release: about 6 years ago - 4 dependent repositories - 401 downloads last month - 817 stars on GitHub - 1 maintainer
py404 0.1.4
py404 is a CLI tool for finding deadlinks on a website.
5 versions - Latest release: about 1 month ago - 41 downloads last month - 1 stars on GitHub - 1 maintainer
facebooker 1.3.1
An un official facebook api
16 versions - Latest release: about 3 years ago - 1 dependent repositories - 115 downloads last month - 36 stars on GitHub - 1 maintainer
Top 1.7% on pypi.org
trafilatura 1.9.0 💰
Python package and command-line tool designed to gather text on the Web, includes all necessary d...
44 versions - Latest release: 17 days ago - 71 dependent packages - 63 dependent repositories - 476 thousand downloads last month - 2,688 stars on GitHub - 1 maintainer
spider-client 0.0.20
Python SDK for Spider Cloud API
12 versions - Latest release: 5 days ago - 1 dependent package - 4.94 thousand downloads last month - 0 stars on GitHub - 1 maintainer
Top 3.8% on pypi.org
random-user-agent 1.0.1
A package to get random user agents based filters provided by user
5 versions - Latest release: over 5 years ago - 24 dependent packages - 94 dependent repositories - 161 thousand downloads last month - 92 stars on GitHub - 1 maintainer
Top 9.3% on pypi.org
page_finder 0.1.9
Find which links on a web page are pagination links
10 versions - Latest release: over 7 years ago - 14 dependent repositories - 490 downloads last month - 30 stars on GitHub - 2 maintainers
page_clustering 0.0.1
Online k-means clustering of web pages
1 version - Latest release: almost 8 years ago - 13 dependent repositories - 239 downloads last month - 35 stars on GitHub - 2 maintainers
taiwanlottery 1.5.1
Taiwan Lottery Crawler 台灣彩券爬蟲
12 versions - Latest release: 5 months ago - 166 downloads last month - 19 stars on GitHub - 1 maintainer
kameleo.local-api-client 3.2.0
This Python package provides convenient access to the Local API REST interface of the Kameleo Cli...
12 versions - Latest release: 6 days ago - 1 dependent repositories - 517 downloads last month - 47 stars on GitHub - 1 maintainer
nlpia2-wikipedia 1.5.16
Updated version of `wikipedia` package because original repo has been abandoned since 2014.
14 versions - Latest release: 2 months ago - 2 dependent packages - 1 dependent repositories - 1.92 thousand downloads last month - 0 stars on GitLab.com - 1 maintainer
Top 3.4% on pypi.org
pywebcopy 7.0.2
Python library to clone/archive pages or sites from the Internet.
17 versions - Latest release: about 2 years ago - 2 dependent packages - 40 dependent repositories - 7.66 thousand downloads last month - 491 stars on GitHub - 1 maintainer
yandex-images-crawler 1.1.0
Crawler/parser for Yandex Images
2 versions - Latest release: 6 months ago - 45 downloads last month - 0 stars on GitHub - 1 maintainer
livepopulartimes 1.3
LivePopularTimes: A Google Maps scraper
4 versions - Latest release: about 3 years ago - 1 dependent repositories - 776 downloads last month - 26 stars on GitHub - 1 maintainer
jmcomic 2.5.11
Python API For JMComic (禁漫天堂)
112 versions - Latest release: 20 days ago - 1 dependent package - 15.2 thousand downloads last month - 435 stars on GitHub - 1 maintainer
Top 2.1% on pypi.org
google-play-scraper 1.2.6
Google-Play-Scraper provides APIs to easily crawl the Google Play Store for Python without any ex...
38 versions - Latest release: 4 months ago - 4 dependent packages - 118 dependent repositories - 137 thousand downloads last month - 682 stars on GitHub - 1 maintainer
epubcrawler 2023.7.9.2
EpubCrawler，用于抓取网页内容并制作 EPUB 的小工具
24 versions - Latest release: 10 months ago - 1 dependent package - 1 dependent repositories - 99 downloads last month - 23 stars on GitHub - 1 maintainer
deadlinks 0.3.5 💰
Health checks for your documentation links.
10 versions - Latest release: 5 months ago - 1 dependent package - 2 dependent repositories - 60 downloads last month - 93 stars on GitHub - 1 maintainer
Top 5.4% on pypi.org
courlan 1.1.0
Clean, filter and sample URLs to optimize data collection – includes spam, content type and langu...
27 versions - Latest release: 19 days ago - 8 dependent packages - 31 dependent repositories - 475 thousand downloads last month - 65 stars on GitHub - 1 maintainer
spotifyscraper 1.0.5
Spotify Web Player Scraper using python, scrape and download song and cover from Spotify.
6 versions - Latest release: about 4 years ago - 1 dependent repositories - 104 downloads last month - 124 stars on GitHub - 1 maintainer
Top 7.2% on pypi.org
bilix 0.18.8
⚡️Lightning-fast asynchronous download tool for bilibili and more
85 versions - Latest release: 4 months ago - 2 dependent packages - 1 dependent repositories - 1.33 thousand downloads last month - 1,501 stars on GitHub - 1 maintainer
Top 8.8% on pypi.org
baiduspider 1.0.0
BaiduSpider，一个爬取百度的利器
34 versions - Latest release: almost 3 years ago - 1 dependent package - 1 dependent repositories - 446 downloads last month - 934 stars on GitHub - 1 maintainer
Top 3.8% on pypi.org
autoscraper 1.1.14 💰
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
16 versions - Latest release: almost 2 years ago - 2 dependent packages - 36 dependent repositories - 3.5 thousand downloads last month - 5,947 stars on GitHub - 1 maintainer
concurrentfloodscraper 1.0.1
A concurrent flood web scraper.
2 versions - Latest release: over 7 years ago - 1 dependent repositories - 43 downloads last month - 0 stars on GitHub - 1 maintainer
Top 7.6% on pypi.org
comiccrawler 2024.4.11
An image crawler, including multiple modules and GUI.
181 versions - Latest release: about 1 month ago - 2 dependent repositories - 2.07 thousand downloads last month - 258 stars on GitHub - 1 maintainer
Top 9.1% on pypi.org
proxybroker2 2.0.0a4 💰
The New (auto rotate) Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS.
1 version - Latest release: over 1 year ago - 1 dependent repositories - 277 downloads last month - 663 stars on GitHub - 1 maintainer
crawl4us 0.1.3 💰
A Python web crawler looking wildly for tables
4 versions - Latest release: about 6 years ago - 51 downloads last month - 1 stars on GitHub - 2 maintainers
insta-feed-checker 1.0.0
The fastest and simplest way to check someone's instagram feeds
1 version - Latest release: 6 months ago - 16 downloads last month - 1 stars on GitHub - 1 maintainer
Top 4.6% on pypi.org
frontera 0.8.1
A scalable frontier for web crawlers
22 versions - Latest release: about 5 years ago - 2 dependent packages - 13 dependent repositories - 902 downloads last month - 1,278 stars on GitHub - 3 maintainers
pyaair 0.0.0
American Airlines scraper in Python
1 version - Latest release: 22 days ago - 1 maintainer
pyzill 0.0.2
Zillow scraper in Python
3 versions - Latest release: 13 days ago - 344 downloads last month - 8 stars on GitHub - 1 maintainer
secretscraper 1.3.9
SecretScraper is a web scraper tool that can scrape the content through target websites and extra...
19 versions - Latest release: 19 days ago - 1.51 thousand downloads last month - 18 stars on GitHub - 1 maintainer
Related Keywords
python 203 scraper 125 spider 110 scraping 87 scrapy 67 crawling 50 python3 49 web 45 web-scraping 32 api 27 asyncio 24 requests 24 selenium 24 proxy 22 data 22 downloader 21 webscraping 21 parser 20 html 19 framework 19 news 18 crawl 16 hacktoberfest 15 twitter 15 distributed 14 aiohttp 14 web-crawler 14 redis 13 bot 13 search 12 cli 12 async 12 crawler-python 11 automation 11 beautifulsoup 10 web-scraping-python 10 proxies 10 scrape 10 http 10 beautifulsoup4 9 pypi 9 bs4 9 proxypool 8 webscraper 8 news-aggregator 8 nlp 8 website 8 scrapper 7 search engine 7 machine-learning 7 security 7 webcrawler 7 mongodb 7 爬虫 7 osint 7 lxml 7 google 7 data-science 6 web-scraper 6 crawling-framework 6 extractor 6 pdf 6 scrapy-redis 6 webdriver 6 webcrawling 6 pip 6 search-engine 6 sitemap 5 web-spider 5 elasticsearch 5 ptt 5 instagram 5 wiki 5 selenium-python 5 zhihu 5 news-crawler 5 document 5 video 5 frontier 5 scraping-framework 5 web-crawling 5 proxy-server 5 proxy-list 5 image 5 bilibili 5 data-mining 5 download 5 asynchronous 5 sync 4 xpath 4 novel 4 webui 4 csv 4 scraping-python 4 tools 4 lib 4 douyin 4 fastapi 4 文档 4 python-3 4