Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

packagist.org "crawler" keyword

crwlr/robots-txt v1.1.1
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
6 versions - Latest release: over 1 year ago - 1 dependent package - 1 dependent repositories - 6.05 thousand downloads total - 8 stars on GitHub - 1 maintainer
Top 0.8% on packagist.org
spatie/robots-txt 2.2.0 💰
Determine if a page may be crawled from robots.txt and robots meta tags
20 versions - Latest release: about 1 month ago - 19 dependent packages - 813 dependent repositories - 8.05 million downloads total - 213 stars on GitHub - 1 maintainer
mcstreetguy/crawler
An advanced web-crawler written in PHP.
Latest release: about 1 month ago - 6 downloads total - 3 stars on GitHub - 1 maintainer
Top 10.0% on packagist.org
ropendev/curl 1.0.3
PHP POO cURL wrapper (PSR friendly) : light and simple.
4 versions - Latest release: over 5 years ago - 3 dependent packages - 8 dependent repositories - 6.4 thousand downloads total - 3 stars on GitHub - 1 maintainer
milu2003/flarum-ext-dxc
DXC采集 flraum接口
Latest release: about 1 month ago - 1 dependent repositories - 8 downloads total - 0 stars on GitHub - 1 maintainer
webcoast/versatile-crawler v1.1.1
A versatile and extendable crawler for TYPO3 CMS
4 versions - Latest release: about 3 years ago - 6.48 thousand downloads total - 7 stars on GitHub - 1 maintainer
crwlr/crawler v1.7.2
Web crawling and scraping library.
38 versions - Latest release: 2 months ago - 3 dependent packages - 1 dependent repositories - 3.36 thousand downloads total - 300 stars on GitHub - 1 maintainer
leowebguy/craft-crawler-detect 2.0.0
Craft plugin for detecting bots/crawlers/spiders
5 versions - Latest release: 2 months ago - 4.96 thousand downloads total - 1 stars on GitHub - 1 maintainer
ixnode/php-web-crawler 0.1.24
PHP Web Crawler - This PHP class allows you to crawl recursively a given html page (or a given ht...
25 versions - Latest release: 3 months ago - 25 downloads total - 2 stars on GitHub - 1 maintainer
shel/crawler 2.4.1 💰
Allows crawling of sitemaps and node-trees
11 versions - Latest release: over 1 year ago - 1.2 thousand downloads total - 7 stars on GitHub - 1 maintainer
pmaxs/crawler v1.1.1
HTTP crawler on non-blocking sockets
4 versions - Latest release: about 2 months ago - 74 downloads total - 0 stars on GitHub - 1 maintainer
Top 0.2% on packagist.org
jaybizzle/crawler-detect v1.2.118
CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
158 versions - Latest release: about 1 month ago - 170 dependent packages - 8,221 dependent repositories - 59.2 million downloads total - 1,918 stars on GitHub - 2 maintainers
Top 0.6% on packagist.org
jaeger/querylist V4.4.3 💰
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on p...
26 versions - Latest release: about 1 month ago - 96 dependent packages - 279 dependent repositories - 555 thousand downloads total - 2,606 stars on GitHub - 1 maintainer
piedweb/crawler 0.1.783
Web Crawler to check few SEO basics.
53 versions - Latest release: 3 months ago - 70 downloads total - 1 stars on GitHub - 1 maintainer
Top 7.0% on packagist.org
tomasnorre/crawler 11.0.7 💰
Crawler extension for TYPO3
70 versions - Latest release: over 1 year ago - 1 dependent package - 2 dependent repositories - 204 thousand downloads total - 55 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
aoepeople/crawler 11.0.7 💰
Crawler extension for TYPO3
70 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 280 thousand downloads total - 55 stars on GitHub - 2 maintainers
urguardian4ngel/flarum-ext-fb-crawler v0.1.0-beta.5
An extension that allows the Facebook Crawler to crawl protected pages.
1 version - Latest release: over 7 years ago - 104 downloads total - 0 stars on GitHub - 1 maintainer
nguyenanhung/template-crawler-package v1.0.1
Structure Repository Template for Crawler Package
2 versions - Latest release: 9 months ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
mehrabx/web-crawler v1.0
A web crawler package
1 version - Latest release: over 2 years ago - 1 downloads total - 3 stars on GitHub - 1 maintainer
delphiki/aprr 2.0.1
A simple crawler to get formated data from APRR website.
5 versions - Latest release: over 7 years ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
dz0x44/9pay-spider-telco v1.9
9Pay Crawler for Telco Promotion
10 versions - Latest release: about 4 years ago - 31 downloads total - 0 stars on GitHub - 1 maintainer
vangxh/beanbun
Beanbun 是用 PHP 编写的多进程网络爬虫框架,具有良好的开放性、高可扩展性
Latest release: about 1 month ago - 2 downloads total - 0 stars on GitHub - 1 maintainer
Top 2.2% on packagist.org
jyggen/curl v4.0.0
A simple and lightweight cURL library with support for asynchronous requests.
13 versions - Latest release: over 7 years ago - 9 dependent packages - 40 dependent repositories - 142 thousand downloads total - 72 stars on GitHub - 1 maintainer
lexxtor/easy-php-crawler
Simple yet flexible URL crawler.
Latest release: about 1 month ago - 8 downloads total - 1 stars on GitHub - 1 maintainer
pagemunch/pagemunch
A PHP wrapper for the PageMunch link unfurling API
Latest release: about 1 month ago - 4 dependent repositories - 31 downloads total - 6 stars on GitHub - 1 maintainer
Top 0.3% on packagist.org
spatie/crawler 8.2.0 💰
Crawl all internal links found on a website
104 versions - Latest release: 4 months ago - 45 dependent packages - 858 dependent repositories - 8.36 million downloads total - 2,472 stars on GitHub - 1 maintainer
heimrichhannot/crawler 6.0.0
Crawl all internal links found on a website
92 versions - Latest release: over 3 years ago - 1.41 thousand downloads total - 1 stars on GitHub - 1 maintainer
Top 0.7% on packagist.org
wa72/htmlpagedom v3.0.2
jQuery-inspired DOM manipulation extension for Symfony's Crawler
30 versions - Latest release: 6 months ago - 51 dependent packages - 125 dependent repositories - 2.67 million downloads total - 345 stars on GitHub - 1 maintainer
Top 1.1% on packagist.org
jaybizzle/laravel-crawler-detect v1.3.0
A Laravel package to detect web crawlers via the user agent
6 versions - Latest release: almost 2 years ago - 17 dependent packages - 183 dependent repositories - 1.57 million downloads total - 299 stars on GitHub - 1 maintainer
puzzle/crawler
web crawler
Latest release: about 1 month ago - 4 dependent repositories - 13 downloads total - 0 stars on GitHub - 1 maintainer
thingston/crawler 0.7.0
Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.
10 versions - Latest release: over 5 years ago - 9 downloads total - 2 stars on GitHub - 1 maintainer
eightwire/magento2-module-primer 0.1.14
A cache primer extension for Magento 2
15 versions - Latest release: over 4 years ago - 15.8 thousand downloads total - 19 stars on GitHub - 1 maintainer
hissezhaut/http-status-check 1.0.2
CLI tool to crawl a website and check HTTP status code
4 versions - Latest release: over 7 years ago - 9 downloads total - 0 stars on GitHub - 1 maintainer
dachcom-digital/dynamic-search-data-provider-crawler v3.0.1
A Spider Crawler Extension for Pimcore Dynamic Search.
16 versions - Latest release: 6 months ago - 12.8 thousand downloads total - 8 stars on GitHub - 1 maintainer
alrik11es/scrappy
A free proxy list based downloader for your crawling projects in PHP
Latest release: about 1 month ago - 2.15 thousand downloads total - 1 stars on GitHub - 1 maintainer
pyaesoneaung/2d-crawler v1.1.0
2D crawler for set.or.th
4 versions - Latest release: 12 months ago - 54 downloads total - 3 stars on GitHub - 1 maintainer
fireguard/correios
Utilities for interaction with the post office of Brazil
Latest release: about 1 month ago - 6 downloads total - 1 stars on GitHub - 1 maintainer
depa/depa-middleware-redirect 1.0.8
Laminas mezzio middleware for redirecting
7 versions - Latest release: 4 months ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
diggin/diggin-robotrules 0.10.0
parser/handler for Robots Exclusion Protocol (robots.txt and more)
5 versions - Latest release: over 8 years ago - 5 dependent repositories - 184 downloads total - 7 stars on GitHub - 1 maintainer
crawlbase/crawlbase v1.0.0
A lightweight, dependency free PHP class that acts as wrapper for Crawlbase API
1 version - Latest release: 12 months ago - 604 downloads total - 9 stars on GitHub - 1 maintainer
filchos/imago
A library for reading and transforming content in a chainable way
Latest release: about 1 month ago - 29 downloads total - 0 stars on GitHub - 1 maintainer
Top 1.6% on packagist.org
vdb/php-spider v0.7.2
A configurable and extensible PHP web spider
18 versions - Latest release: 6 months ago - 8 dependent packages - 23 dependent repositories - 144 thousand downloads total - 1,325 stars on GitHub - 1 maintainer
kynkai/search-engine-crawler v1.0
SearchEngineCrawler Dump the data from the search engines
1 version - Latest release: over 4 years ago - 13 downloads total - 2 stars on GitHub - 1 maintainer
fran-diaz/itseo 0.1.1
Includes methods that extract SEO info from url or HTML code
1 version - Latest release: over 10 years ago - 4 dependent repositories - 22 downloads total - 3 stars on GitHub - 1 maintainer
io-jaegers/valkyrier
Latest release: about 1 month ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
danielbehrendt/web-scraper v1.1.0
This package can be used to scrape elements from websites.
7 versions - Latest release: about 4 years ago - 3 downloads total - 1 stars on GitLab.com - 1 maintainer
tetreum/xupopter
Xupopter crawlea las principales páginas de compraventa de inmuebles de España como son Idealista...
Latest release: about 1 month ago - 45 downloads total - 11 stars on GitHub - 1 maintainer
proxycrawl/proxycrawl 3.0.0
A lightweight, dependency free PHP class that acts as wrapper for ProxyCrawl API
13 versions - Latest release: almost 3 years ago - 62.2 thousand downloads total - 19 stars on GitHub - 1 maintainer
codeblog/crawlerdetect v1.0.0
A CodeBlog CrawlerDetect class is easy and quick to use. Detecting bots/crawlers/spiders via the ...
1 version - Latest release: about 5 years ago - 54 downloads total - 9 stars on GitHub - 1 maintainer
baqend/spider 1.0.0
URL spider which crawls a page and all its subpages
1 version - Latest release: about 6 years ago - 437 downloads total - 6 stars on GitHub - 1 maintainer
mr4-lc/content-protected 0.0.1
Laravel content protected
1 version - Latest release: 7 months ago - 1 downloads total - 0 stars on GitHub - 1 maintainer
Top 7.8% on packagist.org
sleeping-owl/apist 1.3.7
Package to provide api-like access to foreign sites based on html parsing
11 versions - Latest release: about 9 years ago - 12 dependent repositories - 4.42 thousand downloads total - 311 stars on GitHub - 1 maintainer
sleimanx2/grawler 0.2.4
A guided html crawler with media meta extraction
15 versions - Latest release: over 4 years ago - 295 downloads total - 13 stars on GitHub - 1 maintainer
coooold/crawler v1.0.0
Most powerful, popular and production crawling/scraping package for PHP
1 version - Latest release: over 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
opensearchserver/opensearchserver 3.0.20
PHP library for OpenSearchServer: professionnal search engine, crawlers (web, file, database), RE...
21 versions - Latest release: about 6 years ago - 7 dependent repositories - 61.7 thousand downloads total - 47 stars on GitHub - 1 maintainer
sohib/crawler v1.0.1
Scan Http URLs from a url containing an HTML and get Json or Xml or Array
3 versions - Latest release: over 8 years ago - 24 downloads total - 0 stars on GitHub - 1 maintainer
teamtnt/crawler
Distributed Crawler Architecture
Latest release: about 1 month ago - 1 downloads total - 8 stars on GitHub - 1 maintainer
hissezhaut/laravel-link-checker v2.0.1
Check all links in a Laravel app
7 versions - Latest release: over 7 years ago - 11 downloads total - 0 stars on GitHub - 1 maintainer
nekhbet/crawlers-laravel-pulse
Laravel Pulse Card to show crawlers
Latest release: about 1 month ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
bdspider/crawler 0.5.1
crawler,spider for phpQuery。
8 versions - Latest release: over 5 years ago - 46 downloads total - 0 stars on GitHub - 1 maintainer
gaolujie1989/webmagic
A scalable web crawler framework for php.
Latest release: about 1 month ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
allotmentandy/socialmedialinkextractor
extracts social media links for an array of links eg. twitter, facebook etc.
Latest release: about 1 month ago - 87 downloads total - 0 stars on GitHub - 1 maintainer
kiwa/sitemap 2.0.0 💰
Integrates the Bit&Black Sitemap into a Kiwa website to create sitemap files seamlessly.
11 versions - Latest release: 4 months ago - 1 dependent package - 3.24 thousand downloads total - 2 maintainers
dayrev/extractor v1.2.2
Web Page Content Extractor
7 versions - Latest release: about 7 years ago - 11 downloads total - 2 stars on GitHub - 1 maintainer
fgsl/crawler 0.0.1
Fgsl Crawler
1 version - Latest release: over 6 years ago - 8 downloads total - 0 stars on GitHub - 1 maintainer
pabeda-bt/php-crawler
Crawl all internal links in a website
Latest release: about 1 month ago - 38 downloads total - 0 stars on GitHub - 1 maintainer
upstream-team/laravel-crawler-detect 0.1.10
Wrapper for the crawler detection package
6 versions - Latest release: 7 months ago - 3 downloads total - 1 stars on GitHub - 1 maintainer
filipsedivy/kontrola-tachometru 0.1.0 💰
🚘 MDČR - kontrola tachometru
1 version - Latest release: almost 5 years ago - 3 downloads total - 0 stars on GitHub - 1 maintainer
Top 3.2% on packagist.org
crossjoin/browscap v3.0.5
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as devic...
19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
edwinhuish/querylist V5.0.9
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on D...
22 versions - Latest release: about 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
nggiahao/crawler 0.1.0
Package này có nhiêm vụ thu thập dữ liệu từ các website khác sử dụng Guzzle, Phantomjs hay Puppet...
1 version - Latest release: over 3 years ago - 8 downloads total - 4 stars on GitHub - 1 maintainer
phantazm/robotdetector
Detect robots and crawlers.
Latest release: about 1 month ago - 4 dependent repositories - 1.87 thousand downloads total - 5 stars on GitHub - 1 maintainer
haikson/sitemap-generator
Sitemap crawler and generator class
Latest release: about 1 month ago - 0 downloads total - 68 stars on GitHub - 1 maintainer
blogdaren/webman-phpcreeper v1.0.4
PHPCreeper plugin for webman
5 versions - Latest release: about 1 month ago - 171 downloads total - 8 stars on GitHub - 1 maintainer
brunodebarros/http-request 1.0.2
A web crawler, built to be easily dropped into PHP applications, which behaves just like a regula...
3 versions - Latest release: about 9 years ago - 1 dependent package - 4 dependent repositories - 195 downloads total - 8 stars on GitHub - 1 maintainer
creode/craft-page-crawler 1.1.0
This will allow a page to be crawled for useful content during an indexing process.
10 versions - Latest release: about 1 year ago - 39 downloads total - 0 stars on GitHub - 1 maintainer
wangta69/crawler 1.0.0
crawler
1 version - Latest release: about 6 years ago - 14 downloads total - 1 stars on GitHub - 1 maintainer
visuellverstehen/t3fetch 1.5.0
Fetches a website (including all subpages), so the TYPO3 cache gets filled.
14 versions - Latest release: 3 months ago - 14.1 thousand downloads total - 7 stars on GitHub - 1 maintainer
proclnas/curl-rox 3.0.0
Just another curl wrapper for webCrawling purposes
3 versions - Latest release: about 5 years ago - 17 downloads total - 9 stars on GitHub - 1 maintainer
luyadev/luya-module-crawleradmin 1.0.0-beta8
Yii2 LUYA Crawler Admin-Module
4 versions - Latest release: almost 8 years ago - 562 downloads total - 1 maintainer
alexndreazevedo/crawler
Social PHP Crawler
Latest release: about 1 month ago - 4 dependent repositories - 6 downloads total - 0 stars on GitHub - 1 maintainer
vakata/downloader 1.0.2 💰
A PHP class for making a static copy (or indexing) a site
3 versions - Latest release: 2 months ago - 180 downloads total - 0 stars on GitHub - 1 maintainer
ognestraz/crawler
Crawler
Latest release: about 1 month ago - 6 downloads total - 1 maintainer
harm-smits/webcrawler-verifier 2.1.1
PHP library providing functionality to verify that user-agents are who they claim to be.
6 versions - Latest release: over 2 years ago - 1 downloads total - 0 stars on GitHub - 1 maintainer
lefuturiste/google-photos-album-crawler v1.1
Google Photos public album crawler/fetcher
2 versions - Latest release: almost 4 years ago - 62 downloads total - 8 stars on GitHub - 1 maintainer
pithyone/zhihu-crawler 3.5.0
轻量级知乎爬虫
17 versions - Latest release: over 5 years ago - 62 downloads total - 25 stars on GitHub - 1 maintainer
webysther/packagist-mirror 1.1.1 💰
Build mirror of packagist
7 versions - Latest release: over 4 years ago - 150 downloads total - 179 stars on GitHub - 1 maintainer
vanry/baike-spider
A simple spider crawling web pages from baike.baidu.com
Latest release: about 1 month ago - 6 downloads total - 2 stars on GitHub - 1 maintainer
radarlog/banks-spyder 1.0
Banks Spyder is free software to automatically generate bank statements in PDF format from your a...
1 version - Latest release: over 8 years ago - 2 downloads total - 0 stars on GitHub - 1 maintainer
agilekeys/point-picker
Crawler package for online shopping
Latest release: about 1 month ago - 800 downloads total - 0 stars on GitHub - 1 maintainer
snippetify/laravel-sniptbot 1.0.2
Sniptbot for laravel
3 versions - Latest release: almost 4 years ago - 4 downloads total - 1 stars on GitHub - 1 maintainer
mvrc/php-email-crawler 💰
PHP Website Email Crawler, for crawling websites for emails.
Latest release: about 1 month ago - 7 downloads total - 3 stars on GitHub - 1 maintainer
louvian/pururin-crawler 0.0.1
Pururin Crawler
1 version - Latest release: over 6 years ago - 12 downloads total - 4 stars on GitHub - 1 maintainer
grab/spider 0.2
PHP async scrapper used multi curl and reactphp, and proxy inspired by python grab
1 version - Latest release: about 7 years ago - 114 downloads total - 10 stars on GitHub - 1 maintainer
sigmie/crawler v0.1.18
Sigmie Crawler is a PHP package for crawling websites and exporting their HTML contents.
19 versions - Latest release: over 3 years ago - 25 downloads total - 0 stars on GitHub - 1 maintainer
nelexa/roach-php-bundle 1.1.1
Symfony bundle for roach-php/core
4 versions - Latest release: over 1 year ago - 564 downloads total - 9 stars on GitHub - 1 maintainer
laurentvw/scrapher v2.3.1
A web scraper for PHP to easily extract data from web pages
12 versions - Latest release: about 6 years ago - 1 dependent package - 3 dependent repositories - 2.51 thousand downloads total - 18 stars on GitHub - 1 maintainer
laurentvw/lavacrawler v2.3.1
A web scraper for PHP to easily extract data from web pages
12 versions - Latest release: about 6 years ago - 4 dependent repositories - 95 downloads total - 18 stars on GitHub - 1 maintainer
maxlen/webcrawler
Search engine crawlers
Latest release: about 1 month ago - 10 downloads total - 0 stars on GitHub - 1 maintainer
zanysoft/laravel-crawler-detect 1.0
A Laravel package to detect web crawlers via the user agent
1 version - Latest release: over 3 years ago - 357 downloads total - 1 stars on GitHub - 1 maintainer