Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
packagist.org "crawler" keyword
51degrees/fiftyone.devicedetection 4.4.2
Device detection engines for 51Degrees Pipeline API. Parse HTTP headers to detect hardware, opera...50 versions - Latest release: almost 2 years ago - 1 dependent package - 1 dependent repositories - 6.93 thousand downloads total - 0 stars on GitHub - 1 maintainer
68publishers/crawler-client-php v1.2.0
:spider_web: PHP Client for https://github.com/68publishers/crawler4 versions - Latest release: 10 months ago - 280 downloads total - 1 stars on GitHub - 1 maintainer
abelhalo/zolcrawler
Zol CrawlerLatest release: 30 days ago - 12 downloads total - 0 stars on GitHub - 1 maintainer
acanto/crawler 6.0.2
Crawl all internal links found on a website1 version - Latest release: about 3 years ago - 1 dependent package - 2 dependent repositories - 2.72 thousand downloads total - 0 stars on GitLab.com - 1 maintainer
adabra/curl v4.0.4
A simple and lightweight cURL library with support for asynchronous requests.17 versions - Latest release: over 3 years ago - 1 dependent package - 29 downloads total - 0 stars on GitHub - 1 maintainer
agilekeys/point-picker
Crawler package for online shoppingLatest release: 22 days ago - 800 downloads total - 0 stars on GitHub - 1 maintainer
airmole/tjustb-edusys v1.1.0
Tianjin college,USTB education system HTTP client15 versions - Latest release: 2 months ago - 24 downloads total - 1 stars on GitHub - 1 maintainer
akane0151/target-crawler 0.9.7
target crawler - A simple Laravel crawler package to read element of any webpage.6 versions - Latest release: about 1 year ago - 9 downloads total - 0 stars on GitHub - 1 maintainer
alc/sitemap-crawler v2.0.0
Crawl sitemap.xml recursively and return collected urls as array2 versions - Latest release: about 3 years ago - 2 dependent packages - 2 dependent repositories - 2.43 thousand downloads total - 5 stars on GitHub - 1 maintainer
aleksandrhorkavyi/yii2-crawler removed
Crawler based on Selenium webdriverLatest release: about 1 year ago - 14 downloads total - 0 stars on GitHub - 1 maintainer
aleksandrhorkavyi/yii2-crawler-dashboard removed
Crawler dashboardLatest release: about 1 year ago - 2 downloads total - 0 stars on GitHub - 1 maintainer
alexndreazevedo/crawler
Social PHP CrawlerLatest release: 21 days ago - 4 dependent repositories - 6 downloads total - 0 stars on GitHub - 1 maintainer
alikdex/yii2-crawler-detect 1.0
Yii2 extension for CrawlerDetect library1 version - Latest release: almost 8 years ago - 1 dependent repositories - 18 thousand downloads total - 13 stars on GitHub - 1 maintainer
allotmentandy/socialmedialinkextractor
extracts social media links for an array of links eg. twitter, facebook etc.Latest release: 20 days ago - 87 downloads total - 0 stars on GitHub - 1 maintainer
alrik11es/scrappy
A free proxy list based downloader for your crawling projects in PHPLatest release: 17 days ago - 2.14 thousand downloads total - 1 stars on GitHub - 1 maintainer
amir2b/instagram
Fetch data from InstagramLatest release: 13 days ago - 5 downloads total - 0 stars on GitHub - 1 maintainer
andrew-svirin/resource-crawler-bundle
Crawler for any sort of resources.Latest release: 27 days ago - 55 downloads total - 3 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
68 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 279 thousand downloads total - 53 stars on GitHub - 2 maintainers
aoepeople/crawler 11.0.7 π°
Crawler extension for TYPO368 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 279 thousand downloads total - 53 stars on GitHub - 2 maintainers
apachish/crawler 0.0.1
crawler in site1 version - Latest release: about 3 years ago - 44 downloads total - 0 stars on GitHub - 1 maintainer
appstract/laravel-crawl-render 0.1.4
Pre-render your site for web crawlers5 versions - Latest release: about 6 years ago - 36 downloads total - 11 stars on GitHub - 1 maintainer
artplus_f/browscap
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as devic...Latest release: 1 day ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
awanesia/scrap v2.0.1
A web scraper for PHP to easily extract data from web pages -> party of laurentvw1 version - Latest release: about 2 months ago - 3 downloads total - 0 stars on GitHub - 1 maintainer
awesam86/imlinscraper v1.0.1
Image And Link Scraper2 versions - Latest release: almost 6 years ago - 20 downloads total - 0 stars on GitHub - 1 maintainer
baqend/spider 1.0.0
URL spider which crawls a page and all its subpages1 version - Latest release: about 6 years ago - 434 downloads total - 6 stars on GitHub - 1 maintainer
baraja-core/webcrawler v1.3.3
Simple package to load list of urls and make sitemap.15 versions - Latest release: 10 months ago - 21 downloads total - 5 stars on GitHub - 1 maintainer
bdspider/crawler 0.5.1
crawler,spider for phpQueryγ8 versions - Latest release: over 5 years ago - 46 downloads total - 0 stars on GitHub - 1 maintainer
bilginpro/agency-iha v1.0.3
Crawler for IHA News Agency in Turkey.4 versions - Latest release: over 4 years ago - 471 downloads total - 5 stars on GitHub - 1 maintainer
bitandblack/sitemap 2.0.6 π°
Creates a sitemap.xml by parsing the whole website.12 versions - Latest release: 2 months ago - 1 dependent package - 1 dependent repositories - 1.88 thousand downloads total - 2 maintainers
blackfire/player v2.6.0
A powerful web crawler and web scraper with Blackfire support98 versions - Latest release: 7 days ago - 1 dependent repositories - 16 thousand downloads total - 484 stars on GitHub - 1 maintainer
blogdaren/phpcreeper v1.8.9
A new generation of multi-process async event-driven spider engine based on Workerman56 versions - Latest release: 5 days ago - 1 dependent package - 2 dependent repositories - 623 downloads total - 117 stars on GitHub - 1 maintainer
blogdaren/webman-phpcreeper v1.0.4
PHPCreeper plugin for webman5 versions - Latest release: 17 days ago - 165 downloads total - 8 stars on GitHub - 1 maintainer
blueways/bw-cache-uri v1.1.6
TYPO3 extension to parse remote content and save to tt_content element8 versions - Latest release: 7 months ago - 395 downloads total - 0 stars on GitHub - 1 maintainer
bogddan/laravel-crawler-detect removed
A Laravel package to detect web crawlers via the user agentLatest release: 12 months ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
bringyourownideas/laravel-sitemap 3.0.0 π°
A simple website crawler & sitemap generator without a headless browser for Laravel 5.8+19 versions - Latest release: over 1 year ago - 4 dependent repositories - 7.66 thousand downloads total - 9 stars on GitHub - 1 maintainer
brittainmedia/phpcrawl 0.10.1
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-...26 versions - Latest release: about 1 year ago - 2 dependent packages - 1 dependent repositories - 2.94 thousand downloads total - 9 stars on GitHub - 1 maintainer
brunodebarros/http-request 1.0.2
A web crawler, built to be easily dropped into PHP applications, which behaves just like a regula...3 versions - Latest release: about 9 years ago - 1 dependent package - 4 dependent repositories - 195 downloads total - 8 stars on GitHub - 1 maintainer
c2is/walker
A basic wrapper around Goutte to crawl a websiteLatest release: 12 days ago - 5 dependent repositories - 28 downloads total - 0 stars on GitHub - 1 maintainer
c4pone/yolo_crawler
An event based domain crawlerLatest release: 12 days ago - 4 dependent repositories - 39 downloads total - 2 stars on GitHub - 1 maintainer
cable8mm/water-melon v1.1.3
Water Melon is simple melon.com api sdk for php5 versions - Latest release: 2 months ago - 2.47 thousand downloads total - 16 stars on GitHub - 1 maintainer
cacing69/cquery v1.4.0 π°
PHP Scraper with language expression, could be used to scrape data from a website that uses javas...6 versions - Latest release: 8 months ago - 1 dependent package - 17 downloads total - 2 stars on GitHub - 1 maintainer
cdaguerre/robots-bundle v1.1.1
Add contextual robots meta headers7 versions - Latest release: over 6 years ago - 15.2 thousand downloads total - 0 stars on GitHub - 1 maintainer
chemaclass/stock-ticker
An API to gather the latest info from your stocksLatest release: 4 days ago - 1 downloads total - 3 stars on GitHub - 1 maintainer
cobaia/marsvin v1.0.0
Marsvin it's a framework to write crawler applications1 version - Latest release: over 11 years ago - 4 dependent repositories - 111 downloads total - 12 stars on GitHub - 1 maintainer
codeblog/crawlerdetect v1.0.0
A CodeBlog CrawlerDetect class is easy and quick to use. Detecting bots/crawlers/spiders via the ...1 version - Latest release: about 5 years ago - 54 downloads total - 9 stars on GitHub - 1 maintainer
Top 5.3% on packagist.org
13 versions - Latest release: over 2 years ago - 1 dependent package - 8 dependent repositories - 9.69 thousand downloads total - 253 stars on GitHub - 1 maintainer
codeguy/arachnid 2.2.1
A crawler to find all unique internal pages on a given website13 versions - Latest release: over 2 years ago - 1 dependent package - 8 dependent repositories - 9.69 thousand downloads total - 253 stars on GitHub - 1 maintainer
commencement.technology/botcheck 1.0.1
The package which identify the Bot or Crawler which help to identify the how many bots or crawler...2 versions - Latest release: over 1 year ago - 3 downloads total - 0 stars on GitHub - 1 maintainer
compolomus/rssreader 2.1
Small library for obtaining RSS feed with caching.3 versions - Latest release: about 2 months ago - 1 dependent repositories - 43 downloads total - 1 stars on GitHub - 1 maintainer
conradoojr/thief-links
Crawler For WebSites of SeriesLatest release: 13 days ago - 12 downloads total - 2 stars on GitHub - 1 maintainer
contextualcode/crawler v2.4.0
Flexible website crawler which stores the results in persistent storage21 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 526 downloads total - 0 stars on GitLab.com - 2 maintainers
coooold/crawler v1.0.0
Most powerful, popular and production crawling/scraping package for PHP1 version - Latest release: over 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
core23/lastfm-api 3.6.0 π°
Last.fm webservice client for php.20 versions - Latest release: 5 months ago - 2 dependent packages - 2.78 thousand downloads total - 13 stars on GitHub - 1 maintainer
core23/lastfm-bundle 1.3.0 π°
This bundle provides services for using the last.fm API with symfony.11 versions - Latest release: 5 months ago - 104 downloads total - 3 stars on GitHub - 1 maintainer
cotapreco/captcha
Our library for CAPTCHA solving (through third-party services, DeathByCaptcha)Latest release: 24 days ago - 144 downloads total - 2 stars on GitHub - 1 maintainer
crawlbase/crawlbase v1.0.0
A lightweight, dependency free PHP class that acts as wrapper for Crawlbase API1 version - Latest release: 12 months ago - 584 downloads total - 7 stars on GitHub - 1 maintainer
crawlzone/crawlzone 4.0.0
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web se...8 versions - Latest release: about 2 years ago - 5 thousand downloads total - 76 stars on GitHub - 1 maintainer
creode/craft-page-crawler 1.1.0
This will allow a page to be crawled for useful content during an indexing process.10 versions - Latest release: about 1 year ago - 39 downloads total - 0 stars on GitHub - 1 maintainer
crispy-computing-machine/supersimplecrawler 1.12
PHP 8 web crawler13 versions - Latest release: 8 months ago - 22 downloads total - 0 stars on GitHub - 1 maintainer
Top 3.2% on packagist.org
19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
crossjoin/browscap v3.0.5
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as devic...19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
crwlr/crawler v1.7.2
Web crawling and scraping library.38 versions - Latest release: about 2 months ago - 1 dependent package - 1 dependent repositories - 2.7 thousand downloads total - 295 stars on GitHub - 1 maintainer
crwlr/crawler-ext-browser v1.2.1
Extension for the crwlr/crawler package containing steps utilizing a headless browser.7 versions - Latest release: 2 months ago - 135 downloads total - 0 stars on GitHub - 1 maintainer
crwlr/robots-txt v1.1.1
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping6 versions - Latest release: over 1 year ago - 1 dependent package - 1 dependent repositories - 5.36 thousand downloads total - 8 stars on GitHub - 1 maintainer
cviniciussdias/google-crawler v2.0.3 π°
A simple Crawler for getting Google results20 versions - Latest release: almost 5 years ago - 132 downloads total - 32 stars on GitHub - 1 maintainer
czpechy/instagramprofilecrawler v1.0.1
Way to get latest news from instagram profile2 versions - Latest release: almost 2 years ago - 1 dependent repositories - 476 downloads total - 6 stars on GitHub - 1 maintainer
dachcom-digital/dynamic-search-data-provider-crawler v3.0.1
A Spider Crawler Extension for Pimcore Dynamic Search.16 versions - Latest release: 5 months ago - 12.1 thousand downloads total - 8 stars on GitHub - 1 maintainer
dachcom-digital/lucene-search v2.3.2
Pimcore 5.x Website Indexer (powered by Zend Search Lucene)26 versions - Latest release: over 4 years ago - 37.3 thousand downloads total - 26 stars on GitHub - 1 maintainer
danack/sitetool 0.1.2 π°
Event based site tool and tester.6 versions - Latest release: almost 6 years ago - 31 downloads total - 5 stars on GitHub - 1 maintainer
danielbehrendt/web-scraper v1.1.0
This package can be used to scrape elements from websites.7 versions - Latest release: about 4 years ago - 3 downloads total - 1 stars on GitLab.com - 1 maintainer
danielbrendel/rfcrawler removed
PHP Reddit Feed CrawlerLatest release: 6 months ago - 49 downloads total - 0 stars on GitHub - 1 maintainer
darvinstudio/darvin-crawler-bundle 6.0.4
This bundle provides console command that detects broken links on your website.5 versions - Latest release: about 3 years ago - 1.05 thousand downloads total - 0 stars on GitHub - 1 maintainer
dayrev/extractor v1.2.2
Web Page Content Extractor7 versions - Latest release: about 7 years ago - 11 downloads total - 2 stars on GitHub - 1 maintainer
delphiki/aprr 2.0.1
A simple crawler to get formated data from APRR website.5 versions - Latest release: about 7 years ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
depa/depa-middleware-redirect 1.0.8
Laminas mezzio middleware for redirecting7 versions - Latest release: 4 months ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
dezento/crawlify 1.0
Fast Concurrent Crawler1 version - Latest release: almost 3 years ago - 23 downloads total - 7 stars on GitHub - 1 maintainer
diggin/diggin-robotrules 0.10.0
parser/handler for Robots Exclusion Protocol (robots.txt and more)5 versions - Latest release: about 8 years ago - 5 dependent repositories - 184 downloads total - 7 stars on GitHub - 1 maintainer
dmoraschi/sitemap-common v1.1.0
Sitemap generator and crawler library2 versions - Latest release: over 7 years ago - 90 downloads total - 0 stars on GitHub - 1 maintainer
dobrosite/crawler 0.2.0
PHP component for recursive traversal of hierarchical data storages2 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 24 downloads total - 0 stars on GitHub - 1 maintainer
dobrosite/http-crawler 0.3.0
PHP component for recursive web sites traversal3 versions - Latest release: about 5 years ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
dr4g0nsr/sitemap-crawler 1.0 removed
Crawler for any type of site using robots.txt and sitemap.xml as the source of URL. Useful for ca...1 version - Latest release: over 1 year ago - 20 downloads total - 2 stars on GitHub - 1 maintainer
dusk-browser/dusk-browser
Laravel Dusk Browser is Laravel Dusk for browsing capabilities only.Latest release: 8 days ago - 48 downloads total - 0 stars on GitHub - 1 maintainer
Top 2.8% on packagist.org
41 versions - Latest release: 9 months ago - 2 dependent packages - 23 dependent repositories - 90.4 thousand downloads total - 351 stars on GitHub - 1 maintainer
duzun/hquery 3.1.0
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies...41 versions - Latest release: 9 months ago - 2 dependent packages - 23 dependent repositories - 90.4 thousand downloads total - 351 stars on GitHub - 1 maintainer
dz0x44/9pay-mi-spider v1.3
9Pay Micro Invest NAV Crawler5 versions - Latest release: over 3 years ago - 213 downloads total - 0 stars on GitHub - 1 maintainer
dz0x44/9pay-spider-telco v1.9
9Pay Crawler for Telco Promotion10 versions - Latest release: about 4 years ago - 31 downloads total - 0 stars on GitHub - 1 maintainer
eddieace/php-simple v1.0.4
HTML DOM parser for php 7.1+5 versions - Latest release: about 3 years ago - 1 dependent repositories - 34.4 thousand downloads total - 14 stars on GitHub - 1 maintainer
edwinhuish/querylist V5.0.9
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on D...22 versions - Latest release: about 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
eightwire/magento2-module-primer 0.1.14
A cache primer extension for Magento 215 versions - Latest release: over 4 years ago - 15.7 thousand downloads total - 19 stars on GitHub - 1 maintainer
elboletaire/crawler
A simple crawler (spider) writen in php just for funLatest release: 25 days ago - 1 dependent repositories - 6.7 thousand downloads total - 41 stars on GitHub - 1 maintainer
Top 9.8% on packagist.org
66 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 89.7 thousand downloads total - 40 stars on GitHub - 1 maintainer
eliashaeussler/cache-warmup 3.0.1 π°
Composer package to warm up website caches, based on a given XML sitemap66 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 89.7 thousand downloads total - 40 stars on GitHub - 1 maintainer
esyon/crawlerblock 1.0.5
This module blocks all kind of bots / crawlers / spiders.6 versions - Latest release: 11 months ago - 795 downloads total - 0 stars on GitHub - 1 maintainer
euskadi31/crawler-detect-bundle v1.0.1
A Symfony 2 crawler detect bundle2 versions - Latest release: about 8 years ago - 48 downloads total - 1 stars on GitHub - 1 maintainer
exfriend/uncurl 1.0.5
Unfurl links in PHP4 versions - Latest release: about 5 years ago - 1 dependent repositories - 196 downloads total - 1 stars on GitHub - 1 maintainer
famdirksen/crawler 3.3.1
Crawl all internal links found on a website - used for custom crawler42 versions - Latest release: over 6 years ago - 13 downloads total - 1 stars on GitHub - 1 maintainer
fgsl/crawler 0.0.1
Fgsl Crawler1 version - Latest release: over 6 years ago - 8 downloads total - 0 stars on GitHub - 1 maintainer
fi1a/crawler
PHP crawlerLatest release: 25 days ago - 0 downloads total - 1 stars on GitHub - 1 maintainer
filchos/imago
A library for reading and transforming content in a chainable wayLatest release: 19 days ago - 29 downloads total - 0 stars on GitHub - 1 maintainer
filipsedivy/kontrola-tachometru 0.1.0 π°
π MDΔR - kontrola tachometru1 version - Latest release: over 4 years ago - 3 downloads total - 0 stars on GitHub - 1 maintainer
fireguard/correios
Utilities for interaction with the post office of BrazilLatest release: 18 days ago - 6 downloads total - 1 stars on GitHub - 1 maintainer
fran-diaz/itseo 0.1.1
Includes methods that extract SEO info from url or HTML code1 version - Latest release: over 10 years ago - 4 dependent repositories - 22 downloads total - 3 stars on GitHub - 1 maintainer
friends-of-hyva/magento2-crawler-session 1.0.1
Prevent crawlers from creating a session2 versions - Latest release: about 1 month ago - 55 downloads total - 6 stars on GitHub - 2 maintainers
gaolujie1989/webmagic
A scalable web crawler framework for php.Latest release: 20 days ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
ghalambaz/googleplay-spider 1
Another GooglePlay Website Scraper based on goutte2 versions - Latest release: over 5 years ago - 231 downloads total - 6 stars on GitHub - 1 maintainer
Related Keywords
php
122
spider
62
scraper
47
scraping
28
laravel
23
crawling
23
parser
22
http
17
website
17
bot
16
symfony
15
seo
15
hacktoberfest
15
curl
15
dom
14
api
13
search
12
html
12
link
12
library
11
spatie
11
scrape
11
yii2
10
Sitemap
9
web
9
detect
9
module
8
crawl
8
composer
8
extension
7
typo3
6
robot
6
user-agent
6
webcrawler
6
luya
6
google
6
scrapper
6
yii
6
robots.txt
6
bundle
6
framework
6
browser
6
parse
5
crawlerdetect
5
php crawler detect
5
crawler detector
5
crawler detect
5
scraping-websites
5
bots
5
robots-txt
5
index
5
domcrawler
5
url
5
xml
5
news
5
extractor
5
asynchronous
4
phpQuery
4
content
4
luya-module
4
request
4
last.fm
4
webservice client
4
lastfm
4
webservice-client
4
symfony-bundle
4
web-crawler
4
QueryList
4
json
4
health
4
php-crawler
4
web crawler
3
wordpress
3
robots
3
webscraping
3
selector
3
multi
3
seo-optimization
3
stock
3
checker
3
indexer
3
rss
3
sitemap
3
helper
3
web-crawling
3
web-scraping
3
async
3
guzzle
3
crwlr
3
device
3
cache-warmup
3
instagram
3
facebook
3
text
3
data
3
matcher
3
internet
3
extract
3
Harvest
3
Match
3