Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
packagist.org "crawler" keyword
Top 0.2% on packagist.org
158 versions - Latest release: 20 days ago - 148 dependent packages - 8,221 dependent repositories - 57.9 million downloads total - 1,911 stars on GitHub - 2 maintainers
jaybizzle/crawler-detect v1.2.118
CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent158 versions - Latest release: 20 days ago - 148 dependent packages - 8,221 dependent repositories - 57.9 million downloads total - 1,911 stars on GitHub - 2 maintainers
Top 0.3% on packagist.org
104 versions - Latest release: 3 months ago - 44 dependent packages - 858 dependent repositories - 8.23 million downloads total - 2,471 stars on GitHub - 1 maintainer
spatie/crawler 8.2.0 π°
Crawl all internal links found on a website104 versions - Latest release: 3 months ago - 44 dependent packages - 858 dependent repositories - 8.23 million downloads total - 2,471 stars on GitHub - 1 maintainer
blackfire/player v2.6.0
A powerful web crawler and web scraper with Blackfire support98 versions - Latest release: 7 days ago - 1 dependent repositories - 16 thousand downloads total - 484 stars on GitHub - 1 maintainer
heimrichhannot/crawler 6.0.0
Crawl all internal links found on a website92 versions - Latest release: over 3 years ago - 1.31 thousand downloads total - 1 stars on GitHub - 1 maintainer
Top 8.7% on packagist.org
84 versions - Latest release: about 2 years ago - 10 dependent repositories - 4.31 thousand downloads total - 53 stars on GitHub - 1 maintainer
pablouser1/tikscraper v1.2.7
Get data from TikTok API84 versions - Latest release: about 2 years ago - 10 dependent repositories - 4.31 thousand downloads total - 53 stars on GitHub - 1 maintainer
koffleart/crawler 4.7.2 π°
Crawl all internal links found on a website79 versions - Latest release: about 4 years ago - 2 downloads total - 0 stars on GitHub - 1 maintainer
Top 7.0% on packagist.org
68 versions - Latest release: over 1 year ago - 1 dependent package - 2 dependent repositories - 197 thousand downloads total - 53 stars on GitHub - 1 maintainer
tomasnorre/crawler 11.0.7 π°
Crawler extension for TYPO368 versions - Latest release: over 1 year ago - 1 dependent package - 2 dependent repositories - 197 thousand downloads total - 53 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
68 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 279 thousand downloads total - 53 stars on GitHub - 2 maintainers
aoepeople/crawler 11.0.7 π°
Crawler extension for TYPO368 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 279 thousand downloads total - 53 stars on GitHub - 2 maintainers
Top 9.8% on packagist.org
66 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 89.7 thousand downloads total - 40 stars on GitHub - 1 maintainer
eliashaeussler/cache-warmup 3.0.1 π°
Composer package to warm up website caches, based on a given XML sitemap66 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 89.7 thousand downloads total - 40 stars on GitHub - 1 maintainer
blogdaren/phpcreeper v1.8.9
A new generation of multi-process async event-driven spider engine based on Workerman56 versions - Latest release: 5 days ago - 1 dependent package - 2 dependent repositories - 623 downloads total - 117 stars on GitHub - 1 maintainer
piedweb/crawler 0.1.783
Web Crawler to check few SEO basics.53 versions - Latest release: 3 months ago - 70 downloads total - 1 stars on GitHub - 1 maintainer
vormkracht10/laravel-seo-scanner v3.11.1 π°
Laravel package to check if you used important SEO tags in your website.51 versions - Latest release: about 1 month ago - 7.41 thousand downloads total - 168 stars on GitHub - 1 maintainer
kaishiyoku/hera-rss-crawler 6.0.1 π°
Modern library to handle RSS/Atom feeds50 versions - Latest release: about 1 month ago - 2 dependent repositories - 1.81 thousand downloads total - 1 stars on GitHub - 1 maintainer
51degrees/fiftyone.devicedetection 4.4.2
Device detection engines for 51Degrees Pipeline API. Parse HTTP headers to detect hardware, opera...50 versions - Latest release: almost 2 years ago - 1 dependent package - 1 dependent repositories - 6.93 thousand downloads total - 0 stars on GitHub - 1 maintainer
nguyenanhung/my-crawler v1.1.3
Crawler helper, library - Basic, Simple and Lightweight45 versions - Latest release: about 1 month ago - 2 dependent packages - 1 dependent repositories - 725 downloads total - 1 stars on GitHub - 1 maintainer
famdirksen/crawler 3.3.1
Crawl all internal links found on a website - used for custom crawler42 versions - Latest release: over 6 years ago - 13 downloads total - 1 stars on GitHub - 1 maintainer
Top 2.8% on packagist.org
41 versions - Latest release: 9 months ago - 2 dependent packages - 23 dependent repositories - 90.4 thousand downloads total - 351 stars on GitHub - 1 maintainer
duzun/hquery 3.1.0
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies...41 versions - Latest release: 9 months ago - 2 dependent packages - 23 dependent repositories - 90.4 thousand downloads total - 351 stars on GitHub - 1 maintainer
luyadev/luya-module-crawler 3.7.2
An full search page crawler to enable complex and customized searching abilities.40 versions - Latest release: 7 months ago - 1 dependent repositories - 32.7 thousand downloads total - 7 stars on GitHub - 1 maintainer
zephir/luya-module-crawler 3.7.2
An full search page crawler to enable complex and customized searching abilities.40 versions - Latest release: 7 months ago - 182 downloads total - 7 stars on GitHub - 1 maintainer
nws/ultra-parser 2.2.3
Laravel package for easy scraping web pages38 versions - Latest release: about 5 years ago - 44 downloads total - 1 maintainer
crwlr/crawler v1.7.2
Web crawling and scraping library.38 versions - Latest release: about 2 months ago - 1 dependent package - 1 dependent repositories - 2.7 thousand downloads total - 295 stars on GitHub - 1 maintainer
mediashare/spider 0.4.7
Spider is a php library for crawling website that allows you to scrape informations & automated a...33 versions - Latest release: over 2 years ago - 112 downloads total - 14 stars on GitHub - 1 maintainer
slote/spider 0.4.7
Spider is a php library for crawling website that allows you to scrape informations & automated a...33 versions - Latest release: over 2 years ago - 4 downloads total - 14 stars on GitHub - 1 maintainer
Top 0.7% on packagist.org
30 versions - Latest release: 5 months ago - 51 dependent packages - 125 dependent repositories - 2.64 million downloads total - 345 stars on GitHub - 1 maintainer
wa72/htmlpagedom v3.0.2
jQuery-inspired DOM manipulation extension for Symfony's Crawler30 versions - Latest release: 5 months ago - 51 dependent packages - 125 dependent repositories - 2.64 million downloads total - 345 stars on GitHub - 1 maintainer
Top 0.6% on packagist.org
26 versions - Latest release: 14 days ago - 94 dependent packages - 279 dependent repositories - 553 thousand downloads total - 2,606 stars on GitHub - 1 maintainer
jaeger/querylist V4.4.3 π°
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on p...26 versions - Latest release: 14 days ago - 94 dependent packages - 279 dependent repositories - 553 thousand downloads total - 2,606 stars on GitHub - 1 maintainer
brittainmedia/phpcrawl 0.10.1
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-...26 versions - Latest release: about 1 year ago - 2 dependent packages - 1 dependent repositories - 2.94 thousand downloads total - 9 stars on GitHub - 1 maintainer
dachcom-digital/lucene-search v2.3.2
Pimcore 5.x Website Indexer (powered by Zend Search Lucene)26 versions - Latest release: over 4 years ago - 37.3 thousand downloads total - 26 stars on GitHub - 1 maintainer
mediashare/crawler 0.2.8
Crawl urls from a webpage and provide a DomCrawler with Scraper Library26 versions - Latest release: over 2 years ago - 2 dependent packages - 1 dependent repositories - 245 downloads total - 3 stars on GitHub - 1 maintainer
yangze/spiderx v2.1.35
php SpiderX25 versions - Latest release: about 2 years ago - 38 downloads total - 14 stars on GitHub - 1 maintainer
ixnode/php-web-crawler 0.1.24
PHP Web Crawler - This PHP class allows you to crawl recursively a given html page (or a given ht...25 versions - Latest release: 3 months ago - 25 downloads total - 2 stars on GitHub - 1 maintainer
godbout/htmlpagedom 3.0.0 π°
jQuery-inspired DOM manipulation extension for Symfony's Crawler25 versions - Latest release: over 3 years ago - 1 dependent package - 3 dependent repositories - 20.1 thousand downloads total - 1 stars on GitHub - 1 maintainer
yosodog/htmlpagedom v2.0.1
jQuery-inspired DOM manipulation extension for Symfony's Crawler24 versions - Latest release: over 4 years ago - 30 downloads total - 0 stars on GitHub - 1 maintainer
Top 3.4% on packagist.org
24 versions - Latest release: over 1 year ago - 2 dependent packages - 12 dependent repositories - 52.1 thousand downloads total - 259 stars on GitHub - 1 maintainer
spatie/laravel-link-checker 4.3.0 π°
Check all links in a Laravel app24 versions - Latest release: over 1 year ago - 2 dependent packages - 12 dependent repositories - 52.1 thousand downloads total - 259 stars on GitHub - 1 maintainer
innmind/crawler 6.1.0
Library to extract meaningful informations out of a webpage23 versions - Latest release: about 3 years ago - 4 dependent packages - 4 dependent repositories - 936 downloads total - 1 stars on GitHub - 1 maintainer
edwinhuish/querylist V5.0.9
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on D...22 versions - Latest release: about 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
Top 5.9% on packagist.org
22 versions - Latest release: 11 months ago - 9 dependent repositories - 47.2 thousand downloads total - 591 stars on GitHub - 1 maintainer
spatie/http-status-check 4.0.0 π°
CLI tool to crawl a website and check HTTP status code22 versions - Latest release: 11 months ago - 9 dependent repositories - 47.2 thousand downloads total - 591 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
21 versions - Latest release: about 6 years ago - 7 dependent repositories - 61.4 thousand downloads total - 47 stars on GitHub - 1 maintainer
opensearchserver/opensearchserver 3.0.20
PHP library for OpenSearchServer: professionnal search engine, crawlers (web, file, database), RE...21 versions - Latest release: about 6 years ago - 7 dependent repositories - 61.4 thousand downloads total - 47 stars on GitHub - 1 maintainer
contextualcode/crawler v2.4.0
Flexible website crawler which stores the results in persistent storage21 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 526 downloads total - 0 stars on GitLab.com - 2 maintainers
nucleos/lastfm 3.6.0 π°
Last.fm webservice client for php.20 versions - Latest release: 5 months ago - 2 dependent packages - 1 dependent repositories - 10.9 thousand downloads total - 14 stars on GitHub - 1 maintainer
inbo/google-crawler v2.0.3
A simple Crawler for getting Google results20 versions - Latest release: almost 5 years ago - 0 downloads total - 1 stars on GitHub - 1 maintainer
core23/lastfm-api 3.6.0 π°
Last.fm webservice client for php.20 versions - Latest release: 5 months ago - 2 dependent packages - 2.78 thousand downloads total - 13 stars on GitHub - 1 maintainer
cviniciussdias/google-crawler v2.0.3 π°
A simple Crawler for getting Google results20 versions - Latest release: almost 5 years ago - 132 downloads total - 32 stars on GitHub - 1 maintainer
jfcherng/wiki-cgroup-crawler 1.2.6 removed
ζ€θ ³ζ¬η¨ζΌζεηΆεΊηΎη§ηε ¬ε ±θ½ζη΅θ©εΊ«οΌδΈ¦ε°η΅ζε²εηΊε€ι¨ζͺζ‘γ20 versions - Latest release: about 5 years ago - 2 downloads total - 1 stars on GitHub
Top 0.8% on packagist.org
20 versions - Latest release: 21 days ago - 18 dependent packages - 813 dependent repositories - 7.84 million downloads total - 210 stars on GitHub - 1 maintainer
spatie/robots-txt 2.2.0 π°
Determine if a page may be crawled from robots.txt and robots meta tags20 versions - Latest release: 21 days ago - 18 dependent packages - 813 dependent repositories - 7.84 million downloads total - 210 stars on GitHub - 1 maintainer
Top 3.2% on packagist.org
19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
crossjoin/browscap v3.0.5
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as devic...19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
bringyourownideas/laravel-sitemap 3.0.0 π°
A simple website crawler & sitemap generator without a headless browser for Laravel 5.8+19 versions - Latest release: over 1 year ago - 4 dependent repositories - 7.66 thousand downloads total - 9 stars on GitHub - 1 maintainer
seosazi/php-html-parser 1.2.11
simple crawl link and parser it19 versions - Latest release: over 2 years ago - 48 downloads total - 2 stars on GitHub - 1 maintainer
sigmie/crawler v0.1.18
Sigmie Crawler is a PHP package for crawling websites and exporting their HTML contents.19 versions - Latest release: over 3 years ago - 25 downloads total - 0 stars on GitHub - 1 maintainer
Top 1.6% on packagist.org
18 versions - Latest release: 5 months ago - 7 dependent packages - 23 dependent repositories - 143 thousand downloads total - 1,324 stars on GitHub - 1 maintainer
vdb/php-spider v0.7.2
A configurable and extensible PHP web spider18 versions - Latest release: 5 months ago - 7 dependent packages - 23 dependent repositories - 143 thousand downloads total - 1,324 stars on GitHub - 1 maintainer
pithyone/zhihu-crawler 3.5.0
θ½»ιηΊ§η₯δΉη¬θ«17 versions - Latest release: over 5 years ago - 62 downloads total - 25 stars on GitHub - 1 maintainer
adabra/curl v4.0.4
A simple and lightweight cURL library with support for asynchronous requests.17 versions - Latest release: over 3 years ago - 1 dependent package - 29 downloads total - 0 stars on GitHub - 1 maintainer
dachcom-digital/dynamic-search-data-provider-crawler v3.0.1
A Spider Crawler Extension for Pimcore Dynamic Search.16 versions - Latest release: 5 months ago - 12.1 thousand downloads total - 8 stars on GitHub - 1 maintainer
eightwire/magento2-module-primer 0.1.14
A cache primer extension for Magento 215 versions - Latest release: over 4 years ago - 15.7 thousand downloads total - 19 stars on GitHub - 1 maintainer
sleimanx2/grawler 0.2.4
A guided html crawler with media meta extraction15 versions - Latest release: over 4 years ago - 295 downloads total - 13 stars on GitHub - 1 maintainer
mediashare/scraper 0.1.6
Scrapes the information from the targeted page and provides a DomCrawler15 versions - Latest release: over 2 years ago - 1 dependent package - 2 dependent repositories - 254 downloads total - 4 stars on GitHub - 1 maintainer
baraja-core/webcrawler v1.3.3
Simple package to load list of urls and make sitemap.15 versions - Latest release: 10 months ago - 21 downloads total - 5 stars on GitHub - 1 maintainer
airmole/tjustb-edusys v1.1.0
Tianjin college,USTB education system HTTP client15 versions - Latest release: 2 months ago - 24 downloads total - 1 stars on GitHub - 1 maintainer
nueip/curl 0.4.2
NuEIP Curl.14 versions - Latest release: over 2 years ago - 233 downloads total - 2 stars on GitHub - 2 maintainers
nadar/crawler 1.7.1 π°
A highly extendible, dependency free Crawler for HTML, PDFS or any other type of Documents.14 versions - Latest release: about 2 years ago - 2 dependent packages - 1 dependent repositories - 18.6 thousand downloads total - 10 stars on GitHub - 1 maintainer
visuellverstehen/t3fetch 1.5.0
Fetches a website (including all subpages), so the TYPO3 cache gets filled.14 versions - Latest release: 3 months ago - 14 thousand downloads total - 7 stars on GitHub - 1 maintainer
Top 6.4% on packagist.org
14 versions - Latest release: almost 8 years ago - 3 dependent packages - 9 dependent repositories - 30.9 thousand downloads total - 10 stars on GitHub - 1 maintainer
tomverran/robots-txt-checker 1.12.1
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a...14 versions - Latest release: almost 8 years ago - 3 dependent packages - 9 dependent repositories - 30.9 thousand downloads total - 10 stars on GitHub - 1 maintainer
crispy-computing-machine/supersimplecrawler 1.12
PHP 8 web crawler13 versions - Latest release: 8 months ago - 22 downloads total - 0 stars on GitHub - 1 maintainer
Top 5.3% on packagist.org
13 versions - Latest release: over 2 years ago - 1 dependent package - 8 dependent repositories - 9.69 thousand downloads total - 253 stars on GitHub - 1 maintainer
codeguy/arachnid 2.2.1
A crawler to find all unique internal pages on a given website13 versions - Latest release: over 2 years ago - 1 dependent package - 8 dependent repositories - 9.69 thousand downloads total - 253 stars on GitHub - 1 maintainer
johnroyer/crawler-php 0.3.6
crawler implement in PHP13 versions - Latest release: 3 months ago - 1 dependent repositories - 60 downloads total - 3 stars on GitHub - 1 maintainer
Top 9.4% on packagist.org
13 versions - Latest release: over 2 years ago - 2 dependent repositories - 20.1 thousand downloads total - 253 stars on GitHub - 1 maintainer
zrashwani/arachnid 2.2.1
A crawler to find all unique internal pages on a given website13 versions - Latest release: over 2 years ago - 2 dependent repositories - 20.1 thousand downloads total - 253 stars on GitHub - 1 maintainer
Top 2.2% on packagist.org
13 versions - Latest release: over 7 years ago - 9 dependent packages - 40 dependent repositories - 142 thousand downloads total - 72 stars on GitHub - 1 maintainer
jyggen/curl v4.0.0
A simple and lightweight cURL library with support for asynchronous requests.13 versions - Latest release: over 7 years ago - 9 dependent packages - 40 dependent repositories - 142 thousand downloads total - 72 stars on GitHub - 1 maintainer
proxycrawl/proxycrawl 3.0.0
A lightweight, dependency free PHP class that acts as wrapper for ProxyCrawl API13 versions - Latest release: almost 3 years ago - 60.4 thousand downloads total - 19 stars on GitHub - 1 maintainer
hedii/php-crawler 2.2.0
A crawler application written with php and Laravel that finds email addresses on the internets.13 versions - Latest release: over 5 years ago - 1.07 thousand downloads total - 132 stars on GitHub - 1 maintainer
rfussien/leboncoin-crawler 2.3.0
Makes data extraction from leboncoin.fr easy13 versions - Latest release: about 7 years ago - 4 dependent repositories - 358 downloads total - 39 stars on GitHub - 1 maintainer
laurentvw/scrapher v2.3.1
A web scraper for PHP to easily extract data from web pages12 versions - Latest release: about 6 years ago - 1 dependent package - 3 dependent repositories - 2.51 thousand downloads total - 18 stars on GitHub - 1 maintainer
laurentvw/lavacrawler v2.3.1
A web scraper for PHP to easily extract data from web pages12 versions - Latest release: about 6 years ago - 4 dependent repositories - 94 downloads total - 18 stars on GitHub - 1 maintainer
bitandblack/sitemap 2.0.6 π°
Creates a sitemap.xml by parsing the whole website.12 versions - Latest release: 2 months ago - 1 dependent package - 1 dependent repositories - 1.88 thousand downloads total - 2 maintainers
sofyco/spider 2.0.0 π°
Crawler + Scraper + Parser library11 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 52 downloads total - 0 stars on GitHub - 1 maintainer
kiwa/sitemap 2.0.0 π°
Integrates the Bit&Black Sitemap into a Kiwa website to create sitemap files seamlessly.11 versions - Latest release: 3 months ago - 1 dependent package - 3.19 thousand downloads total - 2 maintainers
shel/crawler 2.4.1 π°
Allows crawling of sitemaps and node-trees11 versions - Latest release: over 1 year ago - 1.15 thousand downloads total - 7 stars on GitHub - 1 maintainer
core23/lastfm-bundle 1.3.0 π°
This bundle provides services for using the last.fm API with symfony.11 versions - Latest release: 5 months ago - 104 downloads total - 3 stars on GitHub - 1 maintainer
Top 7.8% on packagist.org
11 versions - Latest release: about 9 years ago - 12 dependent repositories - 4.41 thousand downloads total - 312 stars on GitHub - 1 maintainer
sleeping-owl/apist 1.3.7
Package to provide api-like access to foreign sites based on html parsing11 versions - Latest release: about 9 years ago - 12 dependent repositories - 4.41 thousand downloads total - 312 stars on GitHub - 1 maintainer
nucleos/lastfm-bundle 1.3.0 π°
This bundle provides services for using the last.fm API with symfony.11 versions - Latest release: 5 months ago - 884 downloads total - 3 stars on GitHub - 1 maintainer
gyaaniguy/pcrawl 0.01-beta
PHP web scraping and crawling library. With support for multiple clients, fast parsing, debugging...11 versions - Latest release: 11 months ago - 8 downloads total - 2 stars on GitHub - 1 maintainer
upscale/swoole-warmup 1.5.0
URL crawler to warm-up Swoole web-server10 versions - Latest release: over 3 years ago - 19 downloads total - 0 stars on GitHub - 1 maintainer
thingston/crawler 0.7.0
Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.10 versions - Latest release: over 5 years ago - 9 downloads total - 2 stars on GitHub - 1 maintainer
dz0x44/9pay-spider-telco v1.9
9Pay Crawler for Telco Promotion10 versions - Latest release: about 4 years ago - 31 downloads total - 0 stars on GitHub - 1 maintainer
creode/craft-page-crawler 1.1.0
This will allow a page to be crawled for useful content during an indexing process.10 versions - Latest release: about 1 year ago - 39 downloads total - 0 stars on GitHub - 1 maintainer
podcastcrawler/podcastcrawler 1.1.1
PHP library to find podcasts10 versions - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 2.61 thousand downloads total - 39 stars on GitHub - 1 maintainer
panakour/pkscraper v1.2.0
Get whatever data you want.10 versions - Latest release: 5 months ago - 181 downloads total - 1 stars on GitHub - 1 maintainer
spekulatius/spatie-crawler-toolkit-for-laravel 0.5.0 π°
Handy classes for Spatie's crawler when using it with Laravel.9 versions - Latest release: over 1 year ago - 722 downloads total - 18 stars on GitHub - 1 maintainer
Top 9.1% on packagist.org
9 versions - Latest release: about 3 years ago - 3 dependent packages - 25 dependent repositories - 707 thousand downloads total - 1 stars on GitHub - 1 maintainer
vipnytt/useragentparser v1.0.5
User-Agent parser for robot rule sets9 versions - Latest release: about 3 years ago - 3 dependent packages - 25 dependent repositories - 707 thousand downloads total - 1 stars on GitHub - 1 maintainer
snippetify/snippet-sniffer 1.2.4
Crawling and scraping web pages to extract snippets9 versions - Latest release: almost 4 years ago - 1 dependent package - 1 dependent repositories - 9 downloads total - 1 stars on GitHub - 1 maintainer
bdspider/crawler 0.5.1
crawler,spider for phpQueryγ8 versions - Latest release: over 5 years ago - 46 downloads total - 0 stars on GitHub - 1 maintainer
crawlzone/crawlzone 4.0.0
Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web se...8 versions - Latest release: about 2 years ago - 5 thousand downloads total - 76 stars on GitHub - 1 maintainer
macsch15/crawler-detector 1.3.4
Fat-free, standalone and fast web crawler (bot) detector8 versions - Latest release: almost 6 years ago - 1.77 thousand downloads total - 3 stars on GitHub - 1 maintainer
blueways/bw-cache-uri v1.1.6
TYPO3 extension to parse remote content and save to tt_content element8 versions - Latest release: 7 months ago - 395 downloads total - 0 stars on GitHub - 1 maintainer
lobotomised/laravel-autocrawler 1.2.0
A tool to crawl your own laravel installation checking your HTTP status codes8 versions - Latest release: about 1 month ago - 15.8 thousand downloads total - 1 stars on GitHub - 1 maintainer
jmajors/robotstxt v1.7.1
A small package for parsing websites' robots.txt files8 versions - Latest release: about 7 years ago - 1 dependent repositories - 20 downloads total - 3 stars on GitHub - 1 maintainer
webysther/packagist-mirror 1.1.1 π°
Build mirror of packagist7 versions - Latest release: over 4 years ago - 150 downloads total - 179 stars on GitHub - 1 maintainer
crwlr/crawler-ext-browser v1.2.1
Extension for the crwlr/crawler package containing steps utilizing a headless browser.7 versions - Latest release: 2 months ago - 135 downloads total - 0 stars on GitHub - 1 maintainer
schliesser/sitecrawler v2.0.1
TYPO3 sitemap crawler7 versions - Latest release: about 1 year ago - 1 dependent repositories - 19.8 thousand downloads total - 10 stars on GitHub - 1 maintainer
mihaeu/tarantula v1.3.0
Another PHP crawler based on Guzzle.7 versions - Latest release: almost 10 years ago - 4 dependent repositories - 49 downloads total - 15 stars on GitHub - 1 maintainer
dayrev/extractor v1.2.2
Web Page Content Extractor7 versions - Latest release: about 7 years ago - 11 downloads total - 2 stars on GitHub - 1 maintainer
depa/depa-middleware-redirect 1.0.8
Laminas mezzio middleware for redirecting7 versions - Latest release: 4 months ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
Related Keywords
php
122
spider
62
scraper
47
scraping
28
crawling
23
laravel
23
parser
22
website
17
http
17
bot
16
hacktoberfest
15
seo
15
curl
15
symfony
15
dom
14
api
13
link
12
html
12
search
12
spatie
11
library
11
scrape
11
yii2
10
Sitemap
9
detect
9
web
9
module
8
composer
8
crawl
8
extension
7
browser
6
scrapper
6
robots.txt
6
user-agent
6
robot
6
bundle
6
webcrawler
6
framework
6
google
6
yii
6
typo3
6
luya
6
url
5
scraping-websites
5
extractor
5
index
5
xml
5
news
5
domcrawler
5
parse
5
robots-txt
5
bots
5
crawlerdetect
5
crawler detect
5
crawler detector
5
php crawler detect
5
json
4
asynchronous
4
content
4
QueryList
4
phpQuery
4
webservice-client
4
health
4
lastfm
4
webservice client
4
php-crawler
4
request
4
last.fm
4
luya-module
4
symfony-bundle
4
web-crawler
4
mining
3
async
3
Match
3
selector
3
facebook
3
console
3
web-scraping
3
web-crawling
3
multi
3
guzzle
3
sitemap
3
crwlr
3
cache-warmup
3
Harvest
3
indexer
3
extract
3
user agent
3
web crawler
3
device
3
stock
3
robots
3
rss
3
seo-optimization
3
helper
3
cache
3
instagram
3
laravel-package
3
matcher
3
data
3