Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

packagist.org "crawler" keyword

Top 0.2% on packagist.org
jaybizzle/crawler-detect v1.2.118
CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
158 versions - Latest release: 21 days ago - 170 dependent packages - 8,221 dependent repositories - 58.4 million downloads total - 1,911 stars on GitHub - 2 maintainers
Top 0.6% on packagist.org
jaeger/querylist V4.4.3 πŸ’°
Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on p...
26 versions - Latest release: 14 days ago - 96 dependent packages - 279 dependent repositories - 553 thousand downloads total - 2,606 stars on GitHub - 1 maintainer
Top 0.7% on packagist.org
wa72/htmlpagedom v3.0.2
jQuery-inspired DOM manipulation extension for Symfony's Crawler
30 versions - Latest release: 5 months ago - 51 dependent packages - 125 dependent repositories - 2.64 million downloads total - 345 stars on GitHub - 1 maintainer
Top 0.3% on packagist.org
spatie/crawler 8.2.0 πŸ’°
Crawl all internal links found on a website
104 versions - Latest release: 3 months ago - 45 dependent packages - 858 dependent repositories - 8.23 million downloads total - 2,471 stars on GitHub - 1 maintainer
Top 0.8% on packagist.org
spatie/robots-txt 2.2.0 πŸ’°
Determine if a page may be crawled from robots.txt and robots meta tags
20 versions - Latest release: 22 days ago - 19 dependent packages - 813 dependent repositories - 7.91 million downloads total - 212 stars on GitHub - 1 maintainer
Top 1.1% on packagist.org
jaybizzle/laravel-crawler-detect v1.3.0
A Laravel package to detect web crawlers via the user agent
6 versions - Latest release: almost 2 years ago - 17 dependent packages - 183 dependent repositories - 1.55 million downloads total - 298 stars on GitHub - 1 maintainer
Top 2.2% on packagist.org
jyggen/curl v4.0.0
A simple and lightweight cURL library with support for asynchronous requests.
13 versions - Latest release: over 7 years ago - 9 dependent packages - 40 dependent repositories - 142 thousand downloads total - 72 stars on GitHub - 1 maintainer
Top 1.6% on packagist.org
vdb/php-spider v0.7.2
A configurable and extensible PHP web spider
18 versions - Latest release: 5 months ago - 8 dependent packages - 23 dependent repositories - 143 thousand downloads total - 1,324 stars on GitHub - 1 maintainer
Top 5.7% on packagist.org
minicli/curly 0.2.2
Simple Curl Client
4 versions - Latest release: over 1 year ago - 6 dependent packages - 11 dependent repositories - 257 thousand downloads total - 12 stars on GitHub - 1 maintainer
Top 2.8% on packagist.org
duzun/hquery 3.1.0
An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies...
41 versions - Latest release: 9 months ago - 4 dependent packages - 23 dependent repositories - 90.4 thousand downloads total - 351 stars on GitHub - 1 maintainer
innmind/crawler 6.1.0
Library to extract meaningful informations out of a webpage
23 versions - Latest release: about 3 years ago - 4 dependent packages - 4 dependent repositories - 936 downloads total - 1 stars on GitHub - 1 maintainer
Top 3.2% on packagist.org
crossjoin/browscap v3.0.5
The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as devic...
19 versions - Latest release: over 7 years ago - 4 dependent packages - 14 dependent repositories - 234 thousand downloads total - 43 stars on GitHub - 1 maintainer
Top 9.1% on packagist.org
vipnytt/useragentparser v1.0.5
User-Agent parser for robot rule sets
9 versions - Latest release: about 3 years ago - 3 dependent packages - 25 dependent repositories - 707 thousand downloads total - 1 stars on GitHub - 1 maintainer
Top 6.4% on packagist.org
tomverran/robots-txt-checker 1.12.1
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a...
14 versions - Latest release: almost 8 years ago - 3 dependent packages - 9 dependent repositories - 30.9 thousand downloads total - 10 stars on GitHub - 1 maintainer
crwlr/crawler v1.7.2
Web crawling and scraping library.
38 versions - Latest release: about 2 months ago - 3 dependent packages - 1 dependent repositories - 2.7 thousand downloads total - 295 stars on GitHub - 1 maintainer
Top 10.0% on packagist.org
ropendev/curl 1.0.3
PHP POO cURL wrapper (PSR friendly) : light and simple.
4 versions - Latest release: over 5 years ago - 3 dependent packages - 8 dependent repositories - 6.39 thousand downloads total - 3 stars on GitHub - 1 maintainer
brittainmedia/phpcrawl 0.10.1
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-...
26 versions - Latest release: about 1 year ago - 2 dependent packages - 1 dependent repositories - 2.94 thousand downloads total - 9 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
aoepeople/crawler 11.0.7 πŸ’°
Crawler extension for TYPO3
68 versions - Latest release: over 1 year ago - 2 dependent packages - 1 dependent repositories - 279 thousand downloads total - 53 stars on GitHub - 2 maintainers
mediashare/crawler 0.2.8
Crawl urls from a webpage and provide a DomCrawler with Scraper Library
26 versions - Latest release: over 2 years ago - 2 dependent packages - 1 dependent repositories - 245 downloads total - 3 stars on GitHub - 1 maintainer
sfynx-project/tool-crawler-bundle v2.8.2
This Bundle provides handlers to work with xml and xsd files.
4 versions - Latest release: over 5 years ago - 2 dependent packages - 2 dependent repositories - 473 downloads total - 1 stars on GitHub - 1 maintainer
core23/lastfm-api 3.6.0 πŸ’°
Last.fm webservice client for php.
20 versions - Latest release: 5 months ago - 2 dependent packages - 2.78 thousand downloads total - 13 stars on GitHub - 1 maintainer
zephir/luya-module-crawleradmin 1.0.0-beta8
Yii2 LUYA Crawler Admin-Module
4 versions - Latest release: almost 8 years ago - 2 dependent packages - 1 dependent repositories - 143 downloads total - 1 maintainer
nucleos/lastfm 3.6.0 πŸ’°
Last.fm webservice client for php.
20 versions - Latest release: 5 months ago - 2 dependent packages - 1 dependent repositories - 10.9 thousand downloads total - 14 stars on GitHub - 1 maintainer
alc/sitemap-crawler v2.0.0
Crawl sitemap.xml recursively and return collected urls as array
2 versions - Latest release: about 3 years ago - 2 dependent packages - 2 dependent repositories - 2.43 thousand downloads total - 5 stars on GitHub - 1 maintainer
Top 3.4% on packagist.org
spatie/laravel-link-checker 4.3.0 πŸ’°
Check all links in a Laravel app
24 versions - Latest release: over 1 year ago - 2 dependent packages - 12 dependent repositories - 52.1 thousand downloads total - 259 stars on GitHub - 1 maintainer
nadar/crawler 1.7.1 πŸ’°
A highly extendible, dependency free Crawler for HTML, PDFS or any other type of Documents.
14 versions - Latest release: about 2 years ago - 2 dependent packages - 1 dependent repositories - 18.6 thousand downloads total - 10 stars on GitHub - 1 maintainer
nguyenanhung/my-crawler v1.1.3
Crawler helper, library - Basic, Simple and Lightweight
45 versions - Latest release: about 1 month ago - 2 dependent packages - 1 dependent repositories - 725 downloads total - 1 stars on GitHub - 1 maintainer
laurentvw/scrapher v2.3.1
A web scraper for PHP to easily extract data from web pages
12 versions - Latest release: about 6 years ago - 1 dependent package - 3 dependent repositories - 2.51 thousand downloads total - 18 stars on GitHub - 1 maintainer
smart-core/sitemap-bundle
Symfony Sitemap Crawler Generator Bundle.
Latest release: 6 days ago - 1 dependent package - 6 dependent repositories - 307 downloads total - 0 stars on GitHub - 1 maintainer
crwlr/crawler-ext-browser v1.2.1
Extension for the crwlr/crawler package containing steps utilizing a headless browser.
7 versions - Latest release: 2 months ago - 1 dependent package - 135 downloads total - 0 stars on GitHub - 1 maintainer
dobrosite/crawler 0.2.0
PHP component for recursive traversal of hierarchical data storages
2 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 24 downloads total - 0 stars on GitHub - 1 maintainer
Top 7.0% on packagist.org
tomasnorre/crawler 11.0.7 πŸ’°
Crawler extension for TYPO3
68 versions - Latest release: over 1 year ago - 1 dependent package - 2 dependent repositories - 197 thousand downloads total - 53 stars on GitHub - 1 maintainer
brunodebarros/http-request 1.0.2
A web crawler, built to be easily dropped into PHP applications, which behaves just like a regula...
3 versions - Latest release: about 9 years ago - 1 dependent package - 4 dependent repositories - 195 downloads total - 8 stars on GitHub - 1 maintainer
acanto/crawler 6.0.2
Crawl all internal links found on a website
1 version - Latest release: about 3 years ago - 1 dependent package - 2 dependent repositories - 2.72 thousand downloads total - 0 stars on GitLab.com - 1 maintainer
crwlr/robots-txt v1.1.1
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
6 versions - Latest release: over 1 year ago - 1 dependent package - 1 dependent repositories - 5.36 thousand downloads total - 8 stars on GitHub - 1 maintainer
blogdaren/phpcreeper v1.9.0
A new generation of multi-process async event-driven spider engine based on Workerman
57 versions - Latest release: about 10 hours ago - 1 dependent package - 2 dependent repositories - 625 downloads total - 117 stars on GitHub - 1 maintainer
mediamonks/crawler 2.0.0
Crawl your own website with various clients for SEO and indexing purposes.
4 versions - Latest release: over 6 years ago - 1 dependent package - 1 dependent repositories - 565 downloads total - 19 stars on GitHub - 1 maintainer
yidas/tw-stock-crawler 1.2.1
Taiwan Stock Opendata Crawler with realtime - 台灣肑市肑η₯¨θ³‡θ¨Šζ“·ε–器
5 versions - Latest release: 6 months ago - 1 dependent package - 1 dependent repositories - 26 downloads total - 2 stars on GitHub - 1 maintainer
sofyco/spider 2.0.0 πŸ’°
Crawler + Scraper + Parser library
11 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 52 downloads total - 0 stars on GitHub - 1 maintainer
snippetify/snippet-sniffer 1.2.4
Crawling and scraping web pages to extract snippets
9 versions - Latest release: almost 4 years ago - 1 dependent package - 1 dependent repositories - 9 downloads total - 1 stars on GitHub - 1 maintainer
godbout/htmlpagedom 3.0.0 πŸ’°
jQuery-inspired DOM manipulation extension for Symfony's Crawler
25 versions - Latest release: over 3 years ago - 1 dependent package - 3 dependent repositories - 20.1 thousand downloads total - 1 stars on GitHub - 1 maintainer
51degrees/fiftyone.devicedetection 4.4.2
Device detection engines for 51Degrees Pipeline API. Parse HTTP headers to detect hardware, opera...
50 versions - Latest release: almost 2 years ago - 1 dependent package - 1 dependent repositories - 6.93 thousand downloads total - 0 stars on GitHub - 1 maintainer
podcastcrawler/podcastcrawler 1.1.1
PHP library to find podcasts
10 versions - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 2.61 thousand downloads total - 39 stars on GitHub - 1 maintainer
kiwa/sitemap 2.0.0 πŸ’°
Integrates the Bit&Black Sitemap into a Kiwa website to create sitemap files seamlessly.
11 versions - Latest release: 3 months ago - 1 dependent package - 3.19 thousand downloads total - 2 maintainers
Top 5.3% on packagist.org
codeguy/arachnid 2.2.1
A crawler to find all unique internal pages on a given website
13 versions - Latest release: over 2 years ago - 1 dependent package - 8 dependent repositories - 9.69 thousand downloads total - 253 stars on GitHub - 1 maintainer
smart-core/rich-editor-bundle
Rich editor bundle.
Latest release: 6 days ago - 1 dependent package - 7 dependent repositories - 296 downloads total - 0 stars on GitHub - 1 maintainer
mediashare/scraper 0.1.6
Scrapes the information from the targeted page and provides a DomCrawler
15 versions - Latest release: over 2 years ago - 1 dependent package - 2 dependent repositories - 254 downloads total - 4 stars on GitHub - 1 maintainer
bitandblack/sitemap 2.0.6 πŸ’°
Creates a sitemap.xml by parsing the whole website.
12 versions - Latest release: 2 months ago - 1 dependent package - 1 dependent repositories - 1.88 thousand downloads total - 2 maintainers
Top 9.8% on packagist.org
eliashaeussler/cache-warmup 3.0.1 πŸ’°
Composer package to warm up website caches, based on a given XML sitemap
66 versions - Latest release: about 1 month ago - 1 dependent package - 1 dependent repositories - 89.7 thousand downloads total - 40 stars on GitHub - 1 maintainer
cacing69/cquery v1.4.0 πŸ’°
PHP Scraper with language expression, could be used to scrape data from a website that uses javas...
6 versions - Latest release: 8 months ago - 1 dependent package - 17 downloads total - 2 stars on GitHub - 1 maintainer
contextualcode/crawler v2.4.0
Flexible website crawler which stores the results in persistent storage
21 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 526 downloads total - 0 stars on GitLab.com - 2 maintainers
adabra/curl v4.0.4
A simple and lightweight cURL library with support for asynchronous requests.
17 versions - Latest release: over 3 years ago - 1 dependent package - 29 downloads total - 0 stars on GitHub - 1 maintainer
coooold/crawler v1.0.0
Most powerful, popular and production crawling/scraping package for PHP
1 version - Latest release: over 4 years ago - 15 downloads total - 1 stars on GitHub - 1 maintainer
harm-smits/webcrawler-verifier 2.1.1
PHP library providing functionality to verify that user-agents are who they claim to be.
6 versions - Latest release: over 2 years ago - 1 downloads total - 0 stars on GitHub - 1 maintainer
webysther/packagist-mirror 1.1.1 πŸ’°
Build mirror of packagist
7 versions - Latest release: over 4 years ago - 150 downloads total - 179 stars on GitHub - 1 maintainer
delphiki/aprr 2.0.1
A simple crawler to get formated data from APRR website.
5 versions - Latest release: about 7 years ago - 18 downloads total - 0 stars on GitHub - 1 maintainer
blueways/bw-cache-uri v1.1.6
TYPO3 extension to parse remote content and save to tt_content element
8 versions - Latest release: 7 months ago - 395 downloads total - 0 stars on GitHub - 1 maintainer
dachcom-digital/dynamic-search-data-provider-crawler v3.0.1
A Spider Crawler Extension for Pimcore Dynamic Search.
16 versions - Latest release: 5 months ago - 12.1 thousand downloads total - 8 stars on GitHub - 1 maintainer
ngatngay/crawler-php 8.1.0
A simple PHP Web Crawler
3 versions - Latest release: over 1 year ago - 47 downloads total - 0 stars on GitHub - 1 maintainer
oyjz/querylist
PHP采集ε·₯ε…·(ηˆ¬θ™«),基于https://github.com/jae-jae/QueryList。
Latest release: 27 days ago - 6 downloads total - 0 stars on GitHub - 1 maintainer
piedweb/crawler 0.1.783
Web Crawler to check few SEO basics.
53 versions - Latest release: 3 months ago - 70 downloads total - 1 stars on GitHub - 1 maintainer
morphodo/crawler-bundle
HTTP-Crawler for Symfony2
Latest release: 24 days ago - 4 dependent repositories - 5 downloads total - 1 maintainer
gaolujie1989/webmagic
A scalable web crawler framework for php.
Latest release: 20 days ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
yosodog/htmlpagedom v2.0.1
jQuery-inspired DOM manipulation extension for Symfony's Crawler
24 versions - Latest release: over 4 years ago - 30 downloads total - 0 stars on GitHub - 1 maintainer
pithyone/zhihu-crawler 3.5.0
轻量级ηŸ₯δΉŽηˆ¬θ™«
17 versions - Latest release: over 5 years ago - 62 downloads total - 25 stars on GitHub - 1 maintainer
c2is/walker
A basic wrapper around Goutte to crawl a website
Latest release: 13 days ago - 5 dependent repositories - 28 downloads total - 0 stars on GitHub - 1 maintainer
eddieace/php-simple v1.0.4
HTML DOM parser for php 7.1+
5 versions - Latest release: about 3 years ago - 1 dependent repositories - 34.4 thousand downloads total - 14 stars on GitHub - 1 maintainer
lefuturiste/google-photos-album-crawler v1.1
Google Photos public album crawler/fetcher
2 versions - Latest release: almost 4 years ago - 61 downloads total - 8 stars on GitHub - 1 maintainer
mvrc/php-email-crawler πŸ’°
PHP Website Email Crawler, for crawling websites for emails.
Latest release: 23 days ago - 7 downloads total - 3 stars on GitHub - 1 maintainer
68publishers/crawler-client-php v1.2.0
:spider_web: PHP Client for https://github.com/68publishers/crawler
4 versions - Latest release: 10 months ago - 280 downloads total - 1 stars on GitHub - 1 maintainer
seosazi/php-html-parser 1.2.11
simple crawl link and parser it
19 versions - Latest release: over 2 years ago - 48 downloads total - 2 stars on GitHub - 1 maintainer
vanry/baike-spider
A simple spider crawling web pages from baike.baidu.com
Latest release: 22 days ago - 6 downloads total - 2 stars on GitHub - 1 maintainer
thingston/crawler 0.7.0
Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.
10 versions - Latest release: over 5 years ago - 9 downloads total - 2 stars on GitHub - 1 maintainer
radarlog/banks-spyder 1.0
Banks Spyder is free software to automatically generate bank statements in PDF format from your a...
1 version - Latest release: about 8 years ago - 2 downloads total - 0 stars on GitHub - 1 maintainer
Top 9.4% on packagist.org
zrashwani/arachnid 2.2.1
A crawler to find all unique internal pages on a given website
13 versions - Latest release: over 2 years ago - 2 dependent repositories - 20.1 thousand downloads total - 253 stars on GitHub - 1 maintainer
andrew-svirin/resource-crawler-bundle
Crawler for any sort of resources.
Latest release: 27 days ago - 55 downloads total - 3 stars on GitHub - 1 maintainer
yangze/spiderx v2.1.35
php SpiderX
25 versions - Latest release: about 2 years ago - 38 downloads total - 14 stars on GitHub - 1 maintainer
gpupo/camelspider 1.0 abandoned
A class to help handle gateways to processing entities.
1 version - Latest release: almost 9 years ago - 4 dependent repositories - 1 downloads total - 10 stars on GitHub - 1 maintainer
chemaclass/stock-ticker
An API to gather the latest info from your stocks
Latest release: 5 days ago - 1 downloads total - 3 stars on GitHub - 1 maintainer
nekhbet/crawlers-laravel-pulse
Laravel Pulse Card to show crawlers
Latest release: 20 days ago - 0 downloads total - 0 stars on GitHub - 1 maintainer
dz0x44/9pay-spider-telco v1.9
9Pay Crawler for Telco Promotion
10 versions - Latest release: about 4 years ago - 31 downloads total - 0 stars on GitHub - 1 maintainer
tallesairan/crawler 1.0
Most powerful, popular and production crawling/scraping package for PHP
1 version - Latest release: 9 months ago - 4 downloads total - 0 stars on GitHub - 1 maintainer
dusk-browser/dusk-browser
Laravel Dusk Browser is Laravel Dusk for browsing capabilities only.
Latest release: 9 days ago - 48 downloads total - 0 stars on GitHub - 1 maintainer
Top 8.7% on packagist.org
pablouser1/tikscraper v1.2.7
Get data from TikTok API
84 versions - Latest release: about 2 years ago - 10 dependent repositories - 4.31 thousand downloads total - 53 stars on GitHub - 1 maintainer
howie6879/php-google 1.0
Google search results crawler, get google search results that you need.
1 version - Latest release: almost 7 years ago - 537 downloads total - 29 stars on GitHub - 1 maintainer
smochin/instagram-php-crawler 1.0.1
A simple PHP Crawler for Instagram
2 versions - Latest release: about 6 years ago - 1 dependent repositories - 5.82 thousand downloads total - 46 stars on GitHub - 1 maintainer
dr4g0nsr/sitemap-crawler 1.0 removed
Crawler for any type of site using robots.txt and sitemap.xml as the source of URL. Useful for ca...
1 version - Latest release: over 1 year ago - 20 downloads total - 2 stars on GitHub - 1 maintainer
exfriend/uncurl 1.0.5
Unfurl links in PHP
4 versions - Latest release: about 5 years ago - 1 dependent repositories - 196 downloads total - 1 stars on GitHub - 1 maintainer
lndj/lcrawl v3.1.0
δΈ€εͺδΌ˜ι›…ηš„ζ­£ζ–Ήζ•™εŠ‘η³»η»Ÿηˆ¬θ™«γ€‚
7 versions - Latest release: over 7 years ago - 3 dependent repositories - 1.82 thousand downloads total - 114 stars on GitHub - 1 maintainer
sohib/crawler v1.0.1
Scan Http URLs from a url containing an HTML and get Json or Xml or Array
3 versions - Latest release: over 8 years ago - 24 downloads total - 0 stars on GitHub - 1 maintainer
agilekeys/point-picker
Crawler package for online shopping
Latest release: 22 days ago - 800 downloads total - 0 stars on GitHub - 1 maintainer
lenonleite/hacking v0.1
LenonLeite Component - Hakcing packages
1 version - Latest release: almost 7 years ago - 19 downloads total - 0 stars on GitHub - 1 maintainer
alexndreazevedo/crawler
Social PHP Crawler
Latest release: 22 days ago - 4 dependent repositories - 6 downloads total - 0 stars on GitHub - 1 maintainer
zrashwani/news-scrapper 1.0.3
scrapping news data from a webpage using structured data
6 versions - Latest release: over 6 years ago - 256 downloads total - 13 stars on GitHub - 1 maintainer
spacespell/laravel-crawler v0.1.0
Laravel queue based distributed web crawler framework.
1 version - Latest release: over 6 years ago - 18 downloads total - 2 stars on GitHub - 1 maintainer
filchos/imago
A library for reading and transforming content in a chainable way
Latest release: 20 days ago - 29 downloads total - 0 stars on GitHub - 1 maintainer
migliori/sitemap-crawler 0.0.3
Sitemap crawler/generator. For the given URL it will return sitemap XML file with URLs and images.
3 versions - Latest release: about 5 years ago - 266 downloads total - 3 stars on GitHub - 1 maintainer
Top 6.8% on packagist.org
opensearchserver/opensearchserver 3.0.20
PHP library for OpenSearchServer: professionnal search engine, crawlers (web, file, database), RE...
21 versions - Latest release: about 6 years ago - 7 dependent repositories - 61.4 thousand downloads total - 47 stars on GitHub - 1 maintainer
maidmaid/cffie v0.4.0
CFFie query SBB/CFF/FFS connections from https://www.cff.ch.
5 versions - Latest release: about 7 years ago - 19 downloads total - 0 stars on GitHub - 1 maintainer
jafaripur/varzesh3-crawler
Crawler for varzesh3.com livescore
Latest release: 1 day ago - 1 dependent repositories - 48 downloads total - 3 stars on GitHub - 1 maintainer