Top 9.0% dependent packages on npmjs.org
Top 4.4% dependent repos on npmjs.org
Top 3.6% forks on npmjs.org
npmjs.org : supercrawler
A web crawler. Supercrawler automatically crawls websites. Define custom handlers to parse content. Obeys robots.txt, rate limits and concurrency limits.
Registry
-
Source
- JSON
purl: pkg:npm/supercrawler
Keywords:
crawler
, spider
, supercrawler
, distributed-crawler
, robot
, sitemap
, web-crawler
License: Apache-2.0
Latest release: almost 6 years ago
First release: about 9 years ago
Dependent packages: 2
Dependent repositories: 7
Downloads: 76 last month
Stars: 380 on GitHub
Forks: 63 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: about 22 hours ago