Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
npmjs.org "webcrawler" keyword
zoinks 1.1.8
Crawls through provided website, checking for 200 response, content load, ssl cert errors, and more!19 versions - Latest release: over 3 years ago - 1 dependent package - 1 dependent repositories - 13 downloads last month - 1 stars on GitHub - 1 maintainer
@rodrigogs/spiderman 0.0.0
Yet to describe1 version - Latest release: about 2 years ago - 2 dependent packages - 6 downloads last month - 0 stars on GitHub - 1 maintainer
@drorgl/sitemapper 2.1.17
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: almost 4 years ago - 1 dependent package - 1 dependent repositories - 12 downloads last month - 0 stars on GitHub - 1 maintainer
nightmare-screenshot-selector 1.0.2
A Nightmare plugin to easily take screenshots.3 versions - Latest release: about 7 years ago - 3 dependent packages - 10 dependent repositories - 29 downloads last month - 4 stars on GitHub - 1 maintainer
@waynechang65/baha-crawler 2.6.0
A web crawler module designed to scarp data from Bahamut forum.13 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 7 downloads last month - 3 stars on GitHub - 1 maintainer
algolia-webcrawler 3.2.3
Simple node worker that crawls sitemaps in order to keep an algolia index up-to-date.28 versions - Latest release: over 4 years ago - 2 dependent packages - 2 dependent repositories - 87 downloads last month - 46 stars on GitHub - 2 maintainers
@waynechang65/ptt-crawler 2.6.0
A web crawler module designed to scarp data from Ptt.17 versions - Latest release: 4 months ago - 1 dependent package - 3 dependent repositories - 47 downloads last month - 20 stars on GitHub - 1 maintainer
fetchyt 3.0.7
Search musics on youtube and return your links15 versions - Latest release: almost 3 years ago - 2 dependent packages - 7 downloads last month - 2 stars on GitHub - 1 maintainer
url-scraper 1.0.2
Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and wi...3 versions - Latest release: over 7 years ago - 1 dependent repositories - 29 downloads last month - 5 stars on GitHub - 1 maintainer
bing-me-links 1.0.0
Scrapes Baidu, Bing, StartPage, Yahoo, Webcrawler, Qwant and retrieves links in serps13 versions - Latest release: about 7 years ago - 1 dependent package - 1 dependent repositories - 3 downloads last month - 3 stars on GitHub - 1 maintainer
@qbbsh/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - 98 stars on GitHub - 1 maintainer
adibenmat-sitemapper 2.1.19
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: about 3 years ago - 2 dependent packages - 10 downloads last month - 0 stars on GitHub - 1 maintainer
askro 0.1.6
This tool allow you to parse, collect and traverse through the radiation monitoring data provided...6 versions - Latest release: over 9 years ago - 2 dependent packages - 1 dependent repositories - 11 downloads last month - 0 stars on GitHub - 1 maintainer
spiderboi 1.1.0 π°
A web crawler written in TypeScript.3 versions - Latest release: over 5 years ago - 1 dependent repositories - 10 downloads last month - 5 stars on GitHub - 1 maintainer
scrapefrom 2.5.1
Scrape data from any webpage.70 versions - Latest release: 2 months ago - 2 dependent packages - 133 downloads last month - 4 stars on GitHub - 1 maintainer
sitemapper_mos 3.2.9
Parser for XML Sitemaps to be used with Robots.txt and web crawlers6 versions - Latest release: about 2 years ago - 1 dependent package - 119 downloads last month - 0 stars on GitHub - 1 maintainer
@mastixmc/sitemapper 3.2.0
Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)3 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 maintainer
spiking 2.1.0
A lightweight web crawler.2 versions - Latest release: over 4 years ago - 1 dependent repositories - 7 downloads last month - 11 stars on GitHub - 1 maintainer
trackpurchase 2.0.0
λ¨ λͺμ€μ μ½λλ‘ λ€μν μΌν νλ«νΌμμ κ²°μ λ΄μμ κΈμ΄μ€μ!19 versions - Latest release: over 2 years ago - 49 downloads last month - 21 stars on GitHub - 1 maintainer
Top 3.0% on npmjs.org
3 versions - Latest release: almost 13 years ago - 16 dependent packages - 92 dependent repositories - 6.91 thousand downloads last month - 92 stars on GitHub - 2 maintainers
http-agent 0.1.2
A simple agent for performing a sequence of http requests in node.js3 versions - Latest release: almost 13 years ago - 16 dependent packages - 92 dependent repositories - 6.91 thousand downloads last month - 92 stars on GitHub - 2 maintainers
@cabbiepete/sitemapper 3.0.2
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 10 downloads last month - 1 stars on GitHub - 1 maintainer
@context-labs/sitemapper 3.2.10
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: 12 months ago - 4 dependent packages - 736 downloads last month - 98 stars on GitHub - 3 maintainers
spider_jar 1.0.0 unpublished
A simple pack for writting webscrawles that run inside containers1 version - Latest release: over 1 year ago - 0 stars on GitHub - 1 maintainer
@public-square/sitemapper 3.2.5
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: almost 2 years ago - 2 dependent packages - 1 dependent repositories - 9 downloads last month - 98 stars on GitHub - 2 maintainers
Top 2.3% on npmjs.org
44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
sitemapper 3.2.8
Parser for XML Sitemaps to be used with Robots.txt and web crawlers44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
hellobot 1.2.3
A friendly javascript pre-rendering engine - BETA (UNSTABLE)39 versions - Latest release: about 6 years ago - 1 dependent package - 1 dependent repositories - 44 downloads last month - 2 stars on GitHub - 1 maintainer
orbweaver 0.1.4
A simple webcrawler that prints out the URLs of the pages it encounters. Runs in parallel, up to ...5 versions - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 13 downloads last month - 5 stars on GitHub - 1 maintainer
htmlgetty 1.0.1
This is a function that accepts 3 arguments, "url", "tag" and "output", and writes to a file, in ...2 versions - Latest release: almost 2 years ago - 1 dependent package - 18 downloads last month - 0 stars on GitHub - 1 maintainer
behance-node 1.0.0
An asynchronous module for scrapping Behance using JavaScript.4 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 41 downloads last month - 2 stars on GitHub - 1 maintainer
schabbi-webscraper 1.2.1
Lightweight and easy to use crawling solution for websites.28 versions - Latest release: over 1 year ago - 57 downloads last month - 0 stars on GitHub - 1 maintainer
bfh-mensa 1.0.3
Webcrawler script to retrieve the daily menu of the Bern University of Applied Sciences cantina i...1 version - Latest release: over 4 years ago - 1 dependent package - 1 dependent repositories - 9 downloads last month - 4 stars on GitHub - 1 maintainer
@jarronb/site-mapper 1.0.5
Site mapper utilizing puppeteer to generate a xml sitemap6 versions - Latest release: almost 4 years ago - 1 dependent package - 7 downloads last month - 0 stars on GitHub - 1 maintainer
@wmod/sitemapper 3.2.11
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: 4 months ago - 31 downloads last month - 0 stars on GitHub - 1 maintainer
myreadingmanga-unofficial 0.0.10
MyReadingManga WebCrawler. Find By Languages, Genres and Tags.8 versions - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 14 downloads last month - 1 maintainer
readme-crawler 1.0.5 deprecated
Download README files from GitHub repository links6 versions - Latest release: almost 3 years ago - 1 dependent package - 14 downloads last month - 4 stars on GitHub - 1 maintainer
redis-web-crawler 1.0.0 deprecated
Web Crawler to create directed graph of links among connected sites. Runs with Node.js and stores...1 version - Latest release: almost 3 years ago - 5 downloads last month - 1 stars on GitHub - 1 maintainer
baha-crawler 2.0.0 deprecated
A web crawler module designed to scarp data from Bahamut forum.1 version - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 3 downloads last month - 3 stars on GitHub - 1 maintainer
le-manga 0.0.6 deprecated
CLI for download manga and serve it locally.6 versions - Latest release: about 10 years ago - 1 dependent package - 1 dependent repositories - 7 downloads last month - 1 maintainer
ptt-crawler.js 2.0.0 deprecated
A web crawler module designed to scarp data from Ptt.1 version - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 2 downloads last month - 20 stars on GitHub - 1 maintainer
crawlmap 0.3.1 deprecated
A sitemapping tool for crawlers5 versions - Latest release: over 7 years ago - 1 dependent package - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
@wmod/sitemap-url-scrapper 1.0.0 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: 4 months ago - 1 maintainer
@myagizmaktav/sitemapper 3.2.8 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: over 1 year ago - 58 downloads last month - 0 stars on GitHub - 1 maintainer
wwweb 1.0.3 removed
An autonomous webcrawler for indexing robots.txt files.4 versions - Latest release: almost 8 years ago - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
crawl-cli 0.2.0 removed
A Node crawler/scrape for retrieving data from websites11 versions - Latest release: about 7 years ago - 1 dependent package - 1 dependent repositories - 13 downloads last month - 1 maintainer
Related Keywords
crawler
18
javascript
14
parse
14
sitemap
14
robots.txt
14
xml
13
sitemaps
13
crawlers
13
nodejs
12
scraper
11
spider
7
web-crawler
7
scraping
4
scrape
4
cheerio
4
html
3
sitemap-xml
3
webscraper
3
web
3
node
3
search
2
axios
2
ptt
2
bahamut
2
baha
2
puppeteer
2
js
2
headless
2
webscraping
2
url
2
parser
2
scrapper
2
webspider
2
href
1
div
1
p
1
extract
1
title
1
body
1
head
1
footer
1
dom
1
crawl
1
link
1
links
1
find
1
tag
1
get
1
traffic
1
siege
1
load tester
1
load testing
1
SEO
1
phantomjs
1
react
1
universal
1
isomorphic
1
playwright
1
pupetter
1
nsfw
1
hentai
1
github
1
readme
1
webcrawer
1
redis
1
graph
1
redis-client
1
manga
1
downloader
1
software-agent
1
autonomous
1
bot
1
robots
1
expressjs
1
request
1
code
1
Behance
1
Node
1
api
1
behance
1
npm
1
nodejs crawler
1
node crawler
1
mensa
1
cantina
1
food
1
student
1
bfh
1
university
1
just for fun
1
myreadingmanga
1
containers
1
web-bot
1
url-scrapper
1
scrap
1
urlparser
1
scanner
1
urlscrapping
1
bots
1
webcrawling
1