Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
npmjs.org "webcrawler" keyword
algolia-webcrawler 3.2.3
Simple node worker that crawls sitemaps in order to keep an algolia index up-to-date.28 versions - Latest release: over 4 years ago - 2 dependent packages - 2 dependent repositories - 87 downloads last month - 46 stars on GitHub - 2 maintainers
@waynechang65/ptt-crawler 2.6.0
A web crawler module designed to scarp data from Ptt.17 versions - Latest release: 4 months ago - 1 dependent package - 3 dependent repositories - 47 downloads last month - 20 stars on GitHub - 1 maintainer
fetchyt 3.0.7
Search musics on youtube and return your links15 versions - Latest release: almost 3 years ago - 2 dependent packages - 7 downloads last month - 2 stars on GitHub - 1 maintainer
url-scraper 1.0.2
Url scraper which takes the text input and finds the links/urls, scraps them using cheerio and wi...3 versions - Latest release: over 7 years ago - 1 dependent repositories - 29 downloads last month - 5 stars on GitHub - 1 maintainer
bing-me-links 1.0.0
Scrapes Baidu, Bing, StartPage, Yahoo, Webcrawler, Qwant and retrieves links in serps13 versions - Latest release: about 7 years ago - 1 dependent package - 1 dependent repositories - 3 downloads last month - 3 stars on GitHub - 1 maintainer
@qbbsh/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - 98 stars on GitHub - 1 maintainer
adibenmat-sitemapper 2.1.19
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: about 3 years ago - 2 dependent packages - 10 downloads last month - 0 stars on GitHub - 1 maintainer
askro 0.1.6
This tool allow you to parse, collect and traverse through the radiation monitoring data provided...6 versions - Latest release: over 9 years ago - 2 dependent packages - 1 dependent repositories - 11 downloads last month - 0 stars on GitHub - 1 maintainer
spiderboi 1.1.0 π°
A web crawler written in TypeScript.3 versions - Latest release: over 5 years ago - 1 dependent repositories - 10 downloads last month - 5 stars on GitHub - 1 maintainer
scrapefrom 2.5.1
Scrape data from any webpage.70 versions - Latest release: 2 months ago - 2 dependent packages - 133 downloads last month - 4 stars on GitHub - 1 maintainer
sitemapper_mos 3.2.9
Parser for XML Sitemaps to be used with Robots.txt and web crawlers6 versions - Latest release: about 2 years ago - 1 dependent package - 119 downloads last month - 0 stars on GitHub - 1 maintainer
@mastixmc/sitemapper 3.2.0
Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)3 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 maintainer
spiking 2.1.0
A lightweight web crawler.2 versions - Latest release: over 4 years ago - 1 dependent repositories - 7 downloads last month - 11 stars on GitHub - 1 maintainer
trackpurchase 2.0.0
λ¨ λͺμ€μ μ½λλ‘ λ€μν μΌν νλ«νΌμμ κ²°μ λ΄μμ κΈμ΄μ€μ!19 versions - Latest release: over 2 years ago - 49 downloads last month - 21 stars on GitHub - 1 maintainer
Top 3.0% on npmjs.org
3 versions - Latest release: almost 13 years ago - 16 dependent packages - 92 dependent repositories - 6.91 thousand downloads last month - 92 stars on GitHub - 2 maintainers
http-agent 0.1.2
A simple agent for performing a sequence of http requests in node.js3 versions - Latest release: almost 13 years ago - 16 dependent packages - 92 dependent repositories - 6.91 thousand downloads last month - 92 stars on GitHub - 2 maintainers
@cabbiepete/sitemapper 3.0.2
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 10 downloads last month - 1 stars on GitHub - 1 maintainer
@context-labs/sitemapper 3.2.10
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: 12 months ago - 4 dependent packages - 736 downloads last month - 98 stars on GitHub - 3 maintainers
spider_jar 1.0.0 unpublished
A simple pack for writting webscrawles that run inside containers1 version - Latest release: about 1 year ago - 0 stars on GitHub - 1 maintainer
@public-square/sitemapper 3.2.5
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: almost 2 years ago - 2 dependent packages - 1 dependent repositories - 9 downloads last month - 98 stars on GitHub - 2 maintainers
Top 2.3% on npmjs.org
44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
sitemapper 3.2.8
Parser for XML Sitemaps to be used with Robots.txt and web crawlers44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
hellobot 1.2.3
A friendly javascript pre-rendering engine - BETA (UNSTABLE)39 versions - Latest release: about 6 years ago - 1 dependent package - 1 dependent repositories - 44 downloads last month - 2 stars on GitHub - 1 maintainer
orbweaver 0.1.4
A simple webcrawler that prints out the URLs of the pages it encounters. Runs in parallel, up to ...5 versions - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 13 downloads last month - 5 stars on GitHub - 1 maintainer
htmlgetty 1.0.1
This is a function that accepts 3 arguments, "url", "tag" and "output", and writes to a file, in ...2 versions - Latest release: almost 2 years ago - 1 dependent package - 18 downloads last month - 0 stars on GitHub - 1 maintainer
behance-node 1.0.0
An asynchronous module for scrapping Behance using JavaScript.4 versions - Latest release: 3 months ago - 1 dependent package - 1 dependent repositories - 41 downloads last month - 2 stars on GitHub - 1 maintainer
schabbi-webscraper 1.2.1
Lightweight and easy to use crawling solution for websites.28 versions - Latest release: over 1 year ago - 57 downloads last month - 0 stars on GitHub - 1 maintainer
bfh-mensa 1.0.3
Webcrawler script to retrieve the daily menu of the Bern University of Applied Sciences cantina i...1 version - Latest release: over 4 years ago - 1 dependent package - 1 dependent repositories - 9 downloads last month - 4 stars on GitHub - 1 maintainer
@jarronb/site-mapper 1.0.5
Site mapper utilizing puppeteer to generate a xml sitemap6 versions - Latest release: almost 4 years ago - 1 dependent package - 7 downloads last month - 0 stars on GitHub - 1 maintainer
@wmod/sitemapper 3.2.11
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: 4 months ago - 31 downloads last month - 0 stars on GitHub - 1 maintainer
myreadingmanga-unofficial 0.0.10
MyReadingManga WebCrawler. Find By Languages, Genres and Tags.8 versions - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 14 downloads last month - 1 maintainer
readme-crawler 1.0.5 deprecated
Download README files from GitHub repository links6 versions - Latest release: almost 3 years ago - 1 dependent package - 14 downloads last month - 4 stars on GitHub - 1 maintainer
zoinks 1.1.8
Crawls through provided website, checking for 200 response, content load, ssl cert errors, and more!19 versions - Latest release: over 3 years ago - 1 dependent package - 1 dependent repositories - 39 downloads last month - 1 stars on GitHub - 1 maintainer
@drorgl/sitemapper 2.1.17
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: almost 4 years ago - 1 dependent package - 1 dependent repositories - 14 downloads last month - 0 stars on GitHub - 1 maintainer
@rodrigogs/spiderman 0.0.0
Yet to describe1 version - Latest release: about 2 years ago - 2 dependent packages - 4 downloads last month - 0 stars on GitHub - 1 maintainer
nightmare-screenshot-selector 1.0.2
A Nightmare plugin to easily take screenshots.3 versions - Latest release: about 7 years ago - 3 dependent packages - 10 dependent repositories - 33 downloads last month - 4 stars on GitHub - 1 maintainer
@waynechang65/baha-crawler 2.6.0
A web crawler module designed to scarp data from Bahamut forum.13 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 11 downloads last month - 3 stars on GitHub - 1 maintainer
redis-web-crawler 1.0.0 deprecated
Web Crawler to create directed graph of links among connected sites. Runs with Node.js and stores...1 version - Latest release: almost 3 years ago - 5 downloads last month - 1 stars on GitHub - 1 maintainer
baha-crawler 2.0.0 deprecated
A web crawler module designed to scarp data from Bahamut forum.1 version - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 3 downloads last month - 3 stars on GitHub - 1 maintainer
le-manga 0.0.6 deprecated
CLI for download manga and serve it locally.6 versions - Latest release: about 10 years ago - 1 dependent package - 1 dependent repositories - 7 downloads last month - 1 maintainer
ptt-crawler.js 2.0.0 deprecated
A web crawler module designed to scarp data from Ptt.1 version - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 2 downloads last month - 20 stars on GitHub - 1 maintainer
crawlmap 0.3.1 deprecated
A sitemapping tool for crawlers5 versions - Latest release: over 7 years ago - 1 dependent package - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
@wmod/sitemap-url-scrapper 1.0.0 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: 4 months ago - 1 maintainer
@myagizmaktav/sitemapper 3.2.8 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: about 1 year ago - 58 downloads last month - 0 stars on GitHub - 1 maintainer
wwweb 1.0.3 removed
An autonomous webcrawler for indexing robots.txt files.4 versions - Latest release: almost 8 years ago - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
crawl-cli 0.2.0 removed
A Node crawler/scrape for retrieving data from websites11 versions - Latest release: about 7 years ago - 1 dependent package - 1 dependent repositories - 13 downloads last month - 1 maintainer
Related Keywords
crawler
18
robots.txt
14
parse
14
sitemap
14
javascript
14
sitemaps
13
crawlers
13
xml
13
nodejs
12
scraper
11
spider
7
web-crawler
7
scrape
4
cheerio
4
scraping
4
web
3
node
3
webscraper
3
sitemap-xml
3
html
3
js
2
baha
2
webspider
2
scrapper
2
webscraping
2
puppeteer
2
bahamut
2
parser
2
url
2
headless
2
axios
2
ptt
2
search
2
api
1
behance
1
npm
1
nodejs crawler
1
node crawler
1
mensa
1
Node
1
cantina
1
get
1
tag
1
find
1
extract
1
href
1
div
1
p
1
title
1
body
1
head
1
footer
1
dom
1
crawl
1
link
1
links
1
code
1
Behance
1
request
1
expressjs
1
robots
1
bot
1
autonomous
1
software-agent
1
downloader
1
manga
1
redis-client
1
graph
1
redis
1
nightmarejs
1
headless-browsers
1
selector
1
screenshot
1
plugin
1
nightmare
1
framework
1
webscrapper
1
test
1
zoinks
1
webcrawer
1
readme
1
github
1
hentai
1
nsfw
1
myreadingmanga
1
just for fun
1
university
1
bfh
1
student
1
food
1
ΠΠ‘ΠΠ Π
1
SARMS
1
askro
1
radiation level
1
opendata
1
yahoo
1
startpage
1
bing
1
baidu
1
url-scraper
1