Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
npmjs.org "robots.txt" keyword
@kerouac/robots 0.0.2 💰
Kerouac robots exclusion components for Bixby.js.2 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 4 downloads last month - 1 stars on GitHub - 1 maintainer
robots-parser-combinator 1.1.0
A proper robots.txt parser and combinator that works with eulalie2 versions - Latest release: over 8 years ago - 1 dependent repositories - 7 downloads last month - 0 stars on GitHub - 1 maintainer
@yeskiy/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - 31 downloads last month - 0 stars on GitHub - 1 maintainer
express-robots-middleware 1.0.1
Middleware for Express to add a robots.txt1 version - Latest release: almost 8 years ago - 1 dependent package - 1 dependent repositories - 238 downloads last month - 2 stars on GitHub - 1 maintainer
Top 5.2% on npmjs.org
155 versions - Latest release: 23 days ago - 2 dependent packages - 14 dependent repositories - 937 downloads last month - 125 stars on GitHub - 1 maintainer
html-snapshots 3.8.2
A selector-based html snapshot tool using Puppeteer or PhantomJS that sources sitemap.xml, robots...155 versions - Latest release: 23 days ago - 2 dependent packages - 14 dependent repositories - 937 downloads last month - 125 stars on GitHub - 1 maintainer
@wmod/sitemapper 3.2.11
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: 4 months ago - 11 downloads last month - 0 stars on GitHub - 1 maintainer
robot-apocalypse 0.1.2
Install robots.txt files on every site on a staging server, taking into account both PHP and node...3 versions - Latest release: over 6 years ago - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
@itsmatteomanf/astro-robots-txt 0.1.2
Generate a robots.txt file for your Astro site3 versions - Latest release: 22 days ago - 179 downloads last month - 0 stars on GitHub - 1 maintainer
robots-txt-component 0.0.4
Lightweight robots.txt parsing component without any external dependencies for Node.js.3 versions - Latest release: over 1 year ago - 23 downloads last month - 0 stars on GitHub - 1 maintainer
nextjs-seo-manager 1.0.3
## About186 versions - Latest release: over 1 year ago - 3 dependent packages - 385 downloads last month - 0 stars on GitHub - 1 maintainer
koa-robotstxt 1.0.10
Robots.txt bounce middleware for koa8 versions - Latest release: about 7 years ago - 1 dependent package - 1 dependent repositories - 287 downloads last month - 1 stars on GitHub - 1 maintainer
Top 8.9% on npmjs.org
25 versions - Latest release: about 4 years ago - 2 dependent packages - 1 dependent repositories - 313 downloads last month - 10 stars on GitHub - 1 maintainer
next-with-sitemap 2.5.2
Higher order Next.js configuration for generating sitemap.xml and robots.txt25 versions - Latest release: about 4 years ago - 2 dependent packages - 1 dependent repositories - 313 downloads last month - 10 stars on GitHub - 1 maintainer
webcheck-robots 0.1.0
A plugin for Robots Exclusion Standard for webcheck2 versions - Latest release: about 8 years ago - 1 dependent repositories - 7 downloads last month - 0 stars on GitHub - 1 maintainer
Top 1.9% on npmjs.org
11 versions - Latest release: over 1 year ago - 45 dependent packages - 5,176 dependent repositories - 3.38 million downloads last month - 137 stars on GitHub - 1 maintainer
robots-parser 3.0.1
A specification compliant robots.txt parser with wildcard (*) matching support.11 versions - Latest release: over 1 year ago - 45 dependent packages - 5,176 dependent repositories - 3.38 million downloads last month - 137 stars on GitHub - 1 maintainer
robot-webpack-plugin 0.1.0
robots.txt generation for webpack 31 version - Latest release: almost 6 years ago - 1 dependent repositories - 5 downloads last month - 0 stars on GitHub - 1 maintainer
@drorgl/sitemapper 2.1.17
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: almost 4 years ago - 1 dependent package - 1 dependent repositories - 12 downloads last month - 0 stars on GitHub - 1 maintainer
Top 2.5% on npmjs.org
28 versions - Latest release: about 4 years ago - 15 dependent packages - 4,534 dependent repositories - 427 thousand downloads last month - 64 stars on GitHub - 3 maintainers
generate-robotstxt 8.0.3
Awesome generator robots.txt28 versions - Latest release: about 4 years ago - 15 dependent packages - 4,534 dependent repositories - 427 thousand downloads last month - 64 stars on GitHub - 3 maintainers
Top 3.4% on npmjs.org
13 versions - Latest release: almost 2 years ago - 17 dependent packages - 396 dependent repositories - 77.6 thousand downloads last month - 35 stars on GitHub - 3 maintainers
robotstxt-webpack-plugin 8.0.0
A webpack plugin to output a robots.txt file13 versions - Latest release: almost 2 years ago - 17 dependent packages - 396 dependent repositories - 77.6 thousand downloads last month - 35 stars on GitHub - 3 maintainers
Top 9.9% on npmjs.org
3 versions - Latest release: 6 months ago - 2 dependent packages - 3 dependent repositories - 101 downloads last month - 5 stars on GitHub - 1 maintainer
cyborg.txt 0.3.0
robots.txt utilities3 versions - Latest release: 6 months ago - 2 dependent packages - 3 dependent repositories - 101 downloads last month - 5 stars on GitHub - 1 maintainer
parse-robots 3.2.0
Promise based parser for robots.txt files.11 versions - Latest release: about 4 years ago - 1 dependent package - 1 dependent repositories - 106 downloads last month - 3 stars on GitHub - 1 maintainer
Top 3.2% on npmjs.org
6 versions - Latest release: almost 3 years ago - 14 dependent packages - 1,143 dependent repositories - 86.1 thousand downloads last month - 14 stars on GitHub - 1 maintainer
express-robots-txt 1.0.0
Express middleware to serve and generate robots.txt6 versions - Latest release: almost 3 years ago - 14 dependent packages - 1,143 dependent repositories - 86.1 thousand downloads last month - 14 stars on GitHub - 1 maintainer
@deskeen/web-builder-create-robots-txt 0.2.0
Web Builder Module - Create robots.txt file2 versions - Latest release: about 3 years ago - 6 downloads last month - 0 stars on GitHub - 1 maintainer
robots-agent 1.0.0 💰
robots.txt agent with cache.1 version - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 5 downloads last month - 2 stars on GitHub - 1 maintainer
Top 6.5% on npmjs.org
33 versions - Latest release: 9 months ago - 8 dependent packages - 104 dependent repositories - 39.1 thousand downloads last month - 97 stars on GitHub - 1 maintainer
astro-robots-txt 1.0.0
Generate a robots.txt for Astro33 versions - Latest release: 9 months ago - 8 dependent packages - 104 dependent repositories - 39.1 thousand downloads last month - 97 stars on GitHub - 1 maintainer
@mixmaxhq/express-robots 1.0.0
Express middleware for generating your robots.txt2 versions - Latest release: over 4 years ago - 1 dependent package - 1.37 thousand downloads last month - 1 stars on GitHub - 28 maintainers
@qbbsh/sitemapper
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - 98 stars on GitHub - 1 maintainer
env-robotstxt 1.0.4
generate robots.txt based on env content for static websites5 versions - Latest release: over 1 year ago - 1 dependent package - 44 downloads last month - 0 stars on GitHub - 1 maintainer
@firstandthird/robots 1.0.0
robots.txt text generator1 version - Latest release: over 4 years ago - 2 dependent packages - 6 downloads last month - 6 maintainers
Top 7.1% on npmjs.org
89 versions - Latest release: 27 days ago - 2 dependent packages - 10 dependent repositories - 360 downloads last month - 16 stars on GitHub - 1 maintainer
grunt-html-snapshots 7.8.0
The grunt task for html-snapshots89 versions - Latest release: 27 days ago - 2 dependent packages - 10 dependent repositories - 360 downloads last month - 16 stars on GitHub - 1 maintainer
highlightjs-robots-txt 0.9.1
highlight.js syntax definition for Google's robot.txt format2 versions - Latest release: over 4 years ago - 1 dependent package - 1 dependent repositories - 303 downloads last month - 5 stars on GitHub - 1 maintainer
adibenmat-sitemapper 2.1.19
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: about 3 years ago - 2 dependent packages - 10 downloads last month - 0 stars on GitHub - 1 maintainer
bot-detector 1.0.10
It uses the user-agents.org xml file for detecting bots.11 versions - Latest release: almost 9 years ago - 1 dependent package - 1 dependent repositories - 141 downloads last month - 1 maintainer
sitemapper_mos 3.2.9
Parser for XML Sitemaps to be used with Robots.txt and web crawlers6 versions - Latest release: about 2 years ago - 1 dependent package - 119 downloads last month - 0 stars on GitHub - 1 maintainer
@ryan-sandy/no-robots 1.0.3
Deney all robots express middleware4 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 9 downloads last month - 0 stars on GitHub - 1 maintainer
@mastixmc/sitemapper 3.2.0
Parser for XML Sitemaps to be used with Robots.txt and web crawlers. (Extended version by mastixmc)3 versions - Latest release: about 5 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 maintainer
@api.global/typedserver 3.0.37
A TypeScript-based project for easy serving of static files with support for live reloading, comp...36 versions - Latest release: 21 days ago - 11 dependent packages - 513 downloads last month - 1 maintainer
koa-robots.txt 1.0.0
koa middleware to handle different robots.txt responses on different domains1 version - Latest release: about 8 years ago - 3 dependent packages - 2 dependent repositories - 34 downloads last month - 0 stars on GitHub - 1 maintainer
Top 4.4% on npmjs.org
6 versions - Latest release: about 7 years ago - 7 dependent packages - 3,977 dependent repositories - 49.1 thousand downloads last month - 4 stars on GitHub - 1 maintainer
robot-directives 0.4.0
Parse robot directives within HTML meta and/or HTTP headers.6 versions - Latest release: about 7 years ago - 7 dependent packages - 3,977 dependent repositories - 49.1 thousand downloads last month - 4 stars on GitHub - 1 maintainer
@jamsite/plugin-robots-txt 0.0.2
Creates robots.txt page2 versions - Latest release: over 4 years ago - 1 dependent package - 5 downloads last month - 0 stars on GitHub - 1 maintainer
@cabbiepete/sitemapper 3.0.2
Parser for XML Sitemaps to be used with Robots.txt and web crawlers3 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 10 downloads last month - 1 stars on GitHub - 1 maintainer
eleventy-plugin-robotstxt 0.0.4
An Eleventy plugin to generate a robots.txt file for your static site4 versions - Latest release: about 2 months ago - 58 downloads last month - 1 stars on GitHub - 1 maintainer
Top 6.4% on npmjs.org
17 versions - Latest release: about 2 years ago - 2 dependent packages - 9 dependent repositories - 2.09 thousand downloads last month - 11 stars on GitHub - 1 maintainer
robots-txt-parser 2.0.3
A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.17 versions - Latest release: about 2 years ago - 2 dependent packages - 9 dependent repositories - 2.09 thousand downloads last month - 11 stars on GitHub - 1 maintainer
@passmarked/seo 1.0.9
Rules related to checking for any SEO issues on the page given8 versions - Latest release: over 7 years ago - 2 dependent packages - 1 dependent repositories - 19 downloads last month - 10 stars on GitHub - 1 maintainer
@ruhith/generate-robots-text 1.0.1
Generate Robots.txt file with your custom configs1 version - Latest release: almost 3 years ago - 1 dependent package - 3 downloads last month - 0 stars on GitHub - 1 maintainer
@crawlify.io/robots 1.0.1
Parses robots.txt files to provide meaningful, useful output as well as reporting syntax errors.2 versions - Latest release: over 5 years ago - 1 dependent package - 1 dependent repositories - 6 downloads last month - 1 stars on GitHub - 2 maintainers
@context-labs/sitemapper 3.2.10
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: about 1 year ago - 4 dependent packages - 736 downloads last month - 98 stars on GitHub - 3 maintainers
vite-plugin-robots 1.0.5 💰
Vite plugin for generating robots.txt for production and development builds6 versions - Latest release: 3 months ago - 968 downloads last month - 4 stars on GitHub - 1 maintainer
@public-square/sitemapper 3.2.5
Parser for XML Sitemaps to be used with Robots.txt and web crawlers2 versions - Latest release: almost 2 years ago - 2 dependent packages - 1 dependent repositories - 9 downloads last month - 98 stars on GitHub - 2 maintainers
Top 2.3% on npmjs.org
44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
sitemapper 3.2.8
Parser for XML Sitemaps to be used with Robots.txt and web crawlers44 versions - Latest release: 7 months ago - 37 dependent packages - 76 dependent repositories - 88.6 thousand downloads last month - 98 stars on GitHub - 1 maintainer
Top 5.8% on npmjs.org
8 versions - Latest release: about 8 years ago - 2 dependent packages - 22 dependent repositories - 1.53 thousand downloads last month - 16 stars on GitHub - 1 maintainer
express-robots 0.2.0
Express middleware for generating your robots.txt8 versions - Latest release: about 8 years ago - 2 dependent packages - 22 dependent repositories - 1.53 thousand downloads last month - 16 stars on GitHub - 1 maintainer
express-seo-robots 1.0.2
Simple robots.txt file generation for express3 versions - Latest release: over 2 years ago - 1 dependent package - 1.83 thousand downloads last month - 0 stars on GitHub - 1 maintainer
Top 3.9% on npmjs.org
19 versions - Latest release: almost 7 years ago - 18 dependent packages - 44 dependent repositories - 612 downloads last month - 66 stars on GitHub - 1 maintainer
robots 0.10.1
Parser for robots.txt19 versions - Latest release: almost 7 years ago - 18 dependent packages - 44 dependent repositories - 612 downloads last month - 66 stars on GitHub - 1 maintainer
robots-txt 1.0.0
Check user agents against robots.txt files9 versions - Latest release: almost 8 years ago - 1 dependent package - 1 dependent repositories - 20 downloads last month - 7 stars on GitHub - 4 maintainers
kerouac-robotstxt 0.7.0 💰
robots.txt middleware for Kerouac.9 versions - Latest release: 3 months ago - 2 dependent packages - 5 dependent repositories - 16 downloads last month - 0 stars on GitHub - 1 maintainer
robotstxt 0.0.4-7
a robotstxt parser for node.js17 versions - Latest release: over 11 years ago - 1 dependent package - 2 dependent repositories - 56 downloads last month - 1 maintainer
@jarronb/site-mapper 1.0.5
Site mapper utilizing puppeteer to generate a xml sitemap6 versions - Latest release: almost 4 years ago - 1 dependent package - 7 downloads last month - 0 stars on GitHub - 1 maintainer
Top 9.3% on npmjs.org
2 versions - Latest release: almost 5 years ago - 1 dependent package - 94 dependent repositories - 1.24 thousand downloads last month - 8 stars on GitHub - 1 maintainer
vuepress-plugin-robots 1.0.1
Robots.txt file generator plugin for vuepress2 versions - Latest release: almost 5 years ago - 1 dependent package - 94 dependent repositories - 1.24 thousand downloads last month - 8 stars on GitHub - 1 maintainer
@haroldadmin/robots-parser 1.0.0
Parser for robots.txt1 version - Latest release: 2 months ago - 21 downloads last month - 1 maintainer
express-robots.txt 1.0.3
Express middleware to handle different robots.txt responses on different domains4 versions - Latest release: almost 9 years ago - 1 dependent package - 1 dependent repositories - 291 downloads last month - 0 stars on GitHub - 1 maintainer
crawlmap 0.3.1 deprecated
A sitemapping tool for crawlers5 versions - Latest release: over 7 years ago - 1 dependent package - 1 dependent repositories - 8 downloads last month - 0 stars on GitHub - 1 maintainer
@wmod/sitemap-url-scrapper 1.0.0 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: 4 months ago - 1 maintainer
@myagizmaktav/sitemapper 3.2.8 removed
Parser for XML Sitemaps to be used with Robots.txt and web crawlers1 version - Latest release: over 1 year ago - 58 downloads last month - 0 stars on GitHub - 1 maintainer
robogen 0.0.2 removed
Robots.txt generator done right.2 versions - Latest release: over 7 years ago - 1 dependent repositories - 15 downloads last month - 3 stars on GitHub - 1 maintainer
sitemap-and-robots 1.0.1 unpublished
XML sitemap and robots.txt generator plugin for Node Reactor CMS.2 versions - Latest release: over 4 years ago
Related Keywords
robots
23
sitemap
23
sitemaps
17
crawlers
17
parse
16
xml
15
webcrawler
15
robots-txt
14
seo
11
javascript
8
parser
7
robotstxt
6
express
6
html
5
sitemap-xml
5
nodejs
4
crawler
4
robots-parser
3
vue
3
robots-exclusion-standard
3
user-agent
3
scraper
3
generate
3
crawl
3
spider
2
bot
2
robot
2
webpack
2
txt
2
koa
2
cli
2
robots-generator
2
nextjs
2
web
2
meta
2
middelware
2
spiders
2
html-snapshots
2
sitemap.xml
2
astro-integration
2
middleware
2
SEO
2
astro-component
2
selector
2
kerouac
2
astro
2
snapshots
2
react
2
node-js
2
nofollow
1
suite
1
metadata
1
bing
1
http
1
header
1
google
1
compression (gzip, deflate, brotli)
1
feeds
1
proxy
1
express middleware
1
cors
1
SSL
1
HTTP server
1
typed requests
1
compression
1
live reload
1
static file serving
1
TypeScript
1
no
1
test
1
standards
1
security
1
passmark
1
passmarked
1
robots-txt-parser
1
vite
1
vite-plugin
1
angular
1
11ty
1
vuepress-plugin
1
vuepress
1
robots.txt generator for vuepress
1
eleventy
1
generator
1
eleventy-plugin
1
jamstack-plugin
1
jamstack
1
headers
1
npm-package
1
attributes
1
noindex
1
police
1
webpack3
1
webcheck
1
next.js
1
structured data
1
file management
1
ssr
1
gatsby
1
next
1