Ecosyste.ms: Packages
An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.
rubygems.org "crawler" keyword
ronin-web-spider 0.1.0 💰
ronin-web-spider is a collection of common web spidering routines using the spidr gem.3 versions - Latest release: over 1 year ago - 1 dependent package - 146 dependent repositories - 5.06 thousand downloads total - 7 stars on GitHub - 1 maintainer
tors 0.5.0
Yet another torrent searching application for your command line. But this has an option for autom...7 versions - Latest release: over 6 years ago - 11.7 thousand downloads total - 46 stars on GitHub - 1 maintainer
mwcrawler 0.1.2
Essa gema provê uma api ruby para se fazer o scrapping de páginas html do sistema matricula web e...2 versions - Latest release: almost 4 years ago - 1 dependent repositories - 3.13 thousand downloads total - 4 stars on GitHub - 1 maintainer
amber-kit 0.0.5
Useful network toolkit for create your application3 versions - Latest release: almost 7 years ago - 6.7 thousand downloads total - 5 stars on GitHub - 1 maintainer
tanakai 1.7.3
Maintained fork of Kimurai, a modern web scraping framework written in Ruby and based on Capybara...7 versions - Latest release: 5 months ago - 1 dependent repositories - 15.1 thousand downloads total - 260 stars on GitHub - 1 maintainer
Top 8.2% on rubygems.org
16 versions - Latest release: about 1 year ago - 3 dependent packages - 18 dependent repositories - 35.1 thousand downloads total - 41 stars on GitHub - 1 maintainer
ronin-web 1.0.2 💰
ronin-web is a Ruby library that provides common web security commands and additional libraries.16 versions - Latest release: about 1 year ago - 3 dependent packages - 18 dependent repositories - 35.1 thousand downloads total - 41 stars on GitHub - 1 maintainer
dmm-crawler 0.4.5
Show DMM and DMM.R18's crawled data. e.g. ranking32 versions - Latest release: almost 5 years ago - 1 dependent package - 1 dependent repositories - 58.2 thousand downloads total - 1 stars on GitHub - 1 maintainer
logstash-input-crawler 1.0.0
This plugin need set the initial url.1 version - Latest release: almost 6 years ago - 1.97 thousand downloads total - 1 stars on GitHub - 1 maintainer
proxycrawl 1.0.2
Ruby based client for the ProxyCrawl API that helps developers crawl or scrape thousands of web p...8 versions - Latest release: 11 months ago - 2 dependent repositories - 1.59 million downloads total - 14 stars on GitHub - 1 maintainer
grucrawler 0.0.5
Simple crawler using Redis as backend4 versions - Latest release: over 9 years ago - 10.4 thousand downloads total - 0 stars on GitHub - 1 maintainer
active_proxy 1.0.2
Easy to use ruby proxy fetcher, supports caching and retries3 versions - Latest release: about 5 years ago - 1 dependent repositories - 5.82 thousand downloads total - 1 stars on GitHub - 1 maintainer
gcrawler 0.1.2
Crawling link text and link url by keywords on Google.com.3 versions - Latest release: over 1 year ago - 1 dependent repositories - 1.56 thousand downloads total - 1 stars on GitHub - 1 maintainer
spiderman 2.0.0
your friendly neighborhood web crawler1 version - Latest release: about 4 years ago - 1 dependent repositories - 4.53 thousand downloads total - 17 stars on GitHub - 1 maintainer
Top 2.9% on rubygems.org
34 versions - Latest release: over 1 year ago - 5 dependent packages - 55 dependent repositories - 205 thousand downloads total - 1,303 stars on GitHub - 1 maintainer
wombat 3.0.0 💰
Generic Web crawler with a DSL that parses structured data from web pages34 versions - Latest release: over 1 year ago - 5 dependent packages - 55 dependent repositories - 205 thousand downloads total - 1,303 stars on GitHub - 1 maintainer
event-crawler 0.1.0 💰
Generic Web crawler with a DSL that parses event-related data from web pages1 version - Latest release: over 12 years ago - 4.5 thousand downloads total - 1,303 stars on GitHub - 1 maintainer
spidr_epg 1.0.0 💰
Spidr is a versatile Ruby web spidering library that can spider a site, multiple domains, certain...1 version - Latest release: about 11 years ago - 3.63 thousand downloads total - 792 stars on GitHub - 1 maintainer
Top 2.8% on rubygems.org
28 versions - Latest release: 4 months ago - 15 dependent packages - 89 dependent repositories - 290 thousand downloads total - 792 stars on GitHub - 1 maintainer
spidr 0.7.1 💰
Spidr is a versatile Ruby web spidering library that can spider a site, multiple domains, certain...28 versions - Latest release: 4 months ago - 15 dependent packages - 89 dependent repositories - 290 thousand downloads total - 792 stars on GitHub - 1 maintainer
Top 3.6% on rubygems.org
8 versions - Latest release: over 5 years ago - 3 dependent packages - 80 dependent repositories - 155 thousand downloads total - 999 stars on GitHub - 1 maintainer
kimurai 1.4.0
Modern web scraping framework written in Ruby and based on Capybara/Nokogiri8 versions - Latest release: over 5 years ago - 3 dependent packages - 80 dependent repositories - 155 thousand downloads total - 999 stars on GitHub - 1 maintainer
fluent-plugin-github-activities 0.7.0
This provides ability to crawl public activities of users.8 versions - Latest release: about 7 years ago - 18 thousand downloads total - 3 stars on GitHub - 2 maintainers
daimon_skycrawlers 1.0.0
This is a crawler framework.21 versions - Latest release: over 7 years ago - 1 dependent repositories - 41 thousand downloads total - 1 stars on GitHub - 2 maintainers
snapcrawl 0.5.4 💰
Snapcrawl is a command line utility for crawling a website and saving screenshots.27 versions - Latest release: 10 months ago - 42.8 thousand downloads total - 55 stars on GitHub - 1 maintainer
instagram-crawler 0.3.0
Crawl instagram photos, posts and videos for download.4 versions - Latest release: about 5 years ago - 2 dependent repositories - 7.33 thousand downloads total - 196 stars on GitHub - 1 maintainer
spidercrawl 0.3.9
With the help of Nokogiri, SpiderCrawl will parse each page and return you its title, links, css,...1 version - Latest release: about 8 years ago - 1 dependent package - 1 dependent repositories - 2.69 thousand downloads total - 0 stars on GitHub - 1 maintainer
metabypass 1.0.1
Metabypass | Ruby-based easy implementation for solving any type of captcha by Metabypass2 versions - Latest release: 11 months ago - 369 downloads total - 7 stars on GitHub - 1 maintainer
site_health 0.2.0
Crawl a site and check various health indicators, such as: HTTP 4XX, 5XX status, valid HTML/XML/J...2 versions - Latest release: over 5 years ago - 3.57 thousand downloads total - 1 stars on GitHub - 1 maintainer
sjc_bus_schedule 0.0.2
Makes life easy for who wants to get the bus schedule from SJC website2 versions - Latest release: over 4 years ago - 1 dependent repositories - 3.99 thousand downloads total - 0 stars on GitHub - 1 maintainer
Top 6.4% on rubygems.org
26 versions - Latest release: about 2 months ago - 1 dependent package - 10 dependent repositories - 1.01 million downloads total - 108 stars on GitHub - 1 maintainer
crawler_detect 1.2.4
CrawlerDetect is a library to detect bots/crawlers via the user agent26 versions - Latest release: about 2 months ago - 1 dependent package - 10 dependent repositories - 1.01 million downloads total - 108 stars on GitHub - 1 maintainer
proxy_manager 1.0.1 💰
This is gem for easy usage proxy in your parsers/web-bots. It will manage your proxy list...9 versions - Latest release: about 10 years ago - 1 dependent repositories - 22.4 thousand downloads total - 14 stars on GitHub - 1 maintainer
scrapula 0.6.3
Scrapula is a library for scraping web pages that simplifies some of the common actions t...1 version - Latest release: over 8 years ago - 1 dependent repositories - 2.93 thousand downloads total - 1 stars on GitHub - 1 maintainer
stupid_crawler 0.2.1
Stupid crawler that looks for URLs on a given site. Result is saved as two CSV files one with fou...2 versions - Latest release: over 6 years ago - 3.7 thousand downloads total - 1 stars on GitHub - 1 maintainer
Related Keywords
ruby
21
scraper
10
spider
6
web
5
gem
4
web-scraping
4
proxy
3
web-crawler
3
rubygems
2
http
2
cli
2
webscraping
2
scrapy
2
kimurai
2
nokogiri
2
web-spider
2
rubygem
2
web-scraper
2
dsl
2
spider-links
2
fluentd-plugin
1
fluentd
1
headless-chrome
1
github-api
1
capture
1
screenshot
1
instagram
1
instagram-crawler
1
instagram-scraper
1
2captcha
1
parser
1
crawler-detection
1
bots
1
bot-detection
1
site-health
1
ruby-captcha
1
recaptcha-v3-solver
1
recaptcha-v2-solver
1
recaptcha-solver
1
image-captcha
1
google-recaptcha-solver
1
google-captcha
1
deathbycaptcha
1
captcha-solving
1
captcha-recognition
1
captcha-breaking
1
bypass-recaptcha
1
anticaptcha
1
ai-captcha
1
webcrawler
1
server
1
ronin-rb
1
proxy-server
1
html
1
helpers
1
hacktoberfest
1
chrome-headless
1
middleware
1
http-server
1
torrent-search-engine
1
torrent-downloader
1
ruby-gem
1
websecurity
1
utils
1
recon
1
infosec
1
spider-framework
1
httprb
1
crawler-engine
1
google
1
crawling
1
rails
1
scraping
1
proxycrawl
1
api
1
logstash-plugin
1
logstash-input
1
logstash
1
doujin
1
dojin
1
dmm
1
xml
1