Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

proxy.golang.org "spider" keyword

Top 8.2% on proxy.golang.org
github.com/Nhoya/gOSINT v0.0.0-20190617113449-9c86ed220251
OSINT Swiss Army Knife
1 version - Latest release: almost 5 years ago - 519 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/nhoya/gosint v0.0.0-20190617113449-9c86ed220251
OSINT Swiss Army Knife
2 versions - Latest release: almost 5 years ago - 519 stars on GitHub
Top 9.4% on proxy.golang.org
github.com/injoyai/downloader v0.1.3
Windows下载器,可以下载m3u8资源,如果安装的chrome,运行会自动下载chromedriver.exe,就可以根据网页地址下载网页上的所有m3u8资源
14 versions - Latest release: 4 months ago - 3 stars on GitHub
Top 6.4% on proxy.golang.org
github.com/BeanWei/MusicSpider v0.0.0-20181126200214-39c026f018fa
:trollface: Music Spider. Go 👾 Music Spider 是使用Golang写的音乐聚合爬虫,目前支持的站点包括 网易、QQ、虾米、酷狗、百度。
1 version - Latest release: over 5 years ago - 28 stars on GitHub
Top 9.7% on proxy.golang.org
github.com/suosi-inc/go-pkg-spider v0.24.0
A news website data extraction tool library implemented in Golang with relatively intelligent and...
38 versions - Latest release: 5 months ago - 1 dependent repositories - 53 stars on GitHub
Top 9.0% on proxy.golang.org
github.com/rea1shane/corder v0.1.1
Recorder for Colly program.
2 versions - Latest release: almost 2 years ago - 0 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/sb-child/lsp v0.0.0-20211106041857-87487f793538
python 爬虫, golang 爬虫: 某网站视频爬取下载工具 完美适配 miya(蜜芽)和 yysp(夜夜视频, 夜夜視頻)
1 version - Latest release: over 2 years ago - 25 stars on GitHub
Top 7.4% on proxy.golang.org
github.com/RealAlexandreAI/sticky-hand
✋ Fetch webpage content into text using crawlers or AI at your command.
Latest release: about 24 hours ago - 1 stars on GitHub
Top 7.4% on proxy.golang.org
github.com/realalexandreai/sticky-hand v0.0.6
✋ Fetch webpage content into text using crawlers or AI at your command.
6 versions - Latest release: 19 days ago - 1 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/hunterhug/GoSpider v1.0.13
💐Marmot A Golang HTTP Download
12 versions - Latest release: over 1 year ago - 24 stars on GitHub
Top 6.8% on proxy.golang.org
github.com/malwarize/webpalm/v2 v2.0.18
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive ...
18 versions - Latest release: 3 months ago - 269 stars on GitHub
Top 6.8% on proxy.golang.org
github.com/malwarize/webpalm v1.1.1
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive ...
8 versions - Latest release: 11 months ago - 269 stars on GitHub
Top 6.8% on proxy.golang.org
github.com/Malwarize/webpalm/v2 v2.0.18
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive ...
18 versions - Latest release: 3 months ago - 269 stars on GitHub
Top 6.8% on proxy.golang.org
github.com/Malwarize/webpalm v1.1.1
WebPalm is a powerful command-line tool for website mapping and web scraping. With its recursive ...
8 versions - Latest release: 11 months ago - 269 stars on GitHub
Top 0.4% on proxy.golang.org
github.com/gocolly/colly/v2 v2.1.0
Package colly implements a HTTP scraping framework
3 versions - Latest release: almost 4 years ago - 660 dependent packages - 591 dependent repositories - 19,142 stars on GitHub
Top 0.4% on proxy.golang.org
github.com/gocolly/colly v1.2.0
Package colly implements a HTTP scraping framework
3 versions - Latest release: over 5 years ago - 1,046 dependent packages - 1,411 dependent repositories - 19,142 stars on GitHub
Top 1.1% on proxy.golang.org
github.com/geziyor/geziyor v0.0.0-20230315135110-a242b58aaa65
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
7 versions - Latest release: about 1 year ago - 87 dependent packages - 67 dependent repositories - 1,880 stars on GitHub
Top 6.2% on proxy.golang.org
github.com/sillybobo/marmot v1.0.13
💐Marmot A Golang HTTP Download
12 versions - Latest release: over 1 year ago - 15 stars on
Top 5.9% on proxy.golang.org
github.com/EliseCaro/araneid v0.0.0-20220311070412-fbb5c13b1122
一个基于Glang语言开发的站群系统(蜘蛛池系统)
1 version - Latest release: about 2 years ago - 31 stars on GitHub
Top 1.6% on proxy.golang.org
github.com/henrylee2cn/pholcus v1.3.4
Pholcus(幽灵蛛)是一款纯Go语言编写的支持分布式的高并发、重量级爬虫软件,定位于互联网数据采集,为具备一定Go或JS编程基础的人提供一个只需关注规则定制的功能强大的爬虫工具。 它支持单机...
26 versions - Latest release: over 4 years ago - 4 dependent packages - 5 dependent repositories - 7,353 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/0xinfection/xsrfprobe v2.3.1+incompatible
The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.
6 versions - Latest release: over 4 years ago - 833 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/0xInfection/XSRFProbe v2.3.1+incompatible
The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.
6 versions - Latest release: over 4 years ago - 833 stars on GitHub
Top 2.7% on proxy.golang.org
github.com/jaeles-project/gospider v1.1.6 💰
Gospider - Fast web spider written in Go
12 versions - Latest release: almost 3 years ago - 1 dependent package - 2 dependent repositories - 1,884 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/telanflow/scrago v0.0.0-20201104150845-85a6395b20ba
A micro crawler framework. achieved by GOLANG.
1 version - Latest release: over 3 years ago - 2 stars on GitHub
github.com/hktalent/dht v1.3.5
Package dht implements the bittorrent dht protocol. For more information see http://www.bittorren...
36 versions - Latest release: about 2 years ago - 1 dependent repositories - 7 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/DedSecInside/TorBot v4.0.0+incompatible 💰
Dark Web OSINT Tool
8 versions - Latest release: 7 months ago - 1,898 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/dedsecinside/torbot v4.0.0+incompatible 💰
Dark Web OSINT Tool
8 versions - Latest release: 7 months ago - 1,898 stars on GitHub
Top 4.4% on proxy.golang.org
github.com/andeya/pholcus v1.3.4
Pholcus is a distributed high-concurrency crawler software written in pure golang
1 version - Latest release: over 4 years ago - 7,360 stars on GitHub
Top 7.8% on proxy.golang.org
github.com/star-inc/butterfly v0.0.0-20221108064423-2ef8d7227940
The World-Wide-Web crawler for Apache Solr. (Developing)
1 version - Latest release: over 1 year ago - 2 stars on GitHub
Top 4.9% on proxy.golang.org
github.com/edoardottt/cariddi v1.3.4
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, to...
16 versions - Latest release: about 1 month ago - 740 stars on GitHub
Top 6.3% on proxy.golang.org
github.com/hyperq/jav v0.0.0-20210421082836-1edb4c2927d8
Copyright © 2020 NAME HERE <EMAIL ADDRESS> Licensed under the Apache License, Version 2.0 (the "...
1 version - Latest release: about 3 years ago - 23 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/summeroch/dht v1.0.0
Package dht implements the bittorrent dht protocol. For more information see http://www.bittorren...
2 versions - Latest release: over 2 years ago - 1 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/WindomZ/lagou-jobs v0.0.0-20170826151718-65a5f2b5d639
采集筛选lagou招聘信息,开箱即用,快速并发
1 version - Latest release: over 6 years ago - 2 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/windomz/lagou-jobs v0.0.0-20170826151718-65a5f2b5d639
采集筛选lagou招聘信息,开箱即用,快速并发
1 version - Latest release: over 6 years ago - 2 stars on GitHub
Top 1.5% on proxy.golang.org
github.com/shiyanhui/dht v0.0.0-20201219151056-5a20f3199263
Package dht implements the bittorrent dht protocol. For more information see http://www.bittorren...
1 version - Latest release: over 3 years ago - 6 dependent packages - 25 dependent repositories - 2,601 stars on GitHub
Top 7.7% on proxy.golang.org
github.com/eversinc33/urlExtract v1.0.1
Recursively extract urls from a web page for reconnaissance.
2 versions - Latest release: over 1 year ago - 3 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/logocomune/botdetector v1.0.9
BotDetector is a golang library that detects Bot/Spider/Crawler from user agent
10 versions - Latest release: over 2 years ago - 5 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/neoql/btlet v0.0.0-20190129091513-2c5efcfab5c6
Some toolkits implements part of BT Protocol, like DHT spider.
1 version - Latest release: over 5 years ago - 56 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/edoardottt/Cariddi v1.3.4
Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, to...
16 versions - Latest release: about 1 month ago - 740 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/dineshsprabu/concurrent-web-crawler v0.0.0-20170505055956-d21717df7b11
Concurrent Web Crawler is a light weight configurable crawler with concurrecy enabled. Configrati...
1 version - Latest release: about 7 years ago - 0 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/crawlerclub/crawler v0.2.0
十年磨一剑:Crawler4U, a general purpose focused crawler
2 versions - Latest release: over 5 years ago - 36 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/BeanWei/BWOnlineMusicPlayer v0.0.0-20181126065301-5d3545aca1dd
🎷golang web 在线音乐播放器, 音乐爬虫部分来自我另一个项目 MusicSpider[https://github.com/BeanWei/MusicSpider], 前端UI 来自 ...
1 version - Latest release: over 5 years ago - 12 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/chen622/spider v0.0.0-20190730071540-97750e42b3fb
A web application to get informations from Bilibili/Weibo
1 version - Latest release: almost 5 years ago - 0 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/crawlerclub/x v0.1.0
A crawler framework
1 version - Latest release: almost 7 years ago - 1 dependent package - 2 dependent repositories - 4 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/diinvoke/proxy-pool v0.0.1
go ip proxy pool
1 version - Latest release: over 6 years ago - 10 stars on GitHub
Top 9.6% on proxy.golang.org
github.com/howie6879/ruia v0.8.0
Async Python 3.6+ web scraping micro-framework based on asyncio
10 versions - Latest release: over 3 years ago - 1,713 stars on GitHub
Top 7.2% on proxy.golang.org
github.com/Koshqua/scrapio v0.0.0-20200504235533-2f0050d63ae3
Simple and easy-to-use scraper and crawler in Go.
1 version - Latest release: about 4 years ago - 11 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/hktalent/scrapysite v0.0.6
ScrapySite,go Web Crawler(spider), scraping,intelligence gathering
2 versions - Latest release: about 2 years ago - 3 stars on GitHub
Top 7.1% on proxy.golang.org
github.com/MaoLongLong/actions-starcharts v1.0.1
自动生成星图(stars 趋势)的 action
1 version - Latest release: over 2 years ago - 17 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/telanflow/quick v0.4.1
HTTP Request library for Go (简洁高效的HTTP请求库)
6 versions - Latest release: over 2 years ago - 6 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/KillianMeersman/wander v0.14.0-alpha
Package wander is a scraping library for Go. It aims to provide an easy to use API while also exp...
15 versions - Latest release: almost 4 years ago - 0 stars on GitHub
Top 5.6% on proxy.golang.org
github.com/zhoutk/goTools v1.0.1
create some tools use go lang.
2 versions - Latest release: almost 6 years ago - 70 stars on GitHub
Top 9.0% on proxy.golang.org
github.com/hearecho/MagicSpider v0.0.0-20220422124153-50fb08f6cf08
go语言实现并发爬虫框架,满足一般爬虫功能,只需要设置解析函数,以及根url即可
1 version - Latest release: about 2 years ago - 1 dependent repositories - 6 stars on GitHub
Top 7.1% on proxy.golang.org
github.com/maolonglong/actions-starcharts v1.0.1
自动生成星图(stars 趋势)的 action
1 version - Latest release: over 2 years ago - 17 stars on GitHub
Top 6.8% on proxy.golang.org
github.com/474420502/gcurl v0.10.0
parse curl bash to code of golang
22 versions - Latest release: 7 days ago - 4 dependent packages - 1 dependent repositories - 6 stars on GitHub
Top 5.5% on proxy.golang.org
github.com/twiny/spidy
Domain names collector - Crawl websites and collect domain names along with their availability st...
Latest release: 7 days ago - 110 stars on GitHub
Top 5.5% on proxy.golang.org
github.com/twiny/spidy/v2 v2.0.1
Domain names collector - Crawl websites and collect domain names along with their availability st...
2 versions - Latest release: almost 2 years ago - 110 stars on GitHub
Top 6.1% on proxy.golang.org
github.com/wujunwei928/parse-video v0.0.0-20230506225948-6b68539e6f2b
Golang短视频去水印:抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙,梨视频,acfun,好看视频...
5 versions - Latest release: about 1 year ago - 156 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/peteryangs/article-spider/v3 v3.0.6
文章采集工具 Article collection tool
7 versions - Latest release: about 2 years ago - 132 stars on GitHub
Top 7.6% on proxy.golang.org
github.com/obgnail/scrapyingo
scrapy writen in go
Latest release: 8 days ago - 0 stars on GitHub
Top 5.4% on proxy.golang.org
github.com/PeterYangs/article-spider/v2 v2.0.26
文章采集工具 Article collection tool
27 versions - Latest release: over 2 years ago - 132 stars on GitHub
Top 5.4% on proxy.golang.org
github.com/PeterYangs/article-spider/v3 v3.0.6
文章采集工具 Article collection tool
7 versions - Latest release: about 2 years ago - 132 stars on GitHub
Top 5.4% on proxy.golang.org
github.com/PeterYangs/article-spider v0.0.26
文章采集工具 Article collection tool
26 versions - Latest release: over 2 years ago - 132 stars on GitHub
Top 8.6% on proxy.golang.org
github.com/btjoker/beorc v0.0.0-20180421131135-53b7119e8f2f
文章区图片爬虫
1 version - Latest release: about 6 years ago - 1 stars on GitHub
Top 4.8% on proxy.golang.org
github.com/signal18/replication-manager v2.0.2+incompatible
Signal 18 repman - Replication Manager for MySQL / MariaDB / Percona Server
1 version - Latest release: about 4 years ago - 1 dependent repositories - 574 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/peteryangs/article-spider/v4 v4.0.35
文章采集工具 Article collection tool
36 versions - Latest release: 6 months ago - 133 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/peteryangs/article-spider v0.0.26
文章采集工具 Article collection tool
26 versions - Latest release: over 2 years ago - 132 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/zituocn/esme v0.0.7
A golang sdk package for http requests and task queues
6 versions - Latest release: almost 2 years ago - 2 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/peteryangs/article-spider/v2 v2.0.26
文章采集工具 Article collection tool
27 versions - Latest release: over 2 years ago - 132 stars on GitHub
Top 6.7% on proxy.golang.org
github.com/youngjuning/homebrew-juejin-spider v0.1.0
爬取掘金专栏并转为Markdown保存到本地
2 versions - Latest release: almost 4 years ago - 13 stars on GitHub
Top 5.3% on proxy.golang.org
github.com/go-crawler/douban-movie v0.0.0-20180321035844-73b936237b8a
Golang爬虫 爬取豆瓣电影Top250
1 version - Latest release: about 6 years ago - 1 dependent repositories - 140 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/eroszy/labour v0.0.0-20170409171410-e0a550e88d8c
simple configable spider core application with golang
1 version - Latest release: about 7 years ago - 3 stars on GitHub
Top 5.0% on proxy.golang.org
github.com/infinitbyte/gopa v0.11.0 💰
GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
4 versions - Latest release: over 5 years ago - 294 stars on GitHub
Top 4.9% on proxy.golang.org
github.com/nange/gospider v0.0.0-20200114140849-58e3b2708dd5
GoSpider is a simple go crawler framework. User only need to care about the rules of page, provid...
1 version - Latest release: over 4 years ago - 1 dependent repositories - 352 stars on GitHub
Top 5.4% on proxy.golang.org
github.com/PeterYangs/article-spider/v4 v4.0.35
文章采集工具 Article collection tool
36 versions - Latest release: 6 months ago - 133 stars on GitHub
Top 4.9% on proxy.golang.org
github.com/ma6254/fictiondown v0.1.3
Package FictionDown is a Fiction download tool
3 versions - Latest release: about 4 years ago - 521 stars on GitHub
Top 4.9% on proxy.golang.org
github.com/ma6254/FictionDown v0.1.3
小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对
3 versions - Latest release: about 4 years ago - 521 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/ofasgard/ungoliant v1.0.0
A web reconnaissance tool that proxies its results through Burp or ZAP.
1 version - Latest release: almost 4 years ago - 7 stars on GitHub
Top 5.5% on proxy.golang.org
github.com/luckylittle/blinkist-m4a-downloader v0.0.0-20221012064854-69505e8d18ff 💰
Grabs all of the audio files from all of the Blinkist books
2 versions - Latest release: over 1 year ago - 106 stars on GitHub
Top 9.2% on proxy.golang.org
github.com/resxarcher/paramprobe
Parameter Extractor written in golang
Latest release: 13 days ago - 0 stars on GitHub
Top 9.5% on proxy.golang.org
github.com/fredwu/crawler v1.5.0
A high performance web crawler / scraper in Elixir.
13 versions - Latest release: 7 months ago - 901 stars on GitHub
Top 9.1% on proxy.golang.org
github.com/mlmhl/gcrawler
Package gcrawler provides a very simple and flexible web crawler that supports three running modes:
Latest release: 14 days ago - 0 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/zhshch2002/gospIDer v1.0.0
⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架
1 version - Latest release: about 3 years ago - 185 stars on GitHub
Top 8.0% on proxy.golang.org
github.com/daliyo/whois-spider
顶级域 WHOIS 服务器蜘蛛
Latest release: 15 days ago - 1 stars on GitHub
Top 8.8% on proxy.golang.org
github.com/tech-engine/goscrapy v0.12.2
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
48 versions - Latest release: 17 days ago - 1 dependent repositories - 24 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/s0md3v/photon v1.3.0
Incredibly fast crawler designed for OSINT.
19 versions - Latest release: about 5 years ago - 9,476 stars on GitHub
Top 9.4% on proxy.golang.org
github.com/vinsec/go-spider v0.1.0
Golang implementation, lightweight, multi-threaded web crawler
1 version - Latest release: over 1 year ago - 3 stars on GitHub
Top 4.9% on proxy.golang.org
github.com/zhshch2002/gospider v1.0.0
⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架
1 version - Latest release: about 3 years ago - 1 dependent package - 1 dependent repositories - 185 stars on GitHub
Top 9.4% on proxy.golang.org
github.com/jannchie/zrrk
ZRRK is a Zappy live Room Robot toolKit.
Latest release: 16 days ago - 0 stars on GitHub
Top 8.6% on proxy.golang.org
github.com/ghwuxin/my_spider
this is a spider
Latest release: 18 days ago - 0 stars on GitHub
Top 5.4% on proxy.golang.org
github.com/yields/ant v0.0.0-20220301182912-6c7a55c5708c
Package ant implements a web crawler.
1 version - Latest release: about 2 years ago - 2 dependent repositories - 270 stars on GitHub
github.com/cyclone-github/spider v0.7.1
Cyclone's URL Spider
3 versions - Latest release: 5 months ago - 3 stars on GitHub
github.com/474420502/requests v1.42.0
easy http for spider.
51 versions - Latest release: 9 months ago - 9 dependent packages - 4 dependent repositories - 0 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/generals-space/site-mirror-go v0.0.0-20190605034845-fce466e9cf56
来自[码云](https://gitee.com/generals-space/site-mirror-go) 通用爬虫, 仿站工具, 整站下载
1 version - Latest release: almost 5 years ago - 18 stars on GitHub
Top 9.2% on proxy.golang.org
github.com/ithrael/xspider v0.0.3
XSpider是一款高效的站点爬虫工具,采用深度遍历策略,持续发现并访问页面中的新链接。
3 versions - Latest release: 4 months ago - 0 stars on GitHub
Top 5.3% on proxy.golang.org
github.com/ioannismatzaris/galer v0.0.2 💰
A fast tool to fetch URLs from HTML attributes by crawl-in.
2 versions - Latest release: over 3 years ago - 200 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/dwisiswant0/galer v0.0.2 💰
A fast tool to fetch URLs from HTML attributes by crawl-in.
2 versions - Latest release: over 3 years ago - 172 stars on GitHub
Top 9.2% on proxy.golang.org
github.com/ithrael/XSpider v0.0.3
XSpider是一款高效的站点爬虫工具,采用深度遍历策略,持续发现并访问页面中的新链接。
3 versions - Latest release: 4 months ago - 0 stars on GitHub
Top 7.2% on proxy.golang.org
github.com/xujakai/weatherforgo v0.2.0
基于go的天气系统,包含天气预报、天气预警、降温提醒、带伞提醒
6 versions - Latest release: over 4 years ago - 5 stars on GitHub
Top 8.2% on proxy.golang.org
github.com/ssssssss-team/spider-flow v0.5.0
新一代爬虫平台,以图形化方式定义爬虫流程,不写代码即可完成爬虫。
11 versions - Latest release: about 4 years ago - 7,729 stars on GitHub