Ecosyste.ms: Packages

An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

crates.io "commoncrawl" keyword

ungoliant 2.0.0
The pipeline for the OSCAR corpus.
5 versions - Latest release: over 1 year ago - 2.01 thousand downloads total - 147 stars on GitHub - 2 maintainers
amadeus-commoncrawl 0.4.3
Harmonious distributed data analysis in Rust.
25 versions - Latest release: about 3 years ago - 1 dependent package - 1 dependent repositories - 10.7 thousand downloads total - 466 stars on GitHub - 1 maintainer
tantivy_warc_indexer 0.2.0
Builds a tantivy index from common crawl warc.wet files
1 version - Latest release: almost 3 years ago - 494 downloads total - 8 stars on GitHub - 1 maintainer
Related Keywords
common-crawl 2 corpus-linguistics 1 crawler 1 fasttext 1 language-classification 1 nlp 1 oscar 1 web 1 crawl 1 amadeus 1 data 1 data-analysis 1 data-processing 1 distributed-computing 1 parallel-computing 1 rust 1 stream-processing 1 cli 1 tantivy 1 command-line 1 index 1 search 1