An open API service providing package, version and dependency metadata of many open source software ecosystems and registries.

pypi.org "chunking" keyword

semchunk 4.1.1
A Python library for splitting text into smaller chunks while preserving as much local semantic c...
36 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 3.98 million downloads last month - 667 stars on GitHub - 2 maintainers
llama-index-node-parser-chonkie 0.2.0
llama-index Chonkie integration
4 versions - Latest release: about 2 months ago - 86 downloads last month - 52,537 stars on GitHub - 1 maintainer
tree-sitter-language-pack-hermes-plugin 1.22.2 💰
Parse and extract code intelligence from 372 programming languages with tree-sitter — structure, ...
25 versions - Latest release: 8 days ago - 569 downloads last month - 478 stars on GitHub - 2 maintainers
mseep-claude-context-local 0.1.0
Local semantic code search for Claude Code (MCP) using EmbeddingGemma, FAISS, and AST/tree-sitter...
1 version - Latest release: 11 months ago - 17 downloads last month - 237 stars on GitHub - 1 maintainer
swarmauri_tool_gmail 0.10.0
example community package
55 versions - Latest release: 7 months ago - 982 downloads last month - 88 stars on GitHub - 1 maintainer
btx-lib-mail 4.5.0
Send email from Python or the shell: streamed large attachments (bounded memory), RFC 3030 BDAT/C...
28 versions - Latest release: 7 days ago - 7.42 thousand downloads last month - 1 stars on GitHub - 1 maintainer
doctools-m8 0.1.0
Media-neutral document toolkit: models, revisions, the versioned artifact envelope, the PDF/plain...
1 version - Latest release: 5 days ago - 0 downloads last month - 1 maintainer
embedkit-py 0.1.1
Production embedding pipeline toolkit for building RAG systems with multiple chunking strategies,...
2 versions - Latest release: about 1 month ago - 228 downloads last month - 0 stars on GitHub - 1 maintainer
dot-parser 2.0.0
Document-to-markdown parser and chunker for RAG pipelines
1 version - Latest release: about 2 months ago - 1 maintainer
excelplumber 0.1.0
Edge-case-hardened CSV/Excel parser for RAG chunking and embeddings
1 version - Latest release: about 2 months ago - 0 stars on GitHub - 1 maintainer
context-grid 0.9.5
A lab for grounding pipelines: sweep parser x chunker x embedder x index x reranker on your own d...
6 versions - Latest release: 2 months ago - 302 downloads last month - 0 stars on GitHub - 1 maintainer
chunkerlib 0.1.0
Split binary data into fixed-size chunks. Zero dependencies, fully typed
1 version - Latest release: 2 months ago - 87 downloads last month - 1 stars on GitHub - 1 maintainer
cairn-rag 0.2.0
Cairn reuses unchanged RAG chunks with content-defined identities, cache-aware planning, and tran...
2 versions - Latest release: 2 months ago - 0 downloads last month - 10 stars on GitHub - 1 maintainer
swarmauri_measurement_mutualinformation 0.9.3
Swarmauri Mutual Information Measurement Community Package.
54 versions - Latest release: 7 months ago - 717 downloads last month - 88 stars on GitHub - 1 maintainer
boundary-chunker 0.1.0
Boundary-aware semantic chunking for RAG, using a ModernBERT topic-shift model
1 version - Latest release: 2 months ago - 1 maintainer
burmesenlp 1.2.0
Production-oriented Myanmar (Burmese) NLP preprocessing: syllable, word and sentence segmentation...
3 versions - Latest release: about 1 month ago - 141 downloads last month - 1 maintainer
ragbenchpy 0.1.1
Benchmark chunking and retrieval strategies on your own data: compare cost, latency, and accuracy...
2 versions - Latest release: 3 months ago - 0 stars on GitHub - 1 maintainer
tinyzchunk 0.3.3
GPU-free, tokenizer-free chunker for RAG (EN + PT-BR). Robust to PDF/OCR noise, markdown, code an...
5 versions - Latest release: about 1 month ago - 406 downloads last month - 5 stars on GitHub - 1 maintainer
multixtract 0.1.2
Vendor-neutral PDF/document extraction: text, tables, and vision-described images, chunked and em...
2 versions - Latest release: 2 months ago - 63 downloads last month - 1 stars on GitHub - 1 maintainer
rustchunker 1.0.1
A fast document chunking library for RAG pipelines
2 versions - Latest release: 3 months ago - 34 downloads last month - 0 stars on GitHub - 1 maintainer
ragprepkit 0.1.3
Document preprocessing toolkit for RAG and LLM pipelines: text cleaning, chunking strategies, met...
1 version - Latest release: 3 months ago - 1 maintainer
ingestlib 1.5.0
Self-hosted document intelligence for RAG — parse, classify, split, and extract from PDF/DOCX/PPT...
16 versions - Latest release: about 1 month ago - 248 downloads last month - 5 stars on GitHub - 1 maintainer
ragkit-chunkwise 0.1.1
Token-aware, boundary-respecting text/document chunking for RAG ingestion (stdlib-only).
2 versions - Latest release: 3 months ago - 1 maintainer
langchain-xberg 1.3.8 💰
LangChain document loader for Xberg — extract 88+ file formats into Documents with async, batchin...
39 versions - Latest release: 13 days ago - 2.04 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
llama-index-node-parser-xberg 1.3.8 💰
Structure-aware LlamaIndex node parser that turns xberg native chunks and elements into nodes
39 versions - Latest release: 13 days ago - 2 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
txtai-xberg 1.3.8 💰
Xberg document extraction and chunking pipeline for txtai embeddings
36 versions - Latest release: 13 days ago - 1.85 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
scrapedatshi-mcp 0.6.11
MCP server for the scrapedatshi RAG pipeline API — use scrapedatshi tools directly from Claude De...
28 versions - Latest release: 3 months ago - 117 downloads last month - 0 stars on GitHub - 1 maintainer
cognigraph-chunker 2.2.4
Fast text chunking toolkit: fixed-size, delimiter, semantic, cognition-aware, intent-driven, topo...
2 versions - Latest release: 4 months ago - 29 downloads last month - 1 stars on GitHub - 1 maintainer
chunkbench-rag 0.1.0
A chunk-source-agnostic evaluation harness for RAG chunking strategies
1 version - Latest release: 4 months ago - 17 downloads last month - 9 stars on GitHub - 1 maintainer
arabic-rag-kit 0.2.0
Prepare Arabic (and mixed Arabic/English) documents for RAG and search: normalization, sentence-a...
2 versions - Latest release: 4 months ago - 34 downloads last month - 0 stars on GitHub - 1 maintainer
entropy-chunker 0.1.0
Adaptive Information-Aware Chunking for RAG and Agentic Systems, driven by information density in...
1 version - Latest release: 4 months ago - 15 downloads last month - 0 stars on GitHub - 1 maintainer
adaptive-oci-chunking 0.3.0
Adaptive document chunking for RAG with optional Oracle Cloud Infrastructure integrations.
8 versions - Latest release: 3 months ago - 95 downloads last month - 6 stars on GitHub - 1 maintainer
chunkstore 0.3.0
Content-addressed chunk storage with deduplication
3 versions - Latest release: 4 months ago - 22 downloads last month - 2 stars on GitHub - 1 maintainer
tinychunk 0.1.0
A tiny, zero-dependency text chunking library for Python.
1 version - Latest release: 4 months ago - 15 downloads last month - 1 stars on GitHub - 1 maintainer
document-chunker 0.1.0
Split PDF, DOCX, PPTX, TXT and Markdown into clean text chunks for RAG
1 version - Latest release: 7 days ago - 0 downloads last month - 1 stars on GitHub - 1 maintainer
ragatini 0.1.4
Semantic-velocity document chunker — cuts where meaning changes, then snaps to real sentence/para...
2 versions - Latest release: 4 months ago - 1 maintainer
scrapedatshi 0.13.0
Official Python SDK for the scrapedatshi RAG pipeline API
45 versions - Latest release: 3 months ago - 0 stars on GitHub - 1 maintainer
contextdoctor 0.1.0
A static analyzer for RAG systems and context engineering workflows. ESLint for your context.
1 version - Latest release: 4 months ago - 28 downloads last month - 0 stars on GitHub - 1 maintainer
structchunk 0.1.0
Markdown chunker that respects your document's structure. Tables split at rows (not mid-row), hea...
1 version - Latest release: 4 months ago - 19 downloads last month - 3 stars on GitHub - 1 maintainer
victor-codegraph 0.1.0
Code->CPG chunker: tree-sitter symbol + relation extraction, size-capped chunks, ProximaRecord pr...
2 versions - Latest release: 4 months ago - 195 downloads last month - 5 stars on GitHub - 1 maintainer
konan 0.4.0
Blazingly fast text chunkers in Rust with pythonic bindings
7 versions - Latest release: 3 months ago - 586 downloads last month - 1 stars on GitHub - 1 maintainer
pypreprocess 1.6.1
Preprocess SDK
9 versions - Latest release: 8 months ago - 1 dependent package - 185 downloads last month - 1 maintainer
mcp-pager 0.8.0
Token-aware response paging for MCP servers — chunks large tool responses and delivers them page ...
3 versions - Latest release: 5 months ago - 18 downloads last month - 1 stars on GitHub - 1 maintainer
firecloud-devnet 0.2.1
Private, encrypted, distributed storage across your own machines
3 versions - Latest release: 4 months ago - 17 downloads last month - 0 stars on GitHub - 1 maintainer
koji-ingest 0.5.0
Ingestion pipeline for the koji-db database: parsing, chunking, and multi-vector embeddings
6 versions - Latest release: 4 months ago - 595 downloads last month - 1 maintainer
catalyst-chunker 0.1.8
Invariant-first chunk formation for source-preserving document and code workflows.
9 versions - Latest release: 4 months ago - 120 downloads last month - 0 stars on GitHub - 1 maintainer
contextiq 0.1.1
Turn messy files into agent-ready context.
2 versions - Latest release: 4 months ago - 289 downloads last month - 1 stars on GitHub - 1 maintainer
boundary-smart-splitter 1.0.2
A context-aware, boundary-respecting text splitter for Python.
3 versions - Latest release: 4 months ago - 338 downloads last month - 0 stars on GitHub - 1 maintainer
smantic 0.1.1
Structure-aware semantic chunking, minus the heavyweight stack.
2 versions - Latest release: 5 months ago - 18 downloads last month - 0 stars on GitHub - 1 maintainer
ragcheck-cli 0.2.10
Lighthouse for RAG systems - diagnose and fix your retrieval pipeline
11 versions - Latest release: 5 months ago - 1.2 thousand downloads last month - 0 stars on GitHub - 1 maintainer
less-tokens 0.8.1
Cut LLM prompt token costs by 30-40% with deterministic, training-free lexical compression for Op...
16 versions - Latest release: 7 days ago - 100 downloads last month - 6 stars on GitHub - 1 maintainer
thread-chunks 1.0.5
A python package for performing memory intensive computations in parallel using chunks and checkp...
6 versions - Latest release: over 1 year ago - 42 downloads last month - 3 stars on GitHub - 1 maintainer
chunkin 0.1.4
Document chunking and indexing library for vector stores
5 versions - Latest release: 6 months ago - 16 downloads last month - 1 stars on GitHub - 1 maintainer
Top 9.4% on pypi.org
fastcdc 1.7.0 💰
FastCDC (content defined chunking) in pure Python.
10 versions - Latest release: over 2 years ago - 2 dependent packages - 2 dependent repositories - 31.6 thousand downloads last month - 66 stars on GitHub - 1 maintainer
wizit-open-rag 0.0.9
AI-powered document transcription and semantic chunking for RAG pipelines
9 versions - Latest release: 5 months ago - 61 downloads last month - 1 maintainer
fennec-community 0.1.0
Production-grade RAG framework with Arabic NLP support — chunking, embeddings, LLM interfaces, ve...
1 version - Latest release: 5 months ago - 25 downloads last month - 1 stars on GitHub - 1 maintainer
omnidoc-excel-extractor 0.3.2
Comprehensive Excel extraction SDK — structured chunks for AI/LLM/RAG pipelines
5 versions - Latest release: 5 months ago - 59 downloads last month - 1 maintainer
py-chunks 0.6.4
Parse and chunk 36 document formats for RAG — one compiled Rust engine, zero Python dependencies
23 versions - Latest release: about 1 month ago - 492 downloads last month - 1 stars on GitHub - 1 maintainer
markdown-hero 0.1.0
Markdown processing utilities: strip, chunk, append, break, Word export, lint, and more.
1 version - Latest release: 6 months ago - 252 downloads last month - 20 stars on GitHub - 1 maintainer
ncrfpp 0.1.0
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.
1 version - Latest release: over 4 years ago - 1,901 stars on GitHub - 1 maintainer
ncrf 0.1.0
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.
1 version - Latest release: over 4 years ago - 1,901 stars on GitHub
ncrfpp2 0.1.1
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.
1 version - Latest release: over 4 years ago - 1,901 stars on GitHub - 1 maintainer
ragrails 0.2.1
A modular RAG SDK for ingesting web, document, and API sources, chunking them, and storing embedd...
11 versions - Latest release: 5 months ago - 91 downloads last month - 7 stars on GitHub - 1 maintainer
table2rules 0.6.4
Convert HTML tables to flat, LLM-friendly rules using spatial pathfinding.
10 versions - Latest release: 3 months ago - 71 downloads last month - 3 stars on GitHub - 1 maintainer
chunkshop 0.9.1
Standalone ingest-to-pgvector: source → chunker → embedder → extractor → table. int8 by default.
22 versions - Latest release: 5 months ago - 184 downloads last month - 1 stars on GitHub - 1 maintainer
chunklet-py 3.0.1
High-fidelity context-aware chunking and interactive visualization for RAG. Advanced segmentation...
17 versions - Latest release: 22 days ago - 403 downloads last month - 88 stars on GitHub - 1 maintainer
krira-augment 2.1.13
Production-grade document chunking library for RAG systems - Rust-powered Python library
19 versions - Latest release: 10 months ago - 46 downloads last month - 50 stars on GitHub - 1 maintainer
chunktuner 0.1.2
Auto chunking tuner and MCP server for RAG pipelines
3 versions - Latest release: 6 months ago - 162 downloads last month - 2 stars on GitHub - 1 maintainer
stratum-client 0.1.1
Official Python client for the Stratum document-chunking API.
2 versions - Latest release: 6 months ago - 1 maintainer
preembed 0.1.1
High-performance document preparation toolkit for LLM and RAG pipelines.
2 versions - Latest release: 6 months ago - 992 downloads last month - 0 stars on GitHub - 1 maintainer
docling-serve-client 1.14.0
A Python SDK for interacting with Docling Serve API using Pydantic models
1 version - Latest release: 8 months ago - 253 downloads last month - 0 stars on GitHub - 1 maintainer
rag-forge-core 0.2.3
RAG pipeline primitives: ingestion, retrieval, context management, and security
8 versions - Latest release: 6 months ago - 14 downloads last month - 6 stars on GitHub - 1 maintainer
docprep 0.1.1
Prepare documents into structured, vector-ready data
2 versions - Latest release: 6 months ago - 17 downloads last month - 1 stars on GitHub - 1 maintainer
bigboss 0.1.0
Local folder-to-memory chunking, summary extraction, embedding, and search CLI.
1 version - Latest release: 6 months ago - 26 downloads last month - 1 maintainer
vectordb2 0.1.9
A lightweight Python package for storing and retrieving text using chunking, embedding, and vecto...
11 versions - Latest release: almost 3 years ago - 1 dependent repositories - 258 downloads last month - 799 stars on GitHub - 1 maintainer
longparser 0.1.5
Privacy-first document intelligence engine — converts PDFs, DOCX, PPTX, XLSX, and CSV into AI-rea...
6 versions - Latest release: 6 months ago - 17 downloads last month - 30 stars on GitHub - 1 maintainer
docslicer 0.2.5
Answer questions from documents too large to fit in context, reading only the sections you need
7 versions - Latest release: about 2 months ago - 672 downloads last month - 36 stars on GitHub - 1 maintainer
django-query-chunk 0.0.1
Part QuerySets into small chunks.
1 version - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 367 downloads last month - 1 stars on GitHub - 1 maintainer
coralbricks-context-prep 0.1.0
Build-time context preparation for agentic AI: clean, chunk, embed, enrich, hydrate. No orchestra...
1 version - Latest release: 6 months ago - 36 downloads last month - 53 stars on GitHub - 1 maintainer
openingestion 0.1.5
RAG ingestion pipeline — Chef → Chunker → Refinery → Porter
6 versions - Latest release: 6 months ago - 61 downloads last month - 0 stars on GitHub - 1 maintainer
poma-primecut-nano 0.1.0
Hierarchical document chunking for RAG. Structure in, structure out.
1 version - Latest release: 7 months ago - 45 downloads last month - 9 stars on GitHub - 1 maintainer
batchit 0.4.0
Batch any Python iterator by count and/or elapsed time
5 versions - Latest release: 7 months ago - 10.1 thousand downloads last month - 2 stars on GitHub - 1 maintainer
sim-pdf-chunker 0.0.1
A simple PDF chunking utility for splitting PDF text into manageable pieces
1 version - Latest release: 6 months ago - 1 maintainer
llm-chunk-optimizer 0.1.2
Semantic-aware chunking with provenance tracking for production RAG and LLM data pipelines
2 versions - Latest release: 6 months ago - 59 downloads last month - 1 maintainer
chunk-bench 1.0.0
Benchmark RAG chunking strategies on your own documents. Compare fixed, sliding, paragraph, recur...
1 version - Latest release: 6 months ago - 35 downloads last month - 1 stars on GitHub - 1 maintainer
llm-context-forge 0.1.5
Production-grade LLMOps infrastructure for context window management, token counting, document ch...
2 versions - Latest release: 7 months ago - 333 downloads last month - 1 maintainer
ragpipe-ai 0.1.0
RAG in 3 functions. Ingest any data source into vector databases.
1 version - Latest release: 7 months ago - 52 downloads last month - 3 stars on GitHub - 1 maintainer
chunkweaver 0.4.0
RAG chunker that respects document structure.
5 versions - Latest release: 7 months ago - 48 downloads last month - 0 stars on GitHub - 1 maintainer
stump-cli 0.2.0
Stump: AST-aware code chunking (tree-sitter). Derived from ASTChunk; cite the cAST paper for rese...
2 versions - Latest release: 7 months ago - 32 downloads last month - 0 stars on GitHub - 1 maintainer
omnichunk 2.1.4
Structure-aware deterministic chunking for code, prose, and markup.
16 versions - Latest release: about 2 months ago - 125 downloads last month - 8 stars on GitHub - 1 maintainer
llama-index-node-parser-kreuzberg 0.1.0
Element-aware LlamaIndex node parser for kreuzberg-extracted documents
1 version - Latest release: 7 months ago - 105 downloads last month - 1 maintainer
contexthub 0.1.0
A lightweight context management library for LLM applications
3 versions - Latest release: 7 months ago - 164 downloads last month - 1 maintainer
seedbraid 2.0.5 💰
Reference-based file reconstruction with CDC chunking, SBD1 binary seed format, and IPFS transport
10 versions - Latest release: 7 months ago - 40 downloads last month - 0 stars on GitHub - 1 maintainer
notebooklm-chunker 0.5.0
Turn long PDFs into chunked NotebookLM workflows with Studio outputs
6 versions - Latest release: 7 months ago - 19 downloads last month - 20 stars on GitHub - 1 maintainer
mr-ninja 2.0.0
Large Context Orchestrator for GitLab Duo -- chunk oversized MRs and run multi-agent security ana...
3 versions - Latest release: 7 months ago - 29 downloads last month - 2 stars on GitHub - 1 maintainer
markdownheadertextsplitter 0.1.0
A heading-aware Markdown splitter with a simple local UI
1 version - Latest release: 7 months ago - 164 downloads last month - 1 maintainer
lasr 0.1.0
LASER (Least Action Semantic Router) — globally optimal text chunking for RAG
1 version - Latest release: 8 months ago - 33 downloads last month - 1 maintainer
streamchunk 2.1.0
Adaptive data stream chunker with CPU-aware parallel processing for real-time ETL pipelines
3 versions - Latest release: 6 months ago - 73 downloads last month - 1 stars on GitHub - 1 maintainer
gi-text-summarizer 1.0.0
Lightweight text summarizer with token budgets and smart chunking
1 version - Latest release: 8 months ago - 202 downloads last month - 1 maintainer
docuweave 0.2.1
Layout-aware PDF chunking for production RAG pipelines
6 versions - Latest release: 3 months ago - 48 downloads last month - 13 stars on GitHub - 1 maintainer