pypi.org "chunking" keyword
semchunk 4.1.1
A Python library for splitting text into smaller chunks while preserving as much local semantic c...36 versions - Latest release: 4 months ago - 1 dependent package - 1 dependent repositories - 3.98 million downloads last month - 667 stars on GitHub - 2 maintainers
llama-index-node-parser-chonkie 0.2.0
llama-index Chonkie integration4 versions - Latest release: about 2 months ago - 86 downloads last month - 52,537 stars on GitHub - 1 maintainer
tree-sitter-language-pack-hermes-plugin 1.22.2 💰
Parse and extract code intelligence from 372 programming languages with tree-sitter — structure, ...25 versions - Latest release: 8 days ago - 569 downloads last month - 478 stars on GitHub - 2 maintainers
mseep-claude-context-local 0.1.0
Local semantic code search for Claude Code (MCP) using EmbeddingGemma, FAISS, and AST/tree-sitter...1 version - Latest release: 11 months ago - 17 downloads last month - 237 stars on GitHub - 1 maintainer
swarmauri_tool_gmail 0.10.0
example community package55 versions - Latest release: 7 months ago - 982 downloads last month - 88 stars on GitHub - 1 maintainer
btx-lib-mail 4.5.0
Send email from Python or the shell: streamed large attachments (bounded memory), RFC 3030 BDAT/C...28 versions - Latest release: 7 days ago - 7.42 thousand downloads last month - 1 stars on GitHub - 1 maintainer
doctools-m8 0.1.0
Media-neutral document toolkit: models, revisions, the versioned artifact envelope, the PDF/plain...1 version - Latest release: 5 days ago - 0 downloads last month - 1 maintainer
embedkit-py 0.1.1
Production embedding pipeline toolkit for building RAG systems with multiple chunking strategies,...2 versions - Latest release: about 1 month ago - 228 downloads last month - 0 stars on GitHub - 1 maintainer
dot-parser 2.0.0
Document-to-markdown parser and chunker for RAG pipelines1 version - Latest release: about 2 months ago - 1 maintainer
excelplumber 0.1.0
Edge-case-hardened CSV/Excel parser for RAG chunking and embeddings1 version - Latest release: about 2 months ago - 0 stars on GitHub - 1 maintainer
context-grid 0.9.5
A lab for grounding pipelines: sweep parser x chunker x embedder x index x reranker on your own d...6 versions - Latest release: 2 months ago - 302 downloads last month - 0 stars on GitHub - 1 maintainer
chunkerlib 0.1.0
Split binary data into fixed-size chunks. Zero dependencies, fully typed1 version - Latest release: 2 months ago - 87 downloads last month - 1 stars on GitHub - 1 maintainer
cairn-rag 0.2.0
Cairn reuses unchanged RAG chunks with content-defined identities, cache-aware planning, and tran...2 versions - Latest release: 2 months ago - 0 downloads last month - 10 stars on GitHub - 1 maintainer
swarmauri_measurement_mutualinformation 0.9.3
Swarmauri Mutual Information Measurement Community Package.54 versions - Latest release: 7 months ago - 717 downloads last month - 88 stars on GitHub - 1 maintainer
boundary-chunker 0.1.0
Boundary-aware semantic chunking for RAG, using a ModernBERT topic-shift model1 version - Latest release: 2 months ago - 1 maintainer
burmesenlp 1.2.0
Production-oriented Myanmar (Burmese) NLP preprocessing: syllable, word and sentence segmentation...3 versions - Latest release: about 1 month ago - 141 downloads last month - 1 maintainer
ragbenchpy 0.1.1
Benchmark chunking and retrieval strategies on your own data: compare cost, latency, and accuracy...2 versions - Latest release: 3 months ago - 0 stars on GitHub - 1 maintainer
tinyzchunk 0.3.3
GPU-free, tokenizer-free chunker for RAG (EN + PT-BR). Robust to PDF/OCR noise, markdown, code an...5 versions - Latest release: about 1 month ago - 406 downloads last month - 5 stars on GitHub - 1 maintainer
multixtract 0.1.2
Vendor-neutral PDF/document extraction: text, tables, and vision-described images, chunked and em...2 versions - Latest release: 2 months ago - 63 downloads last month - 1 stars on GitHub - 1 maintainer
rustchunker 1.0.1
A fast document chunking library for RAG pipelines2 versions - Latest release: 3 months ago - 34 downloads last month - 0 stars on GitHub - 1 maintainer
ragprepkit 0.1.3
Document preprocessing toolkit for RAG and LLM pipelines: text cleaning, chunking strategies, met...1 version - Latest release: 3 months ago - 1 maintainer
ingestlib 1.5.0
Self-hosted document intelligence for RAG — parse, classify, split, and extract from PDF/DOCX/PPT...16 versions - Latest release: about 1 month ago - 248 downloads last month - 5 stars on GitHub - 1 maintainer
ragkit-chunkwise 0.1.1
Token-aware, boundary-respecting text/document chunking for RAG ingestion (stdlib-only).2 versions - Latest release: 3 months ago - 1 maintainer
langchain-xberg 1.3.8 💰
LangChain document loader for Xberg — extract 88+ file formats into Documents with async, batchin...39 versions - Latest release: 13 days ago - 2.04 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
llama-index-node-parser-xberg 1.3.8 💰
Structure-aware LlamaIndex node parser that turns xberg native chunks and elements into nodes39 versions - Latest release: 13 days ago - 2 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
txtai-xberg 1.3.8 💰
Xberg document extraction and chunking pipeline for txtai embeddings36 versions - Latest release: 13 days ago - 1.85 thousand downloads last month - 9,427 stars on GitHub - 2 maintainers
scrapedatshi-mcp 0.6.11
MCP server for the scrapedatshi RAG pipeline API — use scrapedatshi tools directly from Claude De...28 versions - Latest release: 3 months ago - 117 downloads last month - 0 stars on GitHub - 1 maintainer
cognigraph-chunker 2.2.4
Fast text chunking toolkit: fixed-size, delimiter, semantic, cognition-aware, intent-driven, topo...2 versions - Latest release: 4 months ago - 29 downloads last month - 1 stars on GitHub - 1 maintainer
chunkbench-rag 0.1.0
A chunk-source-agnostic evaluation harness for RAG chunking strategies1 version - Latest release: 4 months ago - 17 downloads last month - 9 stars on GitHub - 1 maintainer
arabic-rag-kit 0.2.0
Prepare Arabic (and mixed Arabic/English) documents for RAG and search: normalization, sentence-a...2 versions - Latest release: 4 months ago - 34 downloads last month - 0 stars on GitHub - 1 maintainer
entropy-chunker 0.1.0
Adaptive Information-Aware Chunking for RAG and Agentic Systems, driven by information density in...1 version - Latest release: 4 months ago - 15 downloads last month - 0 stars on GitHub - 1 maintainer
adaptive-oci-chunking 0.3.0
Adaptive document chunking for RAG with optional Oracle Cloud Infrastructure integrations.8 versions - Latest release: 3 months ago - 95 downloads last month - 6 stars on GitHub - 1 maintainer
chunkstore 0.3.0
Content-addressed chunk storage with deduplication3 versions - Latest release: 4 months ago - 22 downloads last month - 2 stars on GitHub - 1 maintainer
tinychunk 0.1.0
A tiny, zero-dependency text chunking library for Python.1 version - Latest release: 4 months ago - 15 downloads last month - 1 stars on GitHub - 1 maintainer
document-chunker 0.1.0
Split PDF, DOCX, PPTX, TXT and Markdown into clean text chunks for RAG1 version - Latest release: 7 days ago - 0 downloads last month - 1 stars on GitHub - 1 maintainer
ragatini 0.1.4
Semantic-velocity document chunker — cuts where meaning changes, then snaps to real sentence/para...2 versions - Latest release: 4 months ago - 1 maintainer
scrapedatshi 0.13.0
Official Python SDK for the scrapedatshi RAG pipeline API45 versions - Latest release: 3 months ago - 0 stars on GitHub - 1 maintainer
contextdoctor 0.1.0
A static analyzer for RAG systems and context engineering workflows. ESLint for your context.1 version - Latest release: 4 months ago - 28 downloads last month - 0 stars on GitHub - 1 maintainer
structchunk 0.1.0
Markdown chunker that respects your document's structure. Tables split at rows (not mid-row), hea...1 version - Latest release: 4 months ago - 19 downloads last month - 3 stars on GitHub - 1 maintainer
victor-codegraph 0.1.0
Code->CPG chunker: tree-sitter symbol + relation extraction, size-capped chunks, ProximaRecord pr...2 versions - Latest release: 4 months ago - 195 downloads last month - 5 stars on GitHub - 1 maintainer
konan 0.4.0
Blazingly fast text chunkers in Rust with pythonic bindings7 versions - Latest release: 3 months ago - 586 downloads last month - 1 stars on GitHub - 1 maintainer
pypreprocess 1.6.1
Preprocess SDK9 versions - Latest release: 8 months ago - 1 dependent package - 185 downloads last month - 1 maintainer
mcp-pager 0.8.0
Token-aware response paging for MCP servers — chunks large tool responses and delivers them page ...3 versions - Latest release: 5 months ago - 18 downloads last month - 1 stars on GitHub - 1 maintainer
firecloud-devnet 0.2.1
Private, encrypted, distributed storage across your own machines3 versions - Latest release: 4 months ago - 17 downloads last month - 0 stars on GitHub - 1 maintainer
koji-ingest 0.5.0
Ingestion pipeline for the koji-db database: parsing, chunking, and multi-vector embeddings6 versions - Latest release: 4 months ago - 595 downloads last month - 1 maintainer
catalyst-chunker 0.1.8
Invariant-first chunk formation for source-preserving document and code workflows.9 versions - Latest release: 4 months ago - 120 downloads last month - 0 stars on GitHub - 1 maintainer
contextiq 0.1.1
Turn messy files into agent-ready context.2 versions - Latest release: 4 months ago - 289 downloads last month - 1 stars on GitHub - 1 maintainer
boundary-smart-splitter 1.0.2
A context-aware, boundary-respecting text splitter for Python.3 versions - Latest release: 4 months ago - 338 downloads last month - 0 stars on GitHub - 1 maintainer
smantic 0.1.1
Structure-aware semantic chunking, minus the heavyweight stack.2 versions - Latest release: 5 months ago - 18 downloads last month - 0 stars on GitHub - 1 maintainer
ragcheck-cli 0.2.10
Lighthouse for RAG systems - diagnose and fix your retrieval pipeline11 versions - Latest release: 5 months ago - 1.2 thousand downloads last month - 0 stars on GitHub - 1 maintainer
less-tokens 0.8.1
Cut LLM prompt token costs by 30-40% with deterministic, training-free lexical compression for Op...16 versions - Latest release: 7 days ago - 100 downloads last month - 6 stars on GitHub - 1 maintainer
thread-chunks 1.0.5
A python package for performing memory intensive computations in parallel using chunks and checkp...6 versions - Latest release: over 1 year ago - 42 downloads last month - 3 stars on GitHub - 1 maintainer
chunkin 0.1.4
Document chunking and indexing library for vector stores5 versions - Latest release: 6 months ago - 16 downloads last month - 1 stars on GitHub - 1 maintainer
Top 9.4% on pypi.org
10 versions - Latest release: over 2 years ago - 2 dependent packages - 2 dependent repositories - 31.6 thousand downloads last month - 66 stars on GitHub - 1 maintainer
fastcdc 1.7.0 💰
FastCDC (content defined chunking) in pure Python.10 versions - Latest release: over 2 years ago - 2 dependent packages - 2 dependent repositories - 31.6 thousand downloads last month - 66 stars on GitHub - 1 maintainer
wizit-open-rag 0.0.9
AI-powered document transcription and semantic chunking for RAG pipelines9 versions - Latest release: 5 months ago - 61 downloads last month - 1 maintainer
fennec-community 0.1.0
Production-grade RAG framework with Arabic NLP support — chunking, embeddings, LLM interfaces, ve...1 version - Latest release: 5 months ago - 25 downloads last month - 1 stars on GitHub - 1 maintainer
omnidoc-excel-extractor 0.3.2
Comprehensive Excel extraction SDK — structured chunks for AI/LLM/RAG pipelines5 versions - Latest release: 5 months ago - 59 downloads last month - 1 maintainer
py-chunks 0.6.4
Parse and chunk 36 document formats for RAG — one compiled Rust engine, zero Python dependencies23 versions - Latest release: about 1 month ago - 492 downloads last month - 1 stars on GitHub - 1 maintainer
markdown-hero 0.1.0
Markdown processing utilities: strip, chunk, append, break, Word export, lint, and more.1 version - Latest release: 6 months ago - 252 downloads last month - 20 stars on GitHub - 1 maintainer
ncrfpp 0.1.0
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.1 version - Latest release: over 4 years ago - 1,901 stars on GitHub - 1 maintainer
ncrf 0.1.0
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.1 version - Latest release: over 4 years ago - 1,901 stars on GitHub
ncrfpp2 0.1.1
NCRF++2.0: An Open-source Neural Sequence Labeling and Classification Toolkit.1 version - Latest release: over 4 years ago - 1,901 stars on GitHub - 1 maintainer
ragrails 0.2.1
A modular RAG SDK for ingesting web, document, and API sources, chunking them, and storing embedd...11 versions - Latest release: 5 months ago - 91 downloads last month - 7 stars on GitHub - 1 maintainer
table2rules 0.6.4
Convert HTML tables to flat, LLM-friendly rules using spatial pathfinding.10 versions - Latest release: 3 months ago - 71 downloads last month - 3 stars on GitHub - 1 maintainer
chunkshop 0.9.1
Standalone ingest-to-pgvector: source → chunker → embedder → extractor → table. int8 by default.22 versions - Latest release: 5 months ago - 184 downloads last month - 1 stars on GitHub - 1 maintainer
chunklet-py 3.0.1
High-fidelity context-aware chunking and interactive visualization for RAG. Advanced segmentation...17 versions - Latest release: 22 days ago - 403 downloads last month - 88 stars on GitHub - 1 maintainer
krira-augment 2.1.13
Production-grade document chunking library for RAG systems - Rust-powered Python library19 versions - Latest release: 10 months ago - 46 downloads last month - 50 stars on GitHub - 1 maintainer
chunktuner 0.1.2
Auto chunking tuner and MCP server for RAG pipelines3 versions - Latest release: 6 months ago - 162 downloads last month - 2 stars on GitHub - 1 maintainer
stratum-client 0.1.1
Official Python client for the Stratum document-chunking API.2 versions - Latest release: 6 months ago - 1 maintainer
preembed 0.1.1
High-performance document preparation toolkit for LLM and RAG pipelines.2 versions - Latest release: 6 months ago - 992 downloads last month - 0 stars on GitHub - 1 maintainer
docling-serve-client 1.14.0
A Python SDK for interacting with Docling Serve API using Pydantic models1 version - Latest release: 8 months ago - 253 downloads last month - 0 stars on GitHub - 1 maintainer
rag-forge-core 0.2.3
RAG pipeline primitives: ingestion, retrieval, context management, and security8 versions - Latest release: 6 months ago - 14 downloads last month - 6 stars on GitHub - 1 maintainer
docprep 0.1.1
Prepare documents into structured, vector-ready data2 versions - Latest release: 6 months ago - 17 downloads last month - 1 stars on GitHub - 1 maintainer
bigboss 0.1.0
Local folder-to-memory chunking, summary extraction, embedding, and search CLI.1 version - Latest release: 6 months ago - 26 downloads last month - 1 maintainer
vectordb2 0.1.9
A lightweight Python package for storing and retrieving text using chunking, embedding, and vecto...11 versions - Latest release: almost 3 years ago - 1 dependent repositories - 258 downloads last month - 799 stars on GitHub - 1 maintainer
longparser 0.1.5
Privacy-first document intelligence engine — converts PDFs, DOCX, PPTX, XLSX, and CSV into AI-rea...6 versions - Latest release: 6 months ago - 17 downloads last month - 30 stars on GitHub - 1 maintainer
docslicer 0.2.5
Answer questions from documents too large to fit in context, reading only the sections you need7 versions - Latest release: about 2 months ago - 672 downloads last month - 36 stars on GitHub - 1 maintainer
django-query-chunk 0.0.1
Part QuerySets into small chunks.1 version - Latest release: almost 7 years ago - 1 dependent package - 1 dependent repositories - 367 downloads last month - 1 stars on GitHub - 1 maintainer
coralbricks-context-prep 0.1.0
Build-time context preparation for agentic AI: clean, chunk, embed, enrich, hydrate. No orchestra...1 version - Latest release: 6 months ago - 36 downloads last month - 53 stars on GitHub - 1 maintainer
openingestion 0.1.5
RAG ingestion pipeline — Chef → Chunker → Refinery → Porter6 versions - Latest release: 6 months ago - 61 downloads last month - 0 stars on GitHub - 1 maintainer
poma-primecut-nano 0.1.0
Hierarchical document chunking for RAG. Structure in, structure out.1 version - Latest release: 7 months ago - 45 downloads last month - 9 stars on GitHub - 1 maintainer
batchit 0.4.0
Batch any Python iterator by count and/or elapsed time5 versions - Latest release: 7 months ago - 10.1 thousand downloads last month - 2 stars on GitHub - 1 maintainer
sim-pdf-chunker 0.0.1
A simple PDF chunking utility for splitting PDF text into manageable pieces1 version - Latest release: 6 months ago - 1 maintainer
llm-chunk-optimizer 0.1.2
Semantic-aware chunking with provenance tracking for production RAG and LLM data pipelines2 versions - Latest release: 6 months ago - 59 downloads last month - 1 maintainer
chunk-bench 1.0.0
Benchmark RAG chunking strategies on your own documents. Compare fixed, sliding, paragraph, recur...1 version - Latest release: 6 months ago - 35 downloads last month - 1 stars on GitHub - 1 maintainer
llm-context-forge 0.1.5
Production-grade LLMOps infrastructure for context window management, token counting, document ch...2 versions - Latest release: 7 months ago - 333 downloads last month - 1 maintainer
ragpipe-ai 0.1.0
RAG in 3 functions. Ingest any data source into vector databases.1 version - Latest release: 7 months ago - 52 downloads last month - 3 stars on GitHub - 1 maintainer
chunkweaver 0.4.0
RAG chunker that respects document structure.5 versions - Latest release: 7 months ago - 48 downloads last month - 0 stars on GitHub - 1 maintainer
stump-cli 0.2.0
Stump: AST-aware code chunking (tree-sitter). Derived from ASTChunk; cite the cAST paper for rese...2 versions - Latest release: 7 months ago - 32 downloads last month - 0 stars on GitHub - 1 maintainer
omnichunk 2.1.4
Structure-aware deterministic chunking for code, prose, and markup.16 versions - Latest release: about 2 months ago - 125 downloads last month - 8 stars on GitHub - 1 maintainer
llama-index-node-parser-kreuzberg 0.1.0
Element-aware LlamaIndex node parser for kreuzberg-extracted documents1 version - Latest release: 7 months ago - 105 downloads last month - 1 maintainer
contexthub 0.1.0
A lightweight context management library for LLM applications3 versions - Latest release: 7 months ago - 164 downloads last month - 1 maintainer
seedbraid 2.0.5 💰
Reference-based file reconstruction with CDC chunking, SBD1 binary seed format, and IPFS transport10 versions - Latest release: 7 months ago - 40 downloads last month - 0 stars on GitHub - 1 maintainer
notebooklm-chunker 0.5.0
Turn long PDFs into chunked NotebookLM workflows with Studio outputs6 versions - Latest release: 7 months ago - 19 downloads last month - 20 stars on GitHub - 1 maintainer
mr-ninja 2.0.0
Large Context Orchestrator for GitLab Duo -- chunk oversized MRs and run multi-agent security ana...3 versions - Latest release: 7 months ago - 29 downloads last month - 2 stars on GitHub - 1 maintainer
markdownheadertextsplitter 0.1.0
A heading-aware Markdown splitter with a simple local UI1 version - Latest release: 7 months ago - 164 downloads last month - 1 maintainer
lasr 0.1.0
LASER (Least Action Semantic Router) — globally optimal text chunking for RAG1 version - Latest release: 8 months ago - 33 downloads last month - 1 maintainer
streamchunk 2.1.0
Adaptive data stream chunker with CPU-aware parallel processing for real-time ETL pipelines3 versions - Latest release: 6 months ago - 73 downloads last month - 1 stars on GitHub - 1 maintainer
gi-text-summarizer 1.0.0
Lightweight text summarizer with token budgets and smart chunking1 version - Latest release: 8 months ago - 202 downloads last month - 1 maintainer
docuweave 0.2.1
Layout-aware PDF chunking for production RAG pipelines6 versions - Latest release: 3 months ago - 48 downloads last month - 13 stars on GitHub - 1 maintainer
Related Keywords
nlp
163
rag
132
ai
131
parsing
96
agents
88
llm
87
tools
86
orchestration-framework
84
orchestration
84
monorepo
84
modular
84
metrics
84
measures
84
llm-framework
84
factories
84
tooling
84
vectors
84
swarmauri
59
embeddings
59
pdf
40
python
39
tool
35
retrieval
34
document-processing
27
vector-database
24
ocr
24
retrieval-augmented-generation
23
langchain
22
markdown
22
semantic-search
20
document
19
jupyter
18
text-splitting
17
openai
17
text
16
vector-search
15
notebook
15
semantic-chunking
14
chunk
13
parser
13
vector
13
designed
12
extraction
12
docx
11
text-processing
11
embedding
11
rust
11
ollama
11
llamaindex
10
tree-sitter
10
evaluation
10
mcp
10
community
9
store
9
text-chunking
9
tokenization
9
vectorstore
9
claude
9
semantic
9
docling
9
natural-language-processing
8
documents
8
gemini
8
text-extraction
8
RAG
8
machine-learning
8
anthropic
7
pinecone
7
document-parsing
7
xlsx
7
html
7
text-splitter
7
information-retrieval
7
cli
6
nbconvert
6
tokens
6
example
6
ast
6
context-window
6
client
6
pptx
6
faiss
6
qdrant
6
document-intelligence
6
node
6
deduplication
6
streaming
5
artificial-intelligence
5
pipeline
5
llms
5
NLP
5
kernel
5
code-analysis
5
reranking
5
ingestion
5
context
5
chromadb
5
prompt-engineering
5
elixir
5
model-context-protocol
5