pypi.org : split-markdown4gpt
A Python tool for splitting large Markdown files into smaller sections based on a specified token limit. This is particularly useful for processing large Markdown files with GPT models, as it allows the models to handle the data in manageable chunks.
Registry
-
Source
- Documentation
- JSON
purl: pkg:pypi/split-markdown4gpt
Keywords:
python
, nlp
, markdown
, natural-language-processing
, text-analysis
, openai
, text-summarization
, summarization
, text-processing
, gpt
, data-preprocessing
, mistletoe
, split-text
, text-tokenization
, openai-gpt
, gpt-3
, gpt-4
, gpt-35-turbo
, gpt-35-turbo-16k
, markdown-processing
License: Apache-2.0
Latest release: almost 2 years ago
First release: almost 2 years ago
Dependent repositories: 1
Downloads: 586 last month
Stars: 22 on GitHub
Forks: 2 on GitHub
See more repository details: repos.ecosyste.ms
Last synced: 20 days ago