Recursive Character Splitter

Recursive Character Text Splitter is a hierarchical text chunking algorithm that recursively evaluates a prioritized delimiter list (e.g., `["\n\n", "\n", ". ", " ", ""]`) to segment long documents; while strictly bounding chunk length within target Token limits (`chunk_size`), it maximizes text preservation at the highest semantic boundaries (such as double newlines `\n\n`) and enforces overlapping context (`chunk_overlap`) between consecutive windows.

Universal baseline chunking strategy for general RAG systems, applied across text articles, technical blogs, and documentation.

Naive fixed-length character slicing blindly shears words and sentences midway, creating broken sentence fragments; recursive splitting preserves natural paragraph/sentence integrity, and `chunk_overlap` ensures critical boundary-straddling facts remain intact.

Analyze the trade-offs of `chunk_size` and `chunk_overlap` on embedding semantic density vs duplicate recall overhead?

How to customize delimiter lists for bilingual Chinese-English text to correctly prioritize Chinese punctuation and Markdown headings?

Why must chunk length be evaluated using the target embedding tokenizer rather than Python's raw `len()` character count?

How to tune `chunk_overlap` to prevent context bloat and LLM attention dilution from redundant duplicated segments?

How to apply AST-based code splitters for Python/JS to recursively segment code at class and function boundaries?