sentence
A fixed number of sentences per chunk — for the blocks that are made of sentences.
sentence splits prose into groups of sentences_per_chunk sentences. When you
have a tight token budget and want predictable chunk sizes, this gives you
consistent, sentence-aligned boundaries.
Real input → real output
from py_chunks import get_chunks
# Groups of sentences_per_chunk sentences (default 3), never crossing a block.
# Headings and tables stay whole and carry no sentence count.
for c in get_chunks("notes.md", mode="sentence"):
print(c["content_type"], "|", c["metadata"].get("actual_sentence_count"))
# heading | None
# sentence | 2
# heading | None
# sentence | 2
# table | None
# heading | None
# sentence | 3
# sentence | 1 <- the short final group of a 4-sentence paragraphNote the last group: a four-sentence paragraph at sentences_per_chunk=3
produces a chunk of 3 and a chunk of 1. Groups never span a block, so a short
tail is normal.
Most chunks in this mode are not sentences
sentence mode does not turn the whole document into sentence groups. In
the markdown-family formats it splits paragraphs and leaves every other kind
of block alone: headings, tables, code blocks and lists pass through as
their own typed chunks.
Lists are deliberately not split — cutting a bullet list at "three sentences"
produces fragments that mean nothing. A bullet_list chunk in this mode
carries actual_sentence_count: 0, which is the tell: it was never sentence-split.
What it emits
| Format | content_type values emitted |
|---|---|
| Markdown, HTML, TXT, PDF, EPUB, RTF, EML/MBOX, MSG, ODT, ODP, JSON | sentence, plus heading, bullet_list, code_block and table wherever the document has them |
| IPYNB | code_block for code cells; heading / bullet_list / sentence for markdown cells |
| DOCX, DOC, PPTX family, PPT | sentence only |
| Spreadsheets, CSV/TSV | not supported |
Measured: code_heavy.md → 13 sentence + 22 heading + 17 code_block +
2 table; _stress_big_list.md → 63 bullet_list + 1 sentence +
1 heading; all_round.docx → 11 sentence and nothing else.
If your pipeline assumes content_type == "sentence", a list-heavy Markdown
document will silently drop almost all of its content.
Units and formats
| Family | What gets sentence-split |
|---|---|
| Markdown-family | paragraph blocks only. Headings, lists, tables and code blocks pass through untouched. |
| DOCX, DOC | every paragraph |
| PPTX, PPT | all slide text, split across slides |
| Spreadsheets, CSV/TSV | not supported |
Parameters
| Parameter | Default | Notes |
|---|---|---|
sentences_per_chunk | 3 | Sentences per group. Must be > 0, or sentences_per_chunk must be greater than 0 is raised. |
The same argument means something else for spreadsheets
sentences_per_chunk is also what the unified entry point uses to derive
rows-per-chunk for spreadsheets and CSV — including a sentinel that makes a
deliberate 3 unreachable. See
Chunking Modes → rows per chunk.
Metadata
sentence chunks carry actual_sentence_count (the number this chunk really
holds — the last group of a block is often short), sentences_per_chunk (what
you asked for), source_paragraph_index, heading_path, section_heading,
section_level and chunk_index. Pass-through chunks (heading, table,
code_block) omit the sentence keys entirely; bullet_list chunks keep them
with actual_sentence_count: 0. Full listing in the
Metadata Reference.
When to use it
- Strict token budgets where uniform chunk sizes matter.
- Models or indexes that assume short, even inputs.
- Sentence-level highlighting or citation.