chunk-engine
Chunking Modes

structural (default)

Element-level chunks — one per paragraph, heading, table, list or code block.

View raw

structural produces one chunk per structural element: each heading, paragraph, table, list and code block becomes its own chunk. It is the finest-grained mode and the best choice when you want element-level control over indexing.

Showing examples for Python
— your choice follows you across the docs.

Real input → real output

from py_chunks import get_chunks

# "structural" is what "default" resolves to for prose formats: one chunk per
# block, headings kept as their own chunks.
for c in get_chunks("notes.md", mode="structural"):
    print(c["content_type"], "|", c["metadata"]["section_heading"])

# heading | None
# plain_paragraph | Chunking Notes
# heading | None
# plain_paragraph | Why structure matters
# table | Why structure matters
# heading | None
# plain_paragraph | Modes

default vs structural

For most formats, default is element-level chunking and the two are the same pipeline.

  • DOCX: default is normalized to structural — they're equivalent.

  • Spreadsheets / CSV: default maps to row, and structural is not a valid mode at all (mode must be one of [page_aware, row, semantic, sheet, sliding_window, table] for XLSX).

  • PDF: default and structural are not the same, and the difference is heading classification — the text is identical either way.

    • default ranks type sizes within each page. No pass over the document, so it holds one page at a time: on a 9,000-page PDF that is 630 MB instead of 900 MB. It is not measurably faster — parsing the page content dominates, not the ranking.

      The gap only appears on very large documents

      Re-measured 2026-08-16 on a 9,086-page PDF (Apple M-series, py-chunks 0.6.2): peak RSS 597 MB for default vs 833 MB for structural — the documented ratio holds. On ordinary documents the difference is invisible: a 117-page PDF measures 71 MB vs 73 MB against a ~29 MB interpreter baseline. Choose between these two modes on heading semantics, not on memory, unless your documents run to thousands of pages.

    • structural ranks sizes across the whole document, giving one consistent heading hierarchy. Use it when heading levels matter.

    Neither is strictly better. Per-page ranking finds a title that appears once in a long paper (which document-wide ranking discards as too rare to be structure); document-wide ranking gives levels that mean the same thing on every page. On arxiv_1301.3781_word2vec.pdf the split is visible in the counts: default yields 11 heading chunks and 2 short_disconnected_paragraph, structural yields 10 and 3.

If you're unsure, default is a safe starting point for every format.

Units and formats

FamilyOne chunk per
Markdown, HTML, TXT, PDF, EPUB, RTF, EML/MBOX, MSG, ODT, ODP, JSON, IPYNBmarkdown block — heading, paragraph, list, table or code block
DOCX, DOCparagraph / table (DOCX groups short runs into mixed_content)
PPTX, PPTslide, with short-slide merging
Spreadsheetsnot a valid mode — use row
CSV / TSVnot a valid mode — use row

What it emits

This mode is typed by what each chunk contains, so the set is the widest of any mode:

heading · plain_paragraph · long_single_paragraph · short_disconnected_paragraph · bullet_list · table · code_block · mixed_content (DOCX)

Measured: code_heavy.md → 22 heading + 17 code_block + 13 short_disconnected_paragraph + 2 table; all_round.docx → 8 mixed_content + 2 table + 1 heading; arxiv_1301.3781_word2vec.pdf → 24 plain_paragraph + 19 long_single_paragraph + 10 heading + 10 table + 3 short_disconnected_paragraph.

The full content-type list, with a real example of each, is in the Output Schema.

Parameters

structural and default read none of the four mode parameters.

Metadata

Element chunks carry section_heading, section_level and document_metadata at minimum; DOCX adds page and list information, PDF adds total_pages. Full listing in the Metadata Reference.

When to use it

  • Fine-grained retrieval where each element should be independently searchable.
  • Downstream logic that re-groups elements itself.
  • Precise citations back to a single paragraph or table.

On this page