chunk-engine

Metadata Reference

The exact metadata keys each format emits per mode — so you can filter, branch, and show provenance with confidence.

View raw

Every chunk's metadata is a dictionary whose keys depend on both the format and the mode. Output Schema covers the common keys; this page is the full matrix for when you need to rely on a specific field.

Always use optional access

Treat every field as optional — read it with chunk["metadata"].get(key) (Python), chunk.metadata[key] with a presence check (JS), or chunk.metadata.get(key) on the serde_json::Value (Rust). Keys are identical across languages; only JavaScript's top-level chunk field is camelCase (contentType), while everything inside metadata is passed through untouched.

Document formats

Keys by mode for the prose/binary document families (PDF, DOCX, DOC, PPTX, PPT, Markdown, HTML, TXT).

ModeFormat(s)Notable metadata keys
default / structuralPDFpage_number, is_heading, avg_font_size
default / structuralDOCXsection_heading, section_heading_level, footnotes (list of {id, text}), endnotes, page_number, document_metadata (header_text, footer_text, image_count)
default / structuralDOCsource, chunk_index, total_chunks, paragraph_type (heading/normal/table/list_item), heading_level (1–6 or null), page_number (always null — not in the binary format)
default / structuralMD / HTML / TXTsection_heading, document_metadata.source_type
default / structuralPPTXslide_number, section_heading (when detectable)
sectionPDFpage_number, section_heading, section_level, heading_path, paragraph_count, heading_font_size
sectionDOCXsection_heading, section_heading_level, section_level, heading_path, document_metadata
sectionDOCsource, chunk_index, total_chunks, paragraph_type, heading_level, page_number
sectionMD / HTML / TXT / PPTXsection_heading, section_level, heading_path, paragraph_count
semanticPDFpage_number, paragraph_count, merge_reason
semanticDOCXsection_heading, section_heading_level, paragraph_count, merge_reason, document_metadata
semanticDOCsource, chunk_index, total_chunks, paragraph_type, heading_level, page_number
semanticMD / HTML / TXT / PPTXparagraph_count, merge_reasons (list), primary_merge_reason, keyword_density, avg_block_length (MD/TXT), section_heading, heading_path, chunk_index, document_metadata
sentencePDFsentences_per_chunk, actual_sentence_count, chunk_index, source_paragraph_index
sentenceDOCXsentences_per_chunk, actual_sentence_count, chunk_index, source_paragraph_index, source_paragraph_is_heading, source_paragraph_heading_level, source_paragraph_is_list, source_paragraph_is_table, document_metadata
sentenceDOCsource, chunk_index, total_chunks, paragraph_type, heading_level, page_number
sentenceMD / HTML / TXT / PPTXsentences_per_chunk, actual_sentence_count, chunk_index, source_paragraph_index
sliding_windowPDFwindow_size, overlap, window_index, paragraph_count, paragraph_range, page_number
sliding_windowDOCXwindow_size, overlap, window_index, paragraph_indices, list_item_count, heading_count, paragraph_meta, document_metadata
sliding_windowDOCsource, chunk_index, total_chunks, paragraph_type, heading_level, page_number
sliding_windowMD / HTML / TXT / PPTXwindow_size, overlap, window_index, paragraph_count, paragraph_range
page_awarePDFpage_number, page_break_type, paragraph_count, document_metadata
page_awareDOCXpage_number, page_break_type, paragraph_count, section_heading_level, headings, list_item_count, table_count, document_metadata
page_awareDOCsource, chunk_index, total_chunks, paragraph_type, heading_level, page_number
page_awareMD / HTML / TXTpage_number, page_break_type, paragraph_count
page_awarePPTXslide_numbers, paragraph_count

Spreadsheets (XLSX / XLS)

Modecontent_typeNotable metadata keys
rowrow_documentsheet_name, sheet_index, row_index, header_row, col_count, rows_per_chunk, actual_row_count, chunk_index
tabletable_regionsheet_name, sheet_index, table_name, is_named_table, header_row, start_row, end_row, start_col, end_col, row_count, col_count, chunk_index, is_split, split_part
sheetsheetsheet_name, sheet_index, row_count, col_count, header_row, has_named_tables, named_tables, chunk_index, is_split, split_part
sliding_windowrow_windowsheet_name, sheet_index, window_size, overlap, actual_row_count, window_index, start_row, end_row, header_row, col_count, chunk_index
page_awaresheet_regionsheet_name, sheet_index, has_print_area, print_area_ref, start_row, end_row, start_col, end_col, row_count, col_count, header_row, region_index, chunk_index, is_split, split_part
semanticsemantic_groupsheet_name, sheet_index, category_column, category_value, used_fallback, low_grouping_quality, avg_group_size, start_row, end_row, actual_row_count, header_row, col_count, group_index, chunk_index

XLS vs XLSX

.xls cannot expose named tables or print areas: in table mode is_named_table is always false (heuristic bounding-box detection is used), in sheet mode has_named_tables is always false with an empty named_tables, and page_aware always falls back to the full sheet (has_print_area: false). Every other mode is identical to .xlsx.

CSV / TSV

Modecontent_typeNotable metadata keys
row / default / page_awarerow_grouprow_start, row_end, row_count, col_count, header_row, delimiter_detected, encoding, chunk_index
sliding_windowrow_windowwindow_index, window_size, overlap, row_start, row_end, actual_row_count, col_count, header_row, delimiter_detected, encoding, chunk_index

Image chunks

When list_images is on, each content_type="image" chunk carries format-specific keys:

FormatImage-chunk metadata keys
DOCXimage_name, alt_text, page_number, document_metadata
PPTXslide_number, image_name, alt_text, document_metadata
XLSXsheet_name, sheet_index, image_name, alt_text
HTMLimage_name, alt_text, document_metadata (source_type: "html")
PDFpage_number, image_name, document_metadata (source_type: "pdf", total_pages) — no alt_text; always re-encoded to .png
DOCimage_name, paragraph_index (or null for floating/unanchored images)
PPTimage_name, page_number (1-based slide, or null if it can't be safely attributed)

Enumerated values

Two semantic-mode fields take a fixed set of values:

  • DOCX merge_reason — one of heading_merge, keyword_overlap, reference_continuity, short_paragraph, transition_break.
  • MD / HTML / TXT / PPTX primary_merge_reason — one of reference_continuity, elaboration, example, cause_effect, contrast_continuation, question_answer, definition_expansion, short_paragraph, keyword_overlap, or initial (for singleton chunks).

The full merge_reasons list (every signal that fired) is also present for MD/HTML/TXT/PPTX semantic chunks; primary_merge_reason is just the winning one. See Chunking Modes → semantic.

On this page