Benchmarks
Measured on neutral hardware
The whole 654-file corpus on a neutral x86 box, against the peers that do the same job — Docling and Unstructured, each running its full documented install. Every number re-derives from raw results committed alongside the report.
- Corpus
- 654 files · 32 format directories · the whole corpus, no sampling
- Machine
- Neutral x86_64 VPS · 4 vCPU (AMD EPYC class) · 15 GiB
- OS
- Linux 5.15 · glibc 2.35 · Python 3.13.15
- Runtimes
- py-chunks 0.6.3 (built from the tree; the raw JSON records 0.6.2 because the run predates the version bump) · docling 2.120.1 · unstructured 0.25.2 [all-docs] · LibreOffice present (version not captured in the report)
One run per file on a shared 4-vCPU VPS, so timings are indicative. Competitors run their full documented install — Unstructured with [all-docs], Docling with LibreOffice available.
This describes the competitive figures — coverage and speed — from the 2026-08-18 run. The token-budget section further down is a separate, later run on a different machine and states its own environment; the two are not comparable.
The raw report records Linux-5.15.0-185-generic-x86_64-with-glibc2.35, processor x86_64 and Python 3.13.15. The CPU model, core count and RAM were recorded by the operator at run time and are not in the JSON.
Coverage
Reads formats the others don't
Every tool attempted every file for real. Encrypted documents and deliberately hostile fixtures are excluded from the counts rather than quietly scored as wins or losses.
Format directories with any text extractedformat directories
Individual documentslegitimate files only
This measures one thing: whether the tool returned any text. It is not a quality, structure or speed result, and character counts are a coverage signal rather than a score.
Where the gap is
Format breadth, not magic
The headline is a coverage result. It is worth being specific about which formats produce it.
Only chunk-engine extracted text
All three tools handled
On mainstream office and web formats every tool works. The spread opens on templates, macro-enabled variants, OpenDocument and legacy binary spreadsheets. A corpus of PDF and DOCX will show far less difference than this suggests.
The three documents chunk-engine could not read
Published rather than buried — and it works: the previous run listed five, two of which were non-UTF-8 HTML that both competitors read. That was a real defect, and publishing it is what got it fixed.
- A scanned PDF with no text layer
- nothing to extract — chunk-engine says so and points to list_images — docling recovered 14 characters; unstructured failed
- A 100-page scanned PDF
- no text layer — both competitors failed too
- A malformed .xlsb workbook
- structurally invalid or unsupported — both competitors failed too
Speed
Two orders of magnitude
Measured only on the files all three tools successfully read, so nothing is credited for returning fast because it returned nothing.
| Tool | Median / file | Total |
|---|---|---|
| chunk-engine | 0.001s | 5.7s |
| unstructured | 0.084s84× slower | 622.1s |
| docling | 0.121s121× slower | 2182.2s |
Basis: the 414 files all three tools successfully read (95 MB). Median and total diverge because a handful of large PDFs dominate the totals — the median is the honest headline, the total is shown so the spread is visible rather than hidden.
| Family | n | chunk-engine | docling | unstructured |
|---|---|---|---|---|
| 41 | 0.036s | 15.590s | 1.006s | |
| EPUB | 10 | 0.021s | 1.024s | 14.346s |
| Outlook .msg | 14 | 0.001s | 6.628s | 0.021s |
| PowerPoint (legacy) | 22 | 0.017s | 3.417s | 3.088s |
| Word (legacy) | 24 | 0.001s | 2.887s | 2.083s |
| Spreadsheet | 103 | 0.001s | 0.038s | 0.036s |
Median seconds per file, same basis. Both competitors also pay a one-off model load which the harness warms out before measuring.
Token budgets
The budget holds — audited, not assumed
fit_tokens / fitTokens (0.6.4) re-fit engine chunks to your tokenizer's budget. Every output part is re-counted after the run; a part over budget would be a recorded failure, and there are none. Absolute numbers only — no competitor is run on this axis.
| Pass | Chunks in → out | Time | Chunks/s | Parts over budget |
|---|---|---|---|---|
| Python · budget 512 | 506,303 → 510,657 | 48.86s | 10,362.3 | 0 |
| Python · + overlap 64 | 506,303 → 511,407 | 44.43s | 11,395.5 | 0 |
| JavaScript · budget 512 | 377,597 → 381,951 | 77.19s | 4,891.8 | 0 |
| JavaScript · + overlap 64 | 377,597 → 382,701 | 62.87s | 6,006 | 0 |
Worst case (synthetic, labelled as such): a single 180,000-character whitespace-free CJK run splits into 446 parts in 0.26s — 0 over budget, reassembling losslessly. Before 0.6.4 this input was returned as one chunk, over budget, silently.
Environment: macOS-26.5.1-arm64-arm-64bit-Mach-O · Python 3.13.7 · py-chunks 0.6.4 · Node v24.14.0 · js-chunks 0.6.4 · tokenizers tiktoken 0.14.0 · cl100k_base (Python) and gpt-tokenizer · cl100k_base (JavaScript). Median of 3 timed passes each, every sample committed in the reports; JavaScript passes run in a fresh process apiece. Python and JavaScript use DIFFERENT tokenizer implementations and produce different chunk counts, so the two columns are not a language race and are not presented as one. Run on a different machine than the competitive figures above — the two sets are not comparable. Basis: 649 corpus files read.
Metadata
Provenance shaped to the format
Not part of the competitive run — an engine property you can verify by chunking one file of each kind.
Per-format metadata schemas
Every chunk carries provenance shaped to its format, not a lowest common denominator:
Coverage
Formats × modes
Which chunking modes each format family supports — derived directly from the library source.
| Format family | default | structural | section | semantic | sliding_window | sentence | page_aware | row | table | sheet |
|---|---|---|---|---|---|---|---|---|---|---|
| Word (.docx .doc …) | ||||||||||
| PowerPoint (.pptx .ppt …) | ||||||||||
| PDF (.pdf) | ||||||||||
| Web & Markup (.html .md) | ||||||||||
| Plain text (.txt) | ||||||||||
| Email (.msg .eml) | ||||||||||
| OpenDocument (.odt .odp) | ||||||||||
| eBooks / Notebooks (.epub .ipynb) | ||||||||||
| JSON (.json .jsonl) | ||||||||||
| Rich text (.rtf) | ||||||||||
| Spreadsheets (.xlsx .xls .ods …) | ||||||||||
| Delimited (.csv .tsv) |
Full disclosure
What this does not measure
A benchmark is only worth the caveats printed next to it.
- One run per file, no median-of-N. A median-of-3 was attempted and abandoned because it tripled every LibreOffice conversion. Timings are indicative.
- A shared 4-vCPU VPS. Noisy neighbours widen variance beyond a dedicated machine. A 121× median gap is far outside that noise; a 1.2× difference would not be.
- The harness excludes .json fixtures (a guard against sweeping up its own result files), so JSON coverage is unmeasured on both architectures despite .json being a supported format.
- Character count is a coverage signal, not a quality score. More characters can mean better extraction or more boilerplate.
- Chunk quality, structure fidelity, content-type accuracy and retrieval performance are NOT measured here. Figures for those axes were previously published from a run whose raw results were lost; they were removed rather than restated.
- Both competitors pay a one-off model load, which was warmed out before measuring, so it is excluded from every timing above. It is substantial on a GPU-less VPS — that is a property of the box, not a defect in either tool. No cold-start figure is published here because none was captured in a committed result file.
Every number on this page re-derives from the raw results committed beside their reports — x86_full_654_c7.json and its derived companion for the competitive figures, fit_tokens_bench.json and fit_tokens_bench_js.json for the token-budget section. If a figure cannot be traced to a result file, it does not belong here.