How py-chunks, js-chunks, and rs-chunks relate — what each one can do, and how their output is kept identical.

chunk-engine ships as three packages that share one Rust core. Choose whichever fits your stack; the chunks are the same.

Package	Registry	Install	Import name	How it works
py-chunks	PyPI	`pip install py-chunks`	`py_chunks`	Native Python extension via PyO3
js-chunks	npm	`npm install js-chunks`	`js-chunks`	WASM core — Node, Bun, Deno, browsers
rs-chunks	crates.io	`cargo add rs-chunks`	`chunks_rs`	The Rust engine itself

Where the code lives, and what "reference" means

Two different things are easy to conflate, so, precisely:

rs-chunks is the source of truth for the engine code. The parsing and chunking logic lives there. py_chunks and js-chunks each vendor a copy of the crate (they publish from separate repos and cannot path-depend on a sibling), and a sync script rewrites those copies from rs-chunks and fails the build if either has drifted. A vendored copy that nothing checks is a copy that drifts — that is exactly how three past defects happened.
py-chunks is the behavioural baseline the parity harness compares against. It is the oldest of the three and the one whose output existing indexes were built from, so "unchanged" is defined as "matches py-chunks". The harness dumps chunks from the Rust engine and re-computes them through the Python binding, which is what proves the vendored copies and the binding layers do not distort anything.

So: same code, one direction of code flow, and an independent behavioural check in the other direction. Neither statement makes the other false.

API dialects

The operations are identical; the surface follows each language's conventions.

Concept	Python	JavaScript	Rust
Batch	`get_chunks(...)`	`getChunks(...)`	`get_chunks(...)`
Stream	`stream_chunks(...)`	`streamChunks(...)`	`formats::<fmt>::stream(...)`
Markdown	`get_markdown(...)`	`getMarkdown(...)`	`get_markdown(...)`
Images	`list_images=True`	`listImages: true`	`get_chunks_with_images_from_bytes(...)`
Content type field	`chunk["content_type"]`	`c.contentType`	`c.content_type`
Metadata	`dict`	`Record<string, unknown>`	`serde_json::Value`
Options	keyword args	options object	positional args
Failures	built-in exceptions	thrown `ChunkError`	`Err(ChunkError)`

Metadata keys are snake_case in all three, including JavaScript — only the content_type field itself is camelCased on the JS Chunk.

Rust cannot extract images from a path at the crate root. There is no get_chunks_with_images — only get_chunks_with_images_from_bytes. Read the file yourself, or call formats::docx::chunk_with_images(path, …).

JavaScript has no delimiter or encoding option. CSV files that are semicolon-separated or latin-1 encoded need Python or Rust, or a re-encode before you hand over the bytes.

Parity

Three separate harnesses, three separate claims. They measure different things and it is worth keeping them apart.

Chunk parity — rs ↔ py

3,222 / 3,222 comparisons byte-identical (100%), over every fixture × every mode, last re-verified 2026-08-08.

Every family is identical — OOXML, legacy binary (.doc / .ppt), OpenDocument, email, ebook, PDF, notebook and delimited — including semantic-mode primary_merge_reason: all three engines share the same deterministic tie-break (sort by count descending, then key ascending), so there is no residual non-determinism. get_markdown is compared by the same harness, because a public API once diverged silently while a chunks-only check stayed at zero mismatches.

Image parity — js ↔ py

374 / 375 fixtures identical, comparing image names and image bytes across every format that carries images.

Names matter as much as bytes here. The rs↔py image check compares two native builds, so a target-dependent fault cancels out; this harness is the one that catches a WASM-vs-native difference.

PDF parity — js ↔ py

23 / 24 fixtures identical, comparing chunks, markdown and image bytes across three modes.

The one difference, in both js harnesses

It is the same fixture and the same cause in each: a scanned PDF with no text layer and no embedded page image. Falling back to a page render requires a rasteriser, and there is no PDFium in WASM. Python and Rust return page rasters; JavaScript reports "no extractable text". Both harnesses list the fixture by name and assert the count, so the exception cannot quietly grow to cover something else.

Reproduce the claim

Run from the workspace root, with py_chunks built (maturin develop --release) and js-chunks built (npm run build:wasm && npm run build):

# Chunk parity, rs <-> py — 3,222 / 3,222
cd rs-chunks && cargo run --release --example parity_dump > /tmp/parity.jsonl
cd ../py_chunks && python ../rs-chunks/examples/parity_check.py < /tmp/parity.jsonl

# Image parity, js <-> py — 374 / 375 (dumps from py, then runs the node checker)
python image_parity.py

# PDF parity, js <-> py — 23 / 24
python pdf_parity.py

Adversarial inputs fail with a clean, catchable error and never panic: every public dispatch entry point runs the parse behind a catch_unwind boundary, so a panic inside a third-party parser surfaces as ChunkError::Parse rather than unwinding into your process.

Choosing an SDK

Start from where the bytes already are.

Python if you are behind a web framework, ingesting from S3, or living in a data/ML stack. It has the widest source surface — uploads, file objects and URLs are one call each — plus delimiter/encoding for awkward CSVs, and it is the behavioural baseline, so it is the one to reach for when you need to settle an argument about what the "right" output is.
JavaScript if the document is in a browser, an edge function, or a Node service that already holds it as a File or a Buffer. It is the only SDK that runs client-side, and the only one that can chunk Markdown you parsed elsewhere. Accept that a scanned PDF with no text layer will not work, that paths are Node/Bun only, and that streaming is ergonomic rather than bounded.
Rust if you are embedding chunking in a service and want no FFI, no WASM, and full control. It gives you the format modules directly — per-format options, ChunkOptions, and real incremental streaming for CSV, XLSX and PDF. Accept the positional-argument dispatch API and the bytes-only image entry point.

Mixing them is fine and is the point. Index with Python in a batch job, serve previews from JavaScript in the browser, and both see the same chunks — so long as you pin the same version. The three packages share one version number and are released together.

Next steps

Installation — requirements, wheels, and pinning
Input Sources — every entry point, per SDK
Streaming — the per-format, per-runtime matrix
Error Handling — the full error contract
API Reference — signatures for all three languages

Capability	py-chunks	js-chunks	rs-chunks
Filesystem path	yes	Node / Bun only	yes
Raw bytes	yes	yes	yes
File-like object	yes	—	—
Framework upload (`UploadFile` …)	yes	—	—
HTTP(S) / pre-signed URL	yes	—	—
`Blob` / `File`	—	yes	—
Markdown-string input	—	`chunkPdfMarkdown` family	—
`delimiter` / `encoding` options	yes, on every entry point	no	per-format fns or `ChunkOptions`
Images from a path	yes	yes	no at crate root — per-format `chunk_with_images` (12 formats)
Images from bytes	yes	yes	yes
Streaming entry point	source-agnostic `stream_chunks`	`streamChunks` (all sources)	per-format only — no dispatch-level stream
Streaming from bytes	yes (writes a temp file)	yes	`pdf` and `xlsx` only
Genuinely incremental streaming	PDF, XLSX	none	CSV, XLSX, PDF (`default` mode)
Scanned PDF, no text layer	page rasters (PDFium, vendored)	"no extractable text"	page rasters with `pdf-native` (default on; off for `wasm32`)
Error type	`FileNotFoundError` · `ValueError` · `TypeError` · `RuntimeError` · `OSError` · `OverflowError` (negative counts, some formats)	`ChunkError` for everything, including host `fs` errors (`kind: "io"`, Node's verbatim message)	`ChunkError` (`#[non_exhaustive]`)
Typed mode enum	no — strings	yes, `ChunkMode` union	yes, `ChunkMode` enum (reachable via `ChunkOptions`)
Format-specific chunkers	18 `chunk_` + 18 `stream_chunk_`	—	17 format modules

Languages & parity