Installation
Install chunk-engine for Python (pip), JavaScript (npm), or Rust (cargo).
Pick your language — the install command follows the switcher at the top of the page.
pip install py-chunksPython — py-chunks
- Python 3.9+. The Rust engine ships compiled inside the wheel.
- One runtime dependency:
pypdfium2(installed automatically; bundles PDFium for PDF support).
pip install py-chunksJavaScript — js-chunks
- Node 18+, Bun, Deno, and browsers/bundlers. The engine is compiled to WASM and instantiated lazily on first call.
- Filesystem-path sources are Node/Bun only — elsewhere pass bytes with a
filename.
npm install js-chunksPDF in JavaScript
PDF parsing uses the optional peer dependency
@llamaindex/liteparse-wasm
(the same liteparse version as the Rust engine, so output is byte-identical).
Install it to enable .pdf sources:
npm install @llamaindex/liteparse-wasmIf you already have PDF markdown from another parser, chunk it directly with
chunkPdfMarkdown(markdown, totalPages, opts?) — no peer dependency needed.
Rust — rs-chunks
The reference engine. Published to crates.io as rs-chunks; the library import
name is chunks_rs.
cargo add rs-chunksuse chunks_rs::{get_chunks, get_markdown};PDF feature flag
Native PDF (via the liteparse crate / PDFium) is on by default through the
pdf-native feature. Disable default features for wasm32 targets, where PDF
markdown is produced host-side and fed to pdf::chunk_pdf_markdown.
Verify
from py_chunks import get_chunks, stream_chunks, get_markdown
# Batch — works for every supported format
chunks = get_chunks("document.pdf")
chunks = get_chunks("notes.md", mode="semantic")
chunks = get_chunks("deck.pptx", mode="sliding_window", window_size=3, overlap=1)
for chunk in chunks:
print(chunk["content"], chunk["content_type"], chunk["metadata"])
# Streaming — constant memory over huge files
for chunk in stream_chunks("large.pdf", mode="section"):
handle(chunk)
# Markdown conversion
md = get_markdown("report.docx")