# Introduction

One document chunking engine for RAG — 36 formats, one Rust core, with byte-identical bindings for Python, JavaScript, and Rust.

**chunk-engine** turns any document into clean, retrieval-ready chunks in a
single call. A Rust core does the parsing and segmentation; thin bindings give
you the same API — and the **same output** — from **Python, JavaScript, or
Rust**, across **36 file formats**.

```python
from py_chunks import get_chunks, stream_chunks, get_markdown

# Batch — works for every supported format
chunks = get_chunks("document.pdf")
chunks = get_chunks("notes.md",  mode="semantic")
chunks = get_chunks("deck.pptx", mode="sliding_window", window_size=3, overlap=1)

for chunk in chunks:
    print(chunk["content"], chunk["content_type"], chunk["metadata"])

# Streaming — constant memory over huge files
for chunk in stream_chunks("large.pdf", mode="section"):
    handle(chunk)

# Markdown conversion
md = get_markdown("report.docx")
```

```ts
import { getChunks, streamChunks, getMarkdown } from "js-chunks";

// Batch — works for every supported format
let chunks = await getChunks("./document.pdf");
chunks = await getChunks("./notes.md",  { mode: "semantic" });
chunks = await getChunks("./deck.pptx", { mode: "sliding_window", windowSize: 3, overlap: 1 });

for (const c of chunks) {
  console.log(c.content, c.contentType, c.metadata);
}

// Streaming — constant memory over huge files
for await (const chunk of streamChunks("./large.pdf", { mode: "section" })) {
  handle(chunk);
}

// Markdown conversion
const md = await getMarkdown("./report.docx");
```

```rust
use chunks_rs::{get_chunks, get_markdown};

// Batch — dispatch by extension. Positional args:
// get_chunks(path, mode, window_size, overlap, sentences_per_chunk, paragraphs_per_page)
let chunks = get_chunks("document.pdf", "default", 3, 1, 3, 15)?;
let chunks = get_chunks("notes.md",     "semantic", 3, 1, 3, 15)?;

for c in &chunks {
    println!("[{}] {}", c.content_type, c.content);
    // c.metadata is a serde_json::Value with format-specific provenance
}

// Streaming yields the same chunks, one at a time
use chunks_rs::formats::csv;
for c in csv::stream("data.csv", "row", 10, 5, 1, true, None, "utf-8", true)? {
    let c = c?;
}

// One-shot Markdown conversion
let md = get_markdown("report.docx")?;
```

Every chunk has three fields — `content`, a typed `content_type`, and
`metadata` — ready to embed, index, or feed to an LLM.

## One engine, three languages

| Package | Install | Runtime |
| --- | --- | --- |
| **py-chunks** (PyPI) | `pip install py-chunks` | Python, via PyO3 |
| **js-chunks** (npm) | `npm install js-chunks` | Node · Bun · Deno · browsers, via WASM |
| **rs-chunks** (crates.io) | `cargo add rs-chunks` | Rust — the reference engine |

  The SDKs wrap the same engine and are parity-checked to emit exactly the same
  chunks — 2204/2214 comparisons identical, with the remainder tracing to the
  reference engine's own non-determinism. See [Languages &
  parity](/docs/languages).

## Why chunk-engine

- **Rust-backed speed.** Parsing and chunking run in a compiled core, not a stack
  of interpreted dependencies.
- **One API, every format.** The same entry points work for Word, PowerPoint,
  Excel, PDF, HTML, Markdown, email, eBooks, notebooks, and more.
- **Structure-aware chunking.** Seven document modes and dedicated spreadsheet
  modes keep headings, tables, and lists intact.
- **Streaming built in.** Yield chunks with constant memory over huge files.

## Start here

  - [Installation](/docs/installation) — Install for Python, JavaScript, or Rust.
  - [Quick Start](/docs/quick-start) — Chunk your first document in a few lines.
  - [Languages & parity](/docs/languages) — How the three SDKs relate and stay identical.
  - [API Reference](/docs/api-reference) — Signatures for all three languages.