# Recipes

Practical, copy-pasteable patterns for RAG ingestion, batch processing, and streaming — in Python, JavaScript, or Rust.

Common end-to-end patterns. Every snippet is language-aware — use the tabs (or
the global switcher in the nav) to see Python, JavaScript, or Rust.

## RAG ingestion

Chunk a document into semantically coherent passages, embed each one, and store
it with its metadata. `semantic` mode gives the best embedding boundaries.

```python
from py_chunks import get_chunks

def ingest(path):
    # Semantic chunks make the best embeddings.
    for chunk in get_chunks(path, mode="semantic"):
        vector = embed(chunk["content"])          # your embedding model
        store(vector, text=chunk["content"], meta=chunk["metadata"])
```

```ts
import { getChunks } from "js-chunks";

async function ingest(path: string) {
  // Semantic chunks make the best embeddings.
  for (const chunk of await getChunks(path, { mode: "semantic" })) {
    const vector = await embed(chunk.content);    // your embedding model
    await store(vector, { text: chunk.content, meta: chunk.metadata });
  }
}
```

```rust
use chunks_rs::get_chunks;

fn ingest(path: &str) -> anyhow::Result<()> {
    // Semantic chunks make the best embeddings.
    for c in &get_chunks(path, "semantic", 3, 1, 3, 15)? {
        let vector = embed(&c.content);           // your embedding model
        store(vector, &c.content, &c.metadata);
    }
    Ok(())
}
```

Each chunk carries typed `content_type` and `metadata` (section headings, page
numbers, …), so you can filter results or show provenance at query time — see
the [Output Schema](/docs/output-schema).

## Batch a folder of mixed formats

The same call handles every format, so a mixed folder just works. Isolate
failures per file so one bad document doesn't halt the batch.

```python
from pathlib import Path
from py_chunks import get_chunks

for path in Path("docs").rglob("*"):
    if not path.is_file():
        continue
    try:
        index(path.name, get_chunks(str(path)))
    except Exception as e:
        print("skip", path, e)   # one bad file shouldn't halt the batch
```

```ts
import { readdir } from "node:fs/promises";
import { getChunks } from "js-chunks";

for (const name of await readdir("docs")) {
  try {
    index(name, await getChunks(`docs/${name}`));
  } catch (e) {
    console.warn("skip", name, e);   // one bad file shouldn't halt the batch
  }
}
```

```rust
use std::fs;
use chunks_rs::get_chunks;

for entry in fs::read_dir("docs")? {
    let path = entry?.path();
    let p = path.to_string_lossy();
    match get_chunks(&p, "default", 3, 1, 3, 15) {
        Ok(chunks) => index(&path, &chunks),
        Err(e) => eprintln!("skip {p}: {e}"),  // keep going
    }
}
```

## Stream a large file

`stream_chunks` / `streamChunks` yield chunks one at a time with constant
memory — ideal for large files or streaming an HTTP response.

```python
from py_chunks import stream_chunks

# Constant memory — yields one chunk at a time
for chunk in stream_chunks("data.csv", mode="row"):
    handle(chunk)
```

```ts
import { streamChunks } from "js-chunks";

// Async iterable — one chunk at a time
for await (const chunk of streamChunks("./data.csv", { mode: "row" })) {
  handle(chunk);
}
```

```rust
use chunks_rs::formats::csv;

// stream() is a native Iterator yielding Result<Chunk>
for c in csv::stream("data.csv", "row", 10, 5, 1, true, None, "utf-8", true)? {
    let c = c?;
    handle(c);
}
```

  Streaming yields exactly the same chunks as the batch call, just incrementally.
  In Rust, streaming is per-format via `chunks_rs::formats::*::stream(...)`.

## Convert to Markdown

Get a Markdown string from any supported document (for previews, diffing, or
feeding a Markdown-native tool).

```python
from py_chunks import get_markdown

md = get_markdown("report.docx")           # -> str
md = get_markdown(file_bytes, filename="report.pdf")  # bytes also supported
```

```ts
import { getMarkdown } from "js-chunks";

const md = await getMarkdown("./report.docx");   // -> string
```

```rust
use chunks_rs::get_markdown;

let md = get_markdown("report.docx")?;     // -> String
```

See also [Framework Integration](/docs/framework-integration) for FastAPI,
Flask, Django, Express, and Axum handlers.