chunk-engine (js-chunks) in Express, Fastify, Hono, Next.js, NestJS, and SvelteKit — across Node, Bun, and Deno.

getChunks is polymorphic over its source: pass a Uint8Array / ArrayBuffer / Buffer with a filename, or a named Blob/File (which carries its own name). WASM is instantiated lazily and cached on the first call.

Runtime note

js-chunks runs on Node, Bun, Deno, and browsers. In serverless/edge contexts, run it on a Node.js runtime (not the edge runtime) so the WASM module loads — see the Next.js example below. Bundled browser apps import the web build explicitly: see Installation.

Express (Node)

import express from "express";
import multer from "multer";
import { getChunks } from "js-chunks";

const app = express();
const upload = multer(); // in-memory: exposes req.file.buffer

app.post("/chunk", upload.single("document"), async (req, res) => {
  const chunks = await getChunks(req.file.buffer, {
    filename: req.file.originalname,
  });
  res.json({ chunks });
});

Fastify

import Fastify from "fastify";
import multipart from "@fastify/multipart";
import { getChunks } from "js-chunks";

const app = Fastify();
await app.register(multipart);

app.post("/chunk", async (req) => {
  const file = await req.file();          // @fastify/multipart
  const buffer = await file.toBuffer();
  const chunks = await getChunks(buffer, { filename: file.filename });
  return { chunks };
});

Hono (Node · Bun · Deno · Workers)

Hono's parseBody yields web File objects, so no filename plumbing is needed:

import { Hono } from "hono";
import { getChunks } from "js-chunks";

const app = new Hono();

app.post("/chunk", async (c) => {
  const body = await c.req.parseBody();
  const file = body["document"] as File;   // a named Blob
  const chunks = await getChunks(file);
  return c.json({ chunks });
});

export default app;

Next.js (App Router route handler)

// app/api/chunk/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getChunks } from "js-chunks";

export const runtime = "nodejs"; // the WASM module needs the Node runtime

export async function POST(req: NextRequest) {
  const form = await req.formData();
  const file = form.get("document") as File;
  const chunks = await getChunks(file);    // File is a named Blob
  return NextResponse.json({ chunks });
}

NestJS

import {
  Controller, Post, UploadedFile, UseInterceptors,
} from "@nestjs/common";
import { FileInterceptor } from "@nestjs/platform-express";
import { getChunks } from "js-chunks";

@Controller("chunk")
export class ChunkController {
  @Post()
  @UseInterceptors(FileInterceptor("document"))
  async chunk(@UploadedFile() file: Express.Multer.File) {
    const chunks = await getChunks(file.buffer, {
      filename: file.originalname,
    });
    return { chunks };
  }
}

SvelteKit

// src/routes/chunk/+server.ts
import { json } from "@sveltejs/kit";
import { getChunks } from "js-chunks";

export async function POST({ request }) {
  const form = await request.formData();
  const file = form.get("document") as File;
  const chunks = await getChunks(file);
  return json({ chunks });
}

Streaming an NDJSON response

Forward each chunk as a newline-delimited JSON record instead of buffering one big array. Two shapes cover almost every JS framework: a web ReadableStream (Next.js, Hono, SvelteKit, Deno, Bun, Workers) and a Node stream write loop (Express, Fastify, NestJS).

Next.js route handler (web `ReadableStream`)

// app/api/chunk/stream/route.ts
import { NextRequest } from "next/server";
import { streamChunks, ChunkError } from "js-chunks";

export const runtime = "nodejs";

export async function POST(req: NextRequest) {
  const form = await req.formData();
  const file = form.get("document") as File;

  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      try {
        for await (const chunk of streamChunks(file, { mode: "semantic" })) {
          controller.enqueue(encoder.encode(JSON.stringify(chunk) + "\n"));
        }
      } catch (e) {
        // Headers are already sent — report the failure as a final record
        // rather than trying to change the status code.
        const message = e instanceof ChunkError ? e.message : String(e);
        controller.enqueue(encoder.encode(JSON.stringify({ error: message }) + "\n"));
      } finally {
        controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: { "content-type": "application/x-ndjson" },
  });
}

Express (Node stream)

import express from "express";
import multer from "multer";
import { streamChunks, ChunkError } from "js-chunks";

const app = express();
const upload = multer();

app.post("/chunk/stream", upload.single("document"), async (req, res) => {
  res.setHeader("content-type", "application/x-ndjson");
  res.setHeader("cache-control", "no-cache");

  try {
    const chunks = streamChunks(req.file.buffer, {
      filename: req.file.originalname,
      mode: "semantic",
    });
    for await (const chunk of chunks) {
      // Respect backpressure: wait for drain if the socket is full.
      if (!res.write(JSON.stringify(chunk) + "\n")) {
        await new Promise((resolve) => res.once("drain", resolve));
      }
    }
  } catch (e) {
    const message = e instanceof ChunkError ? e.message : String(e);
    res.write(JSON.stringify({ error: message }) + "\n");
  } finally {
    res.end();
  }
});

What JS streaming does and doesn't give you

streamChunks in js-chunks computes the entire chunk array before it yields the first item — the WASM boundary is a synchronous full parse. So this handler starts writing only after chunking finishes; it does not lower peak memory, and it does not make the first byte arrive sooner than getChunks would have.

What it does give you is a response the client can consume incrementally (useful when the client is slow or wants to render progressively) and a natural place to apply backpressure. See Streaming for the full per-runtime picture.

Errors

Every failure js-chunks raises is a ChunkError — an Error subclass carrying a kind. There are no gaps in that: engine failures, unsupported extensions, bad arguments and host filesystem errors all arrive as ChunkError, so a single catch maps cleanly onto HTTP status codes. (One format skips argument validation instead of raising — see the note below the example.)

import { getChunks, ChunkError } from "js-chunks";

app.post("/chunk", upload.single("document"), async (req, res) => {
  try {
    const chunks = await getChunks(req.file.buffer, {
      filename: req.file.originalname,
    });
    res.json({ chunks });
  } catch (e) {
    if (e instanceof ChunkError) {
      const status = {
        unsupported: 415,     // Unsupported Media Type
        "invalid-arg": 400,   // Bad Request
        parse: 422,           // Unprocessable Entity
        io: 500,              // includes an unreadable path (ENOENT/EACCES)
        unknown: 500,
      }[e.kind];
      return res.status(status).json({ error: e.message, kind: e.kind });
    }
    throw e;   // not a ChunkError — a genuine bug, let it surface
  }
});

`kind` is a clean argument/parse split

overlap >= windowSize, windowSize = 0 and sentencesPerChunk = 0 are validated inside the engine and arrive as kind: "invalid-arg" for every format and on both the path and the bytes route — so the 400-vs-422 split above is accurate without any message matching.

EPUB participates too, as of the current build. It used to be the one format whose chunkers skipped the shared range check, so bad window or sentence arguments returned an empty array instead of raising — in py-chunks as well, so it was never a JavaScript-only quirk. It now raises ChunkError with kind: "invalid-arg" like every other format, before the book is parsed, so an EPUB upload needs no hand-written pre-check and an empty result is no longer ambiguous. See Error Handling.

Serverless & bundle size

The WASM binary is ~2.5 MB (chunks_wasm_bg.wasm, identical in the Node and web builds); the JS wrapper is a few kilobytes on top. That matters in three places:

Lambda / Vercel / Cloud Functions. 2.5 MB fits comfortably inside a zipped deployment package. The module is instantiated on the first call, not at import, and cached from then on — measured at ~9 ms on a warm local Node 22, so it is a real but small cold-start cost. Warm it at module scope if you care:
```
// module scope — runs once per container, not per request
await getChunks(new Uint8Array([0x23]), { filename: "warm.md" }).catch(() => {});
```
Edge runtimes. Use export const runtime = "nodejs". The auto-loader takes the Node path via createRequire; edge runtimes do not provide it.
Browser bundles. The bundler-invisible dynamic import means a bundled browser app must import js-chunks/web explicitly and serve chunks_wasm_bg.wasm as an asset — see Installation. Lazy-load it behind a route split; 2.5 MB in your main bundle is not what you want.

Browser

There's no server round-trip needed — pass the File from an <input type="file"> straight to getChunks:

import { getChunks } from "js-chunks";

input.addEventListener("change", async () => {
  const chunks = await getChunks(input.files[0]); // named File → filename inferred
  render(chunks);
});

This is exactly how the Playground works — chunking runs entirely client-side in WASM, and the file never leaves the browser.

PDF in JS

.pdf works with no extra install — the engine parses PDF in WASM. If you already have PDF markdown from another parser, chunk it with chunkPdfMarkdown(markdown, totalPages). See Installation.

JavaScript

On this page