Skip to main content
ClaudeWave
Skill3.1k repo starsupdated 3d ago

ai-persistence/build-cloudflare-artifact-store

Use when a Cloudflare Worker needs durable byte storage for TanStack AI generated media (images, audio, video, transcripts) — writes a BlobStore backed by R2 and an ArtifactStore backed by D1, composes them onto the generation persistence so withGenerationPersistence persists artifact bytes, and serves them back from a Worker GET route. Includes one-line sketches for S3, GCS, Vercel Blob, Supabase, and a dev filesystem BlobStore.

Install in Claude Code
Copy
git clone --depth 1 https://github.com/TanStack/ai /tmp/ai-persistence-build-cloudflare-artifact-store && cp -r /tmp/ai-persistence-build-cloudflare-artifact-store/packages/ai-persistence/skills/ai-persistence/build-cloudflare-artifact-store ~/.claude/skills/ai-persistence-build-cloudflare-artifact-store
Then start a new Claude Code session; the skill loads automatically.

SKILL.md

# Cloudflare Artifact + Blob Store

`withGenerationPersistence(persistence)` needs only `stores.generationRuns` to track a
generation's lifecycle. Add `stores.artifacts` (metadata) **and** `stores.blobs`
(the bytes) — both, or neither — and the middleware also persists the generated
media: image/audio/TTS/video/transcription bytes land at blob key
`artifacts/<runId>/<artifactId>`, with an `ArtifactRecord` row describing each.

The deliverable is **one file in the Worker** — e.g.
`src/lib/generation-persistence.ts` — exporting a factory that builds an
`AIPersistence` from the request's R2 + D1 bindings, plus a GET route that serves
artifact bytes with `retrieveArtifact` / `retrieveBlob`.

Read the sibling **ai-persistence/build-cloudflare-adapter** skill for the
per-request-binding rule, `wrangler` config shape, D1 migration workflow, and the
chat (generation-run/message) side. This skill covers only the two byte-storage stores and
how to compose them.

## The two contracts

Both come from `@tanstack/ai-persistence`. `defineBlobStore` / `defineArtifactStore`
type an object literal inline (autocomplete + contract checking, no separate
annotation).

```ts
// BlobStore — the byte layer. R2 backs it.
interface BlobStore {
  put: (
    key: string,
    body: BlobBody,
    options?: BlobPutOptions,
  ) => Promise<BlobRecord>
  // metadata + byte accessors; `options.range` reads one slice (for `206`s)
  get: (key: string, options?: BlobGetOptions) => Promise<BlobObject | null>
  head: (key: string) => Promise<BlobRecord | null> // metadata only
  delete: (key: string) => Promise<void> // no-op if absent
  list: (options?: BlobListOptions) => Promise<BlobListPage>
}

// ArtifactStore — the metadata layer. D1 backs it.
interface ArtifactStore {
  save: (record: ArtifactRecord) => Promise<void> // insert or overwrite
  get: (artifactId: string) => Promise<ArtifactRecord | null>
  list: (runId: string) => Promise<Array<ArtifactRecord>> // [] when none
  listForThread: (threadId: string) => Promise<Array<ArtifactRecord>>
  delete: (artifactId: string) => Promise<void>
  deleteForRun: (runId: string) => Promise<void>
}
```

`list` and `listForThread` return records ordered by `createdAt`, then by the
ordinal bytewise order of `artifactId`. Compare UTF-8 bytes from left to right.
Do not use locale collation.

`BlobBody` is `ReadableStream<Uint8Array> | ArrayBuffer | ArrayBufferView |
string | Blob`. The non-stream shapes flow straight into `R2Bucket.put`
unchanged — but a `ReadableStream` body does **not**, in the general case:
workerd's `put` requires a stream with a known length (a `Response` body or the
readable half of a `FixedLengthStream`), and the artifact middleware hands you a
`TransformStream`-wrapped body whenever it had to cap a fetched body as it
drains. Passing that stream to `bucket.put` throws `TypeError: Provided readable
stream must have a known length`.

**When does that actually happen?** The wrapper only exists to enforce
`maxArtifactBytes` during the drain, so the middleware applies it only when
nothing else bounds the transfer:

| Provider response                       | Body handed to `put`              | R2 path             |
| --------------------------------------- | --------------------------------- | ------------------- |
| `content-length`, no `content-encoding` | untouched, declared length intact | `bucket.put` direct |
| chunked (no declared length)            | wrapped, length-less              | multipart           |
| `content-encoding: gzip`                | wrapped, length-less              | multipart           |

A provider CDN normally sends `content-length`, so the first row is the common
case and `bucket.put(key, body)` just works. The recipe below is what makes the
other two rows work: it re-declares the length from
`BlobPutOptions.expectedLength` when the middleware could vouch for one, and
otherwise streams through a multipart upload (one 8 MiB part at a time — flat
memory at any artifact size). Write it once and every response shape is
covered.

`withGenerationPersistence(persistence, { maxArtifactBytes: false })` drops the
ceiling and the wrapper altogether, so even a chunked reply arrives untouched.
It buys nothing extra for R2 (a chunked body has no length to preserve), so
choose it on its own merits: no _application_ limit on what an origin can
stream into your bucket. R2's own limits still apply — 5 GiB per single-shot
put, and 10,000 multipart parts (~80 GiB at the 8 MiB part size below). Keep
the cap when `allowInputUrl` lets callers name the URL.

`BlobPutOptions` is
`{ contentType?, customMetadata?, expectedLength? }`; `BlobGetOptions` is
`{ range?: { offset: number, length?: number } }` and maps onto R2's own
`range`; `BlobListOptions` is `{ prefix?, cursor?, limit? }`; `BlobListPage` is
`{ objects: BlobRecord[], cursor?, truncated? }`.

## 1. BlobStore backed by R2

`R2Object` carries `size`, `etag`, `httpMetadata.contentType`, `customMetadata`,
and `uploaded` (a `Date`). `BlobRecord` wants `createdAt` / `updatedAt` as epoch
ms — R2 tracks only the single `uploaded` instant, so map it to both. `get` /
`head` are the byte-body vs metadata-only split; `R2ObjectBody` already exposes
`body`, `arrayBuffer()`, and `text()`, so a `BlobObject` is essentially the R2
object plus the mapped metadata.

```ts ignore
import { defineBlobStore, resolveBlobRange } from '@tanstack/ai-persistence'
import type { BlobObject, BlobRecord } from '@tanstack/ai-persistence'

// R2 multipart parts must be ≥ 5 MiB and — except for the last — all exactly
// the SAME size, so a part reader has to cut on an exact boundary and carry the
// remainder. 8 MiB × the 10,000-part ceiling puts the multipart path's limit at
// ~80 GiB; raise this for larger objects, and check R2's current object-size
// limits before promising more.
const MULTIPART_PART_SIZE = 8 * 1024 * 1024

/**
 * Cut exactly `limit` bytes off the stream (fewer only at EOF), carrying any
 * overshoot into the next