Skip to content

PRD: AI Platform Maturity & Production Readiness

Document ID: PRD-004 File Path: docs/01-product/prd-ai-platform-maturity.md Version: 1.0.1 Status: In delivery — Phases 1–2 of the derived ticket docs are complete and Phase 3 is partial (22 of 39 tickets done; see the v1.1 index); Phase 3’s remaining ecosystem items are re-prioritized through PRD-005’s lens per the v1.2 milestone Owner: Product / AI Engineering Last Updated: 2026-07-15


PRD-003 closed the deployed-vs-designed gap that made the GA appliance defensible: authentication, crash-safety, an execution driver, a contract-tested API. This PRD closes the next gap — the capability-vs-credible-AI-product gap — surfaced by a 2026-07-09 full-project engineering audit (five parallel deep-dives: AI core, workflow+server, dashboard UI, tools/plugins/sandbox, and DX/CI/deployment/docs).

Where PRD-003 asked “is the platform safe and honest?”, this PRD asks “is it a capable, operable AI product a team would actually build on?” The audit’s one-sentence verdict:

Wovyr has excellent primitives, but three structural problems recur at every layer: sophisticated machinery is built and tested yet never wired into the real run path; the AI core is missing table-stakes production capabilities; and single-node assumptions are baked into surfaces the product markets as multi-tenant.

This document turns the audit’s ~90 findings into numbered workstreams and testable requirements, then hands them to three phased ticket docs (RM-AIM-P1, RM-AIM-P2, RM-AIM-P3).

This is a planning input, not a promise. Requirements graduate to committed work through the roadmap (v1.1) and, where they change a boundary contract, through an ADR.


The audit confirmed PRD-003’s hardening shipped and holds. The remaining gaps are not safety gaps — they are capability, correctness, and operability gaps:

  1. The AI core is a thin loop. run_agent has no context-window management (the full history is cloned into every request and grows unbounded), no tokenizer, sequential-only tool execution, and no native Anthropic provider. Worst of all, cost tracking is hardcoded to $0 for every real provider, which silently disables the per-project quota enforcement PRD-003 built.

  2. The strong machinery is dead code. Container/gVisor/Firecracker sandboxes, the warm SandboxPool, and the FairScheduler are all built and tested — but the run path hardcodes native_only(), so none of them ever run. On Windows the “sandbox” enforces only a timeout. The distributed workflow/queue/lease layer has the same shape (PRD-003 Path A deferred it, correctly).

  3. The RAG stack is missing its middle. No chunking (whole documents get one diluted embedding), no re-ranking, a semantic cache whose key omits the system prompt and embedding-model id (→ wrong-context hits), and no retrieval-quality evaluation.

  4. The evaluation harness is a prototype, not a gate. Substring/exact scoring only; no LLM-as-judge, no baselines, no thresholds, no variance measurement.

  5. The ecosystem has no connective tissue. No MCP (Model Context Protocol) client to reach external tool servers, no plugin authoring SDK/scaffold, a tiny read-biased builtin toolset (no fs_write, code-exec, or web-search).

  6. Operability and DX lag the code. Version pinned at 0.1.0 despite a v0.3.0 tag, no CHANGELOG, no release automation, no published container image; the dashboard and Windows are both absent from CI; the CLI reference doc misdescribes the primary entry point; the dashboard has zero tests and stores bearer tokens in localStorage.

The encouraging counterweight, again: because the abstractions are correct, most fixes are “wire the good primitive onto the path” or “add the missing stage,” not “redesign.”


  • Shipped & hardened (v0.1–v1.0 GA): agent runtime, workflow engine, memory engine, LLM gateway, tool runtime, plugin engine + marketplace, multi-tenancy, events/webhooks, audit, secrets, KMS, the RM-GA-P1..P4 security/durability/ contract floor, and (2026-07-09) the GA-003 residual findings + an S3-compatible backup destination.
  • The gap this PRD closes: the difference between a hardened appliance and a capable, operable, extensible AI product — the AI-core depth, RAG maturity, eval rigor, ecosystem connectivity, sandbox activation, distributed correctness, and DX/release/UI hygiene that the audit found missing.

  • Make the platform’s own production claims true: real cost accounting (so quotas mean something), context/token management (so long runs don’t silently truncate), activated sandboxing (so isolation is real), and durable/graceful server behavior (so a restart isn’t data loss).
  • Make it a credible AI product: native Anthropic support, structured output, guardrails, a real RAG middle (chunking + reranking), and an evaluation gate that can actually catch a regression.
  • Make it extensible: MCP, a plugin authoring SDK, and a richer, ergonomic tool surface.
  • Make it operable and adoptable: consistent versioning + release automation + a published image, the dashboard and Windows in CI, accurate docs, and a UI with tests and a shared component system.
  • Billions-of-memories / thousands-of-concurrent-runs capacity engineering at real scale (PRD-002 post-GA Scale work).
  • Marketplace monetization / billing (PRD-002).
  • Multi-region residency (PRD-002).
  • A cloud-KMS/HSM-backed root key (fast-follow; tracked in GA-003).

  • Requirements are R-<workstream>.<n>, testable, and cite the audit finding(s) they close (§12 traceability). Each maps to one or more tickets in the phase docs.
  • Priority: P0 = correctness/data-loss/quota-integrity blocker, P1 = needed for a credible product, P2 = quality bar, P3 = fast-follow.
  • Severity below mirrors the audit’s own (High/Med/Low).
  • “Done” = implemented, covered by an automated test that fails on regression, and the linked docs/ spec updated (and for wire changes, both SDKs + openapi.yaml).

Twelve workstreams (A–L). Each requirement’s ticket code and phase are in §12.

  • R-A.1 (P0) — Context-window management: add a tokenizer and a token-budgeted history compactor (drop/summarize oldest tool turns) before each model call; stop cloning unbounded history. (Audit: no context mgmt, no token counting.)
  • R-A.2 (P0) — Execute independent tool calls in one turn concurrently, ordered by call id. (Sequential tool loop.)
  • R-A.3 (P1) — Apply the agent manifest’s max_steps as the default budget in run_agent_inner (not only via wovyr-runtime). (max_steps ignored.)
  • R-A.4 (P1) — Step-level error recovery: retry a recoverable model-step error; on budget exhaustion, force a final tool-less answer instead of hard-erroring.
  • R-A.5 (P2) — Richer streaming: emit tool-call-argument and reasoning/thinking events, not just content deltas.
  • R-B.1 (P0) — Per-model price table; compute cost_usd from returned token usage in every provider. (Cost hardcoded $0 → quotas are a no-op.)
  • R-B.2 (P1) — First-class AnthropicProvider (Messages API: native tool-use, system handling, prompt caching, extended thinking). (No native Claude.)
  • R-B.3 (P1) — Add response_format/tool_choice/json_schema to ChatRequest and translate per provider (JSON mode, forced tool). (No structured output.)
  • R-B.4 (P2) — Normalize/validate tool JSON-schema; surface tool-arg parse failures back to the model instead of passing null. (Verbatim pass-through.)
  • R-B.5 (P2) — Multimodal content parts (image/audio) on Message.content.
  • R-B.6 (P3) — Retry jitter + honor Retry-After; distinguish 429 vs 5xx.
  • R-C.1 (P1) — Document chunking with parent-document linkage before embedding.
  • R-C.2 (P1) — A re-ranking stage (cross-encoder or LLM reranker) after RRF.
  • R-C.3 (P1) — Fix the semantic-cache key (include system prompt + tools) and stamp/verify the embedding-model id on every entry. (Wrong-context hits.)
  • R-C.4 (P2) — BM25/TF-IDF in-process keyword search for backend parity.
  • R-C.5 (P2) — Real timestamps on records + range/time metadata filters.
  • R-C.6 (P3) — Incremental re-embedding / embedding-model migration.
  • R-D.1 (P1) — LLM-as-judge + semantic-similarity scoring alongside substring.
  • R-D.2 (P1) — Turn wovyr-eval into a real gate: golden baselines, pass-rate thresholds, variance-over-N, persisted score artifacts compared in CI.
  • R-D.3 (P2) — Evaluate the RAG path (run_agent_with_memory) and manifest max_steps; add retrieval metrics (recall@k / nDCG / MRR).
  • R-E.1 (P0) — Wire SandboxManager::detect() + SandboxPool into the agent/ server run path so container/gVisor/Firecracker actually run. (Dead code.)
  • R-E.2 (P0) — Windows Job Object for memory/CPU/PID limits in the non-Unix native path. (Windows = timeout only.)
  • R-E.3 (P1) — Confined fs_write builtin; R-E.4 (P1) — a sandboxed code-execution tool; R-E.5 (P2) — a #[derive(Tool)]/schemars schema+typed- param ergonomics upgrade.
  • R-E.6 (P2) — Document the platform matrix and fail closed when egress lockdown is unavailable (non-Linux).
  • R-F.1 (P1) — An MCP client tool-source (stdio/HTTP) proxying external tools into ToolRegistry. (No external tool servers at all.)
  • R-F.2 (P1) — A plugin authoring SDK crate + wovyr plugin new scaffold (manifest + wasm build + digest computation + trust snippet). (Format docs only.)
  • R-F.3 (P2) — A container capability loader (reuse ContainerSandbox).
  • R-F.4 (P2) — One-shot wovyr plugin publish (sign + fill digests + emit trust).
  • R-F.5 (P3) — Marketplace OSV/CVE feed keyed on SBOM name@version.
  • R-G.1 (P0) — Graceful shutdown/drain (with_graceful_shutdown + SIGTERM).
  • R-G.2 (P1) — Durable async-run store (or documented non-durability), so a restart doesn’t orphan pollable runs.
  • R-G.3 (P1) — Durable webhook outbox + delivery worker with persisted DLQ.
  • R-G.4 (P1) — API-key lifecycle: created/expires/revoked metadata, a revoke endpoint, rotation, last-used. (Mint-only today.)
  • R-G.5 (P1) — Distributed rate limiting (shared store) for multi-node.
  • R-G.6 (P1) — Per-tenant token quotas; enforce or remove the two dead quota dimensions; per-tenant rate tier.
  • R-G.7 (P2) — Tenant-configurable daily-cost reset boundary (timezone).
  • R-G.8 (P2) — Cache FileApiKeyStore in memory; served OpenAPI; request-path unwrap audit; idempotency-store write-amplification fix; move the ~2,260-line inline lib.rs test suite out.
  • R-H.1 (P0) — Postgres connection pool (deadpool/bb8) + reconnect/health.
  • R-H.2 (P0) — Sub-workflow recursion depth guard / ancestor-cycle detection.
  • R-H.3 (P1) — TLS to Postgres; R-H.4 (P1) — fenced event-sequence generation (DB identity/sequence + lease-token fencing), replacing MAX(seq)+1.
  • R-H.5 (P1) — Loop / for-each (map-over-collection) activity.
  • R-H.6 (P1) — Dynamic (data-driven) fan-out.
  • R-H.7 (P2) — Checkpoint size cap + out-of-line large activity outputs; event- log compaction + paged load; indexed list() columns + SQL-side pagination; fire_at-indexed timers + adaptive dispatch sleep.
  • R-H.8 (P3) — Activity progress events; event-enum schema versioning.
  • R-I.1 (P1) — Content-safety / moderation / PII-redaction hooks on model input and output in the agent loop.
  • R-I.2 (P2) — A prompt template/versioning registry (variables, versions, A/B).
  • R-I.3 (P1) — Default the secrets store to encrypted-at-rest (plaintext becomes the explicit opt-out); R-I.4 (P2) — audit-log time-range + cursor pagination + indexed sink; R-I.5 (P3) — request-scoped secret channel (vsock/stdin) instead of WOVYR_SECRET_* env injection.
  • R-J.1 (P1) — Reconcile versioning: bump workspace/badges/SDKs to the real tag, add a maintained root CHANGELOG.
  • R-J.2 (P1) — Release automation: tag-triggered signed binaries + a published container image (GHCR/Docker Hub) + npm/PyPI SDK publish + generated changelog.
  • R-J.3 (P1) — Add the dashboard (build/lint/test) and a Windows matrix leg to CI.
  • R-J.4 (P2) — SDK parity: async Python client, mutation retry (with Idempotency-Key), a wait_for_completion poll helper, TS paginateAll, coverage/benchmark tracking in CI.
  • R-J.5 (P2) — SDK versioning tied to API version + per-SDK CHANGELOG + server/ SDK skew warning; reconcile the Python PyPI-publish claim.
  • R-J.6 (P2) — Regenerate docs/11-cli/commands.md from the real clap tree; add per-doc shipped/aspirational status front-matter; add a top-of-README 5-minute quickstart; unify the repository URL across manifests.
  • R-J.7 (P3) — Decide/document Go/Java clients (roadmap or non-goal).
  • R-K.1 (P1) — Move bearer token off localStorage (in-memory/session or BFF-issued httpOnly cookie). (XSS-exfiltratable.)
  • R-K.2 (P1) — UI test coverage: service specs (SSE parser, manifest round-trip)
    • a smoke e2e; drop the global skipTests:true.
  • R-K.3 (P2) — Shared component library (StatusPill/Tabs/Modal/Table/empty/ loading/error), replacing the duplicated statusClass/errText patterns and the native confirm().
  • R-K.4 (P2) — Share API types with sdks/typescript (or generate from OpenAPI); replace string-built YAML with a real (de)serializer; central HTTP error handling (no swallowed errors).
  • R-K.5 (P2) — Audit-log viewer surface; R-K.6 (P2) — responsive/mobile breakpoints; R-K.7 (P2) — accessibility pass (label associations, aria on icon buttons, modal focus mgmt).
  • R-K.8 (P3) — Prompt playground; live nav badges (or remove the fakes); i18n decision; icon sprite for bundle hygiene.
  • R-L.1 (P1) — Per-tenant/per-project metric labels (bounded cardinality).
  • R-L.2 (P2) — Queue-depth / in-flight / pending-timer / webhook-DLQ gauges.
  • R-L.3 (P2) — Spans around Postgres/queue/dispatcher operations.
  • R-L.4 (P2) — A systemd unit + install script for the single-node appliance; an operator upgrade/backup/migration runbook.
  • R-L.5 (P3) — SLO / error-budget burn-rate metrics + starter alert rules; Helm HA/TLS templating; a minimal Terraform module (or explicit scope-out).

7. Distributed Scale-Out (folded from PRD-003 Path B)

Section titled “7. Distributed Scale-Out (folded from PRD-003 Path B)”

PRD-003 deferred the distributed platform (Path B) to a “v1.1 Scale-Out” milestone. Several of its wiring tickets overlap this PRD’s correctness work and are absorbed here as P1 items where they are single-node-correctness bugs today (R-G.5 distributed rate limiting, R-H.1 pooling, R-H.4 fencing), and left as PRD-002 capacity work where they are pure scale. The multi-replica shared-catalog promotion (PRD-003 R-5.1/R-5.2) remains gated on Product demand and is tracked, not committed, in v1.1 §Scale-Out.


Three phases, ordered by dependency (calendar dates omitted per house convention). Within a phase, items are parallelizable.

  • Phase 1 — Make production claims true (P0/P1). The fixes that make existing features actually work: R-B.1 (cost table) is the single highest-leverage item — it silently disables quota enforcement everywhere. Plus context/token mgmt, sandbox activation, graceful shutdown, durable async runs, Postgres pool, release reconciliation, and dashboard+Windows CI. → RM-AIM-P1
  • Phase 2 — Credible AI product (P1/P2). Anthropic provider, RAG chunking + reranking, semantic-cache correctness, structured output, guardrails, the eval gate, distributed rate limiting, per-tenant quotas. → RM-AIM-P2
  • Phase 3 — Ecosystem & scale (P2/P3). MCP, plugin SDK, workflow loops/fan-out, encrypted-secret default, UI component library + audit viewer + responsive, SDK parity, docs, systemd/runbooks, observability gauges. → RM-AIM-P3

The trap to avoid: shipping more surface (or more UI) before R-B.1 lands — every cost/quota number the product reports until then is fiction.


This PRD’s scope is met when:

  1. Every real provider reports accurate cost_usd; per-project quotas enforce real spend (WS-B/R-B.1). A long tool loop never silently exceeds the context window (WS-A/R-A.1).
  2. The agent run path uses the strongest available sandbox for the run’s trust class; Windows runs enforce real resource limits (WS-E).
  3. Claude is a first-class provider; structured output and moderation hooks exist (WS-B, WS-I).
  4. Memory chunks + reranks; the semantic cache never serves a wrong-context hit (WS-C). wovyr-eval fails CI on a real quality regression (WS-D).
  5. A server restart loses no pollable run or pending webhook; shutdown drains (WS-G). Multi-node rate limits and quotas are correct (WS-G/WS-H).
  6. Versioning/CHANGELOG/release automation exist and a container image is published; the dashboard and Windows are in CI (WS-J).
  7. MCP and a plugin SDK exist; the UI has tests and a shared component system (WS-F, WS-K).

  • R: R-B.1 (cost table) changes quota behavior from “always $0, never blocks” to “actually blocks” — a behavior change operators may not expect. Mitigation: ship behind a documented rollout; log computed cost before enforcing.
  • R: WS-A context compaction can change agent outputs (summarizing history). Mitigation: make the strategy configurable; default to lossless drop-oldest-tool-turns before summarization.
  • R: WS-E activating strong sandboxes changes latency/perf characteristics. Mitigation: the warm SandboxPool already exists to amortize; benchmark first.
  • A: The trait-port architecture holds (PRD-003 confirmed), so provider/sandbox/ reranker additions are new impls behind existing traits, not redesigns.
  • A: The audit’s file:line evidence (captured 2026-07-09) is current; each ticket re-verifies before implementing.


12. Traceability Matrix — Findings → Requirements → Tickets

Section titled “12. Traceability Matrix — Findings → Requirements → Tickets”

Severity is the audit’s. Phase/ticket codes are authoritative; the phase docs implement exactly these.

Audit finding (abbrev.)SevReqTicketPhase
No context-window mgmt; no tokenizerHighR-A.1AIC-1011
Sequential tool-call executionHighR-A.2AIC-1021
Manifest max_steps ignored by run_agentMedR-A.3AIC-1031
No step-error recovery / budget-exhaust discards workMedR-A.4AIC-2012
Streaming is content-onlyLowR-A.5AIC-2022
Cost tracking hardcoded $0 (no price table)HighR-B.1PRV-1011
No native Anthropic providerHighR-B.2PRV-2012
No structured output / tool_choiceMedR-B.3PRV-2022
Tool-schema pass-through; null-swallowMedR-B.4PRV-2032
No multimodalMedR-B.5PRV-2042
Retry no jitter / ignores Retry-AfterLowR-B.6PRV-2052
Memory: no chunkingHighR-C.1RAG-2012
Memory: no re-rankingHighR-C.2RAG-2022
Semantic-cache key / embedding-model mismatchHighR-C.3RAG-2032
Naive keyword search (no BM25)MedR-C.4RAG-2042
No real timestamps / range filtersLowR-C.5RAG-2052
No incremental re-embeddingLowR-C.6RAG-3013
No retrieval-quality evalMedR-D.3EVL-2032
Eval scoring substring-only (no LLM-judge)HighR-D.1EVL-2012
Eval not a regression gateHighR-D.2EVL-2022
Strong sandboxes dead code (native_only)HighR-E.1SBX-1011
Windows: no resource limitsHighR-E.2SBX-1021
No fs_write builtinMedR-E.3SBX-3013
No code-execution toolMedR-E.4SBX-3023
Custom-tool ergonomics (no derive)MedR-E.5SBX-3033
Egress lockdown Linux-only, silentMedR-E.6SBX-3043
No MCP / external tool serversHighR-F.1ECO-3013
No plugin authoring SDK/scaffoldHighR-F.2ECO-3023
No container/microVM plugin loadersMedR-F.3ECO-3033
Plugin signing UX frictionMedR-F.4ECO-3043
Marketplace scanner static-only (no CVE)MedR-F.5ECO-3053
No graceful shutdown/drainHighR-G.1SRV-1011
Async runs not durableHighR-G.2SRV-1021
Webhook delivery in-process, no outboxHighR-G.3SRV-1031
API keys no expiry/rotation/revocationHighR-G.4SRV-1041
Rate limiter in-process onlyHighR-G.5SRV-2012
Only 2/4 quota dims; no token quotaMedR-G.6SRV-2022
Daily-cost window UTC-onlyMedR-G.7SRV-2032
FileApiKeyStore reads file per requestMedR-G.8SRV-3023
No served OpenAPI; no WebSocketMedR-G.8SRV-3033
lib.rs ~86% inline testMedR-G.8SRV-3043
Idempotency full-file write per reqMedR-G.8SRV-3053
Request-path unwrapsLowR-G.8SRV-3063
In-process concurrency slotsLowR-G.6SRV-3073
Postgres single client, no poolHighR-H.1WFL-1011
Unbounded sub-workflow recursionHighR-H.2WFL-1021
No TLS to PostgresHighR-H.3WFL-1031
Event seq via MAX+1 raceMedR-H.4WFL-1041
No loop/for-eachHighR-H.5WFL-3013
No dynamic fan-outHighR-H.6WFL-3023
Checkpoint write-amp / no payload capMedR-H.7WFL-3033
Event log no compaction / full loadMedR-H.7WFL-3043
list() scans all checkpointsMedR-H.7WFL-3053
Timer/schedule poll O(N) / accuracyMedR-H.7WFL-3063
No activity progress eventsLowR-H.8WFL-3073
Event enum no versioningLowR-H.8WFL-3083
wovyr-runtime ai activity underspecifiedMedR-A.4RUN-2012
Sub-agent observability lost (NullSink, $0)LowR-B.1RUN-2022
No guardrails/moderation/PIIMedR-I.1SAF-2012
No prompt template/versioningMedR-I.2SAF-2022
Default secrets plaintextHighR-I.3SEC-1011
Audit query no time-range/pagination/scaleMedR-I.4SEC-3013
Secret injection env-var leak surfaceLowR-I.5SEC-3023
Version 0.1.0 vs v0.3.0; no CHANGELOGHighR-J.1DX-1011
No release automation / published imageHighR-J.2DX-1021
Dashboard + Windows absent from CIHighR-J.3DX-1031
SDK parity (async py, retry, poll helper)MedR-J.4DX-3013
No coverage/benchmark trackingMedR-J.4DX-3023
SDK versioning strategy absentMedR-J.5DX-3033
commands.md out of syncMedR-J.6DX-3043
Docs status front-matter; quickstart; repo URLLowR-J.6DX-3053
Go/Java clients decisionLowR-J.7DX-3063
Token in localStorageHighR-K.1UI-1011
Zero UI test coverageHighR-K.2UI-1021
No shared component abstractionMedR-K.3UI-3013
API types drift; string-built YAML; swallowed errorsMedR-K.4UI-3023
No audit-log viewerHighR-K.5UI-3033
Weak responsive/mobileMedR-K.6UI-3043
Accessibility thinMedR-K.7UI-3053
Playground / nav badges / i18n / icon spriteLowR-K.8UI-3063
No per-tenant metricsMedR-L.1OBS-2012
No queue-depth/DLQ gaugesMedR-L.2OBS-3013
Uneven trace coverageLowR-L.3OBS-3023
No systemd/install; no upgrade runbookHighR-L.4DEP-3013
No SLO burn; Helm HA/TLS; TerraformLowR-L.5DEP-3023

VersionDateDescription
1.0.12026-07-15Status updated Draft → In delivery: Phases 1–2 complete, Phase 3 partial (22/39), remainder re-prioritized through PRD-005; no content changed
1.0.02026-07-09Initial AI-platform-maturity PRD: a 2026-07-09 five-front engineering audit’s ~90 findings mapped to 12 workstreams / testable requirements, phased into three ticket docs (RM-AIM-P1/P2/P3), with the distributed Scale-Out fold-in and a full findings→requirements→ticket traceability matrix