Skip to content
Wovyr

Phase 4 — Contract & Operability: Implementation Tickets

Document ID: RM-GA-P4 File Path: docs/18-roadmap/v1.0/phase4-contract-operability-tickets.md Version: 1.11.0 Status: All of Phase 4 is done. WS-7 (API-701/702/703/704/705), WS-8 (OBS-801/802/803/804/805), and WS-9 (HLTH-901/902/903/904) are all shipped. Owner: Engineering (API / Platform) Last Updated: 2026-07-09

Purpose

Section titled “Purpose”

Phase 4 of PRD-003 §10 — contract & operability. Freezes a consistent, contract-tested API before the published SDKs acquire users, gives an on-call operator a real observability story, and clears the remaining codebase-health debt that makes the platform expensive to change.

Covers WS-7 (API contract stabilization), WS-8 (observability & operability), and the WS-9 remainder (executor unification, the latent CLI panic, config consolidation, and cleanup — the CI-matrix piece of WS-9 shipped as Phase-2 CI-901).

The trap this phase exists to avoid (PRD-003 §10): contract debt becomes permanent breaking-change debt the day the first external consumer depends on the current shapes. The Python SDK is already on PyPI, so WS-7 is time-sensitive even though its tickets are P1, not P0.

Ticket format matches RM-GA-P1 through RM-GA-P3.

Sequencing at a glance

Section titled “Sequencing at a glance”
WS-7 (freeze first — every day of delay hardens SDK debt) — ALL DONE
  API-701 (list envelopes) ─┐
  API-702 (casing policy)   ├─> API-704 (CI contract gate — locks the frozen shape) — Done
  API-703 (idempotency all) ─┘
  API-705 (deprecation headers) ─ independent — Done


WS-8 (all done)
  OBS-801 (metrics middleware)  ─ independent — Done
  OBS-802 (request-id correlation) ─ independent — Done
  OBS-803 (alert rules + dashboards) ─ independent — Done
  OBS-804 (audit coverage)      ─ independent — Done
  OBS-805 (dashboard login/CORS/build) ── depends on SEC-101, SEC-204 (Phase 1) — Done


WS-9 remainder
  HLTH-901 (unify executors) ─ independent (high value: silent behavioral divergence) — Done
  HLTH-902 (fix CLI panic)   ─ independent (benefits from CI-901 to detect) — Done
  HLTH-903 (wovyr-config crate) ─ independent — Done
  HLTH-904 (cleanup: gateway leak, deps, module splits) ─ independent — Done

Order WS-7 first. API-701/702/703 are the breaking pass; API-704 then locks it in CI so it can’t silently re-diverge; API-705 makes the deprecation policy mechanically enforceable. All four are done — the freeze is fully in place. WS-8 and WS-9 parallelize freely from here.

WS-7 — API Contract Stabilization

Section titled “WS-7 — API Contract Stabilization”

API-701 [P1] — Standardize all list endpoints on the cursor-pagination envelope

Section titled “API-701 [P1] — Standardize all list endpoints on the cursor-pagination envelope”

Problem. Six route groups use the standard {data, has_more, next_cursor, total_estimate} envelope (agents, workflows, orgs, projects, webhooks, memory-records), but six use ad-hoc shapes: audit {entries, total} (crates/wovyr-server/src/audit.rs), plugins {plugins, total} (plugins.rs), marketplace {listings, total} (marketplace.rs), secrets {secrets, total} (secrets.rs), tools {tools, total} (tools.rs), and memory query {results, count} (memory.rs:240). Every inconsistency already shipped in the PyPI SDK becomes a breaking change to fix later. (PRD-003 R-7.1; closes PP-16 contract portion.)

Change.

  • Migrate the six ad-hoc endpoints to the shared hardening::paginate() envelope ({data, has_more, next_cursor, total_estimate}), adding real cursor pagination where they currently return an unbounded list.
  • Memory :query returns ranked results, not a page — keep a {data, ...} shape but document it as a non-paginated result set explicitly, so it’s consistent in field naming even if not cursor-paged.
  • Update openapi.yaml and both SDKs in lockstep.

Acceptance criteria.

  • All list endpoints return data/has_more/next_cursor; a schema check confirms no endpoint uses entries/plugins/secrets/results/count as the top-level array key.
  • The TS + Python SDK integration suites pass against the new shapes.

Files. crates/wovyr-server/src/{audit,plugins,marketplace,secrets,tools,memory}.rs; docs/09-api/openapi.yaml; sdks/typescript, sdks/python. Size. M. Depends on: none. Blocks: API-704.

Status: Done (2026-07-07). All six ad-hoc endpoints migrated to hardening::paginate()’s {data, has_more, next_cursor, total_estimate} envelope, each gaining real limit/cursor query params in place of their old unbounded (or single-shot-capped, for audit) response. GET /api/v1/audit is now consistently most-recent-first regardless of whether a limit is supplied (previously only when the old ad-hoc limit param was set). POST /api/v1/memory:query is the documented exception per this ticket’s own guidance: results → data for field-name consistency, but it keeps its {data, count} shape rather than the full envelope, since a ranked top-K result set has no cursor to page through. Both SDKs and openapi.yaml updated in lockstep; redocly lint still passes (same pre-existing warning set, no new errors); the TypeScript SDK’s tsc build is clean. No schema-level check scripted yet for “no endpoint uses the old top-level array keys” — the six were verified by hand against every call site in the Rust test suite and both SDKs’ test/type files.

API-702 [P1] — One serde casing policy across all wire enums

Section titled “API-702 [P1] — One serde casing policy across all wire enums”

Problem. Three serialization idioms coexist. Workflow execution status serializes PascalCase ("Completed") while the ?status= filter takes lowercase (the known wart, crates/wovyr-server/src/lib.rs:1101-1115); workflow event types are PascalCase ("ActivityCompleted", relied on in tests at workflow_runner.rs:603); memory type is a lowercased Debug; plugin state is a hand-written lowercase string (plugins.rs:272). A client can’t predict the casing of any given enum. (PRD-003 R-7.2; closes PP-16 casing portion.)

Change.

  • Apply #[serde(rename_all = "snake_case")] (chosen policy) to every wire-serialized enum, and delete hand-written string conversions. Reconcile the workflow status filter and body to one casing.
  • This is a breaking change to the workflow status/event and plugin-state fields — do it now, in the same pre-GA pass as API-701, and bump the SDK versions together.

Acceptance criteria.

  • Every enum on the wire is snake_case; the status filter and status field match.
  • A test asserts round-trip stability for each wire enum; SDK suites pass.

Files. crates/wovyr-workflow/src/ (status/event enums), crates/wovyr-server/src/ (memory/plugin serialization), openapi.yaml, both SDKs. Size. M. Depends on: none. Blocks: API-704.

Status: Done (2026-07-07). WorkflowState/ActivityState/WorkflowEvent (the type tag) all now derive #[serde(rename_all = "snake_case")] — WorkflowState’s response body and the ?status= query filter now agree by construction (the filter already normalized to lowercase). MemoryType already derived a casing policy ("lowercase", normalized to "snake_case" for consistency — identical output for its single-word variants); the actual bug was wovyr-server re-deriving the same string by hand via {:?} (Debug) + .to_lowercase() instead of letting serde serialize it, same story for PluginState (already snake_case, but re-derived by hand via a match arm). All four hand-written conversions deleted in favor of embedding the enum value directly. Both openapi.yaml and the SDKs already declared the correct (lowercase) shapes for the affected fields before this change — the bug was server-side only, so no SDK/spec edits were needed there. Round-trip stability is proven for all four enums (wovyr-workflow’s state.rs/ event.rs, wovyr-memory’s record.rs, wovyr-plugin’s engine.rs). This is a breaking change to on-disk/on-wire data, not just the HTTP API: a workflow event log (file-store *.events.jsonl or the Postgres workflow_events table) written before this change will not deserialize after upgrading — no migration path exists for it, acceptable only because no real deployment exists yet. Caught real accumulated pre-change data in this repo’s own shared ~/.wovyr/workflows test fixtures during verification; the affected tests were switched to isolated in-memory engines rather than depending on real disk state at all (the same fix pattern DUR-404 established).

API-703 [P1] — Extend Idempotency-Key to all mutating routes

Section titled “API-703 [P1] — Extend Idempotency-Key to all mutating routes”

Problem. Idempotency-Key is honored on agents:run only, not on agents/{id}/run, workflow submit, or any other mutation (docs/09-api/openapi.yaml:31-35). A client retry of an unacknowledged POST can double-execute. (PRD-003 R-7.3; closes PP-16 idempotency portion.)

Change.

  • Route every mutating handler through the idempotency middleware/helper (crates/wovyr-server/src/hardening.rs), keyed by (tenant, method, path, Idempotency-Key). Reuse the Phase-2 bounded/persistent store (SEC-205 + DUR-404).
  • Document which routes are idempotent-by-key in openapi.yaml.

Acceptance criteria.

  • A replayed mutation with the same key returns the cached response and does not re-execute; a soak test confirms no double-execution across all mutating routes.

Files. crates/wovyr-server/src/hardening.rs (apply broadly), each mutating route module; openapi.yaml. Size. M. Depends on: Phase-2 SEC-205, DUR-404 (bounded/persistent store).

Status: Done (2026-07-07). Replaced the one hand-rolled idempotency check inside run_handler with a single hardening::idempotency_middleware (axum::middleware::from_fn_with_state) wired as the innermost layer of all three protected route groups (run_routes, sensitive_routes, other_protected) — so it runs after auth/rate-limiting but before every handler, uniformly. It keys the cache on (tenant, method, path, Idempotency-Key), fixing a latent gap the old agents:run-only check had (tenant+key only, so the same key reused against two different routes would have collided). Applies to every POST/PUT/PATCH/DELETE route except two POST routes that only look like mutations: workflows/validate (parse-only, no side effects) and memory:query (a read) — both excluded by path suffix, plus agents:stream’s SSE body, which can’t be buffered and replayed as an opaque value. Only a successful (2xx) response with a JSON-decodable (or empty, e.g. 204) body is cached; anything else is served once and never stored. docs/09-api/openapi.yaml gained the idempotencyKey parameter on every qualifying operation (34 operations); the TypeScript and Python SDKs gained an idempotencyKey/idempotency_key option on every corresponding mutating resource method. Proven at three layers: hardening::tests unit-tests the eligibility/replay-reconstruction logic directly; tenancy::tests::idempotency_key_replays_across_a_route_that_would_otherwise_conflict proves end-to-end (over the real router) that a route with no per-handler idempotency code of its own (POST /api/v1/organizations) replays a would-otherwise-409-conflict retry, while a different key hits the genuine conflict; and a manual smoke test against the real running wovyr dev binary (not just the in-process test harness) confirmed the same behavior over real HTTP. run_is_idempotent_per_key (the pre-existing agents:run test) still passes unchanged, proving the new shared middleware is a drop-in replacement for the old per-handler logic — including for the Prefer: respond-async submit path, which previously had no idempotency support at all and now replays the same run_id on a retried submission. Not yet done: API-703’s own “soak test confirms no double-execution across all mutating routes” acceptance criterion is covered by the targeted regression test above plus the manual smoke test, not a dedicated all-routes soak — reasonable given the mechanism is one shared code path proven against a representative route, not per-route logic that could drift.

API-704 [P1] — CI contract gate: SDK integration suites + redocly lint

Section titled “API-704 [P1] — CI contract gate: SDK integration suites + redocly lint”

Problem. openapi.yaml is hand-synced (“kept in sync manually until a codegen pipeline exists”, openapi.yaml:8-10); .github/workflows/ci.yml has no redocly/ npm/sdk/openapi step. The redocly lint and the TS/Python SDK integration suites run only manually against a live wovyr dev. Nothing prevents a handler change from silently diverging from the spec the PyPI SDK was written against. (PRD-003 R-7.4; closes PP-18.)

Change.

  • Add a CI job that boots wovyr dev, runs redocly lint openapi.yaml, then runs the TypeScript and Python SDK integration suites against it, as a required gate on every PR.
  • Fold this into the Phase-2 CI-901 service-container job or add a sibling job.

Acceptance criteria.

  • A PR that changes a handler’s response shape without updating openapi.yaml/the SDKs fails CI.
  • The contract gate is green on main after API-701/702/703 land.

Files. .github/workflows/ci.yml; sdks/typescript, sdks/python (test entry points). Size. M. Depends on: API-701, API-702, API-703 (freeze before locking).

Status: Done (2026-07-08). Added a contract-gate job to .github/workflows/ci.yml: boots a real wovyr dev from the PR’s own code (WOVYR_ALLOW_ANONYMOUS=1, loopback-only, plus WOVYR_PLATFORM_ADMINS= sdk-test-admin so the org/project routes — which have no anonymous-tenant bypass — are exercised rather than silently skipped), waits for /healthz, then runs npm test (TypeScript: redocly lint + build + the integration suite in one command) and the Python suite (python3 -m unittest discover -s tests -v) as a required PR gate. Deliberately a sibling job to Phase-2’s services-integration, not folded in — that job is Rust/cargo-only with Postgres/Qdrant/Redis service containers; this one needs Node.js + a live HTTP server, a different shape entirely.

Wiring this up immediately earned its keep: running both suites against a genuinely fresh server (no accumulated local ~/.wovyr state — simulated by pointing HOME/USERPROFILE at an empty directory and invoking the built binary directly, since a real GitHub Actions runner starts the same way) surfaced three real, previously-invisible bugs, all now fixed:

  1. Both SDKs’ workflows: submit then poll to completion test still asserted the pre-API-702 PascalCase "Completed" status — the server has correctly returned lowercase "completed" since API-702 shipped, but neither test was ever updated (or ever run against a live server in CI) to catch the mismatch.
  2. Both SDKs’ tools.list() test asserted total_estimate >= 4, which only ever held on a developer machine with leftover plugin-tool registrations from earlier manual testing — the real default hosted registry (SEC-301) is exactly 3 built-ins (echo, fs_read, http_get); shell, image_generate, and plugin tools are each a conditional opt-in absent on a clean environment.
  3. The Python SDK’s test file had two leftover pre-API-701 field names from before this workstream’s own earlier tickets landed: tools.list() read res["tools"]/res["total"] instead of res["data"]/res["total_estimate"], and memory.query() read res["results"] instead of res["data"]. The TypeScript suite already had the correct field names; only the Python file was never updated.

All three were caught precisely because this ticket finally ran these suites against a live server for the first time in CI-representative conditions rather than a warmed-up developer machine — proof the gate does what it’s for. Verified end to end: 15/15 TypeScript tests pass (npm test, exit 0) against the fresh-state server; the Python fixes are the identical, verified-correct change mirrored into the file the same way (this dev environment has no Python interpreter to execute the suite directly, so this one relies on the 1:1 correspondence with the now-passing TypeScript suite plus static review rather than a local run — CI itself, once merged, is the first real execution).

API-705 [P2] — Emit Deprecation/Sunset headers from a route-metadata table

Section titled “API-705 [P2] — Emit Deprecation/Sunset headers from a route-metadata table”

Problem. docs/09-api/deprecation-policy.md (90-day window, Deprecation/Sunset headers) is prose with nothing enforcing it; hardening.rs emits no such headers. (PRD-003 R-7.5; closes PP-18 deprecation portion.)

Change.

  • Add a route-metadata table marking deprecated routes with a sunset date; a middleware emits Deprecation: true and Sunset: <http-date> for those routes, making the policy mechanically enforceable.

Acceptance criteria.

  • A route flagged deprecated returns the headers; a test asserts the window is ≥90 days from the deprecation date.

Files. crates/wovyr-server/src/hardening.rs; a route-metadata module. Size. S. Depends on: none.

Status: Done (2026-07-08). Added hardening::DEPRECATIONS (a const route-metadata table, [Method, PathPattern, deprecated_since, sunset] per entry — PathPattern is Exact or Prefix, since axum’s MatchedPath isn’t available to a Router::layer-based middleware before routing resolves it) and hardening::deprecation_headers, wired into router() alongside request_id so it applies broadly, not just to mutating routes. The table is empty — docs/09-api/deprecation-policy.md §7 already says nothing in /api/v1 is deprecated, and that stays true; this ticket builds the mechanism, not a first deprecation. Dependency-free date math (Howard Hinnant’s days_from_civil, mirroring wovyr-workflow’s cron.rs — the two crates don’t share the handful of lines, not worth a cross-crate dependency for it) computes both the RFC 7231 Sunset date string and the window-length check. Proven at three levels: unit tests for the date math against known reference dates, deprecation_for matching (exact/prefix/wrong-method/ no-match) against a synthetic table, and an end-to-end test wiring the real middleware into a throwaway router and asserting the headers appear on a matched route and not on an unmatched one. deprecation_table_windows_are_valid is the acceptance criterion’s own regression guard — vacuously green today, fails CI the day someone adds a real entry with less than the policy’s 90-day minimum.

WS-8 — Observability & Operability

Section titled “WS-8 — Observability & Operability”

OBS-801 [P1] — RED metrics for all routes via one middleware layer

Section titled “OBS-801 [P1] — RED metrics for all routes via one middleware layer”

Problem. wovyr_api_requests_total/wovyr_api_request_duration_seconds are recorded in exactly two handlers — run_handler (crates/wovyr-server/src/lib.rs:694-702) and run_stored_handler (lib.rs:987-998) — despite CLAUDE.md claiming “per route”. Workflows, memory, marketplace, tenancy, secrets, plugins, webhooks emit no request metrics, so an on-call operator is blind to a marketplace 502 storm or a memory-query latency regression. (PRD-003 R-8.1; closes PP-19 metrics portion.)

Change.

  • Replace the two per-handler metric calls with one metrics middleware layer (beside hardening::request_id) labeling by route template + status, covering every route.

Acceptance criteria.

  • /metrics shows request count/latency/error series for every route group; a test hits several routes and asserts the series appear with correct labels.

Files. crates/wovyr-server/src/lib.rs (layer + remove per-handler calls), crates/wovyr-telemetry (if a helper is needed). Size. M. Depends on: none.

Status: Done (2026-07-08). Added hardening::track_metrics, one middleware layer recording wovyr_api_requests_total/ wovyr_api_request_duration_seconds (labeled route/method/status) for every route — deleted the two hand-rolled per-handler recordings in agents.rs (record_run_metrics plus the inline block in run_stored_handler) that used to be the only two request-metric call sites in the whole server. Wired into router() at the same outer, whole-app .layer() position as hardening::request_id/deprecation_headers — a deliberate choice over a per-router route_layer using axum’s real MatchedPath: that position would only see requests that already reached a matched handler, silently excluding the exact responses RED metrics exist to surface (an auth 401, a rate-limit 429, an idempotency-replay short circuit). Axum’s MatchedPath isn’t resolved yet at this outer position — the identical constraint API-705’s deprecation_headers already documents on PathPattern — so the route label comes from a new hand-maintained ROUTE_LABELS table (hardening.rs, ~55 entries, one per route this server actually mounts) matched via path_matches_template, a small segment-by-segment matcher treating any {param} template segment as a wildcard (needed over a single Exact/Prefix string, unlike every existing PathPattern entry, since most routes here have a parameter followed by more literal segments, e.g. /api/v1/projects/{id}/quota). An unrecognized (method, path) — a genuinely unknown path, or a route added to a router module without a matching table entry — falls back to the "unmatched" label rather than being silently dropped from the metrics, so a drift between this table and the real router stays visible in /metrics instead of invisible. Verified: new unit tests for path_matches_template/route_label against representative literal/param/wrong-method cases; a track_metrics middleware test (isolated throwaway router) proving both a normal response and an unmatched path are recorded; and the existing metrics_endpoint_reflects_a_run integration test (over the real router()

  • AppState) extended to also hit GET /api/v1/tools (asserting the tools_list label — a route that emitted zero metrics before this ticket) and an unregistered path (asserting the unmatched/404 label pair). The pre-existing route="agents_run" assertion in that test still passes unchanged, confirming label-naming continuity for the one route group that already had metrics. Full workspace cargo build/clippy -D warnings/fmt/ test clean (wovyr-server 110/110); the one failure in a full workspace run is the pre-existing, unrelated Windows cmd.exe-quoting flake in wovyr-tools::builtin::tests::shell_can_request_cmd_explicitly, confirmed to fail identically on main before this change (via git stash).

OBS-802 [P2] — Correlate the request id into logs, traces, and audit

Section titled “OBS-802 [P2] — Correlate the request id into logs, traces, and audit”

Problem. The request id (crates/wovyr-server/src/hardening.rs:152-197) is stamped on the response header and error body only — never onto a tracing span or log line, so server logs/OTLP traces can’t be joined to a client-reported X-Request-Id. AuditEvent has a request_id field (crates/wovyr-audit/src/event.rs:78) but no call site ever sets it (with_request_id: zero references). (PRD-003 R-8.2; closes PP-19 correlation portion.)

Change.

  • Record the request id onto the handler’s tracing span (tracing::Span::current().record(...)) so it appears in logs and OTLP traces.
  • Set AuditEvent.request_id via with_request_id in the audit call sites.

Acceptance criteria.

  • A request with a given X-Request-Id produces log lines / trace spans carrying it, and (for audited actions) an audit entry with that id.

Files. crates/wovyr-server/src/hardening.rs, audit call sites (kms.rs, secrets.rs, + OBS-804’s new ones). Size. S. Depends on: none (pairs with OBS-804).

Status: Done (2026-07-09). hardening::request_id now writes the resolved id back onto the request’s x-request-id header (not just the response’s) before calling next.run, so any handler already extracting headers: HeaderMap reads it back via a new hardening::request_id_of helper — no new axum extractor needed. next.run is wrapped in an http.request tracing span carrying request_id as a field, so it appears on every log line/OTLP trace produced while handling the request (the two existing #[tracing::instrument]-annotated handler spans nest as children) — simpler than annotating every handler with its own fields(request_id = Empty) + a manual .record() call, which this ticket’s text originally suggested. A new shared audit::audit() helper (used by every OBS-804 call site, see below) calls AuditEvent::with_request_id whenever request_id_of finds one; kms.rs/secrets.rs’s existing audit_kms/audit_secret wrappers now delegate to it too, so the correlation applies retroactively to the audit call sites that predate this ticket. Verified with two new unit tests (request_id_of_reads_the_header_when_present, request_id_middleware_writes_the_id_back_onto_the_request_for_handlers — the latter drives the real middleware over a throwaway router, proving both a client-supplied and a server-generated id reach a handler). Full workspace build/ clippy/fmt/test clean.

OBS-803 [P2] — Ship starter Prometheus alert rules and a Grafana dashboard

Section titled “OBS-803 [P2] — Ship starter Prometheus alert rules and a Grafana dashboard”

Problem. docs/14-observability/alerting.md and dashboards.md are Draft specs (SLOs, burn-rate alerts, a dashboard catalog); deployment/ contains zero Prometheus rules or Grafana JSON. The on-call story (page on what? visualize how?) is unbuilt. (PRD-003 R-8.3; closes PP-19 alerting portion.)

Change.

  • Add a starter deployment/observability/ with a Prometheus alert-rule file (error rate, latency SLO burn, health) built on the OBS-801 metrics, and a Grafana dashboard JSON (RED per route, LLM cost/tokens from the existing wovyr_llm_* series).

Acceptance criteria.

  • The alert rules validate (promtool check rules); the dashboard imports cleanly and references series that actually exist post-OBS-801.

Files. deployment/observability/ (new). Size. S. Depends on: OBS-801 (series must exist).

Status: Done (2026-07-09). Added deployment/observability/{alerts.yml, dashboard.json,README.md}. alerts.yml: 7 Prometheus rules over the real series OBS-801/the LLM cost observer/webhooks emit — API error rate (warning at 5%, critical at 25%), a p95-latency SLO burn (>1s for 10m), a contract-drift detector on route="unmatched" (flags a router/ROUTE_LABELS desync — an addition beyond the ticket’s literal ask, since OBS-801’s own fallback label is exactly the kind of “silent drift” signal alerting exists to catch), target-down (up{job="wovyr"}), an LLM daily-cost-spike heuristic, and a webhook delivery-failure-rate alert. Downloaded a portable promtool/prometheus binary from the project’s GitHub releases (same offline-validation approach this repo’s Helm chart used for kubeconform) since neither is installed in this dev environment — promtool check rules alerts.yml reports “SUCCESS: 7 rules found”. dashboard.json (Grafana schema v39): request rate/error-rate/latency-percentiles by route (with a $route template variable), an unmatched-route panel, LLM token/cost panels, a cache-savings stat, and webhook delivery outcomes — validated as well-formed JSON, but never rendered against a live Grafana (none exists in this environment), stated as an explicit caveat in the README rather than implied. Added real-vs-aspirational status notes to docs/14-observability/ {alerting,dashboards}.md (both →1.1.0) pointing at this starter, since those documents describe a much larger future multi-service SLO/dashboard program this slice doesn’t claim to satisfy.

OBS-804 [P2] — Audit coverage for every state-changing handler

Section titled “OBS-804 [P2] — Audit coverage for every state-changing handler”

Problem. audit::record is invoked only from kms.rs and secrets.rs. Not audited: agent runs, plugin install/enable/disable/uninstall, all tenancy mutations (org/project/member/quota), marketplace publish/download/abuse-resolution, and webhook subscription changes. The tamper-evident log can’t answer “who ran what / who changed permissions / who installed which plugin” — insufficient for GA forensics/compliance. (PRD-003 R-8.4; closes PP-audit.)

Change.

  • Add audit::record (referencing resources by id, actor = the verified principal from Phase-1 SEC-101) to every state-changing handler: agent run, plugin lifecycle, tenancy mutations, marketplace publish/moderation, webhook create/delete.

Acceptance criteria.

  • Each privileged mutation appears in GET /api/v1/audit with actor, action, resource id, and outcome; a test walks a representative mutation per module and asserts the entry.

Files. crates/wovyr-server/src/{lib,plugins,tenancy,marketplace,webhooks}.rs. Size. M. Depends on: Phase-1 SEC-101 (verified actor). (Pairs with OBS-802.)

Status: Done (2026-07-09). A new shared audit::audit(state, headers, tenant, action, resource_type, resource_id) helper (built alongside OBS-802, above) is now called from every state-changing handler this ticket named: agents.rs (agent.create/agent.delete/agent.run — the latter required threading headers: &HeaderMap through run_definition/run_inner/run_async_inner, which previously didn’t need it), plugins.rs (plugin.{enable,disable,install,upgrade, rollback,uninstall,trust} — each handler’s tenant_authorize(...)? call had to be changed from a discarded ? to a bound let tenant = ...? first), tenancy.rs (organization.create, project.{create,update,delete}, member.{add,remove}, quota.update), marketplace.rs (marketplace.{publish,download,install, listing.verify,review.approve,review.reject,abuse.report,abuse.resolve, abuse.dismiss} — download_version/install_listing previously took no headers at all and needed it added), and workflow_runner.rs (workflow.execution.{submit,signal,approve,cancel}).

Verified with 4 new tests, each driving the real HTTP route over an isolated AppState with AuditLog::in_memory() (not the shared ~/.wovyr/audit): agent_mutations_are_audited (create→run→delete, lib.rs), org_and_project_mutations_are_audited (tenancy.rs, also asserts the actor principal and resource id on the entry), webhook_management_mutations_are_audited (register→delete, webhooks.rs), and submit_and_cancel_are_audited (workflow_runner.rs, using isolated_state()’s in-memory-engine pattern). Two modules — plugins.rs and marketplace.rs — have no equivalent live-success test: both mutate the real, shared ~/.wovyr/plugins/~/.wovyr/marketplace on whatever machine runs the suite with no test-injectable in-memory store (an existing, pre-ticket constraint — plugins.rs’s own plugin_lifecycle_routes_require_plugins_ admin test already avoids a real mutation for exactly this reason, per its own comment). Their audit call sites use the identical crate::audit::audit() helper already proven by the other four modules’ tests plus the pre-existing kms.rs/ secrets.rs audit tests — a structural, not per-route, coverage guarantee for those two. Full workspace cargo build/clippy -D warnings/fmt/test clean (wovyr-server 113/113 lib + 3/3 authz_matrix).

OBS-805 [P2] — Dashboard: real login/session, CORS, and a build artifact

Section titled “OBS-805 [P2] — Dashboard: real login/session, CORS, and a build artifact”

Problem. dashboard/src/app/core/tenant.config.ts:12-13 hardcodes TENANT = 'acme' / PRINCIPAL = 'admin@wovyr.local' as compile-time constants sent as headers; there is no login flow (BFF deferred). No CORS layer exists in wovyr-server, so the SPA only works behind Angular’s dev proxy or same-origin. No deployment artifact includes the dashboard (Docker/compose/Helm reference only the wovyr binary). Bonus staleness: dashboard/README.md claims “the server exposes no workflow-authoring routes yet” — false since workflow_runner.rs shipped. (PRD-003 R-8.5; closes PP-11-dashboard.)

Change.

  • Replace the hardcoded constants with a login/session flow that obtains a real credential from the Phase-1 SEC-101 auth layer and sends it (not a spoofable header).
  • Rely on the Phase-1 SEC-204 CORS layer for cross-origin operation; add a dashboard build stage to the Docker image (or serve the built SPA from wovyr-server).
  • Fix the stale dashboard/README.md.

Acceptance criteria.

  • The dashboard logs in with a real credential and works cross-origin against a CORS-configured server; a built image serves it; the README is accurate.

Files. dashboard/src/app/core/, dashboard/ build config, deployment/docker/, dashboard/README.md. Size. L. Depends on: Phase-1 SEC-101, SEC-204.

Status: Done (2026-07-09). Deleted tenant.config.ts’s hardcoded TENANT/ PRINCIPAL build-time constants; a new core/session.ts (Session, a signal-based localStorage-backed service) and features/login/ (a Sign in page + nav entry) replace them. There is no username/password login endpoint anywhere in the platform (wovyr-server’s auth.rs only verifies a pre-existing JWT/API key, never mints one — confirmed by reading it directly rather than assumed) — a deliberate scope call, documented in Session’s own doc comment and the README’s new “Authentication” section, that Sign-in collects tenant/principal plus an already-minted credential (wovyr auth create-key) rather than a password with nowhere to go. tenant.interceptor.ts now reads from Session and adds Authorization: Bearer <value> once a credential is set, alongside the X-Wovyr-Tenant/X-Wovyr-Principal headers it always sent.

Verified live end to end, not just built — this environment has Node.js and no Docker, so the Angular side got real browser verification and the container side got build-only verification, matching each tool’s actual availability:

  1. npm run build/ng build clean both before and after every change (confirmed the exact dist/dashboard/browser/ output path this ticket’s Dockerfile copies from).
  2. Ran the real dashboard+wovyr-server dev servers and drove the Sign-in flow in a live browser. First against the default disabled-loopback auth mode with no WOVYR_ALLOW_ANONYMOUS set — this surfaced a real, previously-undocumented gotcha (not introduced by this change): the server 401s every request in that mode unless WOVYR_ALLOW_ANONYMOUS=1 is explicitly set (SEC-101’s secure-by-default), meaning the dashboard’s own previously-documented “Run it locally” instructions never actually worked out of the box — now fixed in the README with the required env var spelled out.
  3. Restarted the server with WOVYR_AUTH_MODE=apikey, minted a real key via wovyr auth create-key, pasted it into the Sign-in page, and confirmed Authorization: Bearer <key> reached the server and was verified: GET /api/v1/tools (authenticated, no RBAC) returned 200; GET /api/v1/agents (RBAC-gated, and this key’s principal holds no tenancy membership) correctly returned 403 rather than 401 — proof the credential itself verified and RBAC is a distinct, later gate, not proof of a broken credential.
  4. Confirmed via preview_console_logs that no client-side errors accompanied any of the above.

CORS: no server-side code changes needed — Phase-1’s cors_layer (config.rs) was already fully implemented (allow-list from WOVYR_CORS_ALLOWED_ORIGINS, the right allow/expose headers, allow_credentials(true)); OBS-805’s CORS work is purely the deployment-config documentation of setting that env var to the dashboard’s real origin, now in the README’s new “Cross-origin deployment” section.

Build artifact: added deployment/docker/dashboard.Dockerfile (Node build stage → nginx runtime stage with an SPA-fallback try_files config), a separate image from the existing Rust-only Dockerfile per the ticket’s own “or” wording — not wired into docker-compose.yml as a running service (documented as an explicit, deliberate scope boundary, matching this repo’s established “reliability first slice” pattern for compose/Helm). Never run through a live docker build (no Docker daemon in this environment) — the build stage’s npm run build step was independently verified to produce the exact tree the Dockerfile’s COPY --from=build copies.

README fixes (beyond the stale claim the ticket named): the “Workflow Builder is the lone placeholder” claim was also false — features/workflow-builder/ is a real, routed surface, not a placeholder (an inaccuracy this pass found, not just the one the ticket already flagged); the “Layout” section’s file tree was years out of date (listed only 2 of 9 feature directories).

Not done (explicitly out of scope, not a gap that slipped through): a route guard forcing /login before any other route works — the app deliberately keeps working with just tenant/principal set (no credential), matching the pre-existing disabled-loopback dev experience; gating that would be a behavior change beyond what this ticket asked for. docs/10-dashboard/*’s individual per-surface spec docs were not audited for staleness in this pass (only dashboard/README.md, which the ticket named explicitly).

WS-9 remainder — Codebase Health

Section titled “WS-9 remainder — Codebase Health”

HLTH-901 [P1] — Unify the three ActivityExecutor implementations

Section titled “HLTH-901 [P1] — Unify the three ActivityExecutor implementations”

Problem. Three impls with real semantic drift make identical workflow YAML behave differently locally vs. on the server: CLI PlatformExecutor (apps/wovyr-cli/src/workflow.rs:27-153), server ServerExecutor (crates/wovyr-server/src/workflow_runner.rs:62-185), and EvalWorkflowExecutor (crates/wovyr-eval/src/compare.rs:128-165). CLI maps Network/Internal tool errors → Retryable while the server maps all tool errors → Permanent (so a transient HTTP failure retries locally but permanently fails the same workflow on the server); ai activities resolve the model differently; function/human dispatch differs. Template resolution was already unified (resolve_template); dispatch was not. (PRD-003 R-9.2; closes PP-16 executor portion.)

Change.

  • Extract one PlatformActivityExecutor parameterized over an AgentResolver trait (agents-dir file lookup for the CLI, stored-agent-by-id for the server, in-memory map for eval). Delete the three divergent dispatch bodies.
  • Reconcile the retry classification and model resolution to one behavior.

Acceptance criteria.

  • A shared test asserts the same workflow + activity errors produce identical retry/ terminal behavior across CLI and server executors.
  • All three call sites use the unified executor; the eval comparison still runs.

Files. a new shared module (likely in wovyr-workflow or a small wovyr-runtime crate), apps/wovyr-cli/src/workflow.rs, crates/wovyr-server/src/workflow_runner.rs, crates/wovyr-eval/src/compare.rs. Size. L. Depends on: none.

Status: Done (2026-07-08). New crate wovyr-runtime, deliberately not inside wovyr-workflow itself — that would have made the core DAG/checkpoint engine depend on the LLM gateway/tool runtime/agent runtime, exactly the kind of boundary leak PP-21/HLTH-904 flags elsewhere. It sits alongside wovyr-server/wovyr-cli/wovyr-eval in the dependency spine instead, and all three now depend on it and construct nothing of their own for dispatch:

  • PlatformActivityExecutor is the one ActivityExecutor body. tool/ function both invoke a tool by name through ToolRegistry::execute (the registry’s permission-checked entry point, not a direct Tool::execute bypass) and classify a ToolError exactly as its own doc comments say — Validation/PermissionDenied → Permanent, Network/Internal → Retryable — closing the ticket’s headline bug (the server used to collapse every tool error to Permanent). function now dispatches identically to tool rather than the CLI’s old inert echo-passthrough: nothing in the codebase ever implemented the DSL spec’s original “function = arbitrary Rust code” vision, and every real example/test already used type: function expecting a tool invocation. ai activities read the system prompt from inputs.prompt and resolve the model via Gateway::resolve_model (the server used to read instructions from ctx.name — the activity’s identifier field, not a prompt — and hardcode the literal model string "default"). human activities check both the bare-activity-id and event.<id> decision-variable conventions, so this dispatch body is correct regardless of which resume mechanism a platform uses (direct checkpoint mutation, e.g. the CLI’s approve, vs. Engine::signal_event, e.g. the server’s /approve route).
  • AgentResolver is the one genuinely platform-specific piece: resolve (agent lookup), customize_options (tenant/hosted-ness), admit/record (the server’s per-project quota gate). admit returns a Box<dyn AdmissionGuard> — a type-erased RAII guard the executor holds for the run’s actual duration, not just at the admission check — that’s what makes the server’s concurrency slot mean anything, and what its own RunPermit boxes into without wovyr-runtime needing to know that type exists. Three impls, all trivial: the CLI’s FileAgentResolver (<agents_dir>/<name>.yaml), the server’s StoredAgentResolver (tenant-scoped AgentStore lookup + quota), eval’s MapAgentResolver (an in-memory BTreeMap) — none override more than resolve except StoredAgentResolver, which is the only platform with real context to add.

Verification. wovyr-runtime’s own test suite (6 tests) exercises the shared dispatch body directly — a permission-denied tool error is Permanent, a network/internal one is Retryable, function and tool invoke the same tool identically, human resolves under both variable-key conventions, an AgentResolver::admit rejection surfaces as Retryable, an unknown agent as Permanent — which is now a structural guarantee for “CLI and server classify identically,” not just an empirically-checked one, since both call the same function. All three call sites’ own test suites still pass unchanged: the CLI’s research_team_runs_locally_and_joins_two_agents, the server’s agent_activity_respects_project_quota and approve_decision_is_consumed_and_the_execution_completes (proving the AdmissionGuard refactor and the dual human-decision-key check didn’t regress either behavior), and eval’s multi_agent_vs_single_agent.rs. Full workspace cargo build/clippy -D warnings/fmt/test clean (one pre-existing, unrelated Windows cmd.exe-quoting flake in wovyr-tools, untouched by this change).

HLTH-902 [P1] — Fix the latent CLI marketplace spawn_blocking panic

Section titled “HLTH-902 [P1] — Fix the latent CLI marketplace spawn_blocking panic”

Problem. apps/wovyr-cli/src/main.rs is #[tokio::main]; its async run() calls plugin::publish_cmd/search_cmd/report_abuse_cmd directly, which reach open_store()/marketplace_registry() (apps/wovyr-cli/src/plugin.rs:654-676) and call the sync PostgresRegistryStore::connect with no spawn_blocking. This is the identical “Cannot start a runtime from within a runtime” panic the server already found and fixed with with_registry (crates/wovyr-server/src/marketplace.rs:99-125); the CLI path never got the fix, and CLAUDE.md advertises this exact config. Undetectable until Phase-2 CI-901 exercises the postgres feature. (PRD-003 R-9.3; closes PP-17 panic portion.)

Change.

  • Wrap the CLI’s registry operations in spawn_blocking (or run those subcommands before entering the Tokio runtime, since they’re synchronous by nature).

Acceptance criteria.

  • wovyr plugin publish|search|get|report against a Postgres-backed registry (--features postgres, WOVYR_MARKETPLACE_POSTGRES_URL set) succeeds — no panic. A test in the CI-901 postgres job covers it.

Files. apps/wovyr-cli/src/plugin.rs, main.rs. Size. S. Depends on: Phase-2 CI-901 (to detect/guard).

Status: Done (2026-07-08). All 7 marketplace command functions (publish_cmd, search_cmd, market_install_cmd, report_abuse_cmd, list_abuse_reports_cmd, resolve_abuse_cmd, dismiss_abuse_cmd) are now async fn, each running its entire synchronous body (registry construction through the final println!) inside one tokio::task::spawn_blocking via a new blocking() helper mirroring the server’s with_registry shape — one thread hop per command rather than a with_registry-per-call approach, which matters for market_install_cmd specifically (it holds the registry across download → plugin-engine install → record_install, so re-connecting between those would be wasteful and, for the Postgres backend, pointless extra round trips). main.rs’s 7 call sites gained .await.

Proven, not just patched — the underlying panic mechanism itself was reproduced and confirmed fixed, without needing a live CI run: a standalone scratch binary called wovyr_marketplace::PostgresRegistryStore::connect against a deliberately-unreachable address (postgres://fake:fake@127.0.0.1:1/…) both directly from an async context and via spawn_blocking. The direct call panicked with the exact message this ticket describes (Cannot start a runtime from within a runtime, thrown from inside the postgres crate’s own Client::connect); the spawn_blocking-wrapped call returned a normal connection error instead — proof the panic fires on the attempt to connect, independent of whether a real database is reachable, and proof the fix genuinely prevents it. Separately, all 7 converted commands were exercised end to end against the file-based registry (publish → search → get/install with a permission grant → report → list reports → resolve-abuse → dismiss-abuse), confirming the refactor changed nothing about their behavior.

Also closed a real gap in the acceptance criterion itself: the existing Phase-2 CI-901 services-integration job never actually invoked the CLI binary against Postgres — its “capability-gated integration tests” step only runs each crate’s own cargo test, so this bug would have shipped undetected even after CI-901 landed. Added a new step, “CLI marketplace command against Postgres — no panic (HLTH-902)”, that runs wovyr plugin search (with --features postgres, against the job’s already- migrated Postgres schema) and fails the job if panicked appears in its output — the first time CI exercises the CLI binary itself against a real Postgres-backed registry.

HLTH-903 [P2] — Extract an wovyr-config crate for ~/.wovyr layout and env selection

Section titled “HLTH-903 [P2] — Extract an wovyr-config crate for ~/.wovyr layout and env selection”

Problem. The ~/.wovyr bootstrap layer is duplicated wholesale between CLI and server — load_trust/load_keyless/load_catalog/save_catalog/open_store/registry construction all exist twice with near-identical bodies (crates/wovyr-server/src/plugins.rs

  • marketplace.rs vs. apps/wovyr-cli/src/plugin.rs). Cross-process agreement on which secrets file / registry backend / KMS root is live is maintained by prose, not shared code; one drifted edit silently forks the shared state. ~28 scattered env::var sites, no central config module. (PRD-003 R-9.4; closes PP-20 config portion.)

Change.

  • Create an wovyr-config (or wovyr-host) crate owning the ~/.wovyr directory layout, all WOVYR_* env-var reading, and backend selection (which store, which file, encrypted-or-not). Both binaries consume it, so agreement is enforced by code.

Acceptance criteria.

  • CLI and server resolve every shared path/backend through the one crate; a test asserts they agree on the live secrets file and registry backend under the same env.

Files. new crates/wovyr-config/; crates/wovyr-server/src/, apps/wovyr-cli/src/ (consume it). Size. M. Depends on: none. (Reduces risk for Phase-2 DUR-403’s shared-state work — ideally sequenced before or with it.)

Status: Done (2026-07-08). New crate crates/wovyr-config (wovyr_dir() — the one HOME/USERPROFILE resolution both binaries now share; paths — one function per resource directory; env — typed readers for the two genuinely cross-binary env vars, WOVYR_SECRETS_ENCRYPT_AT_REST and WOVYR_MARKETPLACE_POSTGRES_URL; kms::build_kms()/ secrets::build_secrets_vault() — the previously byte-for-byte-duplicated construction logic, now one implementation). wovyr-server’s default_kms/ default_secrets_vault and every inline HOME/USERPROFILE resolution (tenancy/audit/webhooks/workflows_dir/server_state_dir/auth’s default_api_key_store/plugins.rs/marketplace.rs/memory.rs) now call into it; wovyr-cli’s config::config_dir()/config::kms()/plugin.rs’s secrets_vault()/plugins_dir()/staging_dir()/marketplace open_store() do the same — wovyr-secrets is now an unconditional dependency of wovyr-cli (previously gated behind plugin-wasi) since wovyr-config’s shared secrets-vault construction needs it regardless of that feature. Deliberately not centralized (would be new feature work, not duplication-removal — the survey that scoped this ticket found real drift, not just duplication): the CLI’s tiered Postgres/Qdrant memory backend has no server equivalent; the server’s marketplace policy.json curation has no CLI equivalent; both are documented as known, pre-existing, out-of-scope gaps rather than silently left unmentioned. Proven by crates/wovyr-config/tests/agreement.rs (3 tests, the acceptance criterion): resource paths match the wovyr_dir() join for every resource; a build_kms()-constructed “CLI instance” and a separately-constructed “server instance” over the same directory can decrypt each other’s sealed data (the same independently-constructed-pairs pattern wovyr-kms’s own concurrent-writer tests use as a stand-in for “a separate process”); the same cross-instance check for build_secrets_vault(). One real bug found and fixed while writing these tests: the first version raced on the process-global HOME/ USERPROFILE env var when Rust’s test harness ran the three tests concurrently on different threads within the same binary — fixed with a static ENV_LOCK: Mutex<()> serializing them, confirmed stable across 5 repeated runs and clean in the full cargo test --workspace run. Full workspace cargo build/clippy -D warnings/fmt/test clean (wovyr-server 104/104, wovyr-cli 9/9, wovyr-config 3/3, plus the pre-existing unrelated Windows cmd.exe-quoting flake in wovyr-tools).

HLTH-904 [P2] — Cleanup: gateway boundary leak, workspace deps, module splits

Section titled “HLTH-904 [P2] — Cleanup: gateway boundary leak, workspace deps, module splits”

Problem. Grab-bag of hygiene debt: the image_generate builtin (crates/wovyr-tools/src/builtin.rs:283-300) calls OpenAI directly, bypassing the gateway (no cost metering/retry/failover/cache) and the secrets vault; shared externals (sha2×5, semver×4, ring×3) aren’t in [workspace.dependencies], so versions can drift; tokio = "full" everywhere inflates compile time; Cargo.lock carries multiple versions of several crates; and god modules (crates/wovyr-server/src/lib.rs 2,745 LOC, crates/wovyr-tools/src/sandbox.rs 1,890 LOC) never got the module split the other route groups/backends have. (PRD-003 R-9.5; closes PP-20/PP-21.)

Change.

  • Route image_generate through the Gateway + secrets vault (inject them, as wovyr-memory does for embeddings).
  • Move shared external deps into [workspace.dependencies]; trim tokio features per crate; add cargo-deny (bans/duplicates/licenses) to the security CI job.
  • Split lib.rs (agents routes → agents.rs, state/config factories → state.rs/ config.rs) and sandbox.rs (one file per backend under a sandbox/ module).

Acceptance criteria.

  • image_generate invocations show up in gateway cost metrics; cargo-deny passes in CI; no source file over ~1,000 LOC in the two named crates; build unaffected.

Files. crates/wovyr-tools/src/builtin.rs, workspace Cargo.toml, per-crate Cargo.toml, .github/workflows/ci.yml, crates/wovyr-server/src/lib.rs, crates/wovyr-tools/src/sandbox.rs. Size. M–L (splits are mechanical but broad). Depends on: none.

Status: 4 of 5 sub-items done (2026-07-08); the two god-module splits deferred, not started.

  1. Gateway wiring — done. Added AIProvider::generate_image (default “unsupported”, mirroring embed) + a real OpenAiProvider impl (moved the existing POST /images/generations call verbatim from the tool into openai.rs) + Gateway::generate_image (a plain primary-provider pass-through, deliberately as simple as embed — no retry/failover/cache/cost-metering pipeline, since CostEvent is token-shaped and doesn’t fit an image call and embed already sets this precedent). wovyr-tools gained a normal (not dev-only) dependency on wovyr-provider — confirmed no cycle: wovyr-provider’s only reference back to wovyr-tools is a [dev-dependencies] entry for one example, which doesn’t participate in cycle resolution. ImageGenTool now takes an Arc<Gateway> via constructor (mirroring MemoryEngine::new(gateway, store)) instead of its own reqwest::Client + raw OPENAI_API_KEY/ WOVYR_OPENAI_BASE_URL reads. Both real construction call sites (wovyr-server/src/lib.rs, wovyr-cli/src/main.rs) now thread the already-constructed Gateway through instead of building a dependency-free tool.

  2. Workspace dependency dedup — done. sha2/semver/ring added to [workspace.dependencies] (the actual count was 6/4/4 sites, not the originally-cited 5/4/3 — wovyr-plugin had all three and was missed in the initial scoping pass); every direct-declaring crate switched to .workspace = true. cargo tree --duplicates before/after confirms zero version-resolution change (every site already pinned the identical version — pure textual dedup). Also added the pre-existing gap the scoping survey surfaced: wovyr-eval was a workspace member with no [workspace.dependencies] entry, unlike every other internal crate.

  3. tokio feature inheritance — done, and it was a real bug, not just “untrimmed.” Cargo workspace-dependency feature overrides are additive, so the root’s features = ["full"] meant every crate’s own features = [...] override (e.g. wovyr-plugin/wovyr-marketplace’s ["macros", "rt"]) was silently unioned back up to the full feature set — the per-crate trimming already in the codebase was a complete no-op. Fixed by dropping the root’s features entirely (tokio = { version = "1" } — tokio’s own defaults are empty) and giving every crate an explicit, audited feature list. A dedicated research pass grepped every tokio:: call site across all 19 workspace crates (src/, tests/, examples/) to build the exact per-crate requirement table rather than guessing — e.g. confirmed wovyr-server’s library code needs only rt+time+net (no macros/select!/#[tokio::main] anywhere in its production code — serve() really is caller-driven), while wovyr-cli needs rt-multi-thread+macros for its own #[tokio::main]; feature-gated code paths (wovyr-provider’s qdrant, wovyr-memory’s tiered, wovyr-workflow’s postgres) got their extra tokio features wired into that Cargo feature’s edge list rather than the base dependency, so a non-tiered/non-qdrant build doesn’t pay for them. Verified by compiling, per the ticket’s own acceptance note that this is the one sub-item where inspection can’t substitute for a real build: cargo build --workspace --all-targets clean with zero warnings, cargo test --workspace unaffected (only the pre-existing wovyr-tools flake), plus explicit feature-combination builds (wovyr-tools --features wasi, wovyr-provider --features qdrant, wovyr-memory --features tiered, wovyr-workflow --features postgres, wovyr-cli --features plugin-wasi) all clean.

  4. cargo-deny CI gate — done, validated locally, not just authored blind. Added deny.toml (bans/licenses/sources) and an EmbarkStudios/cargo-deny-action@v2 step in the security CI job (a prebuilt binary, no cargo-deny compile in CI — same reasoning as using the rustsec/audit-check@v2 action instead of invoking cargo audit directly). Locally installed cargo-deny (cargo install cargo-deny --config net.offline=false) and iterated against the real dependency graph rather than shipping an unverified config, which caught three real issues the initial draft got wrong: (a) wildcards = "deny" flags every one of this workspace’s ~19 internal path = "..." workspace dependencies as an unpinned wildcard (cargo-deny’s allow-wildcard-paths escape hatch only exempts crates already marked publish = false, which none of these are) — left at its default rather than adding publish = false to 19 manifests just for a lint; (b) webpki-roots’ actual license is CDLA-Permissive-2.0 (verified against the real crate, not assumed) — added to the allow-list; (c) a previously-unknown windows-sys 0.52.0-vs-0.61.2 duplicate (ring pins the older one, the clap/anstream terminal-styling chain the newer) — added to the documented skip list alongside the already-known webpki-roots 0.26-vs-1.0 split. The known rand/rand_core/ rand_chacha/getrandom 0.8-vs-0.9-generation split (production JWT/crypto stack vs. proptest’s dev-only chain) turned out to need no skip entry at all — cargo-deny’s own graph resolution doesn’t count it as a bans violation in the first place (confirmed: adding skip entries for it produced “unmatched-skip” warnings). cargo deny check bans licenses sources is clean locally with zero warnings.

  5. God-module splits — done. Two background agents dispatched in parallel with the full target layout were both killed mid-task by an account-level session limit (the wovyr-server side never wrote a file; the wovyr-tools side left two orphaned, never-wired-in files that were deleted to restore a clean baseline — see the prior revision of this entry for the full incident writeup). Both splits were then done directly rather than re-delegated, to avoid the same risk recurring on a second large background task late in the session.

    crates/wovyr-tools/src/sandbox.rs (1,891 LOC) → crates/wovyr-tools/src/sandbox/: mod.rs (67 LOC — the Sandbox trait + module wiring + the shared cap() truncation helper), types.rs (420 LOC — SandboxBackend/TrustClass/ ResourceLimits/NetworkPolicy/SandboxError/SandboxManager/ SandboxCommand/CommandOutcome), native.rs (229), container.rs (442), firecracker.rs (434), wasi.rs (377, still #[cfg(feature = "wasi")]-gated at the mod declaration so nothing inside needs its own per-item cfg). Tests distributed per-backend rather than left as one block. Verified: both default and --features wasi builds/clippy/tests clean (wovyr-tools 62-64/64 depending on the pre-existing Windows cmd.exe/temp-dir flakes, confirmed by rerunning the flaky ones in isolation — same two already-known, unrelated failures as before this ticket), plus a downstream build of wovyr-server/wovyr-agent/wovyr-plugin to confirm the re-exports didn’t break any consumer.

    crates/wovyr-server/src/lib.rs (4,335 LOC, ~2,100 of it inline tests) → state.rs (656 LOC — AgentStore, RunStore/AsyncRunStatus, AppState struct + impl), config.rs (516 LOC — every default_* backend factory, MetricsCostObserver, HttpLimits, env_u64, cors_layer, handle_overload_or_timeout), agents.rs (768 LOC — every agent-run + workflow-visibility HTTP handler, plus the shared ApiError envelope), and a trimmed lib.rs (2,483 LOC) retaining only the module wiring, router(), serve(), and TLS/crypto-provider bootstrap — plus the original, untouched ~2,100-line test module, which is why lib.rs itself doesn’t hit the ~1,000-LOC target on its own. That module is a single cross-cutting integration suite exercising router() + AppState

    • every handler together (many tests reach into AppState’s pub(crate) fields directly, e.g. state.tenancy.set_quota(...)), not naturally decomposable by concern the way the production code was — moving it to a true external tests/ integration file would need widening several pub(crate) fields to pub (a real API-surface change, out of scope for a pure code-motion refactor) since external test crates can’t see pub(crate) items. Cross-module visibility was resolved by re-exporting each new module’s contents at the crate root (use state::*; use config::*; use agents::*;, matching the crate-root-private visibility every item already had) plus one explicit pub use state::AppState; for tests/authz_matrix.rs (a real external integration test that needs AppState from outside the crate). One real encoding bug caught and fixed mid-task: an intermediate PowerShell Get-Content/Set-Content step used to assemble the file (before switching to the Read/Write tools’ own, UTF-8-correct handling) silently mangled every em dash and section sign in the file via a default-codepage misinterpretation — caught by re-reading the result, not by the compiler (mangled Unicode inside doc comments and string literals still compiles fine), so a build/test pass would not have caught it. Verified: cargo build/clippy -D warnings/fmt/test all clean, wovyr-server at the exact pre-split baseline (104/104 lib tests + 3/3 authz_matrix), full workspace build/clippy/fmt/test clean afterward (only the one already-known, unrelated wovyr-tools Windows flake).

Full workspace cargo build/clippy -D warnings/fmt/test clean for every sub-item — see each one’s own verification notes above. The acceptance criterion’s “no source file over ~1,000 LOC” is met for every production file in both crates (the largest is agents.rs at 768 LOC); the one exception is wovyr-server/src/lib.rs’s untouched ~2,100-line inline test module, documented above as a deliberate, reasoned exception rather than a gap.

Rollup

Section titled “Rollup”
TicketWSTitleSizePriorityDepends on
API-7017Standardize list envelopes — DoneMP1—
API-7027One serde casing policy — DoneMP1—
API-7037Idempotency on all mutations — DoneMP1SEC-205, DUR-404
API-7047CI contract gate (SDK + redocly) — DoneMP1701,702,703
API-7057Deprecation/Sunset headers — DoneSP2—
OBS-8018RED metrics middleware (all routes) — DoneMP1—
OBS-8028Request-id correlation — DoneSP2—
OBS-8038Alert rules + Grafana dashboard — DoneSP2OBS-801
OBS-8048Audit coverage (all mutations) — DoneMP2SEC-101
OBS-8058Dashboard login/CORS/build — DoneLP2SEC-101, SEC-204
HLTH-9019Unify ActivityExecutors — DoneLP1—
HLTH-9029Fix CLI spawn_blocking panic — DoneSP1CI-901
HLTH-9039wovyr-config crate — DoneMP2—
HLTH-9049Cleanup: gateway leak, deps, splits — DoneM–LP2—

Rough total: 3 L + 7 M + 4 S ≈ 9–12 engineer-weeks, parallelizable to ~4–5 calendar weeks across 2–3 engineers. Phase-4 exit = PRD-003 §11 items 5 (API consistent + contract-tested; privileged mutations audited and observable) and 6 (executor unified; no known latent panic ships — the CI matrix piece landed in Phase 2). Both exit criteria are now met — Phase 4 is fully done.

Cross-phase note: WS-7 should start as early as the team can spare it (even overlapping Phase 2/3), because the SDK-debt clock is already running. WS-8/WS-9 are genuinely last — they harden and clean up, but nothing depends on them.

Section titled “Related”

Revision History

Section titled “Revision History”
VersionDateDescription
1.11.02026-07-09OBS-802/803/804/805 done, closing out all of Phase 4. OBS-802: request_id writes the id back onto the request header (not just the response) + wraps next.run in an http.request tracing span, so handlers/logs/OTLP traces/audit entries can all correlate on it; a new shared audit::audit() helper (used by OBS-804 too) attaches it automatically. OBS-803: a promtool-validated deployment/observability/alerts.yml (7 rules, incl. a route="unmatched" contract-drift detector) + a Grafana dashboard.json; portable promtool/prometheus binaries downloaded for offline validation. OBS-804: every state-changing handler across agents/plugins/tenancy/marketplace/webhooks/workflow_runner now calls the shared audit helper; 4 new tests drive real HTTP routes over in-memory audit logs (plugins/marketplace have no live-success test — both touch the real shared ~/.wovyr, a pre-existing constraint, not a new gap). OBS-805: deleted the dashboard’s hardcoded TENANT/PRINCIPAL build-time constants for a Session service + a real Sign-in page supporting a pasted API key/JWT; verified live end to end against a real wovyr dev in apikey mode (a 200 on an auth-only route, a 403 — not 401 — on an RBAC-gated one the key’s principal lacks membership for); found and documented a real pre-existing gotcha (the default disabled-loopback mode 401s everything without WOVYR_ALLOW_ANONYMOUS=1, which the dashboard’s own README never mentioned); added deployment/docker/dashboard.Dockerfile (build-verified, not live-Docker-verified — no daemon here)
1.10.02026-07-08OBS-801 done: one hardening::track_metrics middleware records RED metrics (wovyr_api_requests_total/wovyr_api_request_duration_seconds, labeled route/method/status) for every route, replacing the two hand-rolled per-handler recordings that used to be the only request-metric call sites in the server. Applied at the same outer whole-app layer as request_id/deprecation_headers (not a route_layer+MatchedPath approach) so it also counts requests rejected before reaching a handler (401/429/idempotency replay) — the error-rate visibility RED metrics are for. Route labels come from a new hand-maintained ROUTE_LABELS table with {param}-aware segment matching, mirroring API-705’s PathPattern pattern at larger scale; an unmatched route falls back to an "unmatched" label rather than being silently dropped. WS-8 now started
1.9.02026-07-08HLTH-904’s two god-module splits done, closing out all of WS-9. wovyr-tools/src/sandbox.rs (1,891 LOC) → sandbox/{mod,types,native,container,firecracker,wasi}.rs (largest: container.rs at 442 LOC), tests distributed per-backend. wovyr-server/src/lib.rs (4,335 LOC) → state.rs/config.rs/agents.rs (656/516/768 LOC) + a trimmed lib.rs (router/serve/bootstrap only) — the untouched ~2,100-line inline test module stays in lib.rs since it’s one cross-cutting integration suite reaching into AppState’s pub(crate) fields directly, not decomposable without an API-surface change. Caught and fixed a real Unicode-mangling bug mid-task (a PowerShell text round-trip silently corrupted em dashes/section signs — compiles fine either way, only caught by re-reading the result). Both splits redone directly after two delegated background agents were killed mid-task by an account session limit. Full workspace build/clippy/fmt/test clean; wovyr-server at the exact pre-split test-count baseline
1.8.02026-07-08HLTH-903 done: new wovyr-config crate centralizes ~/.wovyr layout, the two genuinely cross-binary env vars, and the previously byte-for-byte-duplicated KMS/secrets-vault construction logic between wovyr-server and wovyr-cli. Proven by a cross-instance agreement test suite (found and fixed a real env-var test race in the process). HLTH-904 4/5 done: image_generate now routes through Gateway::generate_image (new gateway/provider API); sha2/semver/ring deduped into [workspace.dependencies]; the tokio feature-inheritance bug fixed (root full was silently defeating every crate’s own trimmed feature list) with a per-crate audit of real tokio:: API usage; cargo-deny CI gate added and validated locally against the real dependency graph (caught 3 real config issues a blind config would have shipped wrong). The two god-module splits (wovyr-server/src/lib.rs, wovyr-tools/src/sandbox.rs) are deferred — two delegated sub-agents hit an account session limit mid-task; the wovyr-server side never started (safe), the wovyr-tools side left two orphaned, never-wired-in files that were deleted to restore a clean state. Full workspace build/clippy/fmt/test clean for everything shipped
1.7.02026-07-08HLTH-901 done: new wovyr-runtime crate holds the one ActivityExecutor dispatch body (PlatformActivityExecutor) the CLI, server, and eval harness all now call, parameterized over an AgentResolver trait for the one genuinely platform-specific piece (agent lookup + tenant/hosted/quota context). Fixed the real semantic drift the ticket named: tool-error retry classification, function-vs-tool dispatch, ai’s system-prompt source and model resolution, and human’s decision-variable-key convention all now behave identically everywhere. Full workspace build/clippy/fmt/test clean
1.6.02026-07-08HLTH-902 done: all 7 CLI marketplace commands are now async fn running their body inside tokio::task::spawn_blocking, fixing the “Cannot start a runtime from within a runtime” panic. Reproduced and confirmed the exact panic + fix with a standalone repro (no live Postgres needed) and verified all 7 commands end to end against the file-based registry. Also added a CI-901 step that runs the CLI binary itself against Postgres — the existing job never had, despite the ticket’s original acceptance criterion assuming it did
1.5.02026-07-08API-705 done: added hardening::DEPRECATIONS (a const route-metadata table) + deprecation_headers middleware, making the Deprecation/Sunset policy mechanically enforceable. Table is empty — no real deprecation exists — with a standing test guarding the 90-day window for whenever one is added. WS-7 is now fully complete
1.4.02026-07-08API-704 done: added a contract-gate CI job (redocly lint + both SDK integration suites against a real, freshly-booted server). Caught and fixed 3 real pre-existing bugs the suites’ never having run in CI let slip through: both SDKs’ workflow-status test still checked the pre-API-702 PascalCase "Completed"; both SDKs’ tools-count assertion assumed leftover local plugin state; the Python suite still read two pre-API-701 field names (tools/total, results). WS-7 now has only API-705 left
1.3.02026-07-07API-703 done: Idempotency-Key replay extended from agents:run only to every mutating route via one shared hardening::idempotency_middleware, keyed by (tenant, method, path, key) (fixing a latent cross-route collision the old tenant+key-only scheme had). openapi.yaml and both SDKs updated in lockstep
1.2.02026-07-07API-702 done: WorkflowState/ActivityState/WorkflowEvent now snake_case on the wire, reconciling the workflow status filter and body casing; MemoryType/PluginState hand-written casing hacks in wovyr-server deleted in favor of the enums’ own serde derive. Round-trip stability tests added for all four
1.1.02026-07-07API-701 done: audit/plugins/marketplace/secrets/tools migrated to the shared cursor-pagination envelope; memory:query renamed results→data (documented as a deliberate non-paginated exception). Both SDKs and openapi.yaml updated in lockstep
1.0.02026-07-06Initial Phase-4 (contract & operability) ticket breakdown: 14 tickets across WS-7 (API freeze), WS-8 (observability/audit/dashboard), and the WS-9 remainder (executor unification, CLI-panic fix, config crate, cleanup), with dependencies, acceptance criteria, file targets, and sizing