Skip to content
Wovyr

Roadmap: v0.2 — Memory, Tools & Gateway Hardened

Document ID: RM-002
File Path: docs/18-roadmap/v0.2.md
Version: 1.1.0
Status: Exit criteria met (RAG agent, durable support workflow, chaos tests, performance NFRs); memory Postgres/Qdrant tiered backend landed behind the tiered feature (with capability-gated live integration tests); all in-scope engine tracks (incl. intra-workflow parallel activity execution) complete
Owner: Product Team
Last Updated: 2026-06-28

1. Theme

Section titled “1. Theme”

Depth and durability. Make the core engines production-shaped: real memory, durable workflows, stronger tool isolation, and a resilient gateway.

2. Goals

Section titled “2. Goals”
  • Memory Engine: PostgreSQL + Qdrant tiers, hybrid retrieval, ranking, compression.
  • Workflow Engine: durable executions with checkpointing, retry, and compensation.
  • Tool Runtime: container + gVisor/microVM sandboxes (sandbox runtime) and worker pools.
  • LLM Gateway: failover, circuit breaking, caching, token/cost accounting.
  • Observability: metrics, tracing, logging baseline (section 14). Done (Prometheus metrics + structured logging + OTLP trace/metric/log export + exemplars behind the otlp feature).

3. In Scope

Section titled “3. In Scope”
AreaDeliverableStatus
MemoryNamespaces, hybrid search, knowledge graph (v1)Done (core + Postgres/Qdrant tiered backend, gated integration tests, MMR diversification, ABAC filtering, compression); knowledge graph deferred to v1
WorkflowDAG + state machine + durable resume + compensation + branching + human approvalDone (core + Postgres durable store + timer/event waiting states + distributed workers + intra-workflow parallel activity execution)
ToolsStrong sandboxes, fair scheduling, autoscalingDone — Native/Container/gVisor/WASI + Firecracker backends + enforcement + warm pooling + autoscaling + egress proxy + fair scheduling (L3 egress bypass-blocking deferred)
GatewayResilience + caching + cost eventsDone (core + Redis-shared breakers + semantic cache incl. Qdrant-backed distributed index + hedging)
OpsDashboards, alerts, SLOsPlanned

Workflow engine core (done): crates/wovyr-workflow — DSL + validation, event-sourced state machine, deterministic DAG scheduler, conditional branching (guarded transitions + branch skipping), per-activity retry, saga compensation (reverse-order rollback), and durable suspend/resume (the Interrupted waiting state — e.g. human approval). The durability ports (EventLog + CheckpointStore) are backed by in-memory/file stores and a PostgresStore (behind the postgres feature) for cross-process/node durable resume, with a capability-gated live test (tests/postgres_store.rs). Timer/event waiting states: built-in wait activities (inputs: {event} / {timer}) suspend durably (activity state Waiting) until Engine::signal_event/fire_timer delivers the signal and resumes. CLI: wovyr workflows validate|run|approve|signal. Distributed workers: a WorkQueue leases each ready execution to one Worker (InMemoryWorkQueue, or the PostgresStore via FOR UPDATE SKIP LOCKED + lease expiry); Engine::start + enqueue hands off to workers that drive via the idempotent resume, with crashed-lease reclaim → exactly-once activity effects. Verified offline (two workers, exactly-once + crash recovery) and live against Postgres (two connections). Intra-workflow parallel activities: a fan-out of simultaneously-ready activities runs concurrently (a JoinSet overlapping executor calls + retry backoffs) against a shared pre-batch variable snapshot, then commits in declaration order so the event log/checkpoint stay deterministic; a failed branch still commits its completed siblings before compensating. The workflow track is complete.

Memory engine core (done): crates/wovyr-memory — namespaces, hybrid retrieval (vector + keyword via RRF), weighted ranking (relevance + recency + importance) with score breakdown, in-memory + file stores; embeds via the LLM Gateway. A tiered durable backend (behind the tiered cargo feature) composes a PostgresStore (system of record + tsvector keyword search) with a QdrantStore (vector ANN over REST) as TieredStore; the engine now supports retrieval pushdown (MemoryStore::supports_pushdown → vector/keyword search at the index, RRF-fused for hybrid) and falls back to in-process ranking for the file/in-memory stores. CLI: wovyr memory put|query, selecting the tiered backend when built with --features tiered-memory and WOVYR_MEMORY_POSTGRES_URL + WOVYR_MEMORY_QDRANT_URL are set. The pushdown/backend logic has offline unit tests, and the live Postgres/Qdrant round-trip is covered by capability-gated integration tests (crates/wovyr-memory/tests/tiered_backend.rs). MMR diversification (MemoryQuery::diversity / CLI --diversity) re-ranks results to trade relevance for less redundancy. ABAC filtering (required_scopes on records / AccessContext on queries; CLI put --require-scope / query --grant) drops records whose scopes the principal lacks, fail-closed. Compression (compress / CLI memory compact) consolidates stale low-importance memories into a gateway-summarized note and deletes the originals (MemoryStore::delete). Remaining: knowledge graph (deferred to v1).

Gateway resilience (done): crates/wovyr-provider — the Gateway now holds an ordered candidate list with per-provider circuit breakers, retry of transient errors with exponential backoff, failover, a response cache, and cost events. The breaker is a CircuitBreaker trait: the default LocalCircuitBreaker is in-process; SharedCircuitBreaker keeps state in a shared BreakerKv so a fleet trips/recovers together — RedisKv behind the redis feature (Gateway::with_redis_breakers), with offline unit tests over InMemoryKv and a capability-gated live test (tests/redis_breaker.rs). The cache supports exact and semantic modes (embedding-similarity match over param-compatible entries, exact check first; hits flagged cache: "semantic"). The semantic cache is a SemanticCacheStore trait: InMemorySemanticCache by default, or a fleet-shared QdrantSemanticCache (qdrant feature / with_qdrant_semantic_cache), with a capability-gated live test (tests/semantic_cache_qdrant.rs). Request hedging (with_hedging, off by default) races a slow candidate against its successors after a delay and returns the first to answer. The gateway resilience track is complete.

Observability baseline (done): crates/wovyr-telemetry — a Prometheus Metrics registry (counters + histograms) and structured (JSON) logging. The server exposes GET /metrics with RED route metrics and wovyr_llm_* cost/token metrics fed by gateway cost events. OTLP trace export (behind the otlp feature): a tracing-opentelemetry layer batches the platform’s spans — agent.run, gateway.chat, workflow.activity, api.agents_run — to an OTLP collector when OTEL_EXPORTER_OTLP_ENDPOINT is set; verified end-to-end against a live collector. Exemplars: histogram buckets carry the observing span’s trace id, rendered in OpenMetrics (render_openmetrics(), served from /metrics on content negotiation) — verified live linking a latency bucket to its exported trace. OTLP metrics + logs: the Metrics registry dual-writes to an OTLP push exporter (periodic reader) via Metrics::with_otlp_export, and an opentelemetry-appender-tracing bridge ships log events as OTLP logs — both behind the otlp feature, verified live against a collector (RED/LLM metrics + spans + log records received). The observability track is complete.

Tool sandbox backends (done): crates/wovyr-tools — the SandboxBackend spectrum, TrustClass isolation floors, and select_backend/SandboxManager (strongest of preference/floor/trust, then capability check; SandboxManager::detect probes the host). Backends: NativeSandbox enforces memory/CPU caps via setrlimit plus timeout + output cap; ContainerSandbox runs Docker/Podman with cgroup memory/CPU/PID limits, a read-only rootfs, a bind-mounted workspace, and NetworkPolicy egress control (--network none on deny), and drives gVisor via --runtime=runsc; FirecrackerSandbox runs commands in a microVM via a one-shot block-device protocol (input/output drives + an /init guest agent, see deployment/firecracker/), capability-gated on firecracker + /dev/kvm and verified live. A WasiSandbox (behind the wasi cargo feature) runs wasm32-wasi modules in an in-process Wasmtime VM with capability-based isolation and memory/fuel/epoch limits. A SandboxPool adds warm pooling + autoscaling: a bounded, pre-warmed set of reusable sandbox instances (semaphore-bounded acquire, a PooledSandbox guard that returns itself on drop), with an AutoscalePolicy (min_warm/max_warm) whose autoscale() refills the warm set under load and evicts idle excess — caller-driven and deterministic. Unit-tested offline over the native backend plus a capability-gated live container-pool test. Container/gVisor/Firecracker execution is covered by capability-gated integration tests (the microVM one verified live); WASI by in-process tests. A non-empty NetworkPolicy allow-list is enforced by an EgressProxy (host-side HTTP CONNECT proxy; allow-listed hosts tunnel, others get 403), wired into the container via HTTPS_PROXY — verified offline and with a live container deny test. Fair scheduling: a FairScheduler admits queued work across tenants by smooth weighted round-robin, bounded to max_in_flight, in front of the pool’s concurrency — no tenant starves capacity; weights bias share. Remaining: L3 bypass-blocking (a workload ignoring the proxy).

Remaining v0.2 areas: ops dashboards/alerts/SLOs (largely belong with the v0.3 dashboard).

4. Out of Scope (deferred)

Section titled “4. Out of Scope (deferred)”
  • Plugin marketplace + third-party plugins → v0.3
  • Full dashboard UX → v0.3
  • Multi-region → v1.0

5. Exit Criteria

Section titled “5. Exit Criteria”
  • RAG agent runs (memory-grounded agent: examples/agents/docs-bot.yaml; retrieval wired into the run loop via the ContextRetriever hook).
  • Durable support workflow runs (examples/workflows/support.yaml: conditional branching + durable human approval via workflows run/approve + saga compensation).
  • Resilience validated by chaos tests: gateway fault injection (crates/wovyr-provider/tests/chaos.rs) — failover, circuit-breaker open + recovery, permanent-error pass-through, cache-shielded outage, total outage — plus an agent-level steady-state test (crates/wovyr-agent/tests/resilience.rs). Fault classes for not-yet-built subsystems (Qdrant/Redis/Postgres/NATS/node loss) remain out of scope.
  • Performance meets baseline NFRs: assertion-style perf tests measure p50/p95/p99 and gate the documented targets — gateway overhead < 8 ms p95 (crates/wovyr-provider/tests/perf.rs, measured ~5 µs) and memory warm retrieval < 30 ms p95 (crates/wovyr-memory/tests/perf.rs, measured ~4–5 ms). API-server / tool warm-start / billion-record scale NFRs await their subsystems (server load harness, warm pooling, sharded stores) and are out of scope.
Section titled “6. Related”

7. Revision History

Section titled “7. Revision History”
VersionDateDescription
1.0.02026-06-27Initial v0.2 roadmap
1.1.02026-06-27Workflow engine core implemented (DAG + durable resume + retry)
1.2.02026-06-27Workflow compensation + memory engine core implemented
1.3.02026-06-27Gateway resilience: failover, circuit breaker, cache, cost events
1.4.02026-06-27Observability baseline: Prometheus metrics + /metrics + JSON logging
1.5.02026-06-27Tool sandbox abstraction: backend spectrum, trust floors, selection
1.6.02026-06-28Sandbox backends implemented: native rlimit enforcement, Container/gVisor (cgroups + network policy), Firecracker config (gated)
1.7.02026-06-28WASI/WASM sandbox backend (Wasmtime, wasi feature): module execution, memory/fuel/epoch limits, capability isolation
1.8.02026-06-28RAG agent example (memory-grounded agent via ContextRetriever)
1.9.02026-06-28Workflow conditional branching + durable human approval; customer-support example
1.10.02026-06-28Chaos tests: gateway fault injection (failover, breaker open/recovery, cache-shielded outage) + agent steady-state; Gateway::with_breaker
1.11.02026-06-28Performance NFR tests (gateway overhead, memory warm retrieval); all v0.2 exit criteria met
1.12.02026-06-28Memory tiered backend: Postgres + Qdrant TieredStore (tiered feature), retrieval pushdown, CLI --features tiered-memory selection
1.13.02026-06-28Tiered backend capability-gated live integration tests (tiered_backend.rs): PG put/get/keyword, Qdrant vector, hybrid via engine
1.14.02026-06-28Redis-shared circuit breaker: CircuitBreaker trait + SharedCircuitBreaker/BreakerKv, RedisKv (redis feature), Gateway::with_redis_breakers; offline + gated live tests
1.15.02026-06-28Semantic response cache (CacheMode::Semantic): canonical-request embedding, in-process cosine match over param-compatible entries, exact-first lookup order
1.16.02026-06-28Request hedging (HedgeConfig/with_hedging): race slow candidates against successors after a delay, first-success wins, composes with failover/breakers
1.17.02026-06-28Qdrant-backed distributed semantic cache (SemanticCacheStore trait + QdrantSemanticCache, qdrant feature); gateway resilience track complete
1.18.02026-06-28Workflow Postgres durable store (PostgresStore: EventLog + CheckpointStore, postgres feature) for cross-process/node resume; gated live test
1.19.02026-06-28Memory MMR diversification (MemoryQuery::diversity / CLI --diversity): greedy maximal-marginal-relevance re-ranking over the weighted scores
1.20.02026-06-28Memory ABAC filtering (required_scopes / AccessContext, CLI --require-scope/--grant): fail-closed scope enforcement, persisted in the tiered backend
1.21.02026-06-28Memory compression (compress / CLI memory compact): gateway-summarized consolidation of stale memories + MemoryStore::delete across all stores; memory v0.2 surface complete (KG → v1)
1.22.02026-06-28Workflow timer/event waiting states: built-in wait activities, Engine::signal_event/fire_timer, ActivityState::Waiting, CLI workflows signal
1.23.02026-06-28Sandbox warm pooling (SandboxPool/PooledSandbox): bounded pre-warmed reusable instances, semaphore concurrency; offline + gated live container-pool tests
1.24.02026-06-28Sandbox pool autoscaling (AutoscalePolicy, autoscale()): warm-set refill under load + idle eviction within [min_warm, max_warm] ≤ max_size, deterministic
1.25.02026-06-28OTLP trace export (otlp feature): tracing-opentelemetry layer + span instrumentation (agent.run/gateway.chat/workflow.activity); verified live against an OTLP collector
1.26.02026-06-28Prometheus exemplars: histogram buckets carry the span trace id, OpenMetrics rendering (render_openmetrics) + /metrics content negotiation; verified live (latency bucket → exported trace)
1.27.02026-06-28Firecracker microVM execution: FirecrackerSandbox::execute via a one-shot block-device protocol + /init guest agent (deployment/firecracker/); verified live (command runs in-guest, output returned)
1.28.02026-06-28Egress proxy (EgressProxy): allow-listing HTTP CONNECT proxy enforcing per-host NetworkPolicy, wired into ContainerSandbox via HTTPS_PROXY; offline + live container deny tests
1.29.02026-06-28Tenant-fair scheduling (FairScheduler): smooth weighted round-robin admission bounded to max_in_flight, pairs with SandboxPool; tools track complete
1.30.02026-06-28Distributed workers: WorkQueue (InMemoryWorkQueue + Postgres SKIP LOCKED leasing), Engine::start, Worker (lease→resume→release, crash-lease reclaim); exactly-once verified offline + live (two Postgres connections)
1.31.02026-06-29Per-tool permission enforcement: ToolRegistry::execute/check_permissions deny ungranted tool permissions (fail-closed), agent manifest permissions allow-list wired into the run loop
1.32.02026-06-29Agent persistence: in-memory AgentStore + REST endpoints (create/list/get/delete + run-by-id POST /api/v1/agents/{id}/run); closes the inline-manifest server deviation
1.33.02026-06-29Token streaming: AIProvider::chat_stream/ChatStreamEvent, gateway streaming (failover + cost-on-done), agent per-chunk Deltas, server SSE endpoint (POST /api/v1/agents:stream); mock streams multi-chunk (OpenAI per-token deferred)
1.34.02026-06-29OpenAI per-token streaming: OpenAiProvider::chat_stream parses upstream SSE into per-chunk Deltas + a StreamAccumulator-assembled Done (incl. incremental tool calls + usage); verified live against ollama (local + cloud)
1.35.02026-06-29Intra-workflow parallel activities: concurrent JoinSet execution of a ready batch with deterministic declaration-order commit (snapshot isolation, sibling-commit-before-compensate); workflow track complete
1.36.02026-06-29OTLP metrics + logs export (otlp feature): registry dual-write to an OTLP push exporter via Metrics::with_otlp_export, opentelemetry-appender-tracing log bridge; verified live against a collector (metrics + spans + logs received); observability track complete