Tool Runtime Overview

Document ID: TRT-001
File Path: docs/07-tool-runtime/overview.md
Version: 1.0.0
Status: Draft
Owner: AI Platform Team
Last Updated: 2026-06-27

1. Purpose

This document specifies the Tool Runtime, the deployable service that executes tools for agents and workflows in the Wovyr AI Platform.

The Runtime is the operational counterpart to the Tool Framework. The framework defines the tool model, SDK, manifest, registry, and permission/sandbox schemas; the Runtime provisions sandboxes, enforces policy, runs the tool, and returns the result — safely and at scale.

2. Scope

The Tool Runtime is responsible for:

A network API to invoke, stream, and cancel tool executions
Resolving tools/versions via the registry and routing to workers
Authorizing each execution against policy
Provisioning isolated sandboxes per execution
Enforcing CPU/memory/disk/time/network limits
Injecting secrets and scoped credentials
Rate limiting and fair scheduling across tenants
Emitting audit, metrics, and traces

The Tool Runtime is not responsible for:

Defining the tool model or SDK — see Tool Framework
Deciding which tool to call — that is the Agent Runtime / planner’s job
Packaging or distributing plugins — see the planned 08-plugin-sdk/

3. Position in the Platform

 Agent Runtime ─┐
 Workflow Engine├──► Tool Runtime ──► Sandbox backends (process / wasm / container / microVM)
 Dashboard      ┘        │          ──► Tool Registry (resolve + version)
                         │          ──► Secret Vault (scoped injection)
                         │          ──► Policy Engine (authorize)
                         └── execution events / audit ──► Event Bus

The Runtime is horizontally scalable. A stateless control plane (API + dispatcher + scheduler) fronts a data plane of execution workers that own sandboxes. See C4 Container §4.6 and Worker Pool.

4. Control Plane vs. Data Plane

Plane	Components	Scaling
Control plane	Execution API, Dispatcher, Permission Engine, Scheduler	Horizontal, stateless
Data plane	Worker Pool, Sandbox Manager, Runtime Adapters	Horizontal, capacity-bound

Separating them lets the API stay highly available while heavy/untrusted execution is isolated on workers that can be drained, recycled, or pinned to hardened nodes.

5. Core Responsibilities

5.1 Dispatch

The Dispatcher resolves the requested tool and version against the registry, selects an appropriate worker (by capability, locality, and load), and routes the execution.

5.2 Authorization

Before any code runs, the Permission Engine evaluates the caller’s grants against the tool’s required permissions via the Policy Engine. Denied executions never reach a sandbox.

5.3 Sandbox Provisioning

The Sandbox Manager allocates an isolated environment using the configured backend (native process, WASI, container, microVM, gVisor, K8s pod, or remote worker) and applies resource/network/filesystem policy. See Sandbox Runtime.

5.4 Execution

A Runtime Adapter injects context and secrets, runs the tool, enforces timeouts, streams output, and collects the result. See Execution API.

5.5 Governance

Every execution is rate-limited, metered, audited, and tenant-isolated. See Security & Isolation.

6. Execution Lifecycle

1. Receive invocation          (REST / gRPC)
2. Authenticate + resolve tenant/principal
3. Resolve tool + version      (Registry)
4. Authorize                   (Permission Engine → Policy Engine)
5. Pre-check rate limit + quota
6. Schedule onto a worker
7. Provision sandbox           (apply resource/network/fs policy)
8. Inject context + secrets
9. Execute (stream output, enforce timeout)
10. Collect result / handle error
11. Destroy sandbox + cleanup
12. Emit audit + metrics + execution event
13. Return result + usage

7. Deployment Modes

Mode	Description
Embedded	Runtime runs in-process in the all-in-one dev binary (native sandbox only)
Standalone	Control plane + worker pool as separate services (enterprise default)
Node-local	A worker per node (DaemonSet) for data-locality-sensitive tools
Remote pool	Dedicated hardened worker fleet for untrusted/third-party tools

The Execution API contract is identical across modes. See Deployment Architecture.

8. Module Organization

service-tool-runtime/
├── api/            # REST + gRPC handlers (invoke / stream / cancel)
├── dispatcher/     # tool resolution + worker routing
├── permissions/    # authorization (Policy Engine client)
├── scheduler/      # fair scheduling, concurrency, queueing
├── worker/         # execution worker (data plane)
├── sandbox/        # sandbox manager + adapters
│   ├── native/
│   ├── wasm/
│   ├── container/
│   ├── microvm/
│   └── remote/
├── secrets/        # scoped secret injection
├── governance/     # rate limits, quotas, tenant isolation, audit
├── telemetry/      # logs, metrics, traces
└── main.rs

The sandbox/ adapters implement the backends enumerated in Tool Framework §31.

9. Non-Functional Requirements

Requirement	Target
Dispatch + authorize overhead	< 10 ms p95
Warm sandbox start (pooled)	< 20 ms p95
Cold sandbox start (microVM)	< 200 ms p95
Concurrent executions per worker	hundreds (backend-dependent)
Availability (control plane)	99.99%
Cross-tenant isolation	hard (zero leakage)

10. Failure Behavior

Failure	Behavior
Tool not found / version missing	`404 tool_not_found`, no execution
Authorization denied	`403 forbidden`, no execution
Sandbox provisioning failure	Retry on another worker, then `503`
Timeout	Cancel sandbox, return `timeout` (retryable)
Tool crash / non-zero exit	Return `tool_error` with captured diagnostics
Worker death mid-execution	Reschedule if idempotent; else surface failure
Resource limit exceeded	Kill sandbox, return `resource_exceeded`

Retry semantics integrate with the Retry Engine; only idempotent tools are retried automatically.

11. Security

All tools run sandboxed; nothing executes on the host or control plane.
Default-deny network and filesystem; access is explicitly granted per tool.
Secrets are injected into the sandbox at runtime and never logged.
Untrusted/third-party tools run on hardened, isolated worker pools.
Every execution is audited (tenant, principal, tool, version, inputs hash, result).

See Security & Isolation and Tool Framework §70.

12. Observability

Each execution emits logs, metrics (queue time, start latency, run duration, success/error rate, resource usage), OpenTelemetry traces, and an audit record. Execution events publish to the Event Bus. See Observability & Ops.

13. Dependencies

15. Future Enhancements

Snapshot/restore sandboxes for sub-millisecond cold starts
GPU-aware scheduling for ML tools
Per-tool warm pools driven by demand prediction
WASM component model for portable tools
Cross-region execution routing for data residency

16. Revision History

Version	Date	Description
1.0.0	2026-06-27	Initial Tool Runtime Overview