Memory Engine API

Document ID: MEM-002
File Path: docs/06-memory-engine/memory-api.md
Version: 1.0.0
Status: Draft
Owner: AI Platform Team
Last Updated: 2026-06-27

1. Purpose

This document defines the external contract of the Memory Engine — the API every caller uses to store and retrieve memory, independent of the underlying storage backends.

The contract is exposed over REST (HTTP/JSON) and gRPC with identical semantics, and maps directly onto the MemoryProvider trait from the Memory System.

2. Design Rules

One schema for all memory types; the backend tier is chosen by the Engine.
Every record is scoped by tenant and a scope (private → public).
Reads return relevance score and the matched version.
Embeddings are generated by the Engine; callers send text, not vectors (vectors optional for advanced callers).
The contract is versioned under /v1.

3. Endpoints (REST)

Method	Path	Purpose
POST	`/v1/memories`	Store a memory
GET	`/v1/memories/{id}`	Fetch a memory by id
PATCH	`/v1/memories/{id}`	Update (creates a new version)
DELETE	`/v1/memories/{id}`	Delete / tombstone
POST	`/v1/memories/query`	Retrieve by hybrid search
GET	`/v1/memories/{id}/versions`	List versions
POST	`/v1/memories/batch`	Bulk store
GET	`/healthz`, `/readyz`, `/metrics`	Operations

gRPC exposes Store, Get, Update, Delete, Query, ListVersions, BatchStore on the MemoryEngine service.

4. Memory Record

{
  "id": "mem_01H...",
  "tenant": "acme",
  "scope": "project",
  "project": "support-bot",
  "agent": "order-assistant",
  "type": "semantic",
  "title": "Refund policy",
  "content": "Refunds are processed within 14 days of purchase.",
  "tags": ["policy", "refunds"],
  "labels": { "source": "handbook", "confidence": "high" },
  "metadata": { "url": "https://intranet/handbook#refunds" },
  "version": 1,
  "created_at": "2026-06-27T10:00:00Z",
  "updated_at": "2026-06-27T10:00:00Z"
}

Field	Notes
`type`	One of working, conversation, workflow, episodic, semantic, shared, organizational, archived
`scope`	private, agent, workflow, project, organization, public
`content`	Text; the Engine embeds it unless `embedding` is supplied
`embedding`	Optional caller-supplied vector (skips embedding generation)
`version`	Server-assigned; increments on update

5. Store Request / Response

Request:

{
  "scope": "project",
  "project": "support-bot",
  "type": "semantic",
  "title": "Refund policy",
  "content": "Refunds are processed within 14 days of purchase.",
  "tags": ["policy", "refunds"]
}

Response:

{ "id": "mem_01H...", "version": 1, "embedded": true, "tier": "warm" }

If embedding generation is deferred (Gateway busy), embedded is false and the record becomes semantically searchable once the async embed completes.

6. Query Request

The query API drives the Retrieval and Ranking pipelines.

{
  "query": "how long do refunds take?",
  "scope": ["project", "organization"],
  "project": "support-bot",
  "types": ["semantic", "episodic"],
  "strategy": "hybrid",
  "filters": {
    "tags": ["refunds"],
    "labels": { "confidence": "high" },
    "created_after": "2026-01-01T00:00:00Z"
  },
  "ranking": {
    "weights": { "relevance": 0.6, "recency": 0.2, "importance": 0.2 }
  },
  "limit": 10,
  "token_budget": 1500,
  "include_graph": true
}

Field	Meaning
`strategy`	`vector`, `keyword`, `hybrid`, or `graph` (see Retrieval)
`filters`	Metadata constraints applied before/after search
`ranking.weights`	Overrides default scoring weights (see Ranking)
`token_budget`	Engine compresses results to fit (see Compression)
`include_graph`	Expand results with related graph entities

7. Query Response

{
  "results": [
    {
      "id": "mem_01H...",
      "type": "semantic",
      "title": "Refund policy",
      "content": "Refunds are processed within 14 days of purchase.",
      "version": 1,
      "score": 0.94,
      "score_breakdown": { "relevance": 0.91, "recency": 0.40, "importance": 0.80 },
      "match": "hybrid"
    }
  ],
  "graph": {
    "entities": [{ "id": "policy:refunds", "type": "policy" }],
    "edges": [{ "from": "policy:refunds", "to": "team:finance", "rel": "owned_by" }]
  },
  "usage": { "embeddings": 1, "compressed": true, "returned_tokens": 1280 },
  "degraded": false
}

degraded: true indicates retrieval fell back (e.g. keyword-only because the vector store was unavailable) — recall may be reduced.

8. Update & Versioning

PATCH /v1/memories/{id} creates a new version rather than mutating in place. Prior versions remain retrievable via GET /v1/memories/{id}/versions. Updates re-embed and re-index automatically.

9. Delete Semantics

DELETE performs a soft delete (tombstone) by default so audit and version history are preserved; a hard=true query parameter permanently purges the record and its index entries (subject to retention policy and permissions).

Reads respect the caller’s permissions across scopes (aligned with Memory System §24):

private < agent < workflow < project < organization < public

A query may request multiple scopes; the Engine returns only records the principal is authorized to read and records this in the audit log.

11. Error Model

{
  "error": {
    "code": "forbidden",
    "message": "Principal may not read organization-scoped memory.",
    "type": "client_error",
    "retryable": false
  }
}

Code	HTTP	Retryable	Meaning
`unauthenticated`	401	no	Missing/invalid credentials
`forbidden`	403	no	Scope/policy denied
`not_found`	404	no	Unknown memory id
`invalid_request`	400	no	Schema validation failed
`embedding_unavailable`	202	yes	Stored; embedding deferred
`storage_unavailable`	503	yes	Backend temporarily unavailable
`quota_exceeded`	429	yes	Tenant memory quota hit

12. Idempotency & Batch

Writes accept an Idempotency-Key to dedupe client retries.
POST /v1/memories/batch ingests up to 1,000 records per call; partial failures are reported per-item without failing the whole batch.

13. Versioning Policy

/v1 is additive-compatible; breaking changes introduce /v2. Storage/tier changes never affect the contract version.

14. Dependencies

06-memory-engine/overview.md
06-memory-engine/storage-architecture.md
09-api (planned: platform REST API)

16. Revision History

Version	Date	Description
1.0.0	2026-06-27	Initial Memory Engine API