Validate and normalize SandboxPolicy before supervisor activation

[shiju-nv]

User Story

As an OpenShell operator, I want every code path that turns policy input into active runtime state - including gateway startup/reload, combined-topology OPA construction, sidecar bootstrap/update, and OPA YAML/protobuf conversion - to enforce one effective-policy contract: validate and normalize a SandboxPolicy at typed-policy boundaries and apply equivalent schema validation and normalization at OPA-native data boundaries, so the policy cannot change meaning between save, serialization, transport, and supervisor activation.

Problem Statement

In Kubernetes sidecar topology, the network supervisor sends policy to the process supervisor in bootstrap and update messages over the local Unix control socket. The sandbox policy loader can also convert protobuf or YAML data into OPA input and the internal policy used for enforcement. Each boundary can interpret policy before validation and normalization unless it checks the policy explicitly.

The known concrete failure is MCP: YAML alias lowering can discard the mcp_versions array while protobuf conversion retains it, giving the same authored policy different runtime meaning. The same failure mode is relevant to any protocol-specific policy field that is dropped or reinterpreted by one loader; the implementation must establish the invariant for the complete policy rather than add an MCP-only exception.

Impact / Why This Matters

Without validation in each policy-loading handler, a protocol-specific option can reach enforcement code even though the policy compiler would reject it. A YAML-loaded policy and a protobuf-loaded policy can then authorize different traffic, and a bootstrap or update path can install a different policy from the one stored by the gateway. The operator may believe the configured policy is active while the supervisor is using a weaker or different interpretation.

This can affect any endpoint whose policy travels through an affected bootstrap, update, YAML, or protobuf path, including REST, GraphQL, WebSocket, generic JSON-RPC, and MCP endpoints. A policy that changes meaning across restart, live update, or serialization format can authorize requests the operator did not approve or reject requests the operator expected to allow.

Proposed Design

OpenShell validates and normalizes the complete SandboxPolicy at each listed typed-policy boundary, before installing supervisor state or constructing enforcement input. At the OPA-native YAML boundary, it validates the complete OPA data schema it accepts and applies equivalent normalization before the data is installed in OPA; it does not need to fabricate a reverse SandboxPolicy for data that has no complete typed representation. The listed boundaries include gateway-provided startup and reload data, combined-topology OPA construction, sidecar bootstrap and update messages, and OPA YAML/protobuf conversion. For Kubernetes sidecar topology, the same check applies to the policy bootstrap and update messages sent from the network-supervisor sidecar to the process supervisor over the local control socket. Each protocol keeps its own policy contract: omitted MCP revisions resolve to the pinned default, valid MCP allowlists use deterministic ordering, and invalid, duplicate, unsupported, or misplaced MCP values are rejected before activation.

All listed policy-loading paths produce the same effective SandboxPolicy for the same authored input, including every protocol-specific field. An endpoint with protocol: json-rpc, REST, GraphQL, or WebSocket retains its own protocol behavior and does not acquire MCP revision state. Failure messages identify which policy load was rejected and remain bounded; they do not include the policy body or credential material.

The policy-advisor /wait endpoint evaluates coverage once for each installed policy snapshot and proposed rule, regardless of protocol, and repeats the check only when a new snapshot is installed. This preserves the comparison while avoiding a full comparison of an unchanged snapshot on every timer tick.

Acceptance Criteria

• The gateway runtime startup/reload path, combined-topology OPA startup/reload, and sidecar control-socket bootstrap/update validate a complete SandboxPolicy before installing supervisor state; OPA-native YAML loading validates the complete schema it accepts before creating enforcement input.
• In Kubernetes topology: sidecar, the process supervisor rejects invalid policy in the initial control-socket bootstrap message before installing it; MCP omission still materializes the pinned revision when omission is valid.
• Policy-update messages sent over the sidecar control socket apply the same validation and normalization as bootstrap for every protocol.
• Where the same authored policy is representable in both formats, OPA protobuf and YAML policy loaders produce the same validated and normalized effective SandboxPolicy and preserve all protocol-specific fields, including the MCP mcp_versions array; OPA-only data files are validated against their own schema before becoming OPA input.
• MCP options on a non-MCP endpoint, explicit empty YAML mcp_versions lists, duplicates, padded values, unsupported revisions, and draft revisions fail before lowering or enforcement; a missing or empty protobuf repeated field follows the documented pinned-default behavior because protobuf does not preserve authoring presence.
• REST, GraphQL, WebSocket, and generic protocol: json-rpc endpoints retain their own behavior, receive validation for their own policy fields, and do not receive MCP revision state.
• Representative policy-load and YAML/protobuf parity tests cover REST, GraphQL, WebSocket, generic JSON-RPC, and MCP policies; MCP tests cover omission, invalid values, and revision-array preservation.
• The policy-advisor GET /v1/proposals/{chunk_id}/wait?timeout=<s> endpoint does not repeat full coverage validation for an unchanged policy snapshot and does recheck a newly installed snapshot for a matching proposed rule.
• Runtime errors are bounded and contain no policy payload, credentials, or raw request body.
• Tests cover sidecar bootstrap/update, YAML/protobuf parity, omission and invalid values, generic JSON-RPC preservation, and unchanged-versus-replaced /wait snapshots.

Alternatives Considered

Validating only when the gateway saves a policy is insufficient because the running supervisors can receive older records, direct protobuf messages, or data produced by another path. Each listed active-runtime boundary must validate the complete policy before it installs supervisor state or creates enforcement input.

Fixing only the known MCP field or trusting protobuf while leaving YAML lowering unchanged would preserve one input path while silently dropping a protocol-specific policy field on another. YAML and protobuf must produce the same validated and normalized SandboxPolicy, and the same rule must apply to sidecar bootstrap and updates.

Agent Investigation

No response

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request