Skip to content

feat: OpenShell sandbox with validated pattern integration - #2

Merged
pmatouse merged 12 commits into
mainfrom
feat/validated-pattern-integration
Jul 29, 2026
Merged

feat: OpenShell sandbox with validated pattern integration#2
pmatouse merged 12 commits into
mainfrom
feat/validated-pattern-integration

Conversation

@sauagarwa

Copy link
Copy Markdown
Collaborator

Summary

  • Bootc-based gateway VM image (replaces virt-customize/libguestfs pipeline)
  • TLS passthrough on gateway route for gRPC/HTTP2 support
  • Operator-based Keycloak via RHBK (Keycloak CR + KeycloakRealmImport CR)
  • Pattern-secrets chart with ExternalSecrets for all inference providers + SSH keys
  • Auto-derive OIDC issuer URL from global.clusterDomain (VP flow) or explicit value (quickstart)
  • Dedicated SSH keypair generation (make generate-keys) — user's personal key never touches the cluster
  • Repo reorganized: build-time charts in image-builder-charts/, pattern-managed charts in charts/
  • Complex Makefile logic extracted into shell scripts (scripts/)
  • Headless E2E test via make test (auto-detects provider, ROPC token fetch, no browser)

Two deployment paths

Path How What it deploys
Validated Pattern make install Operators + Vault + ESO + Keycloak + secrets + sandbox via ArgoCD
Quickstart make keycloak && make sandbox-create ... Manual helm deploys with auto-detected OIDC

Test plan

  • tests/test-oidc-templates.sh — 43/43 helm template validations pass
  • tests/test-bootc-e2e.sh — 28/28 E2E checks pass (Keycloak, golden image, sandbox VM, routes)
  • make test — headless E2E with auto-detected provider from ~/values-secret.yaml

🤖 Generated with Claude Code

sauagarwa and others added 12 commits July 28, 2026 13:22
Helm charts for deploying per-user OpenShell agent sandboxes on
OpenShift Virtualization (KubeVirt) with OIDC authentication and
per-user access control.

Charts:
- openshell-sandbox: per-user sandbox VM with access control proxy
- openshell-gateway: master gateway VM and snapshot
- openshell-gateway-image: pre-baked gateway VM disk image
- openshell-keycloak: optional OIDC provider for development
- nemoclaw-imagestream: NemoClaw sandbox image builder
- nemoclaw-cli-imagestream: NemoClaw CLI image builder

Key features:
- Per-user access control via openresty auth proxy that validates
  JWT preferred_username matches the sandbox owner
- Configurable namespace modes: shared (default) or per-user
- OIDC authentication with any provider (Keycloak, Okta, Entra ID)
- Interactive owner detection from OIDC token with override prompt
- openshell CLI integration (gateway add, login, sandbox management)
- E2E test scripts for access control and multi-user isolation

CLI:
- openshell-saw admin provisioning tool for cluster operations
- Users interact via upstream openshell CLI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add make login step to user flow (OIDC auth before sandbox creation)
- Fix access control docs: openresty auth proxy, not oauth2-proxy
- Update accessControl values (remove cookieSecret, correct image)
- Reorder quick starts: containerDisk first (simpler, no gateway needed)
- Add teardown section
- Rename title to Secure Agent Workspace

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Keep only scripts referenced by charts and Makefile:
oidc-login.sh, test-access-control.sh, test-multiuser-isolation.sh

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The openshell CLI uses a random port for the OIDC callback. Allow
http://localhost:* and http://127.0.0.1:* as redirect URIs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Edge TLS termination strips HTTP/2 ALPN negotiation, breaking gRPC
connections from the openshell CLI. The gateway route now always uses
passthrough so the gateway VM handles TLS directly. The auth proxy is
retained for the dashboard route (HTTP/1.1) only, with its upstream
updated to the dashboard port (18789).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the virt-customize/libguestfs pipeline with a bootc Containerfile
based on quay.io/fedora/fedora-bootc:44. CDI on OpenShift 4.16+ imports
bootc images directly into golden image PVCs — no qcow2 wrapping needed.

Remove the snapshot workflow entirely (master VM + VirtualMachineSnapshot +
VirtualMachineClone). Sandboxes now always clone from the bootc golden image.

Key changes:
- buildconfig.yaml: single-stage bootc Containerfile (no libguestfs)
- Deleted snapshot-job, snapshot-cleanup, snapshot-rbac templates
- Deleted virtualmachineclone.yaml (snapshot-only)
- Simplified cloud-init, Makefile, RBAC, and test templates
- Added scripts/test-bootc-e2e.sh for end-to-end validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move build-time charts to image-builder-charts/helm/ (imagestreams, gateway-image)
- Move CLI to repo root (cli/)
- Move test scripts to tests/
- Remove standalone openshell-gateway chart (replaced by bootc golden image)
- Remove standalone keycloak chart (replaced by operator-based chart)
- Add operator-based Keycloak chart (Keycloak CR + KeycloakRealmImport CR)
- Add .gitignore for generated keys and secrets
- Add overrides/openshell-sandbox.yaml for default sandbox in VP flow
- Move Makefile to root as Makefile-quickstart (included by main Makefile)
- Extract complex make targets into shell scripts:
  sandbox-create.sh, sandbox-logout.sh, sandbox-gui.sh,
  generate-keys.sh, e2e-test.sh
- Add make check-prereqs, make generate-keys, make test
- Update README for both VP and quickstart paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pattern-secrets chart with ExternalSecret resources for each
provider (anthropic, gemini, openai, nvidia, openrouter, vertex,
tavily, brave-search) and SSH keys. Secrets flow through Vault via ESO.

Add openshell-keycloak, pattern-secrets, and openshell-sandbox as
ArgoCD-managed applications in values-prod.yaml.

Populate values-secret.yaml.template with all provider API key fields
and SSH key fields.

Update the sandbox chart:
- Auto-derive OIDC issuer URL from global.clusterDomain (VP flow)
  or use explicit oidc.issuerUrl (quickstart flow)
- Mount ESO-managed provider secret in the setup Job
- Inject API key and SSH public key from secrets at runtime
- Add inference.secretName and oidc.keycloakName/realm values

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… analysis

Rewrite README.md following the rh-ai-quickstart template structure:
table of contents, overview, detailed description, architecture diagram,
requirements (hardware/software/permissions), two deployment paths
(validated pattern + quickstart), validation, deletion, repo structure,
references, technical details, and tags.

Add docs/architecture-alignment.md mapping the NVIDIA Secure Agent
Workspace reference design to this implementation. Identifies full
alignment on VM isolation, OIDC, GitOps, secrets, and agent runtime.
Documents gaps in network policies, policy bundles (Phase II), audit,
and image signing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add three diagrams from the NVIDIA Secure Agent Workspace reference
design: OpenShift reference shape, GitOps policy model, and NFS
storage layout. Images stored in docs/images/ and linked in README
with attribution to the source document.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add the three image build commands (build, build-cli, build-gateway-image)
as a required step before make install. Note that this will be replaced
by pre-built upstream golden images in a future release.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix repo_root() to look for charts/ instead of helm/
- Fix chart_path() to search charts/ and image-builder-charts/helm/
- Fix OIDC issuer detection to query Keycloak CR status (RHBK operator)
  instead of the deleted openshell-gateway helm release
- Fix Keycloak route lookup to use label selector instead of fixed name
- Update default SSH key path to .generated-ssh-keys/sandbox-ssh
- Remove source_mode option (snapshot mode removed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pmatouse
pmatouse merged commit a3ba40d into main Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants