diff --git a/.agents/skills/debug-openshell-cluster/SKILL.md b/.agents/skills/debug-openshell-cluster/SKILL.md index 86395af148..f242acf0b7 100644 --- a/.agents/skills/debug-openshell-cluster/SKILL.md +++ b/.agents/skills/debug-openshell-cluster/SKILL.md @@ -58,7 +58,7 @@ Common findings: - `No active gateway`: register one with `openshell gateway add `. - Connection refused: gateway process is not running, service exposure is wrong, or a port-forward/proxy is not active. -- TLS/certificate errors: the endpoint scheme or trust chain is wrong, a local mTLS bundle does not match the gateway CA, or TLS termination does not match the gateway listener. +- TLS/certificate errors: the endpoint scheme or trust chain is wrong, a CLI mTLS bundle does not match the gateway CA, a sandbox is missing the gateway CA, or TLS termination does not match the gateway listener. Sandboxes should not contain a TLS client certificate or private key. - `Unauthenticated` from an edge or OIDC gateway: refresh stored credentials with `openshell gateway login [name]`, then retry. Use `gateway logout` only when intentionally clearing local credentials. - A direct development endpoint with a private or self-signed certificate can be isolated with `--gateway-endpoint --gateway-insecure`; do not persist or recommend insecure verification for shared gateways. @@ -375,7 +375,9 @@ Less commonly, `UnknownCA` can occur if the gateway's client-verification CA is misconfigured. The default `clientCaFromServerTlsSecret=true` is correct for all configurations — the internal server certificate is always signed by the chart CA (the same CA that signs the client cert), so its `ca.crt` is -the right trust anchor. Only override this if you intentionally mount a +the right trust anchor. Sandbox pods project only `ca.crt` from the copied +Secret; `tls.crt` and `tls.key` are reserved for user clients and must not be +visible in a sandbox. Only override this if you intentionally mount a separate client CA via `server.tls.clientCaSecretName`. Verify the mounted client CA matches the CA that signed the client certificate: diff --git a/.agents/skills/helm-dev-environment/SKILL.md b/.agents/skills/helm-dev-environment/SKILL.md index c577cd577e..900defb072 100644 --- a/.agents/skills/helm-dev-environment/SKILL.md +++ b/.agents/skills/helm-dev-environment/SKILL.md @@ -72,7 +72,7 @@ mise run helm:skaffold:run mise run helm:skaffold:run:sidecar ``` -**Supervisor sidecar topology with TLS/mTLS enabled** (build once and leave running): +**Supervisor sidecar topology with gateway TLS and CLI mTLS enabled** (build once and leave running): ```bash mise run helm:skaffold:run:sidecar-mtls ``` @@ -85,8 +85,9 @@ Binary-aware policy mode runs that sidecar as UID 0 with `SYS_PTRACE` and must be at least `1000` and distinct from the workload UID. The sidecar-mTLS profile reuses `ci/values-sidecar.yaml` and restores `server.disableTls=false` inline for Skaffold. The `pkiInitJob` hook (a pre-install -Job that runs `openshell-gateway generate-certs`) generates mTLS secrets on first -install. The default Skaffold values export gateway and Kubernetes-driver traces to +Job that runs `openshell-gateway generate-certs`) generates the gateway and CLI +TLS secrets on first install. Sandbox pods project only `ca.crt`; they use bearer +tokens, not the CLI client certificate, for gateway authentication. The default Skaffold values export gateway and Kubernetes-driver traces to the collector service installed by `helm:k3s:create`. Envoy Gateway opt-in; see the Optional Add-ons section below. diff --git a/architecture/gateway.md b/architecture/gateway.md index 0430d95159..35b9b0477b 100644 --- a/architecture/gateway.md +++ b/architecture/gateway.md @@ -41,12 +41,12 @@ finalized supervisor session disconnects. ## Protocol and Auth The gateway listens on one service port and multiplexes gRPC and HTTP traffic. -The default local single-user deployment mode is mTLS user authentication: -clients present a certificate signed by the local deployment CA, and the -gateway maps the verified certificate subject to a user principal. Kubernetes -deployments use mTLS for transport only and require OIDC or a trusted access -proxy for user authentication unless the explicit unsafe local-development -`allow_unauthenticated_users` switch is enabled. +When a client CA is configured without OIDC, mTLS user authentication defaults +on independently of the compute driver: clients present a certificate signed +by the deployment CA, and the gateway maps the verified certificate subject to +a user principal. Sandboxes do not receive that client certificate. They +authenticate the gateway with the CA and authenticate their own RPCs with +gateway-minted bearer tokens. When that service port is bound to loopback, the listener can also accept plaintext HTTP on the same port for sandbox service subdomains only. That local browser path is enabled by default and disabled with @@ -188,7 +188,7 @@ Supported auth modes: | Mode | Use | |---|---| -| mTLS user auth | Local single-user Docker, Podman, and VM gateway access. | +| mTLS user auth | Gateway user access with verified client certificates. | | Plaintext | Local development or a trusted reverse proxy boundary. | | Unauthenticated local users | Trusted Kubernetes dev or fully trusted proxy deployments only. | | Cloudflare JWT | Edge-authenticated deployments where Cloudflare Access supplies identity. | @@ -217,8 +217,8 @@ the capability query's admin authorization check. The CLI combines the health and capability results so a reachable gateway with an expired or rejected token is reported as connected but unauthenticated. -Sandbox supervisor RPCs authenticate with explicit sandbox credentials; mTLS -does not grant sandbox identity. Kubernetes deployments use the +Sandbox supervisor RPCs authenticate with explicit sandbox credentials; TLS +authenticates the gateway and does not grant sandbox identity. Kubernetes deployments use the gateway-minted JWT bootstrap path: the supervisor starts with a projected ServiceAccount token, exchanges it for a gateway-minted sandbox JWT, and uses that JWT on subsequent gateway RPCs. @@ -700,7 +700,7 @@ aliases, network names, and the sandbox JWT issuer. `[openshell.gateway]` carries a small set of values (`sandbox_namespace`, `default_image`, -`supervisor_image`, `guest_tls_ca/cert/key`, `client_tls_secret_name`, +`supervisor_image`, `guest_tls_ca`, `client_tls_secret_name`, `host_gateway_ip`, `enable_user_namespaces`) that are inherited into each driver's `[openshell.drivers.]` table when the driver-specific table does not override them. The allowlist is per-driver so a gateway-wide diff --git a/crates/openshell-bootstrap/src/pki.rs b/crates/openshell-bootstrap/src/pki.rs index ed6e839bf6..b1157ab14b 100644 --- a/crates/openshell-bootstrap/src/pki.rs +++ b/crates/openshell-bootstrap/src/pki.rs @@ -89,7 +89,7 @@ pub fn generate_pki(extra_sans: &[String]) -> Result { .into_diagnostic() .wrap_err("failed to sign server certificate")?; - // --- Client cert (shared by CLI and sandbox pods) --- + // --- User client cert (CLI only; sandboxes use bearer identity) --- let client_key = KeyPair::generate() .into_diagnostic() .wrap_err("failed to generate client key")?; diff --git a/crates/openshell-core/src/container_paths.rs b/crates/openshell-core/src/container_paths.rs index c63e4bcdd8..d1c16d38e9 100644 --- a/crates/openshell-core/src/container_paths.rs +++ b/crates/openshell-core/src/container_paths.rs @@ -44,8 +44,6 @@ pub const SUPERVISOR_CONTAINER_DIR: &str = "/opt/openshell/bin"; pub const SUPERVISOR_CONTAINER_BINARY: &str = "/opt/openshell/bin/openshell-sandbox"; pub const TLS_CLIENT_DIR: &str = "/etc/openshell/tls/client"; pub const TLS_CA_MOUNT_PATH: &str = "/etc/openshell/tls/client/ca.crt"; -pub const TLS_CERT_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.crt"; -pub const TLS_KEY_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.key"; pub const SANDBOX_TOKEN_MOUNT_PATH: &str = "/etc/openshell/auth/sandbox.jwt"; pub const UPSTREAM_PROXY_AUTH_MOUNT_PATH: &str = "/etc/openshell/auth/upstream-proxy"; pub const CONTAINER_POLICY_PATH: &str = "/etc/openshell/policy.yaml"; @@ -60,8 +58,6 @@ pub const SUPERVISOR_CA_CERT_PATH: &str = "/etc/openshell-tls/openshell-ca.pem"; pub const SUPERVISOR_CA_BUNDLE_PATH: &str = "/etc/openshell-tls/ca-bundle.pem"; pub const VM_GUEST_TLS_CA_PATH: &str = "/opt/openshell/tls/ca.crt"; -pub const VM_GUEST_TLS_CERT_PATH: &str = "/opt/openshell/tls/tls.crt"; -pub const VM_GUEST_TLS_KEY_PATH: &str = "/opt/openshell/tls/tls.key"; pub const VM_GUEST_SANDBOX_TOKEN_PATH: &str = "/opt/openshell/auth/sandbox.jwt"; pub const VM_GUEST_INIT_DROPIN_DIR: &str = "/opt/openshell/init.d"; pub const VM_GUEST_INIT_DROPIN_MANIFEST: &str = "/opt/openshell/init.d.manifest"; @@ -84,8 +80,6 @@ mod tests { SUPERVISOR_CONTAINER_BINARY, TLS_CLIENT_DIR, TLS_CA_MOUNT_PATH, - TLS_CERT_MOUNT_PATH, - TLS_KEY_MOUNT_PATH, SANDBOX_TOKEN_MOUNT_PATH, UPSTREAM_PROXY_AUTH_MOUNT_PATH, CONTAINER_POLICY_PATH, @@ -98,8 +92,6 @@ mod tests { SUPERVISOR_CA_CERT_PATH, SUPERVISOR_CA_BUNDLE_PATH, VM_GUEST_TLS_CA_PATH, - VM_GUEST_TLS_CERT_PATH, - VM_GUEST_TLS_KEY_PATH, VM_GUEST_SANDBOX_TOKEN_PATH, VM_GUEST_INIT_DROPIN_DIR, VM_GUEST_INIT_DROPIN_MANIFEST, diff --git a/crates/openshell-core/src/driver_utils.rs b/crates/openshell-core/src/driver_utils.rs index f3279d8105..5d0f0015cb 100644 --- a/crates/openshell-core/src/driver_utils.rs +++ b/crates/openshell-core/src/driver_utils.rs @@ -90,22 +90,15 @@ pub const SUPERVISOR_CONTAINER_BINARY: &str = "/opt/openshell/bin/openshell-sand // --------------------------------------------------------------------------- // In-container mount paths for guest TLS materials and the sandbox token. // -// All container-based drivers (Docker, Podman, Kubernetes) mount the gateway's -// mTLS client credentials at these fixed paths inside every sandbox container. -// The supervisor reads these paths on startup to establish its gRPC-over-mTLS -// connection back to the gateway. The paths must remain stable across driver -// versions since the supervisor binary is built and packaged separately. +// Container-based drivers mount the gateway CA at this fixed path inside every +// sandbox container. The supervisor reads it on startup to authenticate the +// gateway TLS endpoint. Sandbox identity is provided separately by a bearer +// token. // --------------------------------------------------------------------------- -/// Container-side mount path for the guest mTLS CA certificate. +/// Container-side mount path for the gateway CA certificate. pub const TLS_CA_MOUNT_PATH: &str = "/etc/openshell/tls/client/ca.crt"; -/// Container-side mount path for the guest mTLS client certificate. -pub const TLS_CERT_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.crt"; - -/// Container-side mount path for the guest mTLS client private key. -pub const TLS_KEY_MOUNT_PATH: &str = "/etc/openshell/tls/client/tls.key"; - /// Container-side mount path for the per-sandbox JWT token. pub const SANDBOX_TOKEN_MOUNT_PATH: &str = "/etc/openshell/auth/sandbox.jwt"; diff --git a/crates/openshell-core/src/grpc_client.rs b/crates/openshell-core/src/grpc_client.rs index 54f0db6902..1aff1ebeb9 100644 --- a/crates/openshell-core/src/grpc_client.rs +++ b/crates/openshell-core/src/grpc_client.rs @@ -36,7 +36,7 @@ use openshell_extension_core::{BearerTokenSlot, ExtensionCredentialStore}; use tonic::Status; use tonic::metadata::AsciiMetadataValue; use tonic::service::interceptor::InterceptedService; -use tonic::transport::{Certificate, Channel, ClientTlsConfig, Endpoint, Identity}; +use tonic::transport::{Certificate, Channel, ClientTlsConfig, Endpoint}; use tracing::{debug, info, warn}; /// Channel type after the [`AuthInterceptor`] is applied. Aliased so the @@ -125,10 +125,9 @@ impl tonic::service::Interceptor for AuthInterceptor { /// Build the plain (un-intercepted) gRPC channel. /// -/// When the endpoint uses `https://`, mTLS is configured using these env vars: +/// When the endpoint uses `https://`, server-authenticated TLS is configured +/// using this env var: /// - `OPENSHELL_TLS_CA` -- path to the CA certificate -/// - `OPENSHELL_TLS_CERT` -- path to the client certificate -/// - `OPENSHELL_TLS_KEY` -- path to the client private key /// /// When the endpoint uses `http://`, a plaintext connection is used (for /// deployments where TLS is disabled, e.g. behind a Cloudflare Tunnel). @@ -147,7 +146,7 @@ async fn build_plain_channel(endpoint: &str) -> Result { let tls_enabled = endpoint.starts_with("https://"); - // TODO: TLS certs are loaded once here and never re-read. The gateway + // TODO: The TLS CA is loaded once here and never re-read. The gateway // server side supports hot-reload (ArcSwap + notify in tls.rs). The // supervisor should do the same so that cert-manager rotations take // effect without restarting the sandbox. @@ -155,26 +154,12 @@ async fn build_plain_channel(endpoint: &str) -> Result { let ca_path = std::env::var(sandbox_env::TLS_CA) .into_diagnostic() .wrap_err("OPENSHELL_TLS_CA is required")?; - let cert_path = std::env::var(sandbox_env::TLS_CERT) - .into_diagnostic() - .wrap_err("OPENSHELL_TLS_CERT is required")?; - let key_path = std::env::var(sandbox_env::TLS_KEY) - .into_diagnostic() - .wrap_err("OPENSHELL_TLS_KEY is required")?; - let ca_pem = std::fs::read(&ca_path) .into_diagnostic() .wrap_err_with(|| format!("failed to read CA cert from {ca_path}"))?; - let cert_pem = std::fs::read(&cert_path) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read client cert from {cert_path}"))?; - let key_pem = std::fs::read(&key_path) - .into_diagnostic() - .wrap_err_with(|| format!("failed to read client key from {key_path}"))?; // Trust only the configured CA — this is the chart's internal CA - // that signs both the gateway's internal server certificate and - // this client's identity certificate. The gateway uses SNI-based + // that signs the gateway's internal server certificate. The gateway uses SNI-based // certificate selection to present this internal cert to supervisor // connections, so no public root trust is needed here. // @@ -183,9 +168,7 @@ async fn build_plain_channel(endpoint: &str) -> Result { // (Docker/Podman drivers), and broadening the trust store would let // an attacker who controls the image + DNS present a publicly valid // certificate and intercept the supervisor→gateway TLS connection. - let mut tls_config = ClientTlsConfig::new() - .ca_certificate(Certificate::from_pem(ca_pem)) - .identity(Identity::from_pem(cert_pem, key_pem)); + let mut tls_config = ClientTlsConfig::new().ca_certificate(Certificate::from_pem(ca_pem)); if let Ok(server_name) = std::env::var(sandbox_env::GATEWAY_TLS_SERVER_NAME) && !server_name.is_empty() { diff --git a/crates/openshell-driver-docker/README.md b/crates/openshell-driver-docker/README.md index bbd7e69b88..35643223ee 100644 --- a/crates/openshell-driver-docker/README.md +++ b/crates/openshell-driver-docker/README.md @@ -178,12 +178,13 @@ assigned address that containers should use for callbacks; package-managed macOS gateways should leave it unset. For HTTPS endpoints, the server certificate must include the endpoint host as a -subject alternative name. Docker sandboxes also need the client TLS bundle -mounted into the container and exposed with: +subject alternative name. Docker sandboxes receive only the gateway CA through: - `OPENSHELL_TLS_CA` -- `OPENSHELL_TLS_CERT` -- `OPENSHELL_TLS_KEY` + +The supervisor authenticates the gateway with this CA and authenticates its +RPCs with a sandbox-scoped bearer token. Client certificates and private keys +are not mounted into sandbox containers. HTTP endpoints reject TLS material because the supervisor would not use it. diff --git a/crates/openshell-driver-docker/src/lib.rs b/crates/openshell-driver-docker/src/lib.rs index 3941819c61..9dace1a275 100644 --- a/crates/openshell-driver-docker/src/lib.rs +++ b/crates/openshell-driver-docker/src/lib.rs @@ -80,8 +80,6 @@ const WATCH_POLL_MAX_BACKOFF: Duration = Duration::from_secs(30); const SUPERVISOR_MOUNT_PATH: &str = openshell_core::driver_utils::SUPERVISOR_CONTAINER_BINARY; const TLS_CA_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CA_MOUNT_PATH; -const TLS_CERT_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CERT_MOUNT_PATH; -const TLS_KEY_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_KEY_MOUNT_PATH; const SANDBOX_TOKEN_MOUNT_PATH: &str = openshell_core::driver_utils::SANDBOX_TOKEN_MOUNT_PATH; const SUPERVISOR_PATH: &str = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"; const HOST_OPENSHELL_INTERNAL: &str = "host.openshell.internal"; @@ -139,13 +137,15 @@ pub struct DockerComputeConfig { /// the full resolution order. pub supervisor_image: Option, - /// Host-side CA certificate for Docker sandbox mTLS. + /// Host-side CA certificate for sandbox-to-gateway TLS. pub guest_tls_ca: Option, - /// Host-side client certificate for Docker sandbox mTLS. + /// Deprecated. Sandboxes authenticate with bearer tokens and must not + /// receive a user client certificate. pub guest_tls_cert: Option, - /// Host-side private key for Docker sandbox mTLS. + /// Deprecated. Sandboxes authenticate with bearer tokens and must not + /// receive a user client private key. pub guest_tls_key: Option, /// Docker bridge network that sandbox containers join. @@ -193,8 +193,6 @@ impl Default for DockerComputeConfig { #[derive(Debug, Clone, PartialEq, Eq)] pub(crate) struct DockerGuestTlsPaths { pub(crate) ca: PathBuf, - pub(crate) cert: PathBuf, - pub(crate) key: PathBuf, } #[derive(Debug, Clone)] @@ -2667,12 +2665,6 @@ fn build_binds( )]; if let Some(tls) = &config.guest_tls { binds.push(format!("{}:{}:ro,z", tls.ca.display(), TLS_CA_MOUNT_PATH)); - binds.push(format!( - "{}:{}:ro,z", - tls.cert.display(), - TLS_CERT_MOUNT_PATH - )); - binds.push(format!("{}:{}:ro,z", tls.key.display(), TLS_KEY_MOUNT_PATH)); } if sandbox .spec @@ -2858,14 +2850,6 @@ fn build_environment_for_oci_user( openshell_core::sandbox_env::TLS_CA.to_string(), TLS_CA_MOUNT_PATH.to_string(), ); - environment.insert( - openshell_core::sandbox_env::TLS_CERT.to_string(), - TLS_CERT_MOUNT_PATH.to_string(), - ); - environment.insert( - openshell_core::sandbox_env::TLS_KEY.to_string(), - TLS_KEY_MOUNT_PATH.to_string(), - ); } environment.remove(openshell_core::sandbox_env::SANDBOX_TOKEN); @@ -4035,8 +4019,6 @@ fn canonicalize_existing_file(path: &Path, description: &str) -> CoreResult bool { docker_config.guest_tls_ca.is_some() - && docker_config.guest_tls_cert.is_some() - && docker_config.guest_tls_key.is_some() } pub(crate) fn docker_guest_tls_paths( @@ -4046,24 +4028,25 @@ pub(crate) fn docker_guest_tls_paths( || docker_config.guest_tls_cert.is_some() || docker_config.guest_tls_key.is_some(); + if docker_config.guest_tls_cert.is_some() || docker_config.guest_tls_key.is_some() { + return Err(Error::config( + "guest_tls_cert and guest_tls_key are no longer supported; sandboxes authenticate to the gateway with bearer tokens", + )); + } + if !docker_config.grpc_endpoint.starts_with("https://") { if tls_flags_provided { return Err(Error::config(format!( - "guest_tls_ca/guest_tls_cert/guest_tls_key were provided but grpc_endpoint is '{}'; TLS materials require an https:// endpoint", + "guest_tls_ca was provided but grpc_endpoint is '{}'; TLS materials require an https:// endpoint", docker_config.grpc_endpoint, ))); } return Ok(None); } - let provided = [ - docker_config.guest_tls_ca.as_ref(), - docker_config.guest_tls_cert.as_ref(), - docker_config.guest_tls_key.as_ref(), - ]; - if provided.iter().all(Option::is_none) { + if docker_config.guest_tls_ca.is_none() { return Err(Error::config( - "docker compute driver requires guest_tls_ca, guest_tls_cert, and guest_tls_key when grpc_endpoint uses https://", + "docker compute driver requires guest_tls_ca when grpc_endpoint uses https://", )); } @@ -4072,21 +4055,8 @@ pub(crate) fn docker_guest_tls_paths( "guest_tls_ca is required when Docker sandbox TLS materials are configured", )); }; - let Some(cert) = docker_config.guest_tls_cert.clone() else { - return Err(Error::config( - "guest_tls_cert is required when Docker sandbox TLS materials are configured", - )); - }; - let Some(key) = docker_config.guest_tls_key.clone() else { - return Err(Error::config( - "guest_tls_key is required when Docker sandbox TLS materials are configured", - )); - }; - Ok(Some(DockerGuestTlsPaths { ca: canonicalize_existing_file(&ca, "docker TLS CA certificate")?, - cert: canonicalize_existing_file(&cert, "docker TLS client certificate")?, - key: canonicalize_existing_file(&key, "docker TLS client private key")?, })) } diff --git a/crates/openshell-driver-docker/src/tests.rs b/crates/openshell-driver-docker/src/tests.rs index fd81c9b638..7d8d10f5f0 100644 --- a/crates/openshell-driver-docker/src/tests.rs +++ b/crates/openshell-driver-docker/src/tests.rs @@ -115,8 +115,6 @@ fn runtime_config() -> DockerDriverRuntimeConfig { supervisor_bin: PathBuf::from("/tmp/openshell-sandbox"), guest_tls: Some(DockerGuestTlsPaths { ca: PathBuf::from("/tmp/ca.crt"), - cert: PathBuf::from("/tmp/tls.crt"), - key: PathBuf::from("/tmp/tls.key"), }), daemon_version: "28.0.0".to_string(), supports_gpu: false, @@ -1070,11 +1068,17 @@ fn container_create_body_sets_driver_owned_pids_limit() { } #[test] -fn build_environment_sets_docker_tls_paths() { +fn build_environment_sets_only_docker_tls_ca() { let env = build_environment(&test_sandbox(), &runtime_config()); assert!(env.contains(&format!("OPENSHELL_TLS_CA={TLS_CA_MOUNT_PATH}"))); - assert!(env.contains(&format!("OPENSHELL_TLS_CERT={TLS_CERT_MOUNT_PATH}"))); - assert!(env.contains(&format!("OPENSHELL_TLS_KEY={TLS_KEY_MOUNT_PATH}"))); + assert!( + !env.iter() + .any(|entry| entry.starts_with("OPENSHELL_TLS_CERT=")) + ); + assert!( + !env.iter() + .any(|entry| entry.starts_with("OPENSHELL_TLS_KEY=")) + ); assert!(env.contains(&"TEMPLATE_ENV=template".to_string())); assert!(env.contains(&"SPEC_ENV=spec".to_string())); assert!(env.contains(&format!( @@ -1411,7 +1415,7 @@ fn build_environment_keeps_telemetry_toggle_driver_controlled() { } #[test] -fn build_binds_uses_docker_tls_directory() { +fn build_binds_mounts_only_docker_tls_ca() { let binds = build_binds(&test_sandbox(), &runtime_config()).unwrap(); let targets = binds .iter() @@ -1419,8 +1423,13 @@ fn build_binds_uses_docker_tls_directory() { .collect::>(); assert!(targets.contains(&SUPERVISOR_MOUNT_PATH.to_string())); assert!(targets.contains(&TLS_CA_MOUNT_PATH.to_string())); - assert!(targets.contains(&TLS_CERT_MOUNT_PATH.to_string())); - assert!(targets.contains(&TLS_KEY_MOUNT_PATH.to_string())); + assert_eq!( + targets + .iter() + .filter(|target| target.starts_with(TLS_MOUNT_DIR)) + .count(), + 1 + ); assert!( targets .iter() @@ -2739,18 +2748,19 @@ fn validate_linux_elf_binary_rejects_non_elf_files() { } #[test] -fn docker_guest_tls_paths_require_all_files_for_https() { +fn docker_guest_tls_paths_accept_ca_only_for_https() { let tempdir = TempDir::new().unwrap(); let ca = tempdir.path().join("ca.crt"); fs::write(&ca, b"ca").unwrap(); - let err = docker_guest_tls_paths(&DockerComputeConfig { + let paths = docker_guest_tls_paths(&DockerComputeConfig { grpc_endpoint: "https://localhost:8443".to_string(), - guest_tls_ca: Some(ca), + guest_tls_ca: Some(ca.clone()), ..Default::default() }) - .unwrap_err(); - assert!(err.to_string().contains("guest_tls_cert")); + .unwrap() + .expect("CA-only TLS paths"); + assert_eq!(paths.ca, ca.canonicalize().unwrap()); } #[test] diff --git a/crates/openshell-driver-kubernetes/src/driver.rs b/crates/openshell-driver-kubernetes/src/driver.rs index 484cd07708..d305cfa8f2 100644 --- a/crates/openshell-driver-kubernetes/src/driver.rs +++ b/crates/openshell-driver-kubernetes/src/driver.rs @@ -3050,16 +3050,6 @@ fn supervisor_sidecar_env( openshell_core::sandbox_env::TLS_CA, &format!("{SIDECAR_CLIENT_TLS_MOUNT_PATH}/ca.crt"), ); - upsert_env( - &mut env, - openshell_core::sandbox_env::TLS_CERT, - &format!("{SIDECAR_CLIENT_TLS_MOUNT_PATH}/tls.crt"), - ); - upsert_env( - &mut env, - openshell_core::sandbox_env::TLS_KEY, - &format!("{SIDECAR_CLIENT_TLS_MOUNT_PATH}/tls.key"), - ); } copy_log_level_env(&mut env, template_environment, spec_environment); upsert_env( @@ -3955,7 +3945,7 @@ fn sandbox_template_to_k8s_with_validated_config( } container.insert("securityContext".to_string(), security_context); - // Mount client TLS secret for mTLS to the server. Gateway identity uses + // Mount the gateway CA for server-authenticated TLS. Sandbox identity uses // the projected ServiceAccount bootstrap token. Provider token grants may // additionally mount the SPIFFE Workload API socket. let mut volume_mounts: Vec = Vec::new(); @@ -4000,7 +3990,7 @@ fn sandbox_template_to_k8s_with_validated_config( serde_json::Value::Array(vec![serde_json::Value::Object(container)]), ); - // Add TLS secret volume. Combined mode uses mode 0400 because the + // Add a CA-only view of the TLS secret. Combined mode uses mode 0400 because the // supervisor starts as root and drops privileges before running workload // children. Sidecar mode keeps the process supervisor non-root, so it uses // pod fsGroup + 0440 to preserve gateway session and SSH control behavior. @@ -4014,7 +4004,11 @@ fn sandbox_template_to_k8s_with_validated_config( "name": CLIENT_TLS_VOLUME_NAME, "secret": { "secretName": params.client_tls_secret_name, - "defaultMode": client_tls_default_mode + "defaultMode": client_tls_default_mode, + "items": [{ + "key": "ca.crt", + "path": "ca.crt" + }] } })); } @@ -4380,24 +4374,14 @@ fn apply_required_env( ssh_socket_path, ); } - // TLS cert paths for sandbox-to-server mTLS. Only set when TLS is enabled - // and the client TLS secret is mounted into the sandbox pod. + // Gateway CA path for sandbox-to-server TLS. Sandbox identity is carried + // by the projected ServiceAccount token and gateway-minted JWT. if tls_enabled { upsert_env( env, openshell_core::sandbox_env::TLS_CA, "/etc/openshell-tls/client/ca.crt", ); - upsert_env( - env, - openshell_core::sandbox_env::TLS_CERT, - "/etc/openshell-tls/client/tls.crt", - ); - upsert_env( - env, - openshell_core::sandbox_env::TLS_KEY, - "/etc/openshell-tls/client/tls.key", - ); } // Projected ServiceAccount token written by kubelet (see the volume // definition in `sandbox_template_to_k8s`). The supervisor reads this @@ -6894,22 +6878,13 @@ mod tests { }; let tls_ca = get_env("OPENSHELL_TLS_CA").expect("OPENSHELL_TLS_CA must be set"); - let tls_cert = get_env("OPENSHELL_TLS_CERT").expect("OPENSHELL_TLS_CERT must be set"); - let tls_key = get_env("OPENSHELL_TLS_KEY").expect("OPENSHELL_TLS_KEY must be set"); - - // All TLS paths must be within the mount path + // The CA is the only TLS material exposed to the sandbox. assert!( tls_ca.starts_with(TLS_MOUNT_PATH), "OPENSHELL_TLS_CA path '{tls_ca}' must start with mount path '{TLS_MOUNT_PATH}'" ); - assert!( - tls_cert.starts_with(TLS_MOUNT_PATH), - "OPENSHELL_TLS_CERT path '{tls_cert}' must start with mount path '{TLS_MOUNT_PATH}'" - ); - assert!( - tls_key.starts_with(TLS_MOUNT_PATH), - "OPENSHELL_TLS_KEY path '{tls_key}' must start with mount path '{TLS_MOUNT_PATH}'" - ); + assert!(get_env("OPENSHELL_TLS_CERT").is_none()); + assert!(get_env("OPENSHELL_TLS_KEY").is_none()); } #[test] @@ -7298,7 +7273,7 @@ mod tests { } #[test] - fn tls_secret_volume_uses_restrictive_default_mode() { + fn tls_secret_volume_projects_only_gateway_ca() { let template = SandboxTemplate::default(); let pod_template = { let params = SandboxPodParams { @@ -7324,7 +7299,11 @@ mod tests { assert_eq!( tls_vol["secret"]["defaultMode"], 256, // 0o400 - "TLS secret volume must use mode 0400 to prevent sandbox user from reading the private key" + "TLS CA volume should remain read-only" + ); + assert_eq!( + tls_vol["secret"]["items"], + serde_json::json!([{"key": "ca.crt", "path": "ca.crt"}]) ); } diff --git a/crates/openshell-driver-podman/README.md b/crates/openshell-driver-podman/README.md index ccd4413b3f..7e68a735a4 100644 --- a/crates/openshell-driver-podman/README.md +++ b/crates/openshell-driver-podman/README.md @@ -180,26 +180,22 @@ supervisor path as an argument to an image-provided shell. ## TLS -When all three Podman TLS paths are set, the driver treats sandbox callbacks as -mTLS callbacks: +When the Podman TLS CA path is set, the driver treats sandbox callbacks as +server-authenticated TLS callbacks: - `OPENSHELL_PODMAN_TLS_CA` -- `OPENSHELL_PODMAN_TLS_CERT` -- `OPENSHELL_PODMAN_TLS_KEY` -The driver validates that the TLS paths are provided as a complete set. Partial -configuration fails early instead of silently falling back to plaintext. +The legacy client certificate and key options are rejected. Sandbox identity is +carried by a gateway-minted bearer token. When enabled, the driver: 1. Switches the auto-detected endpoint scheme from `http://` to `https://`. -2. Bind-mounts the client cert files read-only into the container at +2. Bind-mounts the CA read-only into the container at `/etc/openshell/tls/client/`. -3. Sets `OPENSHELL_TLS_CA`, `OPENSHELL_TLS_CERT`, and `OPENSHELL_TLS_KEY` to - the container-side paths. +3. Sets `OPENSHELL_TLS_CA` to the container-side path. -The supervisor reads these env vars and uses them to establish an mTLS -connection back to the gateway. On SELinux systems, the bind mounts include +The supervisor reads this env var and uses it to authenticate the gateway. On SELinux systems, the bind mount includes Podman's shared relabel option so the container process can read the files. The RPM packaging auto-generates a self-signed PKI on first start via @@ -290,14 +286,14 @@ The standalone `openshell-driver-podman` binary sets the same struct field from ## Credential Injection -Sandboxes authenticate to the gateway via mTLS using client materials bind- -mounted into the container from a Podman secret. No shared per-request secret -is injected as an environment variable. +Sandboxes authenticate to the gateway with a sandbox-scoped bearer token +mounted from a Podman secret. TLS uses a read-only gateway CA and does not expose +a client certificate or private key. | Credential | Mechanism | Visible in `inspect`? | Visible in `/proc//environ`? | |---|---|---|---| -| mTLS client cert/key | Bind-mounted file paths (`OPENSHELL_TLS_*` env vars point at them) | Yes (paths only) | Yes (paths only) | -| Sandbox identity | Plaintext env var | Yes | Yes | +| Gateway CA | Read-only bind mount or Podman secret (`OPENSHELL_TLS_CA` points at it) | Yes (path only) | Yes (path only) | +| Sandbox identity | Root-only Podman secret | No | No | | gRPC endpoint | Plaintext env var, override-protected | Yes | Yes | | Supervisor relay socket path | Plaintext env var, override-protected | Yes | Yes | @@ -389,9 +385,7 @@ Podman resources after out-of-band container removal or label drift. | `OPENSHELL_STOP_TIMEOUT` | `--stop-timeout` | `45` | Container stop timeout in seconds. | | `OPENSHELL_SANDBOX_PIDS_LIMIT` | `--sandbox-pids-limit` | `2048` | Podman cgroup PID limit for sandbox containers. Set `0` to inherit Podman's runtime/default PID limit. | | `OPENSHELL_SUPERVISOR_IMAGE` | `--supervisor-image` | `ghcr.io/nvidia/openshell/supervisor:latest` through the gateway, required standalone | OCI image containing the supervisor binary. | -| `OPENSHELL_PODMAN_TLS_CA` | `--podman-tls-ca` | unset | Host path to the CA certificate mounted for sandbox mTLS. | -| `OPENSHELL_PODMAN_TLS_CERT` | `--podman-tls-cert` | unset | Host path to the client certificate mounted for sandbox mTLS. | -| `OPENSHELL_PODMAN_TLS_KEY` | `--podman-tls-key` | unset | Host path to the client private key mounted for sandbox mTLS. | +| `OPENSHELL_PODMAN_TLS_CA` | `--podman-tls-ca` | unset | Host path to the CA certificate mounted so sandboxes can authenticate the gateway. | | `OPENSHELL_SANDBOX_HTTPS_PROXY` | `--sandbox-https-proxy` | unset | Corporate forward proxy URL for the supervisor's upstream TLS dials, chained with HTTP CONNECT. Credential-free `http://host:port` and `https://host:port` URLs are supported (scheme and port required). For an `https://` proxy the supervisor TLS-wraps the proxy connection, verifying the proxy certificate against the built-in and system roots plus `--sandbox-proxy-ca-bundle`. Plain-HTTP requests always dial directly. | | `OPENSHELL_SANDBOX_NO_PROXY` | `--sandbox-no-proxy` | unset | Comma-separated `NO_PROXY` list (hostnames, domain suffixes, IPs, CIDRs, each with an optional `:port` qualifier) dialed directly instead of through the corporate proxy. IP/CIDR entries also match hostnames through their validated DNS resolution. | | `OPENSHELL_SANDBOX_PROXY_AUTH_FILE` | `--sandbox-proxy-auth-file` | unset | Path to a file containing the proxy credentials as `user:pass`. Staged as a root-only Podman secret so credentials never appear in config or container metadata. Requires the insecure-auth acknowledgement below. | diff --git a/crates/openshell-driver-podman/src/config.rs b/crates/openshell-driver-podman/src/config.rs index 42571f00f1..e5ec8fe5ec 100644 --- a/crates/openshell-driver-podman/src/config.rs +++ b/crates/openshell-driver-podman/src/config.rs @@ -107,16 +107,14 @@ pub struct PodmanComputeConfig { /// Mounted read-only into sandbox containers at /opt/openshell/bin /// using Podman's `type=image` mount. pub supervisor_image: String, - /// Host path to the CA certificate for sandbox mTLS. + /// Host path to the CA certificate for sandbox-to-gateway TLS. /// - /// When all three TLS paths (`guest_tls_ca`, `guest_tls_cert`, - /// `guest_tls_key`) are set, the driver bind-mounts them into sandbox - /// containers and switches the auto-detected endpoint from `http://` - /// to `https://`. + /// When set, the driver bind-mounts the CA into sandbox containers and + /// switches the auto-detected endpoint from `http://` to `https://`. pub guest_tls_ca: Option, - /// Host path to the client certificate for sandbox mTLS. + /// Deprecated. Sandboxes authenticate with bearer tokens. pub guest_tls_cert: Option, - /// Host path to the client private key for sandbox mTLS. + /// Deprecated. Sandboxes authenticate with bearer tokens. pub guest_tls_key: Option, /// Container cgroup PID limit for Podman-managed sandboxes. /// @@ -256,43 +254,33 @@ pub fn parse_id_map_entry( } impl PodmanComputeConfig { - /// Returns `true` when all three TLS paths are configured. + /// Returns `true` when the gateway CA is configured. #[must_use] pub fn tls_enabled(&self) -> bool { - self.guest_tls_ca.is_some() && self.guest_tls_cert.is_some() && self.guest_tls_key.is_some() + self.guest_tls_ca.is_some() } /// Validate TLS configuration consistency. /// - /// Returns `Ok(())` when either all three TLS paths are set (full mTLS) - /// or none are set (plaintext). Returns an error naming the missing - /// fields when only a subset is provided — this prevents silent - /// fallback to plaintext when an operator partially configures mTLS. + /// Client certificates are rejected because sandbox identity is carried + /// by bearer tokens rather than the gateway user's mTLS identity. pub fn validate_tls_config(&self) -> Result<(), crate::client::PodmanApiError> { - let has_ca = self.guest_tls_ca.is_some(); let has_cert = self.guest_tls_cert.is_some(); let has_key = self.guest_tls_key.is_some(); - // All set or none set — both are valid. - if (has_ca && has_cert && has_key) || (!has_ca && !has_cert && !has_key) { + if !has_cert && !has_key { return Ok(()); } - - let mut missing = Vec::new(); - if !has_ca { - missing.push("--podman-tls-ca / OPENSHELL_PODMAN_TLS_CA"); - } - if !has_cert { - missing.push("--podman-tls-cert / OPENSHELL_PODMAN_TLS_CERT"); - } - if !has_key { - missing.push("--podman-tls-key / OPENSHELL_PODMAN_TLS_KEY"); - } - Err(crate::client::PodmanApiError::InvalidInput(format!( - "Partial TLS configuration: all three TLS paths must be provided together. \ - Missing: {}", - missing.join(", ") + "Sandbox client certificates are no longer supported; remove {}", + [ + has_cert.then_some("--podman-tls-cert / OPENSHELL_PODMAN_TLS_CERT"), + has_key.then_some("--podman-tls-key / OPENSHELL_PODMAN_TLS_KEY"), + ] + .into_iter() + .flatten() + .collect::>() + .join(" and ") ))) } @@ -934,107 +922,39 @@ mod tests { } #[test] - fn validate_tls_config_all_set_is_ok() { + fn validate_tls_config_ca_only_is_ok() { let cfg = PodmanComputeConfig { guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), - guest_tls_cert: Some(PathBuf::from("/tls/tls.crt")), - guest_tls_key: Some(PathBuf::from("/tls/tls.key")), ..PodmanComputeConfig::default() }; assert!(cfg.validate_tls_config().is_ok()); + assert!(cfg.tls_enabled()); } #[test] - fn validate_tls_config_only_ca_is_error() { - let cfg = PodmanComputeConfig { - guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), - ..PodmanComputeConfig::default() - }; - let err = cfg - .validate_tls_config() - .expect_err("only CA should be rejected"); - let msg = err.to_string(); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - } - - #[test] - fn validate_tls_config_only_cert_is_error() { - let cfg = PodmanComputeConfig { - guest_tls_cert: Some(PathBuf::from("/tls/tls.crt")), - ..PodmanComputeConfig::default() - }; - let err = cfg - .validate_tls_config() - .expect_err("only cert should be rejected"); - let msg = err.to_string(); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - } - - #[test] - fn validate_tls_config_only_key_is_error() { - let cfg = PodmanComputeConfig { - guest_tls_key: Some(PathBuf::from("/tls/tls.key")), - ..PodmanComputeConfig::default() - }; - let err = cfg - .validate_tls_config() - .expect_err("only key should be rejected"); - let msg = err.to_string(); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); - } - - #[test] - fn validate_tls_config_ca_and_cert_missing_key_is_error() { + fn validate_tls_config_rejects_client_certificate() { let cfg = PodmanComputeConfig { - guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), guest_tls_cert: Some(PathBuf::from("/tls/tls.crt")), ..PodmanComputeConfig::default() }; let err = cfg .validate_tls_config() - .expect_err("missing key should be rejected"); - let msg = err.to_string(); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - } - - #[test] - fn validate_tls_config_ca_and_key_missing_cert_is_error() { - let cfg = PodmanComputeConfig { - guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), - guest_tls_key: Some(PathBuf::from("/tls/tls.key")), - ..PodmanComputeConfig::default() - }; - let err = cfg - .validate_tls_config() - .expect_err("missing cert should be rejected"); + .expect_err("sandbox client certificate should be rejected"); let msg = err.to_string(); assert!(msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); } #[test] - fn validate_tls_config_cert_and_key_missing_ca_is_error() { + fn validate_tls_config_rejects_client_private_key() { let cfg = PodmanComputeConfig { - guest_tls_cert: Some(PathBuf::from("/tls/tls.crt")), guest_tls_key: Some(PathBuf::from("/tls/tls.key")), ..PodmanComputeConfig::default() }; let err = cfg .validate_tls_config() - .expect_err("missing CA should be rejected"); + .expect_err("sandbox client private key should be rejected"); let msg = err.to_string(); - assert!(msg.contains("OPENSHELL_PODMAN_TLS_CA"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_CERT"), "{msg}"); - assert!(!msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); + assert!(msg.contains("OPENSHELL_PODMAN_TLS_KEY"), "{msg}"); } #[test] diff --git a/crates/openshell-driver-podman/src/container.rs b/crates/openshell-driver-podman/src/container.rs index a81ee13e1d..cc0a2f205b 100644 --- a/crates/openshell-driver-podman/src/container.rs +++ b/crates/openshell-driver-podman/src/container.rs @@ -54,13 +54,9 @@ const VOLUME_PREFIX: &str = "openshell-sandbox-"; const TOKEN_SECRET_PREFIX: &str = "openshell-token-"; const PROXY_AUTH_SECRET_PREFIX: &str = "openshell-proxy-auth-"; const TLS_CA_SECRET_PREFIX: &str = "openshell-tls-ca-"; -const TLS_CERT_SECRET_PREFIX: &str = "openshell-tls-cert-"; -const TLS_KEY_SECRET_PREFIX: &str = "openshell-tls-key-"; /// Container-side mount paths for client TLS materials and the sandbox token. const TLS_CA_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CA_MOUNT_PATH; -const TLS_CERT_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_CERT_MOUNT_PATH; -const TLS_KEY_MOUNT_PATH: &str = openshell_core::driver_utils::TLS_KEY_MOUNT_PATH; const SANDBOX_TOKEN_MOUNT_PATH: &str = openshell_core::driver_utils::SANDBOX_TOKEN_MOUNT_PATH; const UPSTREAM_PROXY_AUTH_MOUNT_PATH: &str = openshell_core::driver_utils::UPSTREAM_PROXY_AUTH_MOUNT_PATH; @@ -176,14 +172,10 @@ pub fn proxy_auth_secret_name(sandbox_id: &str) -> String { format!("{PROXY_AUTH_SECRET_PREFIX}{sandbox_id}") } -/// Build per-sandbox Podman secret names for TLS CA, cert, and key. +/// Build the per-sandbox Podman secret name for the gateway CA. #[must_use] -pub fn tls_secret_names(sandbox_id: &str) -> [String; 3] { - [ - format!("{TLS_CA_SECRET_PREFIX}{sandbox_id}"), - format!("{TLS_CERT_SECRET_PREFIX}{sandbox_id}"), - format!("{TLS_KEY_SECRET_PREFIX}{sandbox_id}"), - ] +pub fn tls_secret_names(sandbox_id: &str) -> [String; 1] { + [format!("{TLS_CA_SECRET_PREFIX}{sandbox_id}")] } /// Truncate a container ID to 12 characters (standard short form). @@ -546,22 +538,12 @@ fn build_env( openshell_core::sandbox_env::POLICY_DNS_TRANSPARENT_TCP_CAPABILITY.into(), ); - // 3. TLS client cert paths (when mTLS is enabled). These point to - // the container-side mount paths where the cert files are - // bind-mounted from the host. + // 3. Gateway CA path (when TLS is enabled). if config.tls_enabled() { env.insert( openshell_core::sandbox_env::TLS_CA.into(), TLS_CA_MOUNT_PATH.into(), ); - env.insert( - openshell_core::sandbox_env::TLS_CERT.into(), - TLS_CERT_MOUNT_PATH.into(), - ); - env.insert( - openshell_core::sandbox_env::TLS_KEY.into(), - TLS_KEY_MOUNT_PATH.into(), - ); } if let Some(socket_path) = provider_spiffe_workload_api_socket_env_value(config) { @@ -1015,7 +997,7 @@ pub fn build_container_spec_for_image( image_id: &str, oci_user: &str, supervisor_bin_path: Option<&Path>, - tls_secret_names: Option<&[String; 3]>, + tls_secret_names: Option<&[String; 1]>, ) -> Result { let name = container_name(&sandbox.workspace, &sandbox.name, &sandbox.id); let vol = volume_name(&sandbox.id); @@ -1216,7 +1198,7 @@ pub fn build_container_spec_for_image( mode: 0o400, }); } - if let Some([ca, cert, key]) = tls_secret_names { + if let Some([ca]) = tls_secret_names { secrets.push(SecretMount { source: ca.clone(), target: TLS_CA_MOUNT_PATH.into(), @@ -1224,20 +1206,6 @@ pub fn build_container_spec_for_image( gid: 0, mode: 0o400, }); - secrets.push(SecretMount { - source: cert.clone(), - target: TLS_CERT_MOUNT_PATH.into(), - uid: 0, - gid: 0, - mode: 0o400, - }); - secrets.push(SecretMount { - source: key.clone(), - target: TLS_KEY_MOUNT_PATH.into(), - uid: 0, - gid: 0, - mode: 0o400, - }); } secrets }, @@ -1269,18 +1237,14 @@ pub fn build_container_spec_for_image( destination: openshell_core::container_paths::NETNS_MOUNT_ROOT.into(), options: vec!["rw".into(), "nosuid".into(), "nodev".into()], }]; - // Deliver client TLS materials into the container when mTLS is + // Deliver the gateway CA into the container when TLS is // enabled. When userns remaps UIDs (auto, no-map), bind-mounted // host files are unreadable because the container root maps to a // different host UID. In that case TLS materials are delivered as // Podman secrets (handled in the `secrets` block above); otherwise // use bind mounts. if tls_secret_names.is_none() - && let (Some(ca), Some(cert), Some(key)) = ( - &config.guest_tls_ca, - &config.guest_tls_cert, - &config.guest_tls_key, - ) + && let Some(ca) = &config.guest_tls_ca { let mut ro = vec!["ro".into(), "rbind".into()]; if is_selinux_enabled() { @@ -1292,18 +1256,6 @@ pub fn build_container_spec_for_image( destination: TLS_CA_MOUNT_PATH.into(), options: ro.clone(), }); - m.push(Mount { - kind: "bind".into(), - source: cert.display().to_string(), - destination: TLS_CERT_MOUNT_PATH.into(), - options: ro.clone(), - }); - m.push(Mount { - kind: "bind".into(), - source: key.display().to_string(), - destination: TLS_KEY_MOUNT_PATH.into(), - options: ro, - }); } // Bind-mount the corporate proxy CA bundle read-only when // configured. A CA certificate is not secret, so unlike the proxy @@ -2896,12 +2848,10 @@ mod tests { } #[test] - fn container_spec_includes_tls_mounts_when_configured() { + fn container_spec_includes_only_tls_ca_when_configured() { let sandbox = test_sandbox("tls-id", "tls-name"); let mut config = test_config(); config.guest_tls_ca = Some(std::path::PathBuf::from("/host/ca.crt")); - config.guest_tls_cert = Some(std::path::PathBuf::from("/host/tls.crt")); - config.guest_tls_key = Some(std::path::PathBuf::from("/host/tls.key")); let spec = build_container_spec(&sandbox, &config); @@ -2911,16 +2861,10 @@ mod tests { env_map.get("OPENSHELL_TLS_CA").and_then(|v| v.as_str()), Some("/etc/openshell/tls/client/ca.crt"), ); - assert_eq!( - env_map.get("OPENSHELL_TLS_CERT").and_then(|v| v.as_str()), - Some("/etc/openshell/tls/client/tls.crt"), - ); - assert_eq!( - env_map.get("OPENSHELL_TLS_KEY").and_then(|v| v.as_str()), - Some("/etc/openshell/tls/client/tls.key"), - ); + assert!(env_map.get("OPENSHELL_TLS_CERT").is_none()); + assert!(env_map.get("OPENSHELL_TLS_KEY").is_none()); - // Verify bind mounts exist for all three cert files. + // Verify only the CA bind mount exists. let mounts = spec["mounts"] .as_array() .expect("mounts should be an array"); @@ -2933,14 +2877,7 @@ mod tests { bind_dests.contains(&"/etc/openshell/tls/client/ca.crt"), "should bind-mount CA cert" ); - assert!( - bind_dests.contains(&"/etc/openshell/tls/client/tls.crt"), - "should bind-mount client cert" - ); - assert!( - bind_dests.contains(&"/etc/openshell/tls/client/tls.key"), - "should bind-mount client key" - ); + assert_eq!(bind_dests.len(), 1); // Verify SELinux relabel option is present iff SELinux is enabled. let tls_binds: Vec<&Value> = mounts diff --git a/crates/openshell-driver-podman/src/driver.rs b/crates/openshell-driver-podman/src/driver.rs index 8f7c0d32f6..6df70016c1 100644 --- a/crates/openshell-driver-podman/src/driver.rs +++ b/crates/openshell-driver-podman/src/driver.rs @@ -225,13 +225,9 @@ async fn cleanup_sandbox_proxy_auth_secret(client: &PodmanClient, secret_name: & async fn create_tls_secrets( client: &PodmanClient, config: &PodmanComputeConfig, - names: &[String; 3], + names: &[String; 1], ) -> Result<(), ComputeDriverError> { - let paths = [ - config.guest_tls_ca.as_deref(), - config.guest_tls_cert.as_deref(), - config.guest_tls_key.as_deref(), - ]; + let paths = [config.guest_tls_ca.as_deref()]; let mut created = 0usize; for (name, path) in names.iter().zip(paths.iter()) { let Some(p) = path else { continue }; @@ -256,7 +252,7 @@ async fn create_tls_secrets( Ok(()) } -async fn cleanup_tls_secrets(client: &PodmanClient, names: &[String; 3]) { +async fn cleanup_tls_secrets(client: &PodmanClient, names: &[String]) { for name in names { if let Err(err) = client.remove_secret(name).await { warn!( @@ -2090,8 +2086,6 @@ mod tests { let mut cfg = PodmanComputeConfig { gateway_port: 8080, guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), - guest_tls_cert: Some(PathBuf::from("/tls/tls.crt")), - guest_tls_key: Some(PathBuf::from("/tls/tls.key")), ..PodmanComputeConfig::default() }; if cfg.grpc_endpoint.is_empty() { @@ -2102,26 +2096,14 @@ mod tests { } #[test] - fn partial_tls_config_returns_error() { + fn ca_only_tls_config_is_enabled() { let cfg = PodmanComputeConfig { gateway_port: 8080, guest_tls_ca: Some(PathBuf::from("/tls/ca.crt")), - // guest_tls_cert and guest_tls_key not set — incomplete TLS config. ..PodmanComputeConfig::default() }; - assert!(!cfg.tls_enabled()); - let err = cfg - .validate_tls_config() - .expect_err("partial TLS config should be rejected"); - let msg = err.to_string(); - assert!( - msg.contains("OPENSHELL_PODMAN_TLS_CERT"), - "error should name the missing cert: {msg}" - ); - assert!( - msg.contains("OPENSHELL_PODMAN_TLS_KEY"), - "error should name the missing key: {msg}" - ); + assert!(cfg.tls_enabled()); + cfg.validate_tls_config().expect("CA-only TLS is valid"); } #[test] diff --git a/crates/openshell-driver-vm/README.md b/crates/openshell-driver-vm/README.md index f455beaeb5..8471e62e7a 100644 --- a/crates/openshell-driver-vm/README.md +++ b/crates/openshell-driver-vm/README.md @@ -151,9 +151,7 @@ Select the VM driver with `--drivers vm`, `OPENSHELL_DRIVERS=vm`, or `compute_dr | `mem_mib` | `2048` | Memory per sandbox, in MiB. | | `overlay_disk_mib` | `4096` | Sparse writable overlay disk size per sandbox, in MiB. | | `krun_log_level` | `1` | libkrun verbosity (0-5). | -| `guest_tls_ca` | unset | CA cert for the guest's mTLS client bundle. Required when `grpc_endpoint` uses `https://`. | -| `guest_tls_cert` | unset | Guest client certificate. | -| `guest_tls_key` | unset | Guest client private key. | +| `guest_tls_ca` | unset | CA cert used by the guest to authenticate the gateway. Required when `grpc_endpoint` uses `https://`. Sandbox identity uses a bearer token. | See [`openshell-gateway --help`](../openshell-server/src/cli.rs) for the gateway process flag surface. diff --git a/crates/openshell-driver-vm/src/driver.rs b/crates/openshell-driver-vm/src/driver.rs index 2de65c3add..cab1a41c86 100644 --- a/crates/openshell-driver-vm/src/driver.rs +++ b/crates/openshell-driver-vm/src/driver.rs @@ -149,8 +149,6 @@ const OPENSHELL_HOST_GATEWAY_ALIAS: &str = "host.openshell.internal"; const GVPROXY_HOST_LOOPBACK_ALIAS: &str = OPENSHELL_HOST_GATEWAY_ALIAS; const GUEST_SSH_SOCKET_PATH: &str = openshell_core::container_paths::SSH_SOCKET_PATH; const GUEST_TLS_CA_PATH: &str = openshell_core::container_paths::VM_GUEST_TLS_CA_PATH; -const GUEST_TLS_CERT_PATH: &str = openshell_core::container_paths::VM_GUEST_TLS_CERT_PATH; -const GUEST_TLS_KEY_PATH: &str = openshell_core::container_paths::VM_GUEST_TLS_KEY_PATH; const GUEST_SANDBOX_TOKEN_PATH: &str = openshell_core::container_paths::VM_GUEST_SANDBOX_TOKEN_PATH; const GUEST_INIT_DROPIN_DIR: &str = openshell_core::container_paths::VM_GUEST_INIT_DROPIN_DIR; /// Guest path of the driver-authored manifest enumerating which @@ -186,8 +184,6 @@ static IMAGE_CACHE_BUILD_COUNTER: AtomicU64 = AtomicU64::new(0); #[derive(Debug, Clone)] struct VmDriverTlsPaths { ca: PathBuf, - cert: PathBuf, - key: PathBuf, } #[derive(Debug, Clone)] @@ -312,15 +308,16 @@ impl VmDriverConfig { } fn tls_paths(&self) -> Result, String> { - let provided = [ - self.guest_tls_ca.as_ref(), - self.guest_tls_cert.as_ref(), - self.guest_tls_key.as_ref(), - ]; - if provided.iter().all(Option::is_none) { + if self.guest_tls_cert.is_some() || self.guest_tls_key.is_some() { + return Err( + "sandbox client certificates are no longer supported; remove OPENSHELL_VM_TLS_CERT and OPENSHELL_VM_TLS_KEY" + .to_string(), + ); + } + if self.guest_tls_ca.is_none() { return if self.requires_tls_materials() { Err( - "https:// openshell endpoint requires OPENSHELL_VM_TLS_CA, OPENSHELL_VM_TLS_CERT, and OPENSHELL_VM_TLS_KEY so sandbox VMs can authenticate to the gateway" + "https:// openshell endpoint requires OPENSHELL_VM_TLS_CA so sandbox VMs can authenticate the gateway" .to_string(), ) } else { @@ -333,18 +330,7 @@ impl VmDriverConfig { "OPENSHELL_VM_TLS_CA is required when TLS materials are configured".to_string(), ); }; - let Some(cert) = self.guest_tls_cert.clone() else { - return Err( - "OPENSHELL_VM_TLS_CERT is required when TLS materials are configured".to_string(), - ); - }; - let Some(key) = self.guest_tls_key.clone() else { - return Err( - "OPENSHELL_VM_TLS_KEY is required when TLS materials are configured".to_string(), - ); - }; - - for path in [&ca, &cert, &key] { + for path in [&ca] { if !path.is_file() { return Err(format!( "TLS material '{}' does not exist or is not a file", @@ -353,7 +339,7 @@ impl VmDriverConfig { } } - Ok(Some(VmDriverTlsPaths { ca, cert, key })) + Ok(Some(VmDriverTlsPaths { ca })) } } @@ -4521,14 +4507,6 @@ fn build_guest_environment( openshell_core::sandbox_env::TLS_CA.to_string(), GUEST_TLS_CA_PATH.to_string(), ); - environment.insert( - openshell_core::sandbox_env::TLS_CERT.to_string(), - GUEST_TLS_CERT_PATH.to_string(), - ); - environment.insert( - openshell_core::sandbox_env::TLS_KEY.to_string(), - GUEST_TLS_KEY_PATH.to_string(), - ); } environment.insert( openshell_core::sandbox_env::TELEMETRY_ENABLED.to_string(), @@ -4888,21 +4866,13 @@ fn validate_restored_sandbox_state( #[derive(Debug, Clone)] struct GuestTlsMaterials { ca: Vec, - cert: Vec, - key: Vec, } async fn read_guest_tls_materials(paths: &VmDriverTlsPaths) -> Result { let ca = tokio::fs::read(&paths.ca) .await .map_err(|err| format!("read {}: {err}", paths.ca.display()))?; - let cert = tokio::fs::read(&paths.cert) - .await - .map_err(|err| format!("read {}: {err}", paths.cert.display()))?; - let key = tokio::fs::read(&paths.key) - .await - .map_err(|err| format!("read {}: {err}", paths.key.display()))?; - Ok(GuestTlsMaterials { ca, cert, key }) + Ok(GuestTlsMaterials { ca }) } async fn overlay_template_image_ready(path: &Path, size_bytes: u64) -> Result { @@ -5062,14 +5032,7 @@ fn inject_guest_tls_materials( &overlay_upper_path(GUEST_TLS_CA_PATH), &materials.ca, )?; - write_rootfs_image_file( - overlay_disk, - &overlay_upper_path(GUEST_TLS_CERT_PATH), - &materials.cert, - )?; - let key_path = overlay_upper_path(GUEST_TLS_KEY_PATH); - write_rootfs_image_file(overlay_disk, &key_path, &materials.key)?; - set_rootfs_image_file_mode(overlay_disk, &key_path, 0o600) + Ok(()) } fn inject_guest_sandbox_token(overlay_disk: &Path, token: &str) -> Result<(), String> { @@ -5339,27 +5302,8 @@ fn stage_guest_tls_materials( let ca_path = staging_dir .join("upper") .join(GUEST_TLS_CA_PATH.trim_start_matches('/')); - let cert_path = staging_dir - .join("upper") - .join(GUEST_TLS_CERT_PATH.trim_start_matches('/')); - let key_path = staging_dir - .join("upper") - .join(GUEST_TLS_KEY_PATH.trim_start_matches('/')); fs::write(&ca_path, &materials.ca) .map_err(|err| format!("write guest TLS CA {}: {err}", ca_path.display()))?; - fs::write(&cert_path, &materials.cert) - .map_err(|err| format!("write guest TLS cert {}: {err}", cert_path.display()))?; - fs::write(&key_path, &materials.key) - .map_err(|err| format!("write guest TLS key {}: {err}", key_path.display()))?; - - #[cfg(unix)] - { - use std::os::unix::fs::PermissionsExt as _; - - fs::set_permissions(&key_path, fs::Permissions::from_mode(0o600)) - .map_err(|err| format!("chmod guest TLS key {}: {err}", key_path.display()))?; - } - Ok(()) } @@ -6829,8 +6773,8 @@ mod tests { #[test] fn overlay_upper_path_targets_overlay_upperdir() { assert_eq!( - overlay_upper_path(GUEST_TLS_KEY_PATH), - "/upper/opt/openshell/tls/tls.key" + overlay_upper_path(GUEST_TLS_CA_PATH), + "/upper/opt/openshell/tls/ca.crt" ); } @@ -7490,12 +7434,10 @@ mod tests { } #[test] - fn build_guest_environment_includes_tls_paths_for_https_endpoint() { + fn build_guest_environment_includes_only_tls_ca_for_https_endpoint() { let config = VmDriverConfig { openshell_endpoint: "https://127.0.0.1:8443".to_string(), guest_tls_ca: Some(PathBuf::from("/host/ca.crt")), - guest_tls_cert: Some(PathBuf::from("/host/tls.crt")), - guest_tls_key: Some(PathBuf::from("/host/tls.key")), ..Default::default() }; let sandbox = Sandbox { @@ -7507,8 +7449,14 @@ mod tests { let env = build_guest_environment(&sandbox, &config, None); assert!(env.contains(&format!("OPENSHELL_TLS_CA={GUEST_TLS_CA_PATH}"))); - assert!(env.contains(&format!("OPENSHELL_TLS_CERT={GUEST_TLS_CERT_PATH}"))); - assert!(env.contains(&format!("OPENSHELL_TLS_KEY={GUEST_TLS_KEY_PATH}"))); + assert!( + !env.iter() + .any(|entry| entry.starts_with("OPENSHELL_TLS_CERT=")) + ); + assert!( + !env.iter() + .any(|entry| entry.starts_with("OPENSHELL_TLS_KEY=")) + ); } #[test] @@ -7894,8 +7842,6 @@ mod tests { let err = read_guest_tls_materials(&VmDriverTlsPaths { ca: source_dir.join("ca.crt"), - cert: source_dir.join("tls.crt"), - key: source_dir.join("tls.key"), }) .await .expect_err("missing TLS materials should fail before image injection"); @@ -7907,15 +7853,9 @@ mod tests { #[cfg(unix)] #[test] - fn stage_guest_tls_materials_places_files_in_overlay_upper_with_private_key_mode() { - use std::os::unix::fs::PermissionsExt as _; - + fn stage_guest_tls_materials_places_only_ca_in_overlay_upper() { let base = unique_temp_dir(); - let materials = GuestTlsMaterials { - ca: b"ca".to_vec(), - cert: b"cert".to_vec(), - key: b"key".to_vec(), - }; + let materials = GuestTlsMaterials { ca: b"ca".to_vec() }; stage_guest_tls_materials(&base, &materials).expect("stage TLS materials"); @@ -7927,23 +7867,6 @@ mod tests { .unwrap(), b"ca" ); - assert_eq!( - fs::read( - base.join("upper") - .join(GUEST_TLS_CERT_PATH.trim_start_matches('/')) - ) - .unwrap(), - b"cert" - ); - let key_path = base - .join("upper") - .join(GUEST_TLS_KEY_PATH.trim_start_matches('/')); - assert_eq!(fs::read(&key_path).unwrap(), b"key"); - assert_eq!( - fs::metadata(&key_path).unwrap().permissions().mode() & 0o777, - 0o600 - ); - let _ = std::fs::remove_dir_all(base); } diff --git a/crates/openshell-sandbox/src/main.rs b/crates/openshell-sandbox/src/main.rs index 7108378654..54fe5691a5 100644 --- a/crates/openshell-sandbox/src/main.rs +++ b/crates/openshell-sandbox/src/main.rs @@ -43,7 +43,7 @@ const CLIENT_TLS_DIR: &str = openshell_core::container_paths::CLIENT_TLS_DIR; #[cfg(target_os = "linux")] const SIDECAR_CLIENT_TLS_SUBDIR: &str = "client"; #[cfg(target_os = "linux")] -const CLIENT_TLS_FILES: [&str; 3] = ["ca.crt", "tls.crt", "tls.key"]; +const CLIENT_TLS_FILES: [&str; 1] = ["ca.crt"]; #[cfg(target_os = "linux")] const SIDECAR_STATE_DIR_MODE: u32 = 0o2775; #[cfg(target_os = "linux")] @@ -396,7 +396,7 @@ fn copy_sidecar_client_tls_if_present( let source = source_dir.join(file_name); if !source.exists() { return Err(miette::miette!( - "client TLS source file is missing: {}", + "gateway CA source file is missing: {}", source.display() )); } @@ -405,14 +405,14 @@ fn copy_sidecar_client_tls_if_present( std::fs::remove_file(&dest) .into_diagnostic() .wrap_err_with(|| { - format!("failed to remove stale client TLS file {}", dest.display()) + format!("failed to remove stale gateway CA file {}", dest.display()) })?; } std::fs::copy(&source, &dest) .into_diagnostic() .wrap_err_with(|| { format!( - "failed to copy client TLS file {} to {}", + "failed to copy gateway CA file {} to {}", source.display(), dest.display() ) @@ -422,13 +422,13 @@ fn copy_sidecar_client_tls_if_present( std::fs::set_permissions(&dest, perms) .into_diagnostic() .wrap_err_with(|| { - format!("failed to chmod copied client TLS file {}", dest.display()) + format!("failed to chmod copied gateway CA file {}", dest.display()) })?; chown(&dest, Some(Uid::from_raw(uid)), Some(Gid::from_raw(gid))) .into_diagnostic() .wrap_err_with(|| { format!( - "failed to chown copied client TLS file {} to {uid}:{gid}", + "failed to chown copied gateway CA file {} to {uid}:{gid}", dest.display() ) })?; @@ -824,6 +824,7 @@ mod tests { #[cfg(target_os = "linux")] #[test] fn sidecar_tls_modes_preserve_proxy_owned_parent_and_private_client_dir() { + assert_eq!(CLIENT_TLS_FILES, ["ca.crt"]); assert_eq!(SIDECAR_TLS_DIR_MODE, 0o755); assert_eq!(SIDECAR_TLS_STAGING_DIR_MODE, 0o700); assert_eq!(SIDECAR_CLIENT_TLS_DIR_MODE, 0o750); diff --git a/crates/openshell-server/src/auth/principal.rs b/crates/openshell-server/src/auth/principal.rs index 9567cc62d2..5a1fac04cb 100644 --- a/crates/openshell-server/src/auth/principal.rs +++ b/crates/openshell-server/src/auth/principal.rs @@ -67,9 +67,6 @@ pub enum SandboxIdentitySource { /// Gateway-minted JWT validated against the gateway's signing key. /// Produced by [`super::sandbox_jwt::SandboxJwtAuthenticator`]. BootstrapJwt { issuer: String }, - /// Per-sandbox client certificate. Reserved for channel-bound sandbox - /// identity. - BootstrapCert { fingerprint: String }, /// Driver-native credential used to bootstrap a gateway-minted JWT via /// `IssueSandboxToken`. The named compute driver authenticated only the /// sandbox identity; the gateway still authorizes the exchange. diff --git a/crates/openshell-server/src/auth/sandbox_jwt.rs b/crates/openshell-server/src/auth/sandbox_jwt.rs index 9dc10b8401..9828ed95ed 100644 --- a/crates/openshell-server/src/auth/sandbox_jwt.rs +++ b/crates/openshell-server/src/auth/sandbox_jwt.rs @@ -458,7 +458,9 @@ mod tests { SandboxIdentitySource::BootstrapJwt { issuer: iss } => { assert_eq!(iss, "openshell-gateway:test-gateway"); } - other => panic!("unexpected source: {other:?}"), + other @ SandboxIdentitySource::ComputeDriver { .. } => { + panic!("unexpected source: {other:?}") + } } } _ => panic!("expected Sandbox principal"), diff --git a/crates/openshell-server/src/cli.rs b/crates/openshell-server/src/cli.rs index f22e1355e4..645cebff4f 100644 --- a/crates/openshell-server/src/cli.rs +++ b/crates/openshell-server/src/cli.rs @@ -6,7 +6,6 @@ use clap::parser::ValueSource; use clap::{ArgAction, ArgMatches, Command, CommandFactory, FromArgMatches, Parser}; use miette::{IntoDiagnostic, Result}; -use openshell_core::ComputeDriverKind; use openshell_core::config::{DEFAULT_GATEWAY_NAME, DEFAULT_SERVER_PORT}; use std::net::{IpAddr, SocketAddr}; use std::path::PathBuf; @@ -145,11 +144,10 @@ struct RunArgs { #[arg(long, env = "OPENSHELL_OIDC_ISSUER")] oidc_issuer: Option, - /// Enable mTLS client certificate authentication for local single-user gateways. + /// Enable mTLS client certificate authentication for gateway users. /// - /// When unset, this defaults on for Docker, Podman, and VM gateways that - /// have client certificate verification configured and no OIDC issuer. - /// Kubernetes deployments must use OIDC or fronting-proxy auth instead. + /// When unset, this defaults on when client certificate verification is + /// configured and no OIDC issuer is present. #[arg( long = "enable-mtls-auth", env = "OPENSHELL_ENABLE_MTLS_AUTH", @@ -270,8 +268,6 @@ fn prepare_server_config( let compute_driver = compute_drivers .select(&args.drivers) .map_err(|error| miette::miette!("{error}"))?; - let compute_driver_kind = compute_driver.name().parse::().ok(); - let local_tls = apply_runtime_defaults(args)?; let guest_tls = local_tls.as_ref().map(GuestTlsPaths::from); let local_jwt = defaults::complete_local_jwt_config()?; @@ -279,9 +275,7 @@ fn prepare_server_config( let bind = SocketAddr::new(args.bind_address, args.port); let has_client_ca = args.tls_client_ca.is_some(); - let has_oidc = args.oidc_issuer.is_some(); - let mtls_auth_enabled = - resolve_mtls_auth_enabled(args, matches, file.as_ref(), compute_driver_kind); + let mtls_auth_enabled = resolve_mtls_auth_enabled(args, matches, file.as_ref()); if args.disable_tls && has_client_ca { return Err(miette::miette!( @@ -298,12 +292,6 @@ fn prepare_server_config( "mTLS user authentication requires --tls-client-ca so client certificates can be verified." )); } - if mtls_auth_enabled && matches!(compute_driver_kind, Some(ComputeDriverKind::Kubernetes)) { - return Err(miette::miette!( - "mTLS user authentication is not supported with the Kubernetes compute driver. Configure OIDC or a trusted fronting proxy for user authentication." - )); - } - let tls = if args.disable_tls { None } else { @@ -331,7 +319,11 @@ fn prepare_server_config( Some(openshell_core::TlsConfig { cert_path, key_path, - require_client_auth: has_client_ca && !has_oidc, + // Sandboxes authenticate at the application layer with bearer + // identity, so TLS must permit clients without certificates. + // When present, CLI certificates are still verified and may be + // promoted to users by the independently configured mTLS policy. + require_client_auth: false, client_ca_path: args.tls_client_ca.clone(), external_cert_path: ext_cert, external_key_path: ext_key, @@ -817,23 +809,10 @@ fn normalize_compute_driver_socket_args(args: &mut RunArgs, matches: &ArgMatches } } -fn is_singleplayer_driver(driver: Option) -> bool { - matches!( - driver, - Some( - ComputeDriverKind::Docker - | ComputeDriverKind::Podman - | ComputeDriverKind::Vm - | ComputeDriverKind::Mxc - ) - ) -} - fn resolve_mtls_auth_enabled( args: &RunArgs, matches: &ArgMatches, file: Option<&ConfigFile>, - compute_driver: Option, ) -> bool { let file_configured = file .and_then(|f| f.openshell.gateway.mtls_auth.as_ref()) @@ -846,7 +825,7 @@ fn resolve_mtls_auth_enabled( return false; } - is_singleplayer_driver(compute_driver) + true } #[cfg(test)] @@ -1339,7 +1318,7 @@ mod tests { } #[test] - fn mtls_auth_auto_defaults_for_local_tls_driver() { + fn mtls_auth_auto_defaults_when_client_ca_is_configured() { let _lock = ENV_LOCK .lock() .unwrap_or_else(std::sync::PoisonError::into_inner); @@ -1359,12 +1338,7 @@ mod tests { "/tmp/ca.crt", ]); - assert!(super::resolve_mtls_auth_enabled( - &args, - &matches, - None, - Some(openshell_core::ComputeDriverKind::Docker) - )); + assert!(super::resolve_mtls_auth_enabled(&args, &matches, None)); } #[test] @@ -1399,11 +1373,20 @@ mod tests { assert_eq!(prepared.compute_driver.name(), "docker"); assert!(prepared.config.compute_drivers.is_empty()); assert!(prepared.config.mtls_auth.enabled); + assert!( + !prepared + .config + .tls + .as_ref() + .expect("TLS config") + .require_client_auth, + "sandbox bearer clients must be allowed through the TLS handshake" + ); assert_eq!(REGISTRY_DETECTION_CALLS.load(Ordering::SeqCst), 1); } #[test] - fn mtls_auth_does_not_auto_default_for_kubernetes_driver() { + fn mtls_auth_default_is_driver_independent() { let _lock = ENV_LOCK .lock() .unwrap_or_else(std::sync::PoisonError::into_inner); @@ -1423,12 +1406,7 @@ mod tests { "/tmp/ca.crt", ]); - assert!(!super::resolve_mtls_auth_enabled( - &args, - &matches, - None, - Some(openshell_core::ComputeDriverKind::Kubernetes) - )); + assert!(super::resolve_mtls_auth_enabled(&args, &matches, None)); } #[test] @@ -1463,8 +1441,7 @@ enabled = false assert!(!super::resolve_mtls_auth_enabled( &args, &matches, - Some(&file), - Some(openshell_core::ComputeDriverKind::Docker) + Some(&file) )); } @@ -1701,26 +1678,6 @@ ssh_session_ttl_secs = 1234 assert_eq!(file.openshell.gateway.ssh_session_ttl_secs, Some(1234)); } - #[test] - fn singleplayer_driver_matches_only_one_local_driver() { - for driver in [ - openshell_core::ComputeDriverKind::Docker, - openshell_core::ComputeDriverKind::Podman, - openshell_core::ComputeDriverKind::Vm, - openshell_core::ComputeDriverKind::Mxc, - ] { - assert!( - super::is_singleplayer_driver(Some(driver)), - "{driver} should be singleplayer" - ); - } - - assert!(!super::is_singleplayer_driver(Some( - openshell_core::ComputeDriverKind::Kubernetes - ))); - assert!(!super::is_singleplayer_driver(None)); - } - #[test] fn compute_driver_socket_flag_uses_explicit_driver_name() { let _lock = ENV_LOCK @@ -1960,7 +1917,7 @@ namespace = "agents" "#, ); let merged = crate::config_file::driver_table( - super::ComputeDriverKind::Kubernetes.as_str(), + openshell_core::ComputeDriverKind::Kubernetes.as_str(), &file.openshell.gateway, file.openshell.drivers.get("kubernetes"), ); @@ -1984,7 +1941,7 @@ default_image = "k8s-specific:1.0" "#, ); let merged = crate::config_file::driver_table( - super::ComputeDriverKind::Kubernetes.as_str(), + openshell_core::ComputeDriverKind::Kubernetes.as_str(), &file.openshell.gateway, file.openshell.drivers.get("kubernetes"), ); diff --git a/crates/openshell-server/src/compute/driver_config.rs b/crates/openshell-server/src/compute/driver_config.rs index c25c63ded2..c91325819b 100644 --- a/crates/openshell-server/src/compute/driver_config.rs +++ b/crates/openshell-server/src/compute/driver_config.rs @@ -24,13 +24,11 @@ use std::path::PathBuf; #[derive(Debug, Clone, PartialEq, Eq)] pub struct GuestTlsPaths { ca: PathBuf, - cert: PathBuf, - key: PathBuf, } impl GuestTlsPaths { - pub(crate) fn as_paths(&self) -> (&std::path::Path, &std::path::Path, &std::path::Path) { - (&self.ca, &self.cert, &self.key) + pub(crate) fn as_path(&self) -> &std::path::Path { + &self.ca } } @@ -38,8 +36,6 @@ impl From<&LocalTlsPaths> for GuestTlsPaths { fn from(paths: &LocalTlsPaths) -> Self { Self { ca: paths.ca.clone(), - cert: paths.client_cert.clone(), - key: paths.client_key.clone(), } } } diff --git a/crates/openshell-server/src/compute/driver_config/builtin.rs b/crates/openshell-server/src/compute/driver_config/builtin.rs index dea867237d..8879dce912 100644 --- a/crates/openshell-server/src/compute/driver_config/builtin.rs +++ b/crates/openshell-server/src/compute/driver_config/builtin.rs @@ -62,21 +62,11 @@ fn apply_podman_runtime_defaults( ) { podman.gateway_port = context.gateway_port; apply_podman_env_overrides(podman); - apply_guest_tls_defaults_to_split_fields( - &mut podman.guest_tls_ca, - &mut podman.guest_tls_cert, - &mut podman.guest_tls_key, - context.guest_tls, - ); + apply_guest_tls_ca_default(&mut podman.guest_tls_ca, context.guest_tls); } fn apply_docker_runtime_defaults(cfg: &mut DockerComputeConfig, context: DriverStartupContext<'_>) { - apply_guest_tls_defaults_to_split_fields( - &mut cfg.guest_tls_ca, - &mut cfg.guest_tls_cert, - &mut cfg.guest_tls_key, - context.guest_tls, - ); + apply_guest_tls_ca_default(&mut cfg.guest_tls_ca, context.guest_tls); } fn apply_vm_runtime_defaults(cfg: &mut VmComputeConfig, context: DriverStartupContext<'_>) { @@ -94,28 +84,14 @@ fn apply_vm_runtime_defaults(cfg: &mut VmComputeConfig, context: DriverStartupCo cfg.grpc_endpoint = format!("{scheme}://127.0.0.1:{}", context.gateway_port); } - apply_guest_tls_defaults_to_split_fields( - &mut cfg.guest_tls_ca, - &mut cfg.guest_tls_cert, - &mut cfg.guest_tls_key, - context.guest_tls, - ); + apply_guest_tls_ca_default(&mut cfg.guest_tls_ca, context.guest_tls); } -fn apply_guest_tls_defaults_to_split_fields( - ca: &mut Option, - cert: &mut Option, - key: &mut Option, - defaults: Option<&GuestTlsPaths>, -) { +fn apply_guest_tls_ca_default(ca: &mut Option, defaults: Option<&GuestTlsPaths>) { if ca.is_none() - && cert.is_none() - && key.is_none() && let Some(paths) = defaults { *ca = Some(paths.ca.clone()); - *cert = Some(paths.cert.clone()); - *key = Some(paths.key.clone()); } } diff --git a/crates/openshell-server/src/compute/vm.rs b/crates/openshell-server/src/compute/vm.rs index 6a66fc8aa5..c0799eb5a8 100644 --- a/crates/openshell-server/src/compute/vm.rs +++ b/crates/openshell-server/src/compute/vm.rs @@ -175,8 +175,6 @@ impl Default for VmComputeConfig { #[derive(Debug, Clone, PartialEq, Eq)] pub struct VmGuestTlsPaths { pub ca: PathBuf, - pub cert: PathBuf, - pub key: PathBuf, } /// Resolve the `openshell-driver-vm` binary path. @@ -409,14 +407,14 @@ pub fn compute_driver_guest_tls_paths( return Ok(None); } - let provided = [ - vm_config.guest_tls_ca.as_ref(), - vm_config.guest_tls_cert.as_ref(), - vm_config.guest_tls_key.as_ref(), - ]; - if provided.iter().all(Option::is_none) { + if vm_config.guest_tls_cert.is_some() || vm_config.guest_tls_key.is_some() { return Err(Error::config( - "vm compute driver requires guest_tls_ca, guest_tls_cert, and guest_tls_key when grpc_endpoint uses https://", + "guest_tls_cert and guest_tls_key are no longer supported; sandboxes authenticate to the gateway with bearer tokens", + )); + } + if vm_config.guest_tls_ca.is_none() { + return Err(Error::config( + "vm compute driver requires guest_tls_ca when grpc_endpoint uses https://", )); } @@ -425,18 +423,7 @@ pub fn compute_driver_guest_tls_paths( "guest_tls_ca is required when VM guest TLS materials are configured", )); }; - let Some(cert) = vm_config.guest_tls_cert.clone() else { - return Err(Error::config( - "guest_tls_cert is required when VM guest TLS materials are configured", - )); - }; - let Some(key) = vm_config.guest_tls_key.clone() else { - return Err(Error::config( - "guest_tls_key is required when VM guest TLS materials are configured", - )); - }; - - for path in [&ca, &cert, &key] { + for path in [&ca] { if !path.is_file() { return Err(Error::config(format!( "vm guest TLS material '{}' does not exist or is not a file", @@ -445,7 +432,7 @@ pub fn compute_driver_guest_tls_paths( } } - Ok(Some(VmGuestTlsPaths { ca, cert, key })) + Ok(Some(VmGuestTlsPaths { ca })) } /// Launch the VM compute-driver subprocess, wait for its UDS to come up, @@ -501,8 +488,6 @@ pub async fn spawn( .arg(vm_config.overlay_disk_mib.to_string()); if let Some(tls) = guest_tls_paths { command.arg("--guest-tls-ca").arg(tls.ca); - command.arg("--guest-tls-cert").arg(tls.cert); - command.arg("--guest-tls-key").arg(tls.key); } let mut child = command.spawn().map_err(|e| { @@ -735,47 +720,26 @@ mod tests { }; let err = compute_driver_guest_tls_paths(&vm_config) - .expect_err("https vm endpoints should require an explicit guest client bundle"); - assert!( - err.to_string() - .contains("guest_tls_ca, guest_tls_cert, and guest_tls_key") - ); + .expect_err("https vm endpoints should require an explicit gateway CA"); + assert!(err.to_string().contains("guest_tls_ca")); } #[test] - fn vm_compute_driver_tls_uses_guest_bundle_not_gateway_server_identity() { + fn vm_compute_driver_tls_uses_only_gateway_ca() { let dir = tempdir().unwrap(); - let server_cert = dir.path().join("server.crt"); - let server_key = dir.path().join("server.key"); let guest_ca = dir.path().join("guest-ca.crt"); - let guest_cert = dir.path().join("guest.crt"); - let guest_key = dir.path().join("guest.key"); - for path in [ - &server_cert, - &server_key, - &guest_ca, - &guest_cert, - &guest_key, - ] { - std::fs::write(path, path.display().to_string()).unwrap(); - } + std::fs::write(&guest_ca, guest_ca.display().to_string()).unwrap(); let vm_config = VmComputeConfig { grpc_endpoint: "https://gateway.internal:8443".to_string(), guest_tls_ca: Some(guest_ca.clone()), - guest_tls_cert: Some(guest_cert.clone()), - guest_tls_key: Some(guest_key.clone()), ..Default::default() }; let guest_paths = compute_driver_guest_tls_paths(&vm_config) .unwrap() - .expect("https vm endpoints should pass an explicit guest client bundle"); + .expect("https vm endpoints should pass an explicit gateway CA"); assert_eq!(guest_paths.ca, guest_ca); - assert_eq!(guest_paths.cert, guest_cert); - assert_eq!(guest_paths.key, guest_key); - assert_ne!(guest_paths.cert, server_cert); - assert_ne!(guest_paths.key, server_key); } #[test] diff --git a/crates/openshell-server/src/config_file.rs b/crates/openshell-server/src/config_file.rs index 74b6aad01b..37e937fba0 100644 --- a/crates/openshell-server/src/config_file.rs +++ b/crates/openshell-server/src/config_file.rs @@ -156,10 +156,6 @@ pub struct GatewayFileSection { pub sa_token_ttl_secs: Option, #[serde(default)] pub guest_tls_ca: Option, - #[serde(default)] - pub guest_tls_cert: Option, - #[serde(default)] - pub guest_tls_key: Option, // ── TLS toggle ─────────────────────────────────────────────────────── /// When `true`, the gateway listens on plaintext HTTP and ignores any @@ -471,23 +467,14 @@ fn inheritable_keys(driver_name: &str) -> &'static [&'static str] { "supervisor_image", "host_gateway_ip", "guest_tls_ca", - "guest_tls_cert", - "guest_tls_key", ], Some(ComputeDriverKind::Podman) => &[ "default_image", "supervisor_image", "host_gateway_ip", "guest_tls_ca", - "guest_tls_cert", - "guest_tls_key", - ], - Some(ComputeDriverKind::Vm) => &[ - "default_image", - "guest_tls_ca", - "guest_tls_cert", - "guest_tls_key", ], + Some(ComputeDriverKind::Vm) => &["default_image", "guest_tls_ca"], // MXC reads its own settings from the driver config table and has no // gateway-inherited required fields. Some(ComputeDriverKind::Mxc) | None => &[], @@ -505,8 +492,6 @@ fn gateway_inherited_value(g: &GatewayFileSection, key: &str) -> Option g.enable_user_namespaces.map(toml::Value::Boolean), "sa_token_ttl_secs" => g.sa_token_ttl_secs.map(toml::Value::Integer), "guest_tls_ca" => g.guest_tls_ca.as_deref().map(path_value), - "guest_tls_cert" => g.guest_tls_cert.as_deref().map(path_value), - "guest_tls_key" => g.guest_tls_key.as_deref().map(path_value), _ => None, } } diff --git a/crates/openshell-server/src/lib.rs b/crates/openshell-server/src/lib.rs index c2306b6632..9b28ec3900 100644 --- a/crates/openshell-server/src/lib.rs +++ b/crates/openshell-server/src/lib.rs @@ -1328,12 +1328,12 @@ impl ComputeDriverBuildContext<'_> { self.driver_startup.gateway_tls_enabled } - /// Gateway client credentials that a local driver may mount into guests. + /// Gateway CA certificate that a local driver may mount into guests. #[must_use] - pub fn guest_tls_paths(&self) -> Option<(&Path, &Path, &Path)> { + pub fn guest_tls_ca(&self) -> Option<&Path> { self.driver_startup .guest_tls - .map(compute::driver_config::GuestTlsPaths::as_paths) + .map(compute::driver_config::GuestTlsPaths::as_path) } /// Deserialize the selected driver's merged TOML table. diff --git a/crates/openshell-server/src/multiplex.rs b/crates/openshell-server/src/multiplex.rs index 3ef774e4cb..1ee3291f90 100644 --- a/crates/openshell-server/src/multiplex.rs +++ b/crates/openshell-server/src/multiplex.rs @@ -744,7 +744,6 @@ fn gateway_principal_fields(principal: &Principal) -> BTreeMap { "source".to_string(), match &sandbox.source { SandboxIdentitySource::BootstrapJwt { .. } => "bootstrap_jwt", - SandboxIdentitySource::BootstrapCert { .. } => "bootstrap_cert", SandboxIdentitySource::ComputeDriver { .. } => "compute_driver", } .to_string(), diff --git a/deploy/rpm/CONFIGURATION.md b/deploy/rpm/CONFIGURATION.md index 4fc18e6215..fabc81354b 100644 --- a/deploy/rpm/CONFIGURATION.md +++ b/deploy/rpm/CONFIGURATION.md @@ -182,8 +182,6 @@ certificates that are mounted into sandbox containers: ```toml [openshell.gateway] guest_tls_ca = "/home/user/.local/state/openshell/tls/ca.crt" -guest_tls_cert = "/home/user/.local/state/openshell/tls/client/tls.crt" -guest_tls_key = "/home/user/.local/state/openshell/tls/client/tls.key" ``` Inside the container, the supervisor reads them from: @@ -218,7 +216,7 @@ overrides that persist across package upgrades. | `compute_drivers` | `["podman"]` (RPM default) | When unset, the gateway auto-detects Kubernetes, then Podman, then Docker. The RPM default pins to Podman. | | `default_image` | `ghcr.io/nvidia/openshell-community/sandboxes/base:latest` | Default sandbox image. | | `supervisor_image` | `ghcr.io/nvidia/openshell/supervisor:latest` | Supervisor image mounted into Podman sandboxes. | -| `guest_tls_ca`, `guest_tls_cert`, `guest_tls_key` | auto-generated paths | Client TLS material bind-mounted into sandbox containers. | +| `guest_tls_ca` | auto-generated path | Gateway CA bind-mounted into sandbox containers for server-authenticated TLS. Sandbox identity uses a bearer token. | | `[openshell.gateway.tls]` paths | auto-generated paths | Server TLS certificate, key, and client CA. | | `disable_tls` | unset | Set to `true` to disable TLS. | diff --git a/deploy/rpm/QUICKSTART.md b/deploy/rpm/QUICKSTART.md index 442458d09d..e2f1071016 100644 --- a/deploy/rpm/QUICKSTART.md +++ b/deploy/rpm/QUICKSTART.md @@ -67,9 +67,9 @@ On first start, the gateway automatically generates: > **Note:** The primary gateway listener uses the loopback default, > `127.0.0.1:17670`. The Podman driver requests a separate callback listener -> scoped to the interface its sandboxes can reach. Mutual TLS (mTLS) is -> enabled automatically on first start, requiring a valid client certificate -> for every connection. See CONFIGURATION.md for details. +> scoped to the interface its sandboxes can reach. mTLS user authentication is +> enabled automatically on first start, while sandbox callbacks use the gateway +> CA plus sandbox-scoped bearer tokens. See CONFIGURATION.md for details. Verify the service is running: diff --git a/docs/kubernetes/access-control.mdx b/docs/kubernetes/access-control.mdx index 549cc6679f..81aa3ce224 100644 --- a/docs/kubernetes/access-control.mdx +++ b/docs/kubernetes/access-control.mdx @@ -15,7 +15,7 @@ The OpenShell gateway supports two access-control models for human callers on Ku | OIDC (recommended) | Production deployments. Integrates with an existing identity provider, supports role-based access control, and gives each user their own identity without distributing certificates. | | Reverse-proxy auth termination | An access proxy (Cloudflare Access, ngrok, corporate SSO) authenticates callers in front of the gateway. The gateway trusts the proxy and skips its own client-cert check. | -The Helm chart always generates mTLS certificates at install time. The gateway uses them for transport-layer security regardless of which access-control model you choose. The client bundle in the `openshell-client-tls` secret is used internally by sandbox supervisors, not for granting access to individual users. +The Helm chart generates a gateway TLS certificate and a user client certificate at install time. Sandbox Pods project only `ca.crt` from the client Secret so they can authenticate the gateway; the client certificate and private key are not exposed to sandboxes. Supervisors authenticate their RPCs with gateway-minted sandbox JWTs. For how the CLI resolves gateways and stores credentials, refer to [Gateway Authentication](/reference/gateway-auth). diff --git a/docs/kubernetes/managing-certificates.mdx b/docs/kubernetes/managing-certificates.mdx index c4cb07f57e..10e1b9ff74 100644 --- a/docs/kubernetes/managing-certificates.mdx +++ b/docs/kubernetes/managing-certificates.mdx @@ -8,7 +8,7 @@ keywords: "Generative AI, Cybersecurity, Kubernetes, cert-manager, PKI, TLS, mTL position: 3 --- -The OpenShell gateway uses mTLS certificates for transport between the gateway and sandbox supervisors. These certificates are not Kubernetes user authentication; configure OIDC or a trusted access proxy for user access. The Helm chart supports two ways to provision and manage the certificate bundle: +The OpenShell gateway uses TLS for transport to sandbox supervisors. Sandbox Pods receive the gateway CA, not a client certificate or private key, and authenticate RPCs with sandbox JWTs. The generated client certificate can authenticate user clients when gateway mTLS user authentication is enabled; shared deployments can instead configure OIDC or a trusted access proxy. The Helm chart supports two ways to provision and manage the certificate bundle: | Mode | When to use | |---|---| diff --git a/docs/reference/gateway-auth.mdx b/docs/reference/gateway-auth.mdx index 5969c43e64..e0bab2982d 100644 --- a/docs/reference/gateway-auth.mdx +++ b/docs/reference/gateway-auth.mdx @@ -32,9 +32,9 @@ The CLI uses one of these authentication modes depending on the gateway's config ### mTLS -The default mode for local Docker, Podman, and VM gateways without OIDC. The CLI presents a client certificate during the TLS handshake, and the gateway can map the verified certificate subject to a local user principal when mTLS user authentication is enabled. +The default mode for gateways with a client CA and no OIDC issuer. The CLI presents a client certificate during the TLS handshake, and the gateway can map the verified certificate subject to a local user principal when mTLS user authentication is enabled. -mTLS user authentication is for local single-user gateways. Kubernetes deployments must use OIDC or a trusted access proxy for user authentication; the Helm chart does not render `mtls_auth`. +mTLS user authentication is configured by the gateway and is independent of the compute driver. Shared deployments can still prefer OIDC or a trusted access proxy for user identity and lifecycle management. Set these environment variables before starting the gateway: @@ -43,13 +43,13 @@ Set these environment variables before starting the gateway: | `OPENSHELL_TLS_CERT` | Path to the gateway server certificate. | | `OPENSHELL_TLS_KEY` | Path to the gateway server private key. | | `OPENSHELL_TLS_CLIENT_CA` | Path to the CA certificate that verifies CLI client certificates. | -| `OPENSHELL_ENABLE_MTLS_AUTH` | Set to `true` to authenticate CLI callers from verified client certificates. Defaults on for local Docker, Podman, and VM gateways with no OIDC issuer. | +| `OPENSHELL_ENABLE_MTLS_AUTH` | Set to `true` to authenticate CLI callers from verified client certificates. Defaults on when a client CA is configured and no OIDC issuer is present. | For local access, the server certificate must be valid for the endpoint the CLI uses. Include `localhost`, `127.0.0.1`, and `::1` in the certificate SANs when users connect to a local gateway through loopback. Package-managed local gateways generate this bundle automatically for the `openshell` gateway name. Homebrew registers `https://localhost:17670`; Debian and RPM use `https://127.0.0.1:17670`. When you register a package-managed local gateway with `openshell gateway add --local --name openshell`, the CLI refreshes its mTLS bundle from the package-managed TLS directory. -On Homebrew, the gateway service also mirrors the Docker sandbox client bundle into `$HOME/.local/state/openshell/homebrew/tls` before startup so Docker Desktop can bind-mount the files into sandbox containers. +On Homebrew, the gateway service also mirrors the CA into `$HOME/.local/state/openshell/homebrew/tls` before startup so Docker Desktop can bind-mount gateway trust into sandbox containers. The driver does not mount the user client certificate or private key. The CLI loads its mTLS bundle from `~/.config/openshell/gateways//mtls/`: @@ -191,7 +191,7 @@ to [Manage Workspaces and Access](/sandboxes/manage-workspaces). If `OPENSHELL_OIDC_SCOPES_CLAIM` is set, the gateway also enforces scopes. It accepts space-delimited scope strings such as `scope: "openid sandbox:read"` and JSON arrays such as `scp: ["sandbox:read"]`. Standard OIDC scopes such as `openid`, `profile`, `email`, and `offline_access` are ignored for authorization. `openshell:all` grants access to all scoped methods. -Supervisor-to-gateway RPCs do not use user OIDC tokens or mTLS user identity. Each sandbox supervisor presents a gateway-minted `Authorization: Bearer` token scoped to its sandbox ID. On Kubernetes, the Kubernetes compute driver validates the projected ServiceAccount token with TokenReview, verifies the live pod UID and controlling `Sandbox` ownerReference, and returns the authenticated sandbox ID to the gateway. The gateway verifies that sandbox still exists before minting its JWT. Log upload, policy status, credential environment lookup, inference bundle lookup, and sandbox config sync run with sandbox-restricted scope, while CLI users authenticate with OIDC, edge auth, local mTLS user authentication, or an explicitly enabled unauthenticated local developer mode. `GetInferenceBundle` returns route material that includes provider credentials, so it requires a sandbox principal; user callers manage inference configuration through the user-facing inference APIs instead. +Supervisor-to-gateway RPCs do not use user OIDC tokens or mTLS user identity. TLS authenticates the gateway to the supervisor using the configured CA; no client certificate or private key is mounted into the sandbox. Each sandbox supervisor presents a gateway-minted `Authorization: Bearer` token scoped to its sandbox ID. On Kubernetes, the Kubernetes compute driver validates the projected ServiceAccount token with TokenReview, verifies the live pod UID and controlling `Sandbox` ownerReference, and returns the authenticated sandbox ID to the gateway. The gateway verifies that sandbox still exists before minting its JWT. Log upload, policy status, credential environment lookup, inference bundle lookup, and sandbox config sync run with sandbox-restricted scope, while CLI users authenticate with OIDC, edge auth, local mTLS user authentication, or an explicitly enabled unauthenticated local developer mode. `GetInferenceBundle` returns route material that includes provider credentials, so it requires a sandbox principal; user callers manage inference configuration through the user-facing inference APIs instead. Re-authenticate an OIDC gateway with: diff --git a/docs/reference/gateway-config.mdx b/docs/reference/gateway-config.mdx index c04c0040d0..c7686939f3 100644 --- a/docs/reference/gateway-config.mdx +++ b/docs/reference/gateway-config.mdx @@ -114,8 +114,6 @@ host_gateway_ip = "10.0.0.1" enable_user_namespaces = false sa_token_ttl_secs = 3600 guest_tls_ca = "/etc/openshell/certs/ca.pem" -guest_tls_cert = "/etc/openshell/certs/client.pem" -guest_tls_key = "/etc/openshell/certs/client-key.pem" # Optional gRPC rate limit. Both values must be positive to enable the limit. # Set either value to 0, or omit both, to disable rate limiting. @@ -205,7 +203,7 @@ namespace = "openshell" allow_reference_namespace = false ``` -Local Docker, Podman, and VM gateways can also set `[openshell.gateway.mtls_auth] enabled = true` to authenticate CLI callers from verified client certificates. Kubernetes deployments must leave this unset and use OIDC or a trusted access proxy; the Helm chart does not render this table. +Set `[openshell.gateway.mtls_auth] enabled = true` to authenticate CLI callers from verified client certificates. This is a gateway policy and does not depend on the selected compute driver. Sandbox supervisors never inherit that user identity: they establish server-authenticated TLS with `guest_tls_ca`, then authenticate each gateway RPC with a sandbox bearer token. `[openshell.gateway.tls]` supports optional SNI-based dual-certificate mode for deployments that need separate internal and external server certificates. Set `external_cert_path` and `external_key_path` to point at the external (e.g. ACME/publicly-trusted) certificate and key. List the hostnames that should be served with the external certificate in `external_server_names`. Connections whose TLS SNI hostname matches one of those names receive the external certificate; all other connections (including those with no SNI) receive the primary internal certificate from `cert_path`/`key_path`. Both fields must be set together — providing only one is a configuration error. On Kubernetes with the Helm chart, the external certificate is managed automatically when `certManager.serverIssuerRef.name` is set; the chart populates these fields from the cert-manager-issued external server certificate. @@ -448,7 +446,7 @@ Each example is a complete TOML file for one compute driver. The examples repeat ### Kubernetes -The gateway runs as a Pod and creates sandbox Pods in another namespace. mTLS material for sandboxes is delivered through a Kubernetes Secret rather than host-side file paths. +The gateway runs as a Pod and creates sandbox Pods in another namespace. The gateway CA is projected from a Kubernetes Secret; client certificate and key entries in that Secret are not exposed to sandbox Pods. ```toml [openshell] @@ -591,7 +589,7 @@ the SPIRE OIDC discovery endpoint or its TLS CA. ### Docker -Sandboxes run as containers on a local bridge network. The supervisor binary is bind-mounted from the host (no in-cluster image pull required); guest mTLS material is supplied as host paths. +Sandboxes run as containers on a local bridge network. The supervisor binary and gateway CA are bind-mounted from the host. Sandbox identity comes from a gateway-minted bearer token, not a TLS client certificate. ```toml [openshell] @@ -616,8 +614,6 @@ supervisor_bin = "/usr/local/libexec/openshell/openshell-sandbox" # Defaults to the gateway version; override to pin a specific build. # supervisor_image = "ghcr.io/nvidia/openshell/supervisor:" guest_tls_ca = "/etc/openshell/certs/ca.pem" -guest_tls_cert = "/etc/openshell/certs/client.pem" -guest_tls_key = "/etc/openshell/certs/client-key.pem" network_name = "openshell-docker" host_gateway_ip = "172.17.0.1" ssh_socket_path = "/run/openshell/ssh.sock" @@ -631,7 +627,7 @@ sandbox_pids_limit = 2048 ### Podman -Sandboxes run as Podman containers on a user-mode bridge network. The supervisor image is mounted read-only via Podman's `type=image` mount; guest mTLS material is supplied as host paths. +Sandboxes run as Podman containers on a user-mode bridge network. The supervisor image is mounted read-only via Podman's `type=image` mount, and the gateway CA is supplied as a host path. Sandbox identity comes from a gateway-minted bearer token. ```toml [openshell] @@ -662,8 +658,6 @@ stop_timeout_secs = 45 # Defaults to the gateway version; override to pin a specific build. # supervisor_image = "ghcr.io/nvidia/openshell/supervisor:" guest_tls_ca = "/etc/openshell/certs/ca.pem" -guest_tls_cert = "/etc/openshell/certs/client.pem" -guest_tls_key = "/etc/openshell/certs/client-key.pem" # Unsafe operator override. Host bind mounts, including Podman local-driver # bind-backed volumes, expose gateway-host paths inside sandboxes and can # negate OpenShell isolation and filesystem controls. @@ -790,8 +784,6 @@ vcpus = 2 mem_mib = 2048 overlay_disk_mib = 4096 guest_tls_ca = "/var/lib/openshell/guest-tls/ca.pem" -guest_tls_cert = "/var/lib/openshell/guest-tls/client.pem" -guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem" # Resolved sandbox UID/GID for the rootfs /etc/passwd entry. # Defaults to 10001 when unset; matching GID is used if sandbox_gid is empty. # Any non-root Linux UID/GID is valid. diff --git a/docs/reference/sandbox-compute-drivers.mdx b/docs/reference/sandbox-compute-drivers.mdx index 82aa68a956..f5a647b372 100644 --- a/docs/reference/sandbox-compute-drivers.mdx +++ b/docs/reference/sandbox-compute-drivers.mdx @@ -381,7 +381,7 @@ For maintainer-level implementation details, refer to the [Kubernetes driver REA | `image_pull_secrets` | `server.sandboxImagePullSecrets` | Attach Kubernetes image-pull Secrets to sandbox pods. Managed mode copies these explicitly named Secrets from the configured source namespace into each workspace namespace. In shared and operator modes, the Secrets must already exist in the sandbox namespace. | | `[managed_ssh_ingress]` | `networkPolicy.enabled` | In managed mode, create an SSH ingress policy in every workspace namespace. Helm configures the gateway namespace and pod selector automatically. Operator mode leaves namespace policy management to the platform operator. | | `grpc_endpoint` | `server.grpcEndpoint` | Set the gateway callback endpoint reachable from sandbox pods. | -| `client_tls_secret_name` | `server.tls.clientTlsSecretName` | Mount sandbox client TLS materials from a Kubernetes secret. | +| `client_tls_secret_name` | `server.tls.clientTlsSecretName` | Project only `ca.crt` from the Kubernetes TLS Secret so sandboxes can authenticate the gateway. | | `supervisor_image` | `supervisor.image.repository` / `supervisor.image.tag` | Override the supervisor image that provides the `openshell-sandbox` binary. The default repository with an empty tag uses the version-pinned image built into the gateway. Changing the repository uses the effective gateway image tag, while setting a tag pins that version explicitly. | | `supervisor_image_pull_policy` | `supervisor.image.pullPolicy` | Set the Kubernetes image pull policy for the supervisor image. | | `supervisor_sideload_method` | `supervisor.sideloadMethod` | How the supervisor binary is delivered into sandbox pods. Leave empty to auto-detect from cluster version. Set to `image-volume` to mount the supervisor OCI image directly as a volume (requires Kubernetes 1.33+ with the ImageVolume feature gate; GA in 1.36), or `init-container` to copy it through an init container on older clusters. | diff --git a/docs/security/best-practices.mdx b/docs/security/best-practices.mdx index 1c541f8f8e..1b6b9c126a 100644 --- a/docs/security/best-practices.mdx +++ b/docs/security/best-practices.mdx @@ -256,14 +256,14 @@ The gateway secures communication between the CLI, sandbox workloads, and extern ### mTLS -Gateway transport uses TLS, with client certificate checks available where the deployment provides a client CA. Local single-user Docker, Podman, and VM gateways can use the verified client certificate as user authentication. Kubernetes deployments use the certificate bundle for transport and sandbox supervisor connectivity only; configure OIDC or a trusted access proxy for user authentication. +Gateway transport uses TLS, with client certificate checks available where the deployment provides a client CA. mTLS user authentication is gateway policy and does not depend on the compute driver. Sandbox supervisors receive only the gateway CA and authenticate API calls with gateway-minted sandbox JWTs. | Aspect | Detail | |---|---| -| Default | Local TLS bundles enable mTLS user authentication for single-user local gateways. Helm deployments generate mTLS certificates for transport, while sandbox supervisors authenticate API calls with gateway-minted sandbox JWTs. TLS-enabled loopback gateways also accept plaintext HTTP for sandbox service hostnames by default. | -| What you can change | Configure OIDC or a trusted access proxy for multi-user gateways, set `OPENSHELL_ENABLE_MTLS_AUTH=true` for local single-user gateways, enable `server.auth.allowUnauthenticatedUsers=true` only for trusted local Kubernetes development or a fully trusted proxy, disable TLS only for trusted reverse-proxy setups, or disable loopback service HTTP with `--enable-loopback-service-http=false`. | -| Risk if relaxed | Disabling TLS removes transport-level protection entirely. Allowing unauthenticated users removes the gateway user-auth boundary and must not be exposed to shared or public networks. Treating transport certificates as shared user identity in Kubernetes would collapse user and sandbox trust boundaries. Loopback service HTTP is local-only and rejects cross-origin browser requests, but any local process can still reach exposed service URLs directly. | -| Recommendation | Use local mTLS user authentication only for single-user Docker, Podman, and VM gateways. Use OIDC or a trusted access proxy for Kubernetes and shared deployments. | +| Default | A configured client CA without OIDC enables mTLS user authentication. Sandbox supervisors use CA-only TLS plus gateway-minted sandbox JWTs. TLS-enabled loopback gateways also accept plaintext HTTP for sandbox service hostnames by default. | +| What you can change | Configure mTLS user authentication, OIDC, or a trusted access proxy at the gateway; enable `server.auth.allowUnauthenticatedUsers=true` only for trusted local Kubernetes development or a fully trusted proxy; disable TLS only for trusted reverse-proxy setups; or disable loopback service HTTP with `--enable-loopback-service-http=false`. | +| Risk if relaxed | Disabling TLS removes transport-level protection entirely. Allowing unauthenticated users removes the gateway user-auth boundary and must not be exposed to shared or public networks. Mounting a user client certificate into a sandbox would collapse user and sandbox trust boundaries. Loopback service HTTP is local-only and rejects cross-origin browser requests, but any local process can still reach exposed service URLs directly. | +| Recommendation | Keep sandbox identity separate from user identity: expose only the gateway CA to sandboxes and require sandbox JWTs. Use managed OIDC or a trusted access proxy when certificate distribution is unsuitable for shared users. | ### SSH Tunnel Authentication diff --git a/e2e/rust/e2e-vm.sh b/e2e/rust/e2e-vm.sh index de7e3ad09d..fe0e017e0c 100755 --- a/e2e/rust/e2e-vm.sh +++ b/e2e/rust/e2e-vm.sh @@ -289,8 +289,6 @@ grpc_endpoint = "https://host.openshell.internal:${HOST_PORT}" driver_dir = "${DRIVER_DIR}" state_dir = "${RUN_STATE_DIR}" guest_tls_ca = "${PKI_DIR}/ca.crt" -guest_tls_cert = "${PKI_DIR}/client/tls.crt" -guest_tls_key = "${PKI_DIR}/client/tls.key" EOF fi diff --git a/e2e/with-docker-gateway.sh b/e2e/with-docker-gateway.sh index 0a767576dd..66a02c141d 100755 --- a/e2e/with-docker-gateway.sh +++ b/e2e/with-docker-gateway.sh @@ -520,8 +520,6 @@ GATEWAY_CONFIG="${STATE_DIR}/gateway.toml" printf 'default_image = %s\n' "$(toml_string "${SANDBOX_IMAGE}")" printf 'image_pull_policy = %s\n' "$(toml_string "${SANDBOX_IMAGE_PULL_POLICY}")" printf 'guest_tls_ca = %s\n' "$(toml_string "${PKI_DIR}/ca.crt")" - printf 'guest_tls_cert = %s\n' "$(toml_string "${PKI_DIR}/client/tls.crt")" - printf 'guest_tls_key = %s\n' "$(toml_string "${PKI_DIR}/client/tls.key")" printf 'enable_bind_mounts = true\n' printf 'supervisor_image = %s\n' "$(toml_string "${SUPERVISOR_IMAGE}")" if [ -n "${GATEWAY_HOST_ALIAS_IP}" ]; then @@ -538,8 +536,6 @@ if [ "${OPENSHELL_E2E_EXTERNAL_COMPUTE_DRIVER:-0}" = "1" ]; then printf 'default_image = %s\n' "$(toml_string "${SANDBOX_IMAGE}")" printf 'image_pull_policy = %s\n' "$(toml_string "${SANDBOX_IMAGE_PULL_POLICY}")" printf 'guest_tls_ca = %s\n' "$(toml_string "${PKI_DIR}/ca.crt")" - printf 'guest_tls_cert = %s\n' "$(toml_string "${PKI_DIR}/client/tls.crt")" - printf 'guest_tls_key = %s\n' "$(toml_string "${PKI_DIR}/client/tls.key")" printf 'enable_bind_mounts = true\n' printf 'supervisor_image = %s\n' "$(toml_string "${SUPERVISOR_IMAGE}")" if [ -n "${GATEWAY_HOST_ALIAS_IP}" ]; then diff --git a/e2e/with-podman-gateway.sh b/e2e/with-podman-gateway.sh index fc3419e182..b0bc050a5f 100755 --- a/e2e/with-podman-gateway.sh +++ b/e2e/with-podman-gateway.sh @@ -481,8 +481,6 @@ cp "${ROOT}/deploy/rpm/gateway.toml.default" "${GATEWAY_CONFIG}" printf 'stop_timeout_secs = %s\n' "${PODMAN_STOP_TIMEOUT_SECS}" printf 'supervisor_image = %s\n' "$(toml_string "${SUPERVISOR_IMAGE}")" printf 'guest_tls_ca = %s\n' "$(toml_string "${PKI_DIR}/ca.crt")" - printf 'guest_tls_cert = %s\n' "$(toml_string "${PKI_DIR}/client/tls.crt")" - printf 'guest_tls_key = %s\n' "$(toml_string "${PKI_DIR}/client/tls.key")" printf 'enable_bind_mounts = true\n' if [ -n "${OPENSHELL_E2E_PROVIDER_SPIFFE_SOCKET:-}" ]; then printf 'provider_spiffe_workload_api_socket = %s\n' "$(toml_string "${OPENSHELL_E2E_PROVIDER_SPIFFE_SOCKET}")" diff --git a/rfc/0003-gateway-configuration/README.md b/rfc/0003-gateway-configuration/README.md index 2b7c095065..350283e72b 100644 --- a/rfc/0003-gateway-configuration/README.md +++ b/rfc/0003-gateway-configuration/README.md @@ -136,8 +136,6 @@ network_name = "openshell" supervisor_bin = "/usr/local/libexec/openshell/openshell-sandbox" # optional override supervisor_image = "ghcr.io/nvidia/openshell/supervisor:latest" # used to extract bin guest_tls_ca = "/etc/openshell/certs/ca.pem" -guest_tls_cert = "/etc/openshell/certs/client.pem" -guest_tls_key = "/etc/openshell/certs/client-key.pem" [openshell.drivers.podman] socket_path = "/run/podman/podman.sock" @@ -147,8 +145,6 @@ supervisor_image = "ghcr.io/nvidia/openshell/supervisor:latest" network_name = "openshell" stop_timeout_secs = 10 guest_tls_ca = "/etc/openshell/certs/ca.pem" -guest_tls_cert = "/etc/openshell/certs/client.pem" -guest_tls_key = "/etc/openshell/certs/client-key.pem" [openshell.drivers.vm] state_dir = "/var/lib/openshell/vm" @@ -158,8 +154,6 @@ vcpus = 2 mem_mib = 2048 krun_log_level = 1 guest_tls_ca = "/var/lib/openshell/guest-tls/ca.pem" -guest_tls_cert = "/var/lib/openshell/guest-tls/client.pem" -guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem" ``` ### Driver configuration @@ -167,7 +161,7 @@ guest_tls_key = "/var/lib/openshell/guest-tls/client-key.pem" Each `[openshell.drivers.]` table is extracted from the parsed file and handed to the driver's initialization function as a raw TOML value. The driver is then responsible for: 1. **Parsing** — deserializing the table into its own typed config struct (e.g. `KubernetesComputeConfig`, `DockerComputeConfig`, `PodmanComputeConfig`, `VmComputeConfig`). -2. **Validation** — applying cross-field checks specific to that driver (e.g. requiring TLS triplets when sandbox-side mTLS is enabled). +2. **Validation** — applying cross-field checks specific to that driver (e.g. requiring a CA certificate for server-authenticated sandbox TLS). 3. **Consumption** — using the resulting struct to initialize internal state. Driver authors define and own their config schema. Adding a new driver does not require changes to the gateway's core `Config` struct or to this RFC.