Skip to content

feat(helm): add kubernetes local-dev environment#1158

Merged
TaylorMutch merged 6 commits intomainfrom
kube-support/local-dev/tmutch
May 5, 2026
Merged

feat(helm): add kubernetes local-dev environment#1158
TaylorMutch merged 6 commits intomainfrom
kube-support/local-dev/tmutch

Conversation

@TaylorMutch
Copy link
Copy Markdown
Collaborator

@TaylorMutch TaylorMutch commented May 4, 2026

Summary

Adds a local Kubernetes development environment for OpenShell using k3d (Docker-backed k3s), Skaffold, and the existing Helm chart. Includes optional Envoy Gateway (GRPCRoute / Gateway API), Keycloak OIDC, and cert-manager PKI add-ons.

Related Issue

Changes

  • Add `helm:k3s:*` mise tasks (`create`, `delete`, `start`, `stop`, `status`) backed by `tasks/scripts/helm-k3s-local.sh` — cluster name derived from git branch for per-worktree isolation
  • Add `helm:skaffold:*` mise tasks (`dev`, `run`, `delete`, `diagnose`) for iterative and one-shot deploys
  • Add `helm:gateway:apply` task to activate Envoy Gateway routing after deploy
  • Add `keycloak:k8s:*` mise tasks (`setup`, `teardown`) backed by `tasks/scripts/keycloak-k8s-setup.sh`
  • Add Skaffold config (`deploy/helm/openshell/skaffold.yaml`) with Envoy Gateway, Keycloak, and cert-manager as opt-in valuesFiles
  • Add `pkiInitJob` Helm hook — generates mTLS CA + server/client certs via an alpine/openssl Job on pre-install/pre-upgrade (default PKI path)
  • Add cert-manager PKI support (`deploy/helm/openshell/templates/cert-manager-pki.yaml`, `values-cert-manager.yaml`) as an alternative to `pkiInitJob` — creates namespaced Issuer + CA + server/client Certificates; mutual exclusion with `pkiInitJob` enforced at template render time
  • Add `sshHandshakeSecret` Helm hook for SSH handshake init
  • Add `gateway.yaml` and `grpcroute.yaml` Helm templates for Envoy Gateway integration
  • Add `values-gateway.yaml`, `values-keycloak.yaml`, `values-skaffold.yaml`, `values-cert-manager.yaml` overlay files
  • Add `deploy/kube/manifests/envoy-gateway-openshell.yaml` GatewayClass manifest
  • Add `helm-dev-environment` agent skill documenting the full cluster lifecycle, TLS toggle, and mTLS port-forward setup
  • Remove stale `scripts/bin/k9s` and `scripts/bin/kubectl` wrapper scripts (replaced by mise-managed tools)
  • Add `k3d` and update `k9s` in `mise.toml`

Testing

  • `mise run pre-commit` passes
  • Cluster create → deploy → `sandbox list` verified with `pkiInitJob` (plaintext and mTLS)
  • Cluster create → deploy → `sandbox list` + `sandbox create` verified with cert-manager PKI over mTLS port-forward
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@TaylorMutch TaylorMutch requested a review from a team as a code owner May 4, 2026 22:18
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 4, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 4, 2026

@TaylorMutch
Copy link
Copy Markdown
Collaborator Author

/ok to test 5fac9ca

Comment thread .agents/skills/helm-dev-environment/SKILL.md
Comment thread deploy/helm/openshell/templates/pki-hook.yaml Outdated
Comment thread .agents/skills/helm-dev-environment/SKILL.md Outdated
@TaylorMutch
Copy link
Copy Markdown
Collaborator Author

/ok to test 5900603

@TaylorMutch TaylorMutch requested a review from johntmyers May 5, 2026 16:21
@sjenning
Copy link
Copy Markdown
Contributor

sjenning commented May 5, 2026

lgtm 👍

Copy link
Copy Markdown
Collaborator

@derekwaynecarr derekwaynecarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

* Add support for grpcRoute from Kubernetes Gateway API spec
* Add pkiInitJob to initialize mTLS resources
* Add sshHandshake init job
* Test integration with Envoy Gateway
* Add keycloak integration testing with Skaffold
… setup

Add a TLS behaviour section explaining that values-skaffold.yaml disables
TLS by default, and a port-forward connection guide covering both plaintext
and mTLS modes with the exact commands to extract client certs from the
cluster PKI secret.
Re-add the openshell.issuerSelfSigned helper, the mutual-exclusion guard
in pki-hook.yaml, and the certManager condition in the statefulset volume
mount. Add server.disableTls: false to values-cert-manager.yaml so the
overlay correctly overrides the skaffold dev default. Tested end-to-end
with cert-manager issuing mTLS certs and sandbox create over port-forward.
Use port 8090 for direct port-forward to avoid colliding with the k3d
LB binding on 8080 when Envoy Gateway is active.

Check both server and client TLS secrets before skipping PKI generation.
Previously only the server secret was checked, which would silently skip
generation if a partial cleanup left one half of the pair behind. Now
emits a clear error with a recovery command when partial state is detected.
@TaylorMutch TaylorMutch force-pushed the kube-support/local-dev/tmutch branch from 5900603 to 9d78426 Compare May 5, 2026 17:56
@TaylorMutch
Copy link
Copy Markdown
Collaborator Author

/ok to test 9d78426

@TaylorMutch TaylorMutch merged commit 5116cc2 into main May 5, 2026
25 checks passed
@TaylorMutch TaylorMutch deleted the kube-support/local-dev/tmutch branch May 5, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants