Skip to content

oidc-kubectl.md

Latest commit

 

History

History
244 lines (197 loc) · 10 KB

oidc-kubectl.md

File metadata and controls

244 lines (197 loc) · 10 KB
Raw

kubectl OIDC Login via Dex

This guide explains how to use kubelogin to authenticate kubectl against the Kubernetes API via Dex and GitHub.

OIDC is the default, day-to-day way to reach the cluster and is intentionally read-only (see RBAC). Write/admin access is break-glass only — the root client-certificate kubeconfig kept in the vault (see Break-glass admin access).

Prerequisites

  • A kubeconfig containing the cluster connection details but no administrative user credential
  • A GitHub account that is a member of the devantler-tech organization
  • Your OIDC identity must be bound to the read-only roles (see RBAC)

1 — Install kubelogin

# Homebrew (macOS / Linux)
brew install int128/kubelogin/kubelogin

# Or with krew
kubectl krew install oidc-login

Verify: kubectl oidc-login --help

2 — Add an OIDC user to kubeconfig

Local cluster (platform.lan)

kubectl config set-credentials oidc-local \
  --exec-api-version=client.authentication.k8s.io/v1 \
  --exec-interactive-mode=IfAvailable \
  --exec-command=kubectl \
  --exec-arg=oidc-login \
  --exec-arg=get-token \
  --exec-arg=--oidc-issuer-url=https://dex.platform.lan \
  --exec-arg=--oidc-client-id=kubectl \
  --exec-arg=--oidc-extra-scope=email \
  --exec-arg=--oidc-extra-scope=profile \
  --exec-arg=--oidc-extra-scope=groups \
  --exec-arg=--oidc-extra-scope=audience:server:client_id:public-client \
  --exec-arg=--token-cache-storage=keyring \
  --exec-arg=--certificate-authority-data=$(kubectl -n default get configmap platform-ca -o jsonpath='{.data.ca-certificates\.crt}' | base64 | tr -d '\n')

Note: The --certificate-authority-data flag is only needed for local development, where TLS is signed by the cluster's self-signed cert-manager CA (platform-ca). trust-manager publishes that CA in the platform-ca ConfigMap (key ca-certificates.crt) in every namespace, together with the public CAs it trusts by default, because the bundle sets useDefaultCAs: true. Passing that whole bundle here works; read it with the admin context KSail created. To trust the cluster in your system trust store instead, import the platform CA on its own, not the bundle:

kubectl -n cert-manager get secret platform-ca-key-pair -o jsonpath='{.data.tls\.crt}' | base64 -d >platform-ca.crt

Production cluster (platform.devantler.tech)

kubectl config set-credentials oidc-prod \
  --exec-api-version=client.authentication.k8s.io/v1 \
  --exec-interactive-mode=IfAvailable \
  --exec-command=kubectl \
  --exec-arg=oidc-login \
  --exec-arg=get-token \
  --exec-arg=--oidc-issuer-url=https://dex.platform.devantler.tech \
  --exec-arg=--oidc-client-id=kubectl \
  --exec-arg=--oidc-extra-scope=email \
  --exec-arg=--oidc-extra-scope=profile \
  --exec-arg=--oidc-extra-scope=groups \
  --exec-arg=--oidc-extra-scope=audience:server:client_id:public-client \
  --exec-arg=--token-cache-storage=keyring

3 — Create a context that uses the OIDC user

# Local
kubectl config set-context oidc@local \
  --cluster=local \
  --user=oidc-local

# Production
kubectl config set-context oidc@prod \
  --cluster=prod \
  --user=oidc-prod

4 — Test

kubectl config use-context oidc@local   # or oidc@prod
kubectl get nodes

# Confirm the identity and that access is read-only:
kubectl auth whoami                      # Username: oidc:ned@devantler.tech
kubectl auth can-i list pods             # yes
kubectl auth can-i get secrets           # no  (Secrets are excluded by design)
kubectl auth can-i create deployments    # no  (writes are break-glass only)

On the first run, kubelogin opens a browser window. Log in with GitHub through Dex. Once authenticated, the token is cached locally and refreshed automatically.

Authentication flow

kubectl get pods
      │
      ▼
kubelogin (exec credential plugin)
      │  opens browser → https://dex.{domain}/auth
      │                        │
      │                        ▼
      │                  GitHub OAuth login
      │                        │
      │                        ▼
      │                  Dex issues ID token
      │  ◄─── http://localhost:8000 callback
      │
      ▼
kubectl sends ID token as Bearer header
      │
      ▼
kube-apiserver validates token
  • oidc-issuer-url matches token issuer ✓
  • oidc-client-id matches token audience ✓
  • signature verified against Dex JWKS ✓
  • email claim → Kubernetes username
  • groups claim → Kubernetes groups
      │
      ▼
RBAC: ClusterRoleBindings oidc-view + oidc-cluster-reader
  grant READ-ONLY (view + cluster-reader) to oidc:ned@devantler.tech
  (admin is break-glass via the root cert — not OIDC)

RBAC

OIDC access is read-only. Two ClusterRoleBindings in k8s/bases/infrastructure/cluster-role-bindings/oidc-view.yaml and oidc-cluster-reader.yaml bind the user whose email matches the Dex email claim (oidc:${admin_email}) to:

Binding Role Grants
oidc-view built-in view read all namespaced resources (pods, deployments, configmaps, pod logs, events, …) — excluding Secrets
oidc-cluster-reader cluster-reader read what view omits — mostly cluster-scoped infra (nodes, PVs, storage classes, CRDs, API services, priority/runtime/ingress classes, CSRs) plus RBAC objects and node/pod metrics 
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: "oidc:${admin_email}"   # → oidc:ned@devantler.tech

What is deliberately not granted via OIDC: Secrets (kept in the vault on purpose) and any write/exec verb (create/update/delete, pods/exec, pods/portforward). The cluster-reader ClusterRole is defined in k8s/bases/infrastructure/cluster-roles/cluster-reader.yaml.

The private World at Ruin zone trial has one explicit namespace-scoped exception for the configured administrator: a dedicated Role allows pods/portforward and pods/exec only in world-at-ruin, for the localhost tunnel and short-lived admission-token helper. It adds no Secret API access or workload writes and does not modify either global reader binding. Retire this trial exception before introducing production game credentials or player data.

To grant read-only access to additional users, add more subjects to both bindings (or switch the subject to a Dex group such as oidc:devantler-tech:platform).

Break-glass admin access

There is no admin path via OIDC — by design. When a write or an otherwise-forbidden operation is genuinely required, retrieve the root client-certificate kubeconfig from either the locked iCloud Note on the trusted operator device or the approved OpenBao break-glass path. OpenBao is itself protected by Dex OIDC and restricts the admin role to the configured platform administrator.

The ambient ~/.kube/config must contain only the OIDC reader context. Never merge the root user into it; materialize the break-glass kubeconfig as a short-lived, mode-0600 file instead:

  1. Retrieve the root kubeconfig from the vault and point KUBECONFIG at its short-lived path, e.g.:

    export KUBECONFIG=/private/path/to/root-kubeconfig.yaml
kubectl config use-context admin@prod

  2. Do the minimal change, then switch back to the OIDC context:

    unset KUBECONFIG                       # back to OIDC-only ~/.kube/config
kubectl config use-context oidc@prod

  3. Remove the temporary root kubeconfig from the workstation and verify both kubectl config current-context and kubectl auth whoami use the OIDC identity again.

The root cert authenticates directly against the cluster CA and bypasses OIDC/RBAC role limits (it is cluster-admin), so treat it accordingly: pull it only when needed and never persist it in your day-to-day kubeconfig.

Talos follows the same reader-by-default pattern; see Talos API access.

Last-resort regeneration (if the vault copy is lost): a fresh admin kubeconfig can be minted from the Talos control plane with talosctl --talosconfig <admin-talosconfig> kubeconfig.

Dex client configuration

kubelogin uses the kubectl static client defined in the Dex HelmRelease. This is a public client (no secret required) that uses Dex's cross-client trust (trustedPeers) so the explicit audience:server:client_id:public-client scope issues a token with aud: public-client, matching the kube-apiserver's --oidc-client-id flag. The token cache is stored in the macOS keychain instead of as a bearer-token file under ~/.kube/cache.

Troubleshooting

Symptom Cause Fix
error: You must be logged in to the server (Unauthorized) Token expired or wrong audience Run kubectl oidc-login setup --oidc-issuer-url=... --oidc-client-id=kubectl to verify the flow
Browser doesn't open kubelogin not installed or not in $PATH Verify kubectl oidc-login --help works
x509: certificate signed by unknown authority (local) The local platform-ca is not trusted Pass --certificate-authority-data from the platform-ca ConfigMap, or add the platform CA on its own to your system trust store (how)
Forbidden on a read Email not bound, or resource outside the read-only roles Check kubectl auth whoami; confirm the email matches the oidc-view.yaml / oidc-cluster-reader.yaml subjects
Forbidden on a write Expected — OIDC is read-only Use the break-glass admin cert for writes

References