Operate rollouts

Prod ships via an Argo Rollouts canary with time pauses, not automated analysis. There’s no Prometheus gate on this platform yet, so you decide when to promote or abort. Namespace is <app>-<env>; the Rollout is usually named <app>-api.

Every command on this page works on the node or locally: Access & consoles → kubectl. Running locally also needs the kubectl argo rollouts plugin installed on your machine.

Watch

Follow the canary’s progress:

kubectl argo rollouts get rollout <app>-api -n <app>-<env> --watch

Success: shows the live step, traffic weight, ReplicaSets, and pod readiness. If the plugin isn’t installed, fall back to plain kubectl:

kubectl -n <app>-<env> get rollout -w

To check pod status directly:

kubectl get pods -n <app>-<env> -l app=<app>-api

Promote or abort

# Skip the remaining pause and continue to the next step
kubectl argo rollouts promote <app>-api -n <app>-<env>

# Skip straight to 100% traffic
kubectl argo rollouts promote <app>-api -n <app>-<env> --full

# Cancel the rollout and roll back to the last stable version
kubectl argo rollouts abort <app>-api -n <app>-<env>

# Restart an aborted or failed rollout
kubectl argo rollouts retry rollout <app>-api -n <app>-<env>

Force a restart (config or secret changed)

Nothing about the app changed. A Secret value did (e.g. an Infisical edit). Kubernetes doesn’t hot-reload envFrom/env into already-running containers, so the pods need to restart to pick it up.

kubectl -n <app>-<env> patch rollout <app>-api --type merge \
  -p '{"spec":{"template":{"metadata":{"annotations":{"<domain>/redeploy":"<any-new-value>"}}}}}'

Any change to that annotation’s value gives the pod template a new hash, so Argo Rollouts starts a fresh ReplicaSet immediately. Pick a value you haven’t used before (a timestamp or counter is fine). This annotation is declared in git (workloads/<app>/base/rollout.yaml), so on Argo’s next reconcile selfHeal will reset it back to git’s value. Harmless, since the restart you wanted has already happened by then; worst case it triggers one extra, harmless rollout cycle. Commit the same value to git afterward if you want the annotation itself to stay in sync.

Argo CD also ships a Restart action for Rollout resources (sets spec.restartAt, which Argo Rollouts uses the same way). Application → the <app>-api Rollout → its actions menu. Worth trying if you’re already in the UI, but whether it’s present depends on your Argo CD install; the kubectl patch above always works.

Diagnose a stuck canary

Symptom: the rollout sits at step 1 indefinitely with 0 Ready pods on the new ReplicaSet.

Cause: a bad env var or secret is failing the new pods’ /readyz check. They never become Ready, so the canary can’t finish. Stable pods keep serving in the meantime, so this is safe, not an outage. It is readiness, not a missing metrics stack.

Fix: inspect the failing pods, correct the config, and push again:

kubectl describe pod -n <app>-<env> -l app=<app>-api
kubectl logs -n <app>-<env> -l app=<app>-api --tail=50