Kubernetes Production Deployment Checklist: From Container Image to Reliable Rollout

Overview

A reliable Kubernetes deployment should answer five questions before traffic reaches the new version:

• Is the image trustworthy, reproducible, and suitable for production?
• Will the workload run with the permissions and resources it actually needs?
• Can Kubernetes distinguish a starting container from a ready container and a failed container?
• Can the team observe the rollout and recover quickly if it goes wrong?
• Does someone know how to operate the application after deployment?

Use this Kubernetes production checklist as a release gate rather than a one-time document. The exact configuration will vary by application, cluster, and cloud provider, but the control points remain useful for APIs, workers, web applications, and scheduled workloads.

Keep deployment configuration in version control and make changes through the same delivery path used for other production infrastructure. If Terraform manages cluster or platform resources, review the related Terraform drift detection workflow and confirm that manual changes will not be overwritten unexpectedly.

Checklist by scenario

1. Before building or promoting the container image

• Pin the base image to an intentional version or digest, and record how it will be updated.
• Build the image from a repeatable process. Avoid embedding credentials, tokens, private keys, or environment-specific configuration.
• Run unit tests, integration tests, and security checks before promotion.
• Use a clear image tag that identifies the release. Do not rely on a mutable tag such as latest for a production rollout.
• Confirm that the image starts with the expected command and listens on the documented port.
• Run the container as a non-root user where the application permits it.
• Verify that the image contains only the files and tools needed at runtime.

Image validation is part of the Kubernetes deployment guide, not a separate concern. A clean manifest cannot compensate for an untracked or unreproducible image.

2. Before applying Kubernetes manifests

• Use the current API versions supported by the target cluster and validate manifests before deployment.
• Set labels consistently, including application, component, environment, and release identifiers where useful.
• Define a namespace deliberately. Confirm that quotas, policies, and network controls for that namespace match the workload.
• Review the service account. Grant only the permissions the application requires, and avoid using a broadly privileged default identity.
• Keep configuration separate from the image. Use ConfigMaps for non-sensitive settings and an approved secret-management approach for credentials.
• Set security context values appropriate to the workload, such as non-root execution, read-only root filesystems, dropped Linux capabilities, and a selected seccomp profile where supported.
• Confirm that the Service selector matches the Pod labels exactly. A mismatch can produce a deployment that appears healthy while receiving no traffic.
• Check ingress or gateway rules, TLS configuration, hostnames, timeouts, and routing behavior in the target environment.

For sensitive values, compare the proposed design with the cloud secrets management comparison. The important production question is not only where a secret is stored, but also who can read it, how it is rotated, and whether it can appear in logs.

3. Before setting resource requests and limits

Resource requests influence scheduling; limits constrain usage. Set them from measured behavior when possible, then revisit them after observing the workload under realistic traffic.

resources:
  requests:
    cpu: "250m"
    memory: "256Mi"
  limits:
    cpu: "1"
    memory: "512Mi"

• Use requests that allow the scheduler to place the workload on an appropriate node.
• Set memory values carefully because memory pressure can cause termination when a container exceeds its limit.
• Decide whether CPU limits help or hinder the workload based on its behavior and platform standards.
• Include requests for every production container, including sidecars where applicable.
• Check namespace quotas and limit ranges so the manifest is compatible with admission controls.
• For critical services, evaluate whether a PodDisruptionBudget and multiple replicas are appropriate.

4. Before declaring a Pod healthy

Readiness, liveness, and startup probes have different jobs. Use them to express application behavior rather than copying the same endpoint into every probe.

Probe checks should be lightweight and representative. A readiness failure should normally remove a Pod from service without restarting it. A liveness probe should be harder to fail and should not depend on a database or external service unless restarting the application is genuinely the correct recovery action.

5. During a controlled rollout

1. Deploy to a non-production environment that resembles production closely enough to expose configuration and networking errors.
2. Run smoke tests against the new version, including authentication, critical API paths, background processing, and key integrations.
3. Confirm that the Deployment reaches its intended replica count and that old replicas terminate as expected.
4. Watch application metrics, logs, events, latency, error rates, saturation, and business-level signals during the rollout.
5. Use a staged, canary, or blue-green approach when the impact of a faulty release warrants additional isolation.
6. Record the release identifier, image digest, manifest revision, operator, and start time.

A GitHub Actions CI/CD workflow can automate validation and promotion, but automation should preserve approval points and make rollback actions visible to the person on call.

6. Before closing the deployment

• Confirm that dashboards and alerts identify the new version and its namespace.
• Verify logs are arriving with useful fields such as timestamp, severity, request or correlation ID, and release version.
• Check autoscaling behavior if horizontal or event-driven scaling is configured.
• Confirm that scheduled jobs, consumers, migrations, and one-time tasks will not run twice unexpectedly.
• Update the runbook with the deployment, rollback, dependency, and escalation details.
• Obtain an explicit handoff from the deploying team to the operational owner.

What to double-check

Manifest review

Before approval, inspect the rendered manifests rather than only the source template. Look for unintended namespaces, image tags, replica counts, environment variables, hostnames, volume mounts, and service selectors. Review the effective output produced by Helm, Kustomize, or another templating system because defaults and overlays can change the final resource.

Security and access

Check that the Pod can access only the APIs, files, and network destinations it needs. Review RoleBindings, service account tokens, admission policies, network policies, container privileges, and secret references. Ensure that debugging settings, verbose error responses, and temporary administrative access are removed before production.

Reliability dependencies

Identify what happens when the database, queue, cache, identity provider, or third-party API is slow or unavailable. The application should fail in a controlled way, use bounded retries, and avoid creating a retry storm. Confirm termination behavior, including graceful shutdown time, connection draining, and in-flight request handling.

Rollout and rollback commands

Write down the commands and approvals needed to inspect and reverse a release. A typical Deployment workflow may include:

kubectl rollout status deployment/<name> -n <namespace>
kubectl rollout history deployment/<name> -n <namespace>
kubectl rollout undo deployment/<name> -n <namespace>

These commands are useful only when revision history is retained and the previous image and configuration remain available. Test the rollback path in a non-production environment, and define what happens to database schema changes that are not backward compatible.

Production handoff template

Capture the following before the release is considered complete:

• Application and version: name, image digest, commit, and release owner.
• Cluster and namespace: target context, namespace, and relevant workload names.
• Dependencies: databases, queues, external APIs, secrets, and required network paths.
• Health signals: dashboards, alerts, probe endpoints, and expected baseline behavior.
• Change risk: known limitations, migrations, feature flags, and traffic assumptions.
• Recovery: rollback command, decision threshold, escalation contact, and data-recovery considerations.

Common mistakes

• Using liveness as readiness: A temporary dependency failure can cause unnecessary restarts and amplify an incident.
• Deploying a mutable image tag: The same manifest can produce different runtime code, making diagnosis and rollback difficult.
• Skipping resource requests: The scheduler lacks reliable placement information, and capacity planning becomes less predictable.
• Testing only that Pods are running: A running process may still return errors, fail authentication, or be unreachable through the Service.
• Forgetting graceful termination: Requests can be interrupted when Pods are replaced unless the application handles shutdown correctly.
• Changing application and database schemas together without a compatibility plan: Rollback may restore the old code but not the old data model.
• Allowing manual production edits without recording them: Configuration drift can make the declared deployment different from the running system. Review the Terraform state management guidance when infrastructure changes are managed as code.
• Leaving observability until after release: Without logs, metrics, and traces, the team cannot confidently distinguish a bad deployment from an unrelated platform issue. Use a deliberate monitoring tools comparison when selecting the supporting stack.

When to revisit

Run this checklist before every production release that changes application code, images, configuration, dependencies, traffic patterns, or cluster placement. Revisit it after a security finding, reliability incident, cluster migration, Kubernetes version change, ingress or service-mesh change, major autoscaling adjustment, or change to the secret-management process.

At least twice a year, review the checklist itself. Confirm that the cluster still supports the API versions and security controls used by the manifests, that the ingress and gateway approach remains appropriate, and that the delivery system still records enough information for diagnosis and rollback. Also compare resource requests and limits with observed usage rather than allowing them to remain historical guesses.

For the next deployment, turn this article into a release ticket with explicit owners. Mark image validation, manifest review, probes, resources, security, observability, rollout, rollback, and handoff as separate checks. Do not close the ticket until the operator can answer one practical question: if this release fails five minutes after deployment, what will we look at first, and how will we restore service?