Sheet ⁨02⁩ · ⁨Blog⁩Surveyed ⁨2026⁩

Blog post image for Securing Kubernetes Workloads: Pod Security Admission, OPA Gatekeeper, and Kyverno Compared - The same three security policies written in Pod Security Admission, OPA Gatekeeper and Kyverno, plus where each one runs in the admission pipeline, what failurePolicy really costs you, and why ValidatingAdmissionPolicy changes the decision.

Securing Kubernetes Workloads: Pod Security Admission, OPA Gatekeeper, and Kyverno Compared

Published: Updated: 13 Mins read
Markdown for AI(opens in a new tab)

Kubernetes is permissive by default, and it is worth being blunt about how permissive. Out of the box, a developer can deploy a pod that runs as root, mounts the host filesystem at /, and declares no memory limit at all. The API server accepts it without a warning. If someone then finds a remote code execution bug in that application, the distance between “compromised container” and “compromised node” is very short.

The mechanism for stopping that is admission control, and there are now four credible ways to do it. Picking between them is less about features than about where the check runs and what happens when it breaks.

Why workload security policies matter more than a review process

Misconfiguration, not an unpatched CVE, is what most Kubernetes security incidents come down to. Red Hat’s annual State of Kubernetes Security report has put misconfiguration and exposure at or near the top of respondents’ reported incident causes for several years running, consistently ahead of vulnerabilities found in software.

That points at a process problem rather than a patching problem, and process controls scale badly. A security engineer reviewing pull requests is a real control, but it runs after the fact and only on the changes that go through review. By the time someone reads the diff, the non-compliant workload may have been running in production for a day.

Policy as code moves the check to the one place every change has to pass through. When a request reaches the API server, it goes through authentication, then authorization through RBAC, and then admission control. That third phase is the last point at which an object can be rejected or modified before it is written to etcd, and it does not care whether the request came from a pipeline, a kubectl apply, or someone’s laptop at midnight.

Where each tool actually runs

Before comparing syntax, it is worth being precise about the pipeline, because the position of each tool explains most of its limitations.

The green band is in-process, inside the API server, with no network hop and nothing that can be down. The purple band is the two webhook-based engines, which is what forces the failurePolicy decision in the red panel below. The bottom note explains why PSA cannot require resource limits: it only participates in the validating phase, so the strongest thing it can say is no.

Two details from that diagram do most of the explanatory work.

Mutation happens before validation. So a validating policy always sees the object after every mutating webhook has finished with it. If Kyverno injects a default memory limit and a validating rule then requires one, the order works out. Reverse it and nothing would ever pass.

Validating webhooks run in parallel, mutating webhooks run in sequence. Validation is a pure predicate, so order does not matter and any single rejection rejects the request. Mutation is not: each webhook sees the previous one’s output. Two controllers patching the same field can fight, which is why mutations need to be idempotent, patching only when the field is absent rather than unconditionally.

Pod Security Admission

PSA is built into Kubernetes, enabled by default since 1.23 and GA since 1.25, and it replaced the removed PodSecurityPolicy. It enforces the Pod Security Standards, and it is configured entirely with namespace labels.

There are three cumulative profiles.

ProfileWhat it allowsWhere it fits
privilegedEverything, including every known privilege escalationTrusted infrastructure workloads: CNI plugins, storage drivers, node agents
baselineBlocks known privilege escalations, permits ordinary application shapesA sensible default for most application namespaces
restrictedEnforces hardening: non-root, dropped capabilities, seccomp, no privilege escalationUntrusted workloads and anything handling sensitive data

Each profile can be applied in three modes, and you can use several at once.

  • enforce rejects the pod.
  • warn allows it and returns a warning to whoever submitted it.
  • audit allows it and annotates the audit log.

Tip

Combining the modes is what makes a rollout survivable. Set baseline to enforce so the serious problems are blocked today, and set restricted to warn and audit at the same time. Developers see exactly what would break before it does, you get audit-log data on the real violation rate, and you flip restricted to enforce once that rate reaches zero. Going straight to enforcing restricted on an existing cluster breaks deployments on a Friday afternoon.

Applying it is two labels:

apiVersion: v1
kind: Namespace
metadata:
name: namespace-prod
labels:
pod-security.kubernetes.io/enforce: baseline
pod-security.kubernetes.io/enforce-version: latest
# Both at once: block the serious things now, surface the rest.
pod-security.kubernetes.io/warn: restricted
pod-security.kubernetes.io/audit: restricted

Pinning enforce-version to a specific release rather than latest is worth considering. With latest, a cluster upgrade that tightens a standard can start rejecting workloads that were fine yesterday.

PSA’s limitation is not a configuration gap, it is the design. It understands the Pod Security Standards and nothing else. It cannot restrict which registry an image comes from, cannot require resource limits, cannot check a custom label, and cannot generate a NetworkPolicy for a new namespace. For any of those you need a policy engine, and the reason PSA cannot add a default limit is structural: it only participates in the validating phase, and adding something to an object requires mutation.

OPA Gatekeeper

Gatekeeper is an admission controller built on Open Policy Agent, evaluating policies written in Rego. It splits a policy into two objects, which is the part that confuses people initially and turns out to be the point.

A ConstraintTemplate holds the Rego logic and declares a schema for the parameters the rule accepts. It is a reusable blueprint, and applying one causes Gatekeeper to generate a new CRD named after it.

A Constraint is an instance of that blueprint. It supplies the parameters, such as the list of allowed registries, and scopes the rule to particular namespaces and resource kinds.

The separation means a platform team maintains a library of generic templates while application teams instantiate them with their own values, without either side writing Rego twice. A typical GitOps repository ends up looking like this:

  • Directorygatekeeper/
    • Directorytemplates/
      • k8sallowedrepos.yaml the Rego, written once
      • k8srequiredlabels.yaml
      • k8sblockprivileged.yaml
    • Directoryconstraints/
      • Directoryproduction/
        • allowed-repos.yaml parameters and scope only
        • required-labels.yaml
      • Directorystaging/
        • allowed-repos.yaml
    • Directoryconfig/
      • config.yaml sync the resources policies need to read
      • exempt-namespaces.yaml

Because OPA is a general-purpose engine, the same Rego can validate a Kubernetes manifest, a Terraform plan, and Envoy authorization requests. Gatekeeper also supports external data providers, so a policy can call an HTTP endpoint during admission to check something the cluster does not know, such as whether an image digest appears in an approved-builds database.

The cost is Rego. It is a genuinely capable language and a genuinely unfamiliar one, and debugging a non-obvious Rego rule at the point where it is blocking a deployment is not a pleasant introduction. If nobody on your team writes Rego today, the “one language everywhere” argument is a projection rather than an existing asset.

Kyverno

Kyverno takes the opposite bet. It is built only for Kubernetes, and policies are written in Kubernetes YAML rather than a separate language. If you can write a Deployment, you can read a Kyverno policy on first encounter, which matters a great deal when the person reading it is a developer trying to work out why their deploy was rejected.

Policies use a match and validate structure, with CEL and JMESPath available when the declarative pattern is not enough. Beyond validation, Kyverno does two things Gatekeeper does less naturally: it mutates incoming resources to inject missing configuration, and it generates entirely new resources in response to events, such as creating a default NetworkPolicy and ResourceQuota whenever a namespace appears.

Rolling out a policy safely follows the same shape every time:

  1. Write the policy with failureAction: Audit. It reports violations through PolicyReport objects and blocks nothing.

  2. Query the reports across the cluster with kubectl get polr -A to see the real violation count. This number is almost always higher than the team expects, and finding that out before enforcement is the entire purpose of the step.

  3. Fix the violations, or add explicit exclusions for the workloads that legitimately need an exception. Write down why in the policy annotation, because in six months nobody will remember.

  4. Flip to failureAction: Enforce, scoped to one namespace first rather than cluster-wide.

  5. Expand the scope namespace by namespace, watching the admission-webhook error rate and latency as you go.

Careful here

Kyverno renamed this field. spec.validationFailureAction was deprecated in favour of spec.rules[].validate.failureAction, which is per-rule rather than per-policy, and the values are Audit and Enforce. Plenty of blog posts and copy-pasteable examples still use the old top-level field. It continues to work for now, and it will not forever, so check which version of the syntax you are reading.

The same three policies in each tool

Abstract comparisons are less useful than seeing the same requirement written three ways.

Blocking privileged containers

A privileged container has effectively the same access to the host as a process running directly on the node. Blocking it is the most basic hardening step there is.

One label. The baseline profile already forbids privileged containers, along with host namespaces, host ports and all hostPath volumes.

Terminal window
kubectl label ns app-team pod-security.kubernetes.io/enforce=baseline

Enforcing an image registry

Pulling from arbitrary public registries is a supply chain risk. Images should come from your own registry, where they have been scanned.

Not possible. PSA evaluates the Pod Security Standards only, and image provenance is not part of them. This is the requirement that usually makes teams install a policy engine.

Requiring resource limits

A container with no memory limit can consume every byte on a node and take its neighbours down with it.

Not possible, for the structural reason described earlier. Even validating this is outside the Pod Security Standards, and injecting a default would require mutation, which PSA does not do at all.

Use a LimitRange in the namespace instead. It is built in, it sets defaults for containers that omit them, and it long predates all of this.

The webhook risk nobody mentions until it bites

Both Gatekeeper and Kyverno run as dynamic admission webhooks, outside the API server. That means every matching API request involves a network call from the control plane to a pod in your cluster, and that pod is now on the critical path of resource creation.

Which forces a decision with no good answer. Every webhook registration carries a failurePolicy telling the API server what to do when the webhook times out or errors.

failurePolicy: Ignore lets the request through. Availability wins, the cluster keeps working, and during the outage your policies are simply not enforced. Nothing announces this. Unless you alert on webhook error rates, the first you hear about it is when you find a privileged pod that was admitted during a fifteen-minute window last Tuesday.

failurePolicy: Fail rejects the request. Security wins, nothing non-compliant gets through, and if the policy controller is down you cannot deploy anything. On a cluster-wide rule, that includes the pods you would need to deploy to fix the outage. This failure mode is genuinely circular and has taken clusters down.

The way out is not to pick a side, it is to scope the webhook so Fail is survivable:

# Exclude the namespaces that have to work when policy is broken.
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: NotIn
values: ['kube-system', 'kyverno', 'gatekeeper-system']

Alongside that: run the controller with several replicas and a PodDisruptionBudget, keep the webhook timeout low (timeoutSeconds: 5 or less, because the API server waits on it), and narrow rules so the webhook is only consulted for the resources it actually has policies about. A webhook matching */* is asking to be in the path of every operation in the cluster, including ones you never wrote a policy for.

Mutating webhooks add the ordering problem on top. They run sequentially, each seeing the previous output, so a non-idempotent mutation can fight another controller and cause the API server to re-run the chain. Keep the logic simple and conditional, which is what both the pathTests and +() examples above are doing.

So which one should you use

Work down from the top and stop at the first box that fits. The two green outcomes need nothing installed and have no webhook to fail. The blue panel at the bottom is the setup most production clusters end up with: PSA as a floor that keeps working when a webhook is down, plus one engine above it for what PSA cannot express.

The honest summary is that the tools are less interchangeable than a feature matrix suggests.

EngineWhere it fitsThe real drawback
Pod Security AdmissionYou want root and privileged containers blocked with nothing new to run or monitorUnderstands only the Pod Security Standards. No custom logic, no registry rules, no generation
ValidatingAdmissionPolicyCustom validation without a webhook, in-process, using CELValidation only. No mutation, no resource generation
OPA GatekeeperYou already run OPA for Terraform, CI or an API gateway, and want one policy language across all of itRego is hard to learn and harder to debug. Large constraint libraries consume real memory
KyvernoKubernetes-focused teams wanting mutation, generation and supply chain verification in plain YAMLPolicies are Kubernetes-only. You cannot reuse them to gate anything outside the cluster

The option worth calling out separately is ValidatingAdmissionPolicy, which reached GA in Kubernetes 1.30. It lets you write validation rules in CEL that the API server evaluates in-process, so there is no webhook, no failurePolicy dilemma, and no network latency on the admission path. It cannot mutate or generate, so it does not replace Kyverno for a platform team that relies on those. But for pure validation it removes the entire failure mode this article spent a section on, and it is new enough that most comparison articles predate it.

If I were setting up a cluster today: PSA at baseline everywhere immediately, because it costs nothing and works when everything else is broken. Then restricted in warn and audit mode to find out what would break. Then one engine on top, chosen by whether the team already writes Rego, for the rules PSA cannot express. Running both Gatekeeper and Kyverno is possible and rarely worth two failure domains and two skill sets.

Frequently Asked Questions

No, and expecting it to be causes trouble. PSP could be scoped to a service account through RBAC and could mutate pods to add defaults. PSA does neither: it is namespace-scoped through labels and it is validating only. Most non-trivial PSP setups do not translate directly, and the migration usually ends up as PSA for the standard hardening plus a policy engine for whatever the PSPs did beyond that.

Technically yes. They register separate webhooks and do not conflict. In practice it means two controllers to keep healthy, two failure domains on the admission path, two policy languages for people to learn, and two places to check when a deployment is rejected without an obvious reason. The usual reason to do it is a migration, and the usual mistake is not finishing the migration.

It adds latency to every API request the webhook matches, which is why the rules scoping matters more than the policy count. A webhook matching everything sits in the path of every operation. In steady state a well-scoped webhook adds single-digit milliseconds; the number that hurts is the timeout, because the API server waits for it. Set timeoutSeconds low and alert on the apiserver_admission_webhook_rejection_count and duration metrics.

Both engines have an audit mode, and both have a CLI for testing outside the cluster. Kyverno’s kyverno apply runs a policy against manifest files locally and is easy to put in a pipeline, so a developer finds out at commit time rather than at deploy time. Gatekeeper has gator test for the same purpose. Do both: audit mode tells you the violation rate on what is already running, and the CLI stops new violations before they are submitted.

Nothing, immediately. Admission control only runs on create and update, so an existing pod keeps running until something touches it. That is a trap: the policy looks enforced, and the violations are still there. Kyverno generates PolicyReport objects for existing resources, and Gatekeeper’s audit controller reports violations on a schedule. Check those, or your first sign of the problem is a rescheduled pod that suddenly cannot start.

Yes. They solve different problems at different layers. RBAC decides who may submit the request at all, and admission control decides whether the object they submitted is acceptable. NetworkPolicy governs traffic between pods once they are running, which no admission controller sees. A policy engine can help by generating a default deny-all NetworkPolicy for every new namespace, which is one of the better uses of Kyverno’s generation feature.

For validation, plausibly, over time. Being in-process removes the failure mode that makes webhooks operationally awkward, and CEL is easier to learn than Rego. But it does not mutate or generate resources, and it has no equivalent of Kyverno’s image verification, so it does not replace either engine for a platform team using those. The likely end state is VAP handling the simple validation most clusters need, with an engine layered on for mutation, generation and supply chain checks.

References

Was this useful?

You might also enjoy

More posts on similar topics

Policy as Code with Open Policy Agent: A Technical and Governance Perspective

Policy as Code with Open Policy Agent: A Technical and Governance Perspective

Think about how much stuff modern organizations have running in the cloud these days. It's a lot, right? All those servers, applications, and connections can get pretty complicated to manage. Just cli

Container Image Signing with Cosign: A Hands-On Guide to Secure Your Supply Chain

Container Image Signing with Cosign: A Hands-On Guide to Secure Your Supply Chain

Container images make packaging and running apps consistent across environments, which is genuinely useful. But that consistency also brings new security questions. We need to make sure the images run

GitOps vs. Traditional IaC for Kubernetes: A Comparative Analysis

GitOps vs. Traditional IaC for Kubernetes: A Comparative Analysis

If you're managing modern cloud-native applications, especially with Kubernetes, you know it can be a real puzzle. Getting containers to work together, handling all those configurations, and scaling t

Compliance as Code: Making Security Easier with Terraform and InSpec

Compliance as Code: Making Security Easier with Terraform and InSpec

Hey, so you know how keeping our tech stuff secure and following all the rules can be a real headache these days? With everything moving to the cloud and so many regulations popping up, it's tough to

The Resilience of Timbernetes: An Analysis of In-Place Pod Vertical Scaling in Kubernetes 1.35

The Resilience of Timbernetes: An Analysis of In-Place Pod Vertical Scaling in Kubernetes 1.35

The release of Kubernetes 1.35, officially designated as "Timbernetes," is a definitive shift in how cloud-native orchestration treats compute. This version marks the graduation of In-Place Pod Vertic

Kubernetes Networking Demystified: CNI Plugins, Network Policies, and Pod-to-Pod Communication

Kubernetes Networking Demystified: CNI Plugins, Network Policies, and Pod-to-Pod Communication

If you've spent any time with Kubernetes, you know that networking is often the part that makes people's heads spin. It feels like magic until something breaks, and then you're staring at a maze of vi

6 related posts