Kubernetes Admission Webhooks with Open Policy Agent
Enforcing security standards and governance policies across Kubernetes clusters is critical for multi-tenant organizations. Preventing developers from deploying privileged root containers, missing resource CPU/memory limits, or invalid ingress hostnames cannot rely on manual pull request code reviews.
Kubernetes Admission Webhooks intercept API server requests before objects are persisted into etcd. Open Policy Agent (OPA) Gatekeeper provides a policy engine for Kubernetes, enabling DevOps teams to write declarative policy rules using Rego. Gatekeeper acts as a validating and mutating webhook server, enforcing security compliance, container image signature verification, and label enforcement at scale. This guide details admission control lifecycles, Rego policy syntax, ConstraintTemplates, and continuous cluster auditing.
Mental Model: Static YAML Linters vs Dynamic Kubernetes Admission Control
Static CLI linters (kubeval, conftest) inspect Kubernetes manifest files during Git commit or CI/CD stages. However, static linters cannot inspect runtime cluster state or intercept dynamic API calls made by Helm charts, Kubernetes Operators, or direct kubectl commands.
Dynamic Admission Control operates inside the Kubernetes API Server request pipeline:
When a client submits kubectl apply -f deployment.yaml, the API Server authenticates and authorizes the request, passes the object through Mutating Admission Webhooks (to inject default sidecars or labels), enforces schema validation, and routes the object to Validating Admission Webhooks (OPA Gatekeeper). If Gatekeeper denies the request, the API Server rejects the deployment instantly ($O(1)$ policy enforcement). For K8s operator patterns, review mastering kubernetes custom resource definitions kubebuilder and mastering gitops argocd fluxcd kubernetes.
Quick reference
- Admission webhooks intercept API server requests prior to etcd database persistence.
- OPA Gatekeeper enforces policy compliance dynamically for all kubectl, Helm, and Operator API calls.
- Prevents misconfigured, insecure, or un-restricted workloads from ever entering the cluster.
- Decouples policy definition (Rego code) from application deployment manifests.
- Protects multi-tenant Kubernetes clusters against privilege escalation and resource starvation.
Remember this
Deploy OPA Gatekeeper admission webhooks to enforce dynamic runtime security policies in Kubernetes.
Mutating vs Validating Admission Webhook Execution Lifecycles
The Kubernetes API Server processes admission requests in a strict two-phase sequence:
1. Mutating Phase: Mutating webhooks execute first. They can modify incoming JSON object manifests before schema validation. Common use cases include injecting Istio sidecar proxies or defaulting container resource limits.
2. Schema Validation: The API Server validates the modified JSON manifest against OpenAPI schemas.
3. Validating Phase: Validating webhooks (OPA Gatekeeper) execute last. They evaluate the final modified manifest against declarative policy rules and return an allowed: true or allowed: false JSON decision with detailed rejection reasons.
Quick reference
- Mutating webhooks modify incoming Kubernetes manifests (e.g. injecting sidecars or default labels).
- Validating webhooks evaluate final manifests and approve or deny object creation.
- Webhook calls execute over mutual TLS (mTLS) with strict timeout windows (default 10s).
- FailurePolicy (Fail vs Ignore) determines API Server behavior if the webhook server times out.
- Setting FailurePolicy: Fail prevents insecure deployments during webhook server outages.
Remember this
Configure Mutating webhooks for default injections and Validating webhooks (OPA) for security enforcement.
Writing Declarative Policy Constraints with Open Policy Agent & Rego
OPA policy logic is written in Rego, a high-performance declarative query language:
1package k8srequiredlabels2 3violation[{"msg": msg}] {4 provided := {label | input.review.object.metadata.labels[label]}5 required := {label | label := input.parameters.labels[_]}6 missing := required - provided7 count(missing) > 08 msg := sprintf("You must provide labels: %v", [missing])9}This Rego rule extracts labels from incoming resource requests, compares them against a required parameter list, and generates a violation message if any required labels are missing.
Quick reference
- Rego is a declarative query language designed specifically for policy decision-making.
- Rules evaluate incoming Kubernetes admission request JSON objects (input.review.object).
- Set pattern operations evaluate resource labels, container images, securityContext, and ingress hosts.
- Generates descriptive violation messages returned directly to developers in kubectl CLI outputs.
- Unit test Rego policy rules using opa test before deploying to production clusters.
Remember this
Write modular Rego policy rules to validate container security attributes, labels, and ingress constraints.
Deploying Gatekeeper ConstraintTemplates & Audit Controllers in K8s
Gatekeeper exposes Kubernetes Custom Resource Definitions (CRDs) to manage OPA policies:
1. ConstraintTemplate: Defines the reusable Rego policy logic and CRD schema parameters.
2. Constraint: Instantiates a ConstraintTemplate, targeting specific Kubernetes namespaces or resource kinds (Deployment, Pod, Ingress).
3. Audit Controller: In addition to blocking non-compliant real-time requests, Gatekeeper's background Audit Controller continuously scans existing cluster objects, reporting non-compliant resources in Constraint.status.violations without interrupting running pods.
Quick reference
- ConstraintTemplates encapsulate Rego policy code inside declarative Kubernetes CRDs.
- Constraints apply templates to specific namespaces, resource kinds, or API groups.
- Audit Controller continuously scans pre-existing cluster resources for policy drift.
- Gatekeeper metrics export to Prometheus for real-time policy violation alerting.
- DryRun enforcement action logs violations without blocking live deployment requests during testing.
Remember this
Deploy Gatekeeper ConstraintTemplates and use DryRun modes to test policies before enforcing strict blocks.
Key takeaway
To test OPA Gatekeeper, install Gatekeeper via Helm (helm install gatekeeper gatekeeper/gatekeeper). Apply a K8sRequiredLabels ConstraintTemplate and test a pod deployment missing required labels.
Related Articles
Explore this topic