Examining Gatekeeper: Extracting sample input for OPA Gatekeeper policy development
By Alain Baxter, SADA Sr. Cloud Infrastructure Engineer
This article is part two of a series on OPA Gatekeeper. For additional reading, see the previous article on this topic.
Open Policy Agent (OPA) is a policy engine that simplifies policy enforcement and decouples policy decisions from your application. OPA Gatekeeper integrates OPA and Kubernetes, and builds a framework to take advantage of Kubernetes’ declarative nature to simplify the process of policy management and enforcement in the cluster. OPA Gatekeeper has become an important tool in the Kubernetes ecosystem to provide security controls and guardrails for resources in your cluster.
The open source community maintaining Gatekeeper also provides a library with many useful constraint templates ready to go to kick start your ability to quickly provide value. Google Cloud’s Anthos Config Management Policy Controller, which is a managed and supported install of Gatekeeper in the GKE/Anthos ecosystem, provides a similar constraint template library which contains the open source library along with additional templates developed to cover security requirements like the CIS Kubernetes Benchmark recommendations.
While these existing libraries are excellent to get you started with policy enforcement in your cluster, we have found that as our clients get comfortable with Gatekeeper as a tool, they quickly think of additional ideas for impactful use cases. This is when custom constraint templates generally come into the picture. This article will help you understand what data is available to you when designing new custom policies.
In our previous article on this topic, we looked at how to understand the Rego policy language used to define constraint templates for your OPA Gatekeeper policies.
Data for Gatekeeper Policy development
Constraints are created from a constraint template. This tells Gatekeeper to review resources both at admission time and during its audit cycle. The data provided to the Rego code in both workflows are the same, and come from two potential sources:
- The information of the resource under review, in the
inputobject - (Optionally) A cached copy of existing resources in the cluster, in the
data.inventoryobject.
All this data is coming from the kube-apiserver, either through the admission review process for the input object, or through caching queried data for data.inventory. Since data.inventory is cached data, it’s important to know that it is eventually consistent. This means that referential policies might allow a violation initially if two resources are created at nearly the same time.
The input object does not reflect exactly the resource manifest being reviewed. The kube-apiserver will enrich the object with defaults in many fields, and include additional information about the request, such as the user, operation, existing object, and so on. We have put together a method to extract the actual input and data.inventory exactly as it is provided to your policy code. This method uses Gatekeeper with a custom constraint template and constraints. This is ideally deployed to your lab or development environment, allowing your policy development teams to export examples when needed.
Here is an example of a custom ConstraintTemplate resource:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: blockandexportreview
spec:
crd:
spec:
names:
kind: BlockAndExportReview
validation:
# Schema for the `parameters` field
openAPIV3Schema:
type: object
properties:
exportData:
type: boolean
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package blockandexportreview
violation[{"msg": msg}] {
data_export := get_data
msg := sprintf("Blocked to export objects:\nInput: %v%v", [input, data_export])
}
get_data = data_export {
input.parameters.exportData
data_export := sprintf(" \nData: {\"inventory\": %v}", [data.inventory])
}
get_data = data_export {
not input.parameters.exportData
data_export := ""
}If referential policies are completely disabled for OPA Gatekeeper, any reference to data.inventory is not allowed and it will fail to create. Instead, create a simpler policy, as follows:
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: blockandexportreview
spec:
crd:
spec:
names:
kind: BlockAndExportReview
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package blockandexportreview
violation[{"msg": msg}] {
msg := sprintf("Blocked to export objects:\nInput: %v", [input])
}The policy code here is simple, there is no logic in the violation rule, so if this policy is triggered, it will always trigger a violation. The key is that the input object and (optionally) the data.inventory object, are exported in the policy’s return message, which is then returned to the user from the kube-apiserver in response to their request.
Setting up a special namespace, with a constraint that matches all resources in that namespace, will allow for exporting any namespace scoped resource. For example:
apiVersion: v1
kind: Namespace
metadata:
name: policy-review-exporting
---
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: BlockAndExportReview
metadata:
name: block-and-export-review-input
spec:
match:
scope: Namespaced
namespaces: ["policy-review-exporting"]
parameters:
exportData: falseLet’s take this for a spin and try applying the following deployment object to this namespace:
apiVersion: apps/v1
kind: Deployment
metadata:
name: test-deployment
namespace: policy-review-exporting
spec:
selector:
matchLabels:
app: nginx
replicas: 2
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80We get the following message from the apply:
Error from server ([block-and-export-review-input] Blocked to export objects:
Input: {...SNIP...}): error when creating "test-deployment.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [block-and-export-review-input] Blocked to export objects:
Input: {...SNIP...}Clearing out the JSON input for now, notice that it was returned twice. This is an artifact from how the kube-apiserver returns admission errors.
When you enable “exportData” as true for the constraint, and you’ve configured Gatekeeper to cache existing resources from the cluster, then you’ll be able to see the documented data structure from data.inventory, which is:
data.inventory.cluster[<groupVersion>][<kind>][<name>]for cluster scoped resourcesdata.inventory.namespace[<namespace>][<groupVersion>][<kind>][<name>]for namespace scoped resources
To pretty print the content so that we can inspect it easily, I recommend using jq like this:
echo '{...SNIP...}' | jqLet’s look at the output for the previous test deployment:
{
"parameters": {
"exportData": false
},
"review": {
"_unstable": {
"namespace": {
"apiVersion": "v1",
"kind": "Namespace",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"name\":\"policy-review-exporting\"}}\n"
},
"creationTimestamp": "2022-01-24T21:58:54Z",
"labels": {
"kubernetes.io/metadata.name": "policy-review-exporting"
},
"name": "policy-review-exporting",
"resourceVersion": "16499846",
"uid": "34d86832-c475-481b-8efd-0572b55c8016"
},
"spec": {
"finalizers": [
"kubernetes"
]
},
"status": {
"phase": "Active"
}
}
},
"dryRun": false,
"kind": {
"group": "apps",
"kind": "Deployment",
"version": "v1"
},
"name": "test-deployment",
"namespace": "policy-review-exporting",
"object": {
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"name\":\"test-deployment\",\"namespace\":\"policy-review-exporting\"},\"spec\":{\"replicas\":2,\"selector\":{\"matchLabels\":{\"app\":\"nginx\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"nginx\"}},\"spec\":{\"containers\":[{\"image\":\"nginx:1.14.2\",\"name\":\"nginx\",\"ports\":[{\"containerPort\":80}]}]}}}}\n"
},
"creationTimestamp": "2022-01-24T22:03:36Z",
"generation": 1,
"name": "test-deployment",
"namespace": "policy-review-exporting",
"uid": "bab19531-d27f-41b5-9dc9-0b87e691d57e"
},
"spec": {
"progressDeadlineSeconds": 600,
"replicas": 2,
"revisionHistoryLimit": 10,
"selector": {
"matchLabels": {
"app": "nginx"
}
},
"strategy": {
"rollingUpdate": {
"maxSurge": "25%",
"maxUnavailable": "25%"
},
"type": "RollingUpdate"
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"app": "nginx"
}
},
"spec": {
"containers": [
{
"image": "nginx:1.14.2",
"imagePullPolicy": "IfNotPresent",
"name": "nginx",
"ports": [
{
"containerPort": 80,
"protocol": "TCP"
}
],
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File"
}
],
"dnsPolicy": "ClusterFirst",
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"terminationGracePeriodSeconds": 30
}
}
},
"status": {}
},
"oldObject": null,
"operation": "CREATE",
"options": {
"apiVersion": "meta.k8s.io/v1",
"fieldManager": "kubectl-client-side-apply",
"kind": "CreateOptions"
},
"requestKind": {
"group": "apps",
"kind": "Deployment",
"version": "v1"
},
"requestResource": {
"group": "apps",
"resource": "deployments",
"version": "v1"
},
"resource": {
"group": "apps",
"resource": "deployments",
"version": "v1"
},
"uid": "21a7e572-dbff-406c-961e-67ff70bb75e5",
"userInfo": {
"groups": [
"system:authenticated"
],
"username": "user@example.com"
}
}
}There are some interesting sections here:
- The userInfo of the user who requested the operation
- A stub for the oldObject used when updating an existing object
- Multiple object-type information sections
Notice that in the object section, there are many fields that are not in the original YAML manifest that was applied, but which show up here with configuration. These are defaults for several fields, such as the strategy for upgrading pods from the deployment. This will also reflect any mutations that were applied to the object from any mutating admission webhooks.
Understanding how Gatekeeper sees the input gives a great perspective on what policies can be created and how they might work in a real cluster. Try it out for yourself, with the resources for which you are writing policy.
Conclusion
I hope you’ve found these articles instructive and they have helped you to understand the role of OPA Gatekeeper, how you can use Rego to make your own custom templates, and built an understanding of how it works with its input so that you can build your own policies to get the most benefit from OPA Gatekeeper. Check back for future articles on this topic.
About Alain Baxter
Alain Baxter is a Senior Cloud Infrastructure Engineer at SADA who specializes in delivering Anthos based solutions to our clients. He has worked with Kubernetes on and off for over four years, and is a champion of the devops team culture. He has contributed to the open source OPA Gatekeeper ConstraintTemplate library and engages with the community regularly. Alain can be found on LinkedIn.
About SADA
At SADA, we climb every mountain, clear every hurdle, and turn the improbable into possible — over and over again. Simply put, we propel your organization forward. It’s not enough to migrate to the cloud, it’s what you do once you’re there. Accelerating application development. Advancing productivity and collaboration. Using your data as a competitive edge. When it comes to Google Cloud, we’re not an add-on, we’re a must-have, driving the business performance of our clients with its power. Beyond our expertise and experience, what sets us apart is our people. It’s the spirit that carried us from scrappy origins as one of the Google Cloud launch partners to an award-winning global partner year after year. With a client list that spans healthcare, media and entertainment, retail, manufacturing, public sector and digital natives — we simply get the job done, every step of the way. Visit SADA.com to learn more.
If you’re interested in becoming a part of the SADA team, please visit our careers page.
