Kubernetes 准入控制器： Admission Webhook

Admission controllers are a powerful Kubernetes-native feature that helps you define and customize what is allowed to run on your cluster.

As watchdogs, they can control what’s going into your cluster. They can manage deployments requesting too many resources, enforce pod security policies, and even block vulnerable images from being deployed.

In this article, you’ll learn ​​what admission controllers are in Kubernetes​​​ and how their webhooks can be used to ​​implement image scanning​​.

Admission controllers guard the door to your Kubernetes cluster

---

An admission controller intercepts and processes requests to the Kubernetes API prior to persistence of the object, but after the request is authenticated and authorized.

准入控制器在对象持久化之前拦截并处理对 Kubernetes API 的请求，但在请求经过身份验证和授权之后。 

These controllers are compiled and shipped into the ​​kube-apiserver​​​ binary, and can only be enabled and configured by the cluster administrator using the ​​--enable-admission-plugins​​​ and ​​--admission-control-config-file​​ flags.

Most of the available admission controllers have very specific functions.

For example, ​​LimitRanger​​​ validates that none of the objects in a Kubernetes deployment violate the constraints specified in the ​​LimitRange​​​ object of a ​​Namespace​​. It can also mutate the request to assign default ​​resource limits and requests​​ to Pods that don’t specify any.

This is the set of recommended Kubernetes admission controllers that you may find in any vanilla deployment:

--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,Priority,ResourceQuota,PodSecurityPolicy

 You can check the full list of available admission controllers ​​in the Kubernetes documentation​​.

The power of Kubernetes admission controllers

---

Admission controllers can answer existential questions in the life of a DevOps:

• Is the pod requesting too many resources?
• Are the base images used to spawn the microservice pods secure?
• What is the priority of this deployment compared to the others?
• Which privileges are currently granted to the service account linked to these pods/deployments? Do they adhere to the principle of least privilege?
• Pod 请求的资源是否过多？
• 用于生成微服务 pod 的基础镜像是否安全？
• 与其他部署相比，此部署的优先级是什么？
• 当前授予与这些 pods/deployments 相关联的服务帐户哪些权限？他们是否遵守最小特权原则？

They also can take action.

They can block pods from running if the cluster is out of resources or if the images are not secure. And, as we saw earlier, they can even mutate the request to tweak the resources request from a pod.

You are probably already running several Kubernetes admission controllers that come pre-configured out-of-the-box with your standard deployment. Some aspects of Kubernetes that you may consider built-in are actually enforced by these controllers, for example:

• ​​LimitRanger​​: Manages resource limits and requests, as mentioned before.
• ​​PodSecurityPolicy​​: Acts on creation and modification of the pod and determines if it should be admitted based on the requested security context and the available Pod Security Policies.
• ​​PersistentVolumeClaimResize​​​: Implements additional validations for checking incoming​​PersistentVolumeClaim​​ resize requests.

 Again, all of this is done before the request is persisted in etcd, which means before it is executed. This is what makes Kubernetes admissions controllers such a perfect candidate to deploy preventive security controls on your cluster.

 同样，所有这些都是在请求被持久化到 etcd之前完成的，这意味着在它被执行之前。这就是使 Kubernetes 准入控制器成为在集群上部署预防性安全控制的完美候选者的原因。

介绍

---

在本文中，我们将探讨webhook如何在kubernetes中工作，更具体地说是关于ImagePolicyWebhook。因为有关它的​​kubernetes文档​​有点含糊不清，也没有真正的示例。

本文基于ImagePolicyWebhook 或 ValidatingAdmissionWebhook，实现准入控制器将拒绝所有使用带有latest标签的镜像的Pod。

准入控制器（Admission Control）在授权后对请求做进一步的验证或添加默认参数。不同于授权和认证只关心请求的用户和操作，准入控制还处理请求的内容，并且仅对创建、更新、删除或连接（如代理）等有效，而对读操作无效。

Admission Webhook： 准入控制器Webhook是准入控制插件的一种， 用于拦截所有向APISERVER发送的请求，并且可以修改请求或拒绝请求。

Admission webhook为开发者提供了非常灵活的插件模式，在kubernetes资源持久化之前，管理员通过程序可以对指定资源做校验、修改等操作。例如为资源自动打标签、pod设置默认SA，自动注入sidecar容器等。

相关Webhook准入控制器：

• MutatingAdmissionWebhook：修改资源，理论上可以监听并修改任何经过ApiServer处理的请求

使用kubectl其实就是将json的数据，提交到k8s的api当中，这个控制器就可以拦截的提交的这个请求，然后去修改其中的字段，然后返回回去，最后去持久化，这样就可以在你原有的json上添加东西，修改东西。

• ValidatingAdmissionWebhook：验证资源

• ImagePolicyWebhook：镜像策略，主要验证镜像字段是否满足条件

gitlab去感知有没有提交代码，然后去触发webhook，然后webhook去调用Jenkins的接口，然后去执行job，自动化的去部署。也就是提交代码之后自动部署，在这里webhook就类似于通知的机制一样，所以在这里处理一些轻量级的事件。

准入控制插件：比如部署了一个pod，这个pod你们可能有自己的规范，就希望在每个pod里面加一些字段，所有的部署资源都要加这个字段，你可能会通过一个平台将这些默认字段加上，这是没有问题的，如果没有平台要通过kubectl那么就需要要求所有的人都这样干，但是往往会有些遗漏，为了解决这些问题就是让其自动的去加，不让用户感知到有这么样的一个东西存在，那么webhook就起到了至关重要的作用，就可以拦截所有向k8s发送的请求进行区修改，比如给它打标签，设置默认的sa，比如默认给他添加容器，这些都可以去做。

所以添加默认字段都可以通过它去实现。 

可以看到准入控制在持久化数据之前，在鉴权之后。

ImagePolicyWebhook：去验证镜像仓库地址是否符合规范，如果是外网的dockerhub上面的镜像地址，那么就拒绝了，只允许你走私有仓库去部署，来去规避外部镜像的风险，或者镜像符合某种格式才去帮你部署。

ImagePolicyWebhook优势：

• 如果无法连接Webhook端点，可以指示API服务器拒绝镜像，这非常方便，但是它也会带来问题，例如核心Pod无法调度。

ImagePolicyWebhook劣势：

• 配置涉及更多内容，并且需要访问主节点或apiserver配置，文档尚不明确，可能难以进行更改，更新等。
• 部署并不是那么简单，你需要使用systemd进行部署或将其作为docker容器在主机中运行，更新dns等。

Extending Kubernetes admission controllers with webhooks

---

All of these features we’ve been describing so far are critical to run reliable and secure services.

However, as each organization has their own policies and default set of best practices, such highly specific controls may not be enough.

Fortunately, Kubernetes has you covered.

You can extend and customize the Kubernetes API functionality, without adding complexity to its base code, by using webhooks.

您可以使用 webhooks 扩展和自定义 Kubernetes API 功能，而不会增加其基本代码的复杂性。

The Kubernetes API server will call a registered webhook, which is a rather standard interface. This makes admission controllers easy to integrate with any third-party code.

Kubernetes API 服务器将调用注册的 webhook，这是一个相当标准的接口。这使得准入控制器易于与任何第三方代码集成。 

These three specific admission controllers let you expand the API functionality via webhooks:

• ​​ImagePolicyWebhook​​ to decide if an image should be admitted.
• ​​MutatingAdmissionWebhook​​ to modify a request.
• ​​ValidatingAdmissionWebhook​​ to decide whether the request should be allowed to run at all.

Implementing your own Kubernetes admission controller webhook

---

Now that we know how the admission controllers work, let’s see what marvelous things we can do with this power.

Let’s imagine we want to implement an ​​ImagePolicyWebhook​​.

For starters, we’ll need to make sure that the webhook is enabled when we start ​​kube-apiserver​​:

kube-apiserver --enable-admission-plugins=ImagePolicyWebhook …

We also need to configure the webhook ​​server​​ that will be called by the API server:

kube-apiserver --admission-control-config-file=admission-config.yaml …

 An example ​​admission-config.yaml​​​ contains an ​​AdmissionConfiguration​​ object:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: ImagePolicyWebhook
  configuration:
    imagePolicy:
      kubeConfigFile: <path-to-kubeconfig-file>
      allowTTL: 50
      denyTTL: 50
      retryBackoff: 500
      defaultAllow: true

And then, the webhook ​​server​​ is configured into a kubeconfig file:

# clusters refers to the remote service.
clusters:
- name: name-of-remote-imagepolicy-service
  cluster:
    certificate-authority: /path/to/ca.pem    # CA for verifying the remote service.
    server: https://images.example.com/policy # URL of remote service to query. Must use 'https'.
# users refers to the API server's webhook configuration.
users:
- name: name-of-api-server
  user:
    client-certificate: /path/to/cert.pem # cert for the webhook admission controller to use
    client-key: /path/to/key.pem          # key matching the cert

Please refer to ​​the ImagePolicyWebhook documentation​​ for a detailed description of the configuration options and alternatives.

我们现在可以编写我们的 HTTP 服务器来处理 webhook 请求。

We can now code our HTTP server to attend the webhook requests.

一旦 Kubernetes API 服务器收到部署请求，我们的 webhook 将收到类似于以下内容的 JSON 请求：

Once the Kubernetes API server receives a request for a deployment, our webhook will receive a JSON request similar to:

{
  "apiVersion":"imagepolicy.k8s.io/v1alpha1",
  "kind":"ImageReview",
  "spec":{
    "containers":[
      {
        "image":"myrepo/myimage:v1"
      }
    ],
    "namespace":"mynamespace"
  }
}

checking that image against your image scanner to ensure that it doesn’t contain vulnerabilities or misconfigurations.

In our example, that image does not conform with our company policies, so we would respond with the following JSON payload:

{
  "apiVersion": "imagepolicy.k8s.io/v1alpha1",
  "kind": "ImageReview",
  "status": {
    "allowed": false,
    "reason": "image runs as root"
  }
}

Because we have rejected one part of the request, the entire API request is immediately rejected, the image won’t be deployed, and an error is returned to the end-user.

准入控制器： ImagePolicyWebhook（验证资源的webhook）

---

这里需要去部署webhook服务器了，使用Kubectl经过前面的鉴权，授权，才到了准入控制器的系统里面，当准入控制器处理好之后才会持久化到ETCD里面。持久化ETCD资源才会正确的被创建。

在准入控制当中，拦截向apiserver发送的请求，它可以去修改请求和拒绝请求。

这里是可以去修改请求，webhook为apiserver提供了非常灵活的插件模式，在k8s持久化资源之前，可以通过webhook来对资源做校验和修改等操作。 

需要配置ImagePolicyWebhook，准入控制是需要一个启用的，启用准入控制插件。启用之后还需要指定配置文件，配置文件就指定了去连接镜像服务器的地址。（镜像服务器对验证地址是否满足固定的镜像仓库下载的，比如从dockerhub上拉取的就不让你去部署，这样去规避外部镜像的风险）

1、启用准入控制插件

还是在你的配置文件找到准入控制插件这一行

[root@k8s-master ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml 
 - --enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ImagePolicyWebhook

2、准备配置文件

#在这个目录下面存放着配置文件
[root@k8s-master kubernetes]# mkdir -p image-policy

[root@k8s-master ~]# kubectl get node
The connection to the server 192.168.179.102:6443 was refused - did you specify the right host or port?

[root@k8s-master ~]# docker ps -a | grep apiserver
91e09ac3ba60        1b74e93ece2f                                        "kube-apiserver --ad??   7 seconds ago       Exited (1) 5 seconds ago                           k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_4b056fb9a056a080fe3c2312b0e54715_8
c62ec62fb68f        registry.aliyuncs.com/google_containers/pause:3.2   "/pause"                 7 minutes ago       Up 7 minutes                                       k8s_POD_kube-apiserver-k8s-master_kube-system_4b056fb9a056a080fe3c2312b0e54715_0

[root@k8s-master ~]# docker logs 91e09ac3ba60
Flag --insecure-port has been deprecated, This flag will be removed in a future version.
I0722 11:20:25.147851       1 server.go:625] external host was not specified, using 192.168.179.102
I0722 11:20:25.148118       1 server.go:163] Version: v1.19.0
Error: failed to initialize admission: failed to read plugin config: unable to read admission control configuration from "/etc/kubernetes/image-policy/admission_configuration.yml" [open /etc/kubernetes/image-policy/admission_configuration.yml: no such file or directory]

[root@k8s-master ~]# ls /etc/kubernetes/image-policy/admission_configuration.yml
/etc/kubernetes/image-policy/admission_configuration.yml

可以看到宿主机有这个文件，但是还是提示没有这个文件。因为是kubeadm部署的，既然是容器部署的，那么宿主机上面的文件系统和容器当中的文件系统是隔离的。kube-apiserver是读取不到宿主机上面的文件的。

    - --enable-admission-plugins=NodeRestriction,ImagePolicyWebhook
    - --admission-control-config-file=/etc/kubernetes/image-policy/admission_configuration.yml


    - mountPath: /etc/kubernetes/image-policy
      name: image-policy
      readOnly: true

 volumes:
  - hostPath:
      path: /etc/kubernetes/image-policy
      type: Directory
    name: image-policy

[root@k8s-master ~]# docker ps -a | grep apiserver
ce1cba4ebebd        1b74e93ece2f                                        "kube-apiserver --ad??   20 seconds ago       Exited (1) 18 seconds ago                          k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_0365af43aa782395c87e303ff421139c_3
ba0290848da1        registry.aliyuncs.com/google_containers/pause:3.2   "/pause"                 About a minute ago   Up About a minute                                  k8s_POD_kube-apiserver-k8s-master_kube-system_0365af43aa782395c87e303ff421139c_0
ae653e9fe234        1b74e93ece2f                                        "kube-apiserver --ad??   4 minutes ago        Exited (1) 4 minutes ago                           k8s_kube-apiserver_kube-apiserver-k8s-master_kube-system_4b056fb9a056a080fe3c2312b0e54715_12
c62ec62fb68f        registry.aliyuncs.com/google_containers/pause:3.2   "/pause"                 18 minutes ago       Up 18 minutes                                      k8s_POD_kube-apiserver-k8s-master_kube-system_4b056fb9a056a080fe3c2312b0e54715_0
[root@k8s-master ~]# docker logs ce1cba4ebebd
Flag --insecure-port has been deprecated, This flag will be removed in a future version.
I0722 11:30:17.828946       1 server.go:625] external host was not specified, using 192.168.179.102
I0722 11:30:17.829186       1 server.go:163] Version: v1.19.0
Error: failed to initialize admission: couldn't init admission plugin "ImagePolicyWebhook": invalid configuration: [unable to read client-cert /etc/kubernetes/image-policy/apiserver-client.pem for api-server due to open /etc/kubernetes/image-policy/apiserver-client.pem: no such file or directory, unable to read client-key /etc/kubernetes/image-policy/apiserver-client-key.pem for api-server due to open /etc/kubernetes/image-policy/apiserver-client-key.pem: no such file or directory, cluster has no server defined]

不能是以latest去创建，得带上tag。验证资源的字段，提交的时候主要是去提交镜像相关的信息。

Kubernetes Admission controllers for image scanning

---

Implementing ​​image scanning​​​ should be one of your first steps when implementing ​​Kubernetes security​​. It is a confiable mechanism to detect vulnerabilities and misconfigurations.

If you are following ​​image scanning best practices​​, you probably secured your CI/CD pipeline and your registries already.

So why should you do image scanning with an admission controller too?

 There are two use cases you might be interested in:

Manual deployments can still occur. For example, you may skip protocol to do a manual deploy in a rush, or an attacker that gains access to your cluster deployed images skipping your image scanner.

There are also rules that rely on Kubernetes-specific context. For example, if you want to restrict one image to a specific namespace.

If you want to dig deeper into this topic, don’t miss our article “​​Shielding your Kubernetes runtime with an Admission Controller and image scanning​​.” In it, we cover image scanning with more detail and offer guidelines to help you implement such a solution.

Conclusion

---

Admission controllers are a powerful Kuberentes-native tool that help you enforce your organization policies.

Many basic features of Kubernetes are implemented with the help of Admission controllers, like limits and requests, and pod security policies.

Being able to expand the Kubernetes API server with webhooks opens the door to very interesting features, like implementing image scanning.

They are worth exploring!