K8S安全框架

k8s的权限控制/安全框架主要由3个阶段控制，每个阶段都支持插件方式，通过Api-server配置来启用插件

  • 鉴权 Authentication
  • 授权 Authorization
  • 准入控制 Admission Control

k8S权限访问控制_安全


鉴权 Authentication

鉴权主要是为了验证客户端身份（用户/来访者），k8s Apiserver提供三种客户端身份认证：

  • Https 证书认证：基于CA证书签名的数字证书认证（kubeconfig）
  • Http Token认证：通过一个token来识别用户身份（serviceAccount）
  • Http Base认证： 用户名+密码的方式 1.19版弃用

授权 Authorization

RBAC （Role-Based Access Control）基于角色的访问控制，主要负责用户的授权工作。根绝用户的权限决定允许访问还是拒绝

常见的授权维度为：

  • user: 用户名
  • userGroup: 用户分组
  • 资源 例如 pod services deployment
  • 资源操作方式： get list create update delete
  • 命名空间
  • API组

准入控制 Admission Control

主要是为了功能的解耦和功能的可插拔性，决定那个功能开启，那个功能关闭

Admission Control实际上是一个准入控制器插件列表，发送到Apiserver的请求都需要经过这个列表中的每个准入控制器插件的检查，检查不通过，则拒绝请求。

启用一个准入控制器：

kube-apiserver --enable-admission-plugins=NamespaceLifecycle, LimitRanger...

关闭一个准入控制器：

kube-apiserver --disable-admission-plugins=NamespaceLifecycle, LimitRanger...

查看默认启用：

kubectl exec kube-apiserver-k8s-master -n kube-system -- kube-apiserver -h |grep enable-admission-plugins

# 显示结果如下：
--enable-admission-plugins strings       

# 默认启用的
admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). 

# 其他
Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.


基于角色的权限访问控制 RBAC

在k8s中要通过外部插件新增和操作API对象，需要通过RBAC授权。k8s中的所有API对象都存储在ETCD中，对这些API对象的操作一定是要通过kube-apiserver实现，很重要的原因是需要apiserver授权。k8s的默认策略，可动态配置，即修改立即生效。

主体（subject）

即给谁授权

  • User 用户
  • Group 用户组
  • ServiceAccount 服务账号（pod访问K8s Api）

角色

决定了都有什么权限

  • Role 授权特定命名空间的访问权限
  • ClusterRole 授权所有命名空间的访问权限

角色绑定

  • RoleBinding 将角色绑定到主体
  • ClusterRoleBinding 将集群角色绑定到主体

注： RoleBinding在指定命名空间执行授权，ClusterRoleBinding在集群范围内进行授权

示例1 给用户授权

例如新入职人员，只希望对方先熟悉下k8s集群，只给default命名空间的相关查看权限。步骤如下：

  • 用k8s CA签发客户端证书
  • 生成kubeconfig授权文件
  • 创建RBAC授权策略
  • 指定kubeconfig文件测试权限

文件准备：https://github.com/yangqiqigithub/k8s-rbac

文件解读：

[root@k8s-master rbac]# cat cert.sh 

cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {  
      "kubernetes": {
        "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ],
        "expiry": "87600h"
      }
    }
  }
}
EOF

cat > aliang-csr.json <<EOF
{
  "CN": "aliang", #用户名
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=/etc/kubernetes/pki/ca.crt -ca-key=/etc/kubernetes/pki/ca.key -cnotallow=ca-config.json -profile=kubernetes aliang-csr.json | cfssljson -bare aliang
[root@k8s-master rbac]# cat kubeconfig.sh 

kubectl config set-cluster kubernetes \
  --certificate-authority=/etc/kubernetes/pki/ca.crt \
  --embed-certs=true \
  --server=https://192.168.18.125:6443 \ # APIsever地址
  --kubecnotallow=aliang.kubeconfig
 
# 设置客户端认证
kubectl config set-credentials aliang \
  --client-key=aliang-key.pem \
  --client-certificate=aliang.pem \
  --embed-certs=true \
  --kubecnotallow=aliang.kubeconfig

# 设置默认上下文
kubectl config set-context kubernetes \
  --cluster=kubernetes \
  --user=aliang \
  --kubecnotallow=aliang.kubeconfig

# 设置当前使用配置
kubectl config use-context kubernetes --kubecnotallow=aliang.kubeconfig
[root@k8s-master rbac]# cat rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: ["","apps"] # api组 
  # 例如 apps组 空值代表是核心API组，像namespace pod service pv pvc 都在里边
  resources: ["pods","services","deployments"] # 资源名称 注意用复数
  verbs: ["get", "watch", "list"] # 资源操作方式
- apiGroups: [""] # 可以设定多个规则叠加
  resources: ["pods"]
  verbs: ["delete"]
---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User # 
  name: aliang
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader #
  apiGroup: rbac.authorization.k8s.io
[root@k8s-master rbac]#

查看资源和API组的方式

k8S权限访问控制_安全_02

操作方式

[root@k8s-master ~]# cd rbac/
[root@k8s-master rbac]# ls
aliang-csr.json  ca-config.json  cert.sh  kubeconfig.sh  rbac.yaml
# 用k8s CA签发客户端证书
[root@k8s-master rbac]# bash cert.sh 
2023/02/27 16:18:02 [INFO] generate received request
2023/02/27 16:18:02 [INFO] received CSR
2023/02/27 16:18:02 [INFO] generating key: rsa-2048
2023/02/27 16:18:02 [INFO] encoded CSR
2023/02/27 16:18:02 [INFO] signed certificate with serial number 566522034020276577497226021802714830130514085279
2023/02/27 16:18:02 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").

[root@k8s-master rbac]# ls
aliang.csr  aliang-csr.json  aliang-key.pem  aliang.pem  ca-config.json  cert.sh  kubeconfig.sh  rbac.yaml

# 生成kubeconfig授权文件
[root@k8s-master rbac]# bash kubeconfig.sh 
Cluster "kubernetes" set.
User "aliang" set.
Context "kubernetes" created.
Switched to context "kubernetes".

# 生成aliang.kubeconfig
[root@k8s-master rbac]# ls
aliang.csr  aliang-csr.json  aliang-key.pem  aliang.kubeconfig  aliang.pem  ca-config.json  cert.sh  kubeconfig.sh  rbac.yaml

到目前为止是有了一个用户，但是还没给这个用户授权，先测试一下这个用户的权限
[root@k8s-master rbac]# kubectl get pods --kubecnotallow=aliang.kubeconfig
Error from server (Forbidden): pods is forbidden: User "aliang" cannot list resource "pods" in API group "" in the namespace "default"
会看到现在没有权限

# 创建RBAC授权策略
[root@k8s-master rbac]# kubectl apply -f rbac.yaml 
role.rbac.authorization.k8s.io/pod-reader created
rolebinding.rbac.authorization.k8s.io/read-pods created
# 指定kubeconfig文件测试
[root@k8s-master rbac]# kubectl get pods --kubecnotallow=aliang.kubeconfig
No resources found in default namespace.

示例2 给serviceAccount授权

serviceAccount简称SA，服务账号，用于让集群内的Pod访问k8s的API

例如：如下为一个服务账号创建可查看、创建pod deployment的权限

命令行创建方式

# 创建名称空间
kubectl create namespace app-team1
# 创建服务账号
kubectl create serviceaccount cicd-token -n app-team1
# 创建集群角色
kubectl create clusterrole  deployment-clusterrole -n app-team1  --verb=create,get,list --resource=pods,deployments 
# 将服务账号绑定
kubectl create rolebinding cicd-token -n app-team1 --clusterrole=deployment-clusterrole--serviceaccount=app-team1:cicd-token 
# 测试服务账号权限
kubectl --as=system:serviceaccount:app-team1:cicd-token get pods -n app-team1

yaml创建方式

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cide-token
  namespace: app-team1
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  namespace: app-team1
  name: deployment-clusterrole
rules:
- apiGroups: ["apps"] 
  resources: ["pods","deployments"] 
  verbs: ["get", "watch", "list","create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-token
  namespace: app-team1
subjects:
- kind: ServiceAccount
  name: cicd-toke 
  namespace: app-team1
roleRef:
  kind: ClusterRole
  name: deployment-clusterrole
  apiGroup: rbac.authorization.k8s.io

查看相应的serviceAccount的权限

[root@k8s-master job]# kubectl get clusterrole # 查看clusterrole
NAME                                                                   CREATED AT
admin                                                                  2023-02-21T04:48:59Z
calico-kube-controllers                                                2023-02-21T13:22:58Z
calico-node                                                            2023-02-21T13:22:58Z
cluster-admin                                                          2023-02-21T04:48:59Z
edit                                                                   2023-02-21T04:48:59Z

# 查看具体的admin这个clusterrole拥有的权限
[root@k8s-master job]# kubectl describe clusterrole admin 
Name:         admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
  Resources                                       Non-Resource URLs  Resource Names  Verbs
  ---------                                       -----------------  --------------  -----
  rolebindings.rbac.authorization.k8s.io          []                 []              [create delete deletecollection get list patch update watch]
  roles.rbac.authorization.k8s.io                 []                 []              [create delete deletecollection get list patch update watch]
  configmaps                                      []                 []              [create delete deletecollection patch update get list watch]
  events                                          []                 []              [create delete deletecollection patch update get list watch]
  persistentvolumeclaims                          []                 []              [create delete deletecollection patch update get list watch]
  pods                                            []                 []              [create delete deletecollection patch update get list watch]
  replicationcontrollers/scale                    []                 []              [create delete deletecollection patch update get list watch]
  replicationcontrollers                          []                 []              [create delete deletecollection patch update get list watch]
  services                                        []                 []              [create delete deletecollection patch update get list watch]
  daemonsets.apps                                 []                 []              [create delete deletecollection patch update get list watch]

k8s预先定义好的cluserrole

  • cluster-admin
  • admin
  • edit
  • view