k8s使用nfs动态存储
参考:https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client
1、Kubernetes集群管理员通过提供不同的存储类,可以满足用户不同的服务质量级别、备份策略和任意策略要求的存储需求。动态存储卷供应使用StorageClass进行实现,其允许存储卷按需被创建。如果没有动态存储供应,Kubernetes集群的管理员将不得不通过手工的方式类创建新的存储卷。通过动态存储卷,Kubernetes将能够按照用户的需要,自动创建其需要的存储。
基于StorageClass的动态存储供应整体过程如下图所示:
1)集群管理员预先创建存储类(StorageClass);
2)用户创建使用存储类的持久化存储声明(PVC:PersistentVolumeClaim);
3)存储持久化声明通知系统,它需要一个持久化存储(PV: PersistentVolume);
4)系统读取存储类的信息;
5)系统基于存储类的信息,在后台自动创建PVC需要的PV;
6)用户创建一个使用PVC的Pod;
7)Pod中的应用通过PVC进行数据的持久化;
8)而PVC使用PV进行数据的最终持久化处理。
2、example
0) 首先创建nfs服务
1)创建rbac授权,(如果集群开启了rbac),非default需要授权
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
2)创建nfs的nfs-client-provisioner
# cat deployment-nfs.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: docker.io/xhuaustc/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 1.14.0.4 # nfs服务ip
- name: NFS_PATH
value: /data/nfs/file # nfs路径
volumes:
- name: nfs-client-root
nfs:
server: 1.14.0.4 # nfs服务ip
path: /data/nfs/file # nfs路径
!!PS:10.10.229.43是nfs服务的监听地址,/data/opv是nfs共享的目录。
3)创建storageclass
# cat storageclass-nfs.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
annotations:
storageclass.kubernetes.io/is-default-class: "true" # 设置该storageclass为PVC创建时默认使用的存储机制
provisioner: fuseim.pri/ifs # 匹配deployment中的环境变量'PROVISIONER_NAME'
parameters:
archiveOnDelete: "true" # "false" 删除PVC时不会保留数据,"true"将保留PVC数据
reclaimPolicy: Delete
!!PS:fuseim.pri/ifs为上面deployment上创建的PROVISIONER_NAME。
4)创建PersistenetVolumeClaim
在存储类被正确创建后,就可以创建PersistenetVolumeClaim来请求StorageClass,而StorageClass将会为PersistenetVolumeClaim自动创建一个可用PersistentVolume。PersistenetVolumeClaim是对PersistenetVolume的声明,即PersistenetVolume为存储的提供者,而PersistenetVolumeClaim为存储的消费者。
# cat pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
volume.beta.kubernetes.io/storage-class: managed-nfs-storage
volume.beta.kubernetes.io/storage-provisioner: fuseim.pri/ifs
name: nfs-claim-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
!!PS:nfs-class为上面创建的storageclass的name,10Gi是设置的目录的大小。
5)执行命令
NAMESPACE=`oc project -q`
sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" rbac.yaml
oc create -f rbac.yaml
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner
oc create -f nfs-client-provisioner-deployment.yaml
oc create -f strageclass.yaml
oc create -f nfs-pvc.yaml
# 应用配置
spec:
containers:
- name: test-pod
volumeMounts:
- name: nfs-claim-pvc
mountPath: "/mnt"
volumes:
- name: nfs-pvc
persistentVolumeClaim:
claimName: nfs-claim-pvc
运维虐我千万遍,我对运维如初恋。
1、Kubernetes集群管理员通过提供不同的存储类,可以满足用户不同的服务质量级别、备份策略和任意策略要求的存储需求。动态存储卷供应使用StorageClass进行实现,其允许存储卷按需被创建。如果没有动态存储供应,Kubernetes集群的管理员将不得不通过手工的方式类创建新的存储卷。通过动态存储卷,Kubernetes将能够按照用户的需要,自动创建其需要的存储。
基于StorageClass的动态存储供应整体过程如下图所示:
1)集群管理员预先创建存储类(StorageClass);
2)用户创建使用存储类的持久化存储声明(PVC:PersistentVolumeClaim);
3)存储持久化声明通知系统,它需要一个持久化存储(PV: PersistentVolume);
4)系统读取存储类的信息;
5)系统基于存储类的信息,在后台自动创建PVC需要的PV;
6)用户创建一个使用PVC的Pod;
7)Pod中的应用通过PVC进行数据的持久化;
8)而PVC使用PV进行数据的最终持久化处理。
2、example
0) 首先创建nfs服务
1)创建rbac授权,(如果集群开启了rbac),非default需要授权
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: default
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
2)创建nfs的nfs-client-provisioner
# cat deployment-nfs.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: docker.io/xhuaustc/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: fuseim.pri/ifs
- name: NFS_SERVER
value: 1.14.0.4 # nfs服务ip
- name: NFS_PATH
value: /data/nfs/file # nfs路径
volumes:
- name: nfs-client-root
nfs:
server: 1.14.0.4 # nfs服务ip
path: /data/nfs/file # nfs路径
!!PS:10.10.229.43是nfs服务的监听地址,/data/opv是nfs共享的目录。
3)创建storageclass
# cat storageclass-nfs.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
annotations:
storageclass.kubernetes.io/is-default-class: "true" # 设置该storageclass为PVC创建时默认使用的存储机制
provisioner: fuseim.pri/ifs # 匹配deployment中的环境变量'PROVISIONER_NAME'
parameters:
archiveOnDelete: "true" # "false" 删除PVC时不会保留数据,"true"将保留PVC数据
reclaimPolicy: Delete
!!PS:fuseim.pri/ifs为上面deployment上创建的PROVISIONER_NAME。
4)创建PersistenetVolumeClaim
在存储类被正确创建后,就可以创建PersistenetVolumeClaim来请求StorageClass,而StorageClass将会为PersistenetVolumeClaim自动创建一个可用PersistentVolume。PersistenetVolumeClaim是对PersistenetVolume的声明,即PersistenetVolume为存储的提供者,而PersistenetVolumeClaim为存储的消费者。
# cat pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
annotations:
volume.beta.kubernetes.io/storage-class: managed-nfs-storage
volume.beta.kubernetes.io/storage-provisioner: fuseim.pri/ifs
name: nfs-claim-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
!!PS:nfs-class为上面创建的storageclass的name,10Gi是设置的目录的大小。
5)执行命令
NAMESPACE=`oc project -q`
sed -i'' "s/namespace:.*/namespace: $NAMESPACE/g" rbac.yaml
oc create -f rbac.yaml
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:$NAMESPACE:nfs-client-provisioner
oc create -f nfs-client-provisioner-deployment.yaml
oc create -f strageclass.yaml
oc create -f nfs-pvc.yaml
# 应用配置
spec:
containers:
- name: test-pod
volumeMounts:
- name: nfs-claim-pvc
mountPath: "/mnt"
volumes:
- name: nfs-pvc
persistentVolumeClaim:
claimName: nfs-claim-pvc