一 存储卷的概念和类型

为了保证数据的持久性 必须保证docker容器中的数据存储在容器外部 为了实现数据的持久性存储 在宿主机和容器内做映射 可以保证在容器的生命周期结束 数据依旧可以实现持久性存储 但是在k8s中 由于pod 分布在各个不同的节点上 并不能实现不同节点之间持久性数据的共享 并且在节点故障时 可能会导致数据的丢失 为此 k8s就引入了外部存储卷的功能

k8s的存储卷类型

[root@master service]# kubectl explain pod.spec.volumes  #查看支持的存储类型
KIND:     Pod
VERSION:  v1

FIELDS:
   awsElasticBlockStore	<Object>
     AWSElasticBlockStore represents an AWS Disk resource that is attached to a
     kubelet's host machine and then exposed to the pod. More info:
     https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore

   cephfs	<Object>
     CephFS represents a Ceph FS mount on the host that shares a pod's lifetime

   cinder	<Object>
     Cinder represents a cinder volume attached and mounted on kubelets host
     machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md

   configMap	<Object>
     ConfigMap represents a configMap that should populate this volume

   csi	<Object>
     CSI (Container Storage Interface) represents ephemeral storage that is
     handled by certain external CSI drivers (Beta feature).
常用的分类
  1. emptyDir（临时目录）:Pod删除，数据也会被清除，这种存储成为emptyDir，用于数据的临时存储。多容器数据互通
  2. hostPath (宿主机目录映射):
  3. 本地的SAN (iSCSI,FC)、NAS(nfs,cifs,http)存储
  4. 分布式存储（glusterfs，rbd，cephfs）
  5. 云存储 （EBS，Azure Disk）

二、emptyDir存储卷演示

一个emptyDir 第一次创建是在一个pod被指定到具体node的时候，并且会一直存在在pod的生命周期当中，正如它的名字一样，它初始化是一个空的目录，pod中的容器都可以读写这个目录，这个目录可以被挂在到各个容器相同或者不相同的的路径下。当一个pod因为任何原因被移除的时候，这些数据会被永久删除。注意：一个容器崩溃了不会导致数据的丢失，因为容器的崩溃并不移除pod.

emptyDir 磁盘的作用：
（1）普通空间，基于磁盘的数据存储
（2）作为从崩溃中恢复的备份点
（3）存储那些那些需要长久保存的数据，例web服务中的数据
默认的，emptyDir 磁盘会存储在主机所使用的媒介上，可能是SSD，或者网络硬盘，这主要取决于你的环境。当然，我们也可以将emptyDir.medium的值设置为Memory来告诉Kubernetes 来挂在一个基于内存的目录tmpfs，因为tmpfs速度会比硬盘块度了，但是，当主机重启的时候所有的数据都会丢失。

[root@k8s-master ~]# kubectl explain pods.spec.volumes.emptyDir  #查看emptyDir存储定义
[root@k8s-master ~]# kubectl explain pods.spec.containers.volumeMounts  #查看容器挂载方式
[root@k8s-master ~]# mkdir volumes && cd volumes
[root@k8s-master volumes]# vim pod-vol-demo.yaml   #创建emptyDir的清单
apiVersion: v1
kind: Pod
metadata:
  name: pod-demo
  namespace: default
  labels:
    app: myapp
    tier: frontend
  annotations:
    magedu.com/create-by:"cluster admin"
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    imagePullPolicy: IfNotPresent
    ports:
    - name: http
      containerPort: 80
    volumeMounts:    #在容器内定义挂载存储名称和挂载路径
    - name: html
      mountPath: /usr/share/nginx/html/
  - name: busybox
    image: busybox:latest
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: html
      mountPath: /data/    #在容器内定义挂载存储名称和挂载路径
    command: ['/bin/sh','-c','while true;do echo $(date) >> /data/index.html;sleep 2;done']
  volumes:  #定义存储卷
  - name: html    #定义存储卷名称  
    emptyDir: {}  #定义存储卷类型

[root@master volume]# kubectl apply -f empryDir_pod.yaml 
pod/pod-demo created
[root@master volume]# kubectl get pod -l app=myapp
NAME       READY   STATUS              RESTARTS   AGE
pod-demo   0/2     ContainerCreating   0          13s
[root@master volume]# kubectl get pod -l app=myapp -o wide
NAME       READY   STATUS              RESTARTS   AGE   IP       NODE    NOMINATED NODE   READINESS GATES
pod-demo   0/2     ContainerCreating   0          16s   <none>   node2   <none>           <none>
[root@master volume]# kubectl get pod -l app=myapp -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP             NODE    NOMINATED NODE   READINESS GATES
pod-demo   2/2     Running   0          86s   172.7.104.28   node2   <none>           <none>
[root@master volume]# curl 172.7.104.28
Wed Aug 11 09:55:47 UTC 2021
Wed Aug 11 09:55:49 UTC 2021
Wed Aug 11 09:55:51 UTC 2021
Wed Aug 11 09:55:53 UTC 2021
Wed Aug 11 09:55:55 UTC 2021
Wed Aug 11 09:55:57 UTC 2021
Wed Aug 11 09:55:59 UTC 2021
Wed Aug 11 09:56:01 UTC 2021
Wed Aug 11 09:56:03 UTC 2021
Wed Aug 11 09:56:05 UTC 2021
Wed Aug 11 09:56:07 UTC 2021
Wed Aug 11 09:56:09 UTC 2021
Wed Aug 11 09:56:11 UTC 2021
Wed Aug 11 09:56:13 UTC 2021

三、hostPath存储卷

hostPath宿主机路径，就是把pod所在的宿主机之上的脱离pod中的容器名称空间的之外的宿主机的文件系统的某一目录和pod建立关联关系，在pod删除时，存储数据不会丢失。

（1）查看hostPath存储类型定义
[root@k8s-master volumes]# kubectl explain pods.spec.volumes.hostPath  
KIND:     Pod
VERSION:  v1

RESOURCE: hostPath <Object>

DESCRIPTION:
     HostPath represents a pre-existing file or directory on the host machine
     that is directly exposed to the container. This is generally used for
     system agents or other privileged things that are allowed to see the host
     machine. Most containers will NOT need this. More info:
     https://kubernetes.io/docs/concepts/storage/volumes#hostpath

     Represents a host path mapped into a pod. Host path volumes do not support
     ownership management or SELinux relabeling.

FIELDS:
   path	<string> -required-  #指定宿主机的路径
     Path of the directory on the host. If the path is a symlink, it will follow
     the link to the real path. More info:
     https://kubernetes.io/docs/concepts/storage/volumes#hostpath

   type	<string>
     Type for HostPath Volume Defaults to "" More info:
     https://kubernetes.io/docs/concepts/storage/volumes#hostpath

type：
DirectoryOrCreate  宿主机上不存在创建此目录  
Directory 必须存在挂载目录  
FileOrCreate 宿主机上不存在挂载文件就创建  
File 必须存在文件  

(2)清单定义
apiVersion: v1
kind: Pod
metadata:
  name: volumes-hostpath-demo
spec:
  containers:
  - name: filebeat
    image: ikubernetes/filebeat:5.6.7-alpine
    env:
    - name: REDIS_HOST
      value: redis.ilinux.io:6379
    - name: LOG_LEVEL
      value: info
    volumeMounts:
    - name: varlog
      mountPath: /var/log
    - name: socket
      mountPath: /var/run/docker.sock
    - name: varlibdockercontainers
      mountPath: /var/lib/docker/containers
      readOnly: true
  volumes:
  - name: varlog
    hostPath:
      path: /var/log
  - name: varlibdockercontainers
    hostPath:
      path: /var/lib/docker/containers
  - name: socket
    hostPath:
      path: /var/run/docker.sock

[root@master volume]# kubectl get pod 
NAME                        READY   STATUS    RESTARTS   AGE
volumes-hostpath-demo       1/1     Running   0          7m49s
[root@master volume]# kubectl exec -it volumes-hostpath-demo -- bash
OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown
command terminated with exit code 126
[root@master volume]# kubectl exec -it volumes-hostpath-demo -- sh
/ # ls /var/log/
anaconda                  chrony                    grubby                    ntpstats                  tallylog                  vmware-vmsvc-root.log
audit                     containers                grubby_prune_debug        pods                      tuned                     vmware-vmsvc.log
boot.log                  cron                      lastlog                   rhsm                      vmware-network.1.log      vmware-vmtoolsd-root.log
boot.log-20210804         cron-20210808             maillog                   secure                    vmware-network.2.log      wtmp
boot.log-20210805         dmesg                     maillog-20210808          secure-20210808           vmware-network.3.log      yum.log
btmp                      dmesg.old                 messages                  spooler                   vmware-network.log
calico                    firewalld                 messages-20210808         spooler-20210808          vmware-vgauthsvc.log.0
/ # cat /var/log/yum.log 
Apr 19 12:38:24 Installed: 1:bash-completion-2.1-6.el7.noarch
Apr 19 12:38:40 Installed: 2:vim-filesystem-7.4.160-5.el7.x86_64
Apr 19 12:38:42 Installed: 2:vim-common-7.4.160-5.el7.x86_64
Apr 19 12:38:42 Installed: 14:libpcap-1.5.3-11.el7.x86_64
Apr 19 12:38:42 Installed: gpm-libs-1.20.7-5.el7.x86_64
Apr 19 12:38:42 Installed: 2:vim-enhanced-7.4.160-5.el7.x86_64
Apr 19 12:38:43 Installed: 14:tcpdump-4.9.2-3.el7.x86_64
Apr 19 12:38:43 Installed: lsof-4.87-6.el7.x86_64
Apr 19 12:38:43 Installed: lrzsz-0.12.20-36.el7.x86_64

四、nfs共享存储卷演示

nfs使的我们可以挂在已经存在的共享到的我们的Pod中，和emptyDir不同的是，emptyDir会被删除当我们的Pod被删除的时候，但是nfs不会被删除，仅仅是解除挂在状态而已，这就意味着NFS能够允许我们提前对数据进行处理，而且这些数据可以在Pod之间相互传递.并且，nfs可以同时被多个pod挂在并进行读写

（1）在stor01节点上安装nfs，并配置nfs服务
[root@stor01 ~]# yum install -y nfs-utils  ==》192.168.56.14
[root@stor01 ~]# mkdir /data/volumes -pv
[root@stor01 ~]# vim /etc/exports
/data/volumes 192.168.56.0/24(rw,no_root_squash)
[root@stor01 ~]# systemctl start nfs
[root@stor01 ~]# showmount -e
Export list for stor01:
/data/volumes 192.168.56.0/24

（2）在node01和node02节点上安装nfs-utils，并测试挂载
[root@k8s-node01 ~]# yum install -y nfs-utils
[root@k8s-node02 ~]# yum install -y nfs-utils
[root@k8s-node02 ~]# mount -t nfs stor01:/data/volumes /mnt
[root@k8s-node02 ~]# mount
......
stor01:/data/volumes on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.56.13,local_lock=none,addr=192.168.56.14)
[root@k8s-node02 ~]# umount /mnt/

（3）创建nfs存储卷的使用清单
[root@k8s-master volumes]# cp pod-hostpath-vol.yaml pod-nfs-vol.yaml
[root@k8s-master volumes]# vim pod-nfs-vol.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-vol-nfs
  namespace: default
spec:
  containers:
  - name: myapp
    image: ikubernetes/myapp:v1
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html
  volumes:
    - name: html
      nfs:
        path: /data/volumes
        server: stor01
[root@k8s-master volumes]# kubectl apply -f pod-nfs-vol.yaml 
pod/pod-vol-nfs created
[root@k8s-master volumes]# kubectl get pods -o wide
NAME                     READY     STATUS    RESTARTS   AGE       IP            NODE
pod-vol-nfs              1/1       Running   0          21s       10.244.2.38   k8s-node02

（3）在nfs服务器上创建index.html
[root@stor01 ~]# cd /data/volumes
[root@stor01 volumes ~]# vim index.html
<h1> nfs stor01</h1>
[root@k8s-master volumes]# curl 10.244.2.38
<h1> nfs stor01</h1>
[root@k8s-master volumes]# kubectl delete -f pod-nfs-vol.yaml   #删除nfs相关pod，再重新创建，可以得到数据的持久化存储
pod "pod-vol-nfs" deleted
[root@k8s-master volumes]# kubectl apply -f pod-nfs-vol.yaml