基于kubeadm部署K8S集群一、环境准备

操作系统

IP地址

主机名

组件

CentOS7.5

192.168.200.111

docker-server1

kubeadmkubeletkubectldocker-ce

CentOS7.5

192.168.200.112

docker-server2

kubeadmkubeletkubectldocker-ce

CentOS7.5

192.168.200.113

docker-server3

kubeadmkubeletkubectldocker-ce

注意:所有主机配置推荐CPU2C+  Memory2G+

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_Kubernetes


1.1、主机初始化配置

所有主机配置禁用防火墙和selinux,配置主机名

[root@localhost ~]# iptables -F

[root@localhost ~]# setenforce 0

[root@localhost ~]# systemctl stop firewalld

 

不同主机名称不同(分别为docker-server2docker-server3

[root@localhost ~]# hostname docker-server1

[root@localhost ~]# bash

 

[root@docker-server1 ~]# vim /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

192.168.200.111     docker-server1

192.168.200.112    docker-server1

192.168.200.113    docker-server1

 

[root@docker-server1 ~]# scp /etc/hosts 192.168.200.112:/etc/

[root@docker-server1 ~]# scp /etc/hosts 192.168.200.113:/etc/

 

禁用swap虚拟内存

[root@docker-server1 ~]# vim /etc/fstab

#/dev/mapper/centos-swap swap swap    defaults        0 0    #禁用swap自动挂载

 

[root@docker-server1 ~]# swapoff /dev/mapper/centos-swap

[root@docker-server1 ~]# free -h

              total        used        free      shared  buff/cache   available

Mem:           1.9G        749M        101M         10M        1.1G        906M

Swap:            0B          0B          0B

1.2、部署docker环境

安装docker-ce(所有主机配置)

[root@docker-server1 ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

[root@docker-server1 ~]# yum -y install yum-utils device-mapper-persistent-data lvm2

[root@docker-server1 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

[root@docker-server1 ~]# ls /etc/yum.repos.d/

backup  CentOS-Base.repo  CentOS-Media.repo  docker-ce.repo

 

[root@docker-server1 ~]# yum -y install docker-ce

[root@docker-server1 ~]# systemctl start docker && systemctl enable docker

 

阿里云镜像加速器(所有主机配置)

[root@docker-server1 ~]# cat << END > /etc/docker/daemon.json

{

        "registry-mirrors":[ "https://nyakyfun.mirror.aliyuncs.com" ]

}

END

[root@docker-server1 ~]# systemctl daemon-reload

[root@docker-server1 ~]# systemctl restart docker

 

[root@docker-server1 ~]# docker version

Client: Docker Engine - Community

 Version:           19.03.5

 API version:       1.40

 Go version:        go1.12.12

 Git commit:        633a0ea

 Built:             Wed Nov 13 07:25:41 2019

 OS/Arch:           linux/amd64

 Experimental:      false

 

Server: Docker Engine - Community

 Engine:

  Version:          19.03.5

  API version:      1.40 (minimum version 1.12)

  Go version:       go1.12.12

  Git commit:       633a0ea

  Built:            Wed Nov 13 07:24:18 2019

  OS/Arch:          linux/amd64

  Experimental:     false

 containerd:

  Version:          1.2.10

  GitCommit:        b34a5c8af56e510852c35414db4c1f4fa6172339

 runc:

  Version:          1.0.0-rc8+dev

  GitCommit:        3e425f80a8c931f88e6d94a8c831b9d5aa481657

 docker-init:

  Version:          0.18.0

  GitCommit:        fec3683

 

1.3、相关组件版本说明

组件

版本

说明

kubernetes

1.17.3

主程序

docker

19.03.5

容器

flannel

0.11.0

网络插件

etcd

3.3.15

数据库

coredns

1.6.2

dns组件

kubernetes-dashboard

2.0.0-beta5

web界面

 

二、部署kubernetes集群

2.1、组件介绍

三个节点都需要安装下面三个组件

l  kubeadm:安装工具,使所有的组件都会以容器的方式运行

l  kubectl:客户端连接K8S API工具

l  kubelet:运行在node节点,用来启动容器的工具

2.2、配置阿里云yum

所有主机配置yum

推荐使用阿里云的yum源安装:

[root@docker-server1 ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

EOF

 

[root@docker-server1 ~]# ls /etc/yum.repos.d/

backup  CentOS-Base.repo  CentOS-Media.repo  docker-ce.repo  kubernetes.repo

 

K8S目前最新版本是:1.17.3

[root@docker-server1 ~]# yum -y info kubeadm

已加载插件:fastestmirror, langpacks

Loading mirror speeds from cached hostfile

 * base: mirrors.aliyun.com

 * extras: mirrors.aliyun.com

 * updates: mirrors.aliyun.com

kubernetes/signature                                                                                |  454 B  00:00:00    

https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg 检索密钥

导入 GPG key 0xA7317B0F:

 用户ID     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"

 指纹       : d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f

 来自       : https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg

https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg 检索密钥

kubernetes/signature                                                                                | 1.4 kB  00:00:01 !!!

kubernetes/primary                                                                                  |  64 kB  00:00:00    

kubernetes                                                                                                         469/469

可安装的软件包

名称    kubeadm

架构    x86_64

版本    1.17.3

发布    0

大小    8.7 M

    kubernetes

简介    Command-line utility for administering a Kubernetes cluster.

网址    https://kubernetes.io

协议    ASL 2.0

描述    Command-line utility for administering a Kubernetes cluster.

2.3、安装kubelet kubeadm kubectl

所有主机配置

[root@docker-server1 ~]# yum install -y kubelet kubeadm kubectl

[root@docker-server1 ~]# rpm -qa | grep kube*

kubeadm-1.17.3-0.x86_64

kubelet-1.17.3-0.x86_64

kubernetes-cni-0.7.5-0.x86_64

kubectl-1.17.3-0.x86_64

[root@docker-server1 ~]# systemctl enable kubelet && systemctl start kubelet

2.4、加载内核模块

所有主机配置

[root@docker-server1 ~]# cat > /etc/sysctl.d/k8s.conf << EOF

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_nonlocal_bind = 1

net.ipv4.ip_forward = 1

vm.swappiness=0

EOF

[root@docker-server1 ~]# sysctl –system

 

[root@docker-server1 ~]# vim /etc/sysctl.conf

net.ipv4.ip_forward = 1

[root@docker-server1 ~]# sysctl -p

 

[root@docker-server1 ~]# cat > /etc/sysconfig/modules/ipvs.modules <<EOF

#!/bin/bash

modprobe -- br_netfilter

modprobe -- ip_vs

modprobe -- ip_vs_rr

modprobe -- ip_vs_wrr

modprobe -- ip_vs_sh

modprobe -- nf_conntrack_ipv4

EOF

 

[root@docker-server1 ~]# chmod 755 /etc/sysconfig/modules/ipvs.modules

[root@docker-server1 ~]# bash /etc/sysconfig/modules/ipvs.modules

[root@docker-server1 ~]# lsmod | grep -E "ip_vs|nf_conntrack_ipv4"

ip_vs_sh               12688  0

ip_vs_wrr              12697  0

ip_vs_rr               12600  0

ip_vs                 141432  6 ip_vs_rr,ip_vs_sh,ip_vs_wrr

nf_conntrack_ipv4      15053  2

nf_defrag_ipv4         12729  1 nf_conntrack_ipv4

nf_conntrack          133053  7 ip_vs,nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntr

ack_ipv4libcrc32c              12644  4 xfs,ip_vs,nf_nat,nf_conntrack

 

2.5、配置kubeadm-config.yaml

kubeadm-config.yaml组成部署说明:

l  InitConfiguration:用于定义一些初始化配置,如初始化使用的token以及apiserver地址等;

l  ClusterConfiguration:用于定义apiserveretcdnetworkschedulercontroller-managermaster组件相关配置项

l  KubeletConfiguration:用于定义kubelet组件相关的配置项

l  KubeProxyConfiguration:用于定义kube-proxy组件相关的配置项

 

master节点安装,master 定于为192.168.200.111,通过如下指令创建默认的kubeadm-config.yaml文件:

[root@docker-server1 ~]# kubeadm config print init-defaults  > kubeadm-config.yaml

W0212 21:18:11.685591    2403 validation.go:28] Cannot validate kube-proxy config - no validator is available

W0212 21:18:11.685648    2403 validation.go:28] Cannot validate kubelet config - no validator is available

 

kubeadm-config.yaml配置

[root@docker-server1 ~]# vim kubeadm-config.yaml

apiVersion: kubeadm.k8s.io/v1beta2

bootstrapTokens:

- groups:

  - system:bootstrappers:kubeadm:default-node-token

  token: abcdef.0123456789abcdef

  ttl: 24h0m0s

  usages:

  - signing

  - authentication

kind: InitConfiguration

localAPIEndpoint:

  advertiseAddress: 192.168.200.111  #master节点的IP

  bindPort: 6443

nodeRegistration:

  criSocket: /var/run/dockershim.sock

  name: 192.168.200.111                         #修改为IP地址,如果使用域名,必须保证解析正常

  taints:

  - effect: NoSchedule

    key: node-role.kubernetes.io/master

---

apiServer:

  timeoutForControlPlane: 4m0s

apiVersion: kubeadm.k8s.io/v1beta2

certificatesDir: /etc/kubernetes/pki

clusterName: kubernetes

controllerManager: {}

dns:

  type: CoreDNS

etcd:

  local:

    dataDir: /var/lib/etcd                       #etcd容器的目录挂载到本地的/var/lib/etcd目录下,防止数据丢失

imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers          #镜像仓库地址,可以修改为gcr.azk8s.cn/google_containers

kind: ClusterConfiguration

kubernetesVersion: v1.17.3                     #Kubernetes软件版本

networking:

  dnsDomain: cluster.local

  serviceSubnet: 10.96.0.0/12

  podSubnet: 10.244.0.0/16        #添加这个内容

scheduler: {}

 

2.6、安装master节点

可以预先下载镜像

[root@docker-server1 ~]# kubeadm config images pull --config kubeadm-config.yaml

 

安装matser节点

[root@docker-server1 ~]# kubeadm init --config kubeadm-config.yaml

W0214 15:07:53.469593   65073 validation.go:28] Cannot validate kube-proxy config - no validator is available

W0214 15:07:53.469677   65073 validation.go:28] Cannot validate kubelet config - no validator is available

[init] Using Kubernetes version: v1.17.3

[preflight] Running pre-flight checks

         [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd

". Please follow the guide at https://kubernetes.io/docs/setup/cri/[preflight] Pulling images required for setting up a Kubernetes cluster

[preflight] This might take a minute or two, depending on the speed of your internet connection

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

[kubelet-start] Starting the kubelet

[certs] Using certificateDir folder "/etc/kubernetes/pki"

[certs] Generating "ca" certificate and key

[certs] Generating "apiserver" certificate and key

[certs] apiserver serving cert is signed for DNS names [192.168.200.111 kubernetes kubernetes.default kubernetes.default.sv

c kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.200.111][certs] Generating "apiserver-kubelet-client" certificate and key

[certs] Generating "front-proxy-ca" certificate and key

[certs] Generating "front-proxy-client" certificate and key

[certs] Generating "etcd/ca" certificate and key

[certs] Generating "etcd/server" certificate and key

[certs] etcd/server serving cert is signed for DNS names [192.168.200.111 localhost] and IPs [192.168.200.111 127.0.0.1 ::1

][certs] Generating "etcd/peer" certificate and key

[certs] etcd/peer serving cert is signed for DNS names [192.168.200.111 localhost] and IPs [192.168.200.111 127.0.0.1 ::1]

[certs] Generating "etcd/healthcheck-client" certificate and key

[certs] Generating "apiserver-etcd-client" certificate and key

[certs] Generating "sa" key and public key

[kubeconfig] Using kubeconfig folder "/etc/kubernetes"

[kubeconfig] Writing "admin.conf" kubeconfig file

[kubeconfig] Writing "kubelet.conf" kubeconfig file

[kubeconfig] Writing "controller-manager.conf" kubeconfig file

[kubeconfig] Writing "scheduler.conf" kubeconfig file

[control-plane] Using manifest folder "/etc/kubernetes/manifests"

[control-plane] Creating static Pod manifest for "kube-apiserver"

[control-plane] Creating static Pod manifest for "kube-controller-manager"

W0214 15:12:35.410900   65073 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,R

BAC"[control-plane] Creating static Pod manifest for "kube-scheduler"

W0214 15:12:35.413190   65073 manifests.go:214] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,R

BAC"[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"

[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/ma

nifests". This can take up to 4m0s[apiclient] All control plane components are healthy after 34.504759 seconds

[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace

[kubelet] Creating a ConfigMap "kubelet-config-1.17" in namespace kube-system with the configuration for the kubelets in th

e cluster[upload-certs] Skipping phase. Please see --upload-certs

[mark-control-plane] Marking the node 192.168.200.111 as control-plane by adding the label "node-role.kubernetes.io/master=

''"[mark-control-plane] Marking the node 192.168.200.111 as control-plane by adding the taints [node-role.kubernetes.io/master

:NoSchedule][bootstrap-token] Using token: abcdef.0123456789abcdef

[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles

[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term cer

tificate credentials[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstra

p Token[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster

[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace

[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key

[addons] Applied essential addon: CoreDNS

[addons] Applied essential addon: kube-proxy

 

Your Kubernetes control-plane has initialized successfully!

 

To start using your cluster, you need to run the following as a regular user:

 

#安装完成,在master上操作

  mkdir -p $HOME/.kube

  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

  sudo chown $(id -u):$(id -g) $HOME/.kube/config

 

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

  https://kubernetes.io/docs/concepts/cluster-administration/addons/

 

Then you can join any number of worker nodes by running the following on each as root:

 

#用于添加node节点  

kubeadm join 192.168.200.111:6443 --token abcdef.0123456789abcdef \

    --discovery-token-ca-cert-hash sha256:c2d6067d5c3b12118275958dee222226d09a89fc5fb559687dc989d2508d5a50

 

kubeadm init 主要执行了以下操作:

       [init]:指定版本进行初始化操作

       [preflight] :初始化前的检查和下载所需要的Docker 镜像文件

       [kubelet-start] :生成kubelet 的配置文件”/var/lib/kubelet/config.yaml”,没有这个文件kubelet无法启动,所以初始化之前的kubelet 实际上启动失败。

       [certificates]:生成Kubernetes 使用的证书,存放在/etc/kubernetes/pki 目录中。

       [kubeconfig] :生成 Kubeconfig 文件,存放在/etc/kubernetes 目录中,组件之间通信需要使用对应文件。

       [control-plane]:使用/etc/kubernetes/manifest 目录下的YAML 文件,安装 Master 组件。

       [etcd]:使用/etc/kubernetes/manifest/etcd.yaml 安装Etcd 服务。

       [wait-control-plane]:等待control-plan 部署的Master 组件启动。

       [apiclient]:检查Master 组件服务状态。

       [uploadconfig]:更新配置

       [kubelet]:使用configMap 配置kubelet

       [patchnode]:更新CNI 信息到Node 上,通过注释的方式记录。

       [mark-control-plane]:为当前节点打标签,打了角色Master,和不可调度标签,这样默认就不会使用Master 节点来运行Pod

       [bootstrap-token]:生成token 记录下来,后边使用kubeadm join 往集群中添加节点时会用到

       [addons]:安装附加组件CoreDNS kube-proxy

2.7、查看容器

[root@docker-server1 ~]# docker ps -a

CONTAINER ID        IMAGE                                                           COMMAND                  CREATED             STATUS

              PORTS               NAMES88167513e26b        7d54289267dc                                                    "/usr/local/bin/kube…"   6 minutes ago       Up 6

minutes                            k8s_kube-proxy_kube-proxy-trrsg_kube-system_1d8ad663-c8d8-4429-9bfa-62c0644d048b_004ef064f9de7        registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 m

inutes                            k8s_POD_kube-proxy-trrsg_kube-system_1d8ad663-c8d8-4429-9bfa-62c0644d048b_0116a097c0f34        5eb3b7486872                                                    "kube-controller-man…"   6 minutes ago       Up 6

minutes                            k8s_kube-controller-manager_kube-controller-manager-192.168.200.111_kube-system_655c81bbe85f53920741e98506a879a4_0aa9676158688        303ce5db0e90                                                    "etcd --advertise-cl…"   6 minutes ago       Up 6

minutes                            k8s_etcd_etcd-192.168.200.111_kube-system_263a5d6fc4cb43d1291a4e7fc493a149_02f5d3ee4c848        78c190f736b1                                                    "kube-scheduler --au…"   6 minutes ago       Up 6

minutes                            k8s_kube-scheduler_kube-scheduler-192.168.200.111_kube-system_75516e998e1ab97384d969d8ccd139db_0f5d54e1fe069        0cae8d5cc64c                                                    "kube-apiserver --ad…"   6 minutes ago       Up 6

minutes                            k8s_kube-apiserver_kube-apiserver-192.168.200.111_kube-system_c4b84d01dcb983c440c0474273fb535c_04c4a714c82fe        registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 m

inutes                            k8s_POD_kube-controller-manager-192.168.200.111_kube-system_655c81bbe85f53920741e98506a879a4_06d5de46ad990        registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 m

inutes                            k8s_POD_kube-apiserver-192.168.200.111_kube-system_c4b84d01dcb983c440c0474273fb535c_0a1436b78e49e        registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 m

inutes                            k8s_POD_etcd-192.168.200.111_kube-system_263a5d6fc4cb43d1291a4e7fc493a149_03a6901465499        registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1   "/pause"                 6 minutes ago       Up 6 m

inutes                            k8s_POD_kube-scheduler-192.168.200.111_kube-system_75516e998e1ab97384d969d8ccd139db_0

 

根据提示操作

kubectl 默认会在执行的用户家目录下面的.kube 目录下寻找config 文件。这里是将在初始化时[kubeconfig]步骤生成的admin.conf 拷贝到.kube/config

[root@docker-server1 ~]# mkdir -p $HOME/.kube

[root@docker-server1 ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

[root@docker-server1 ~]# chown $(id -u):$(id -g) $HOME/.kube/config

在该配置文件中,记录了API Server 的访问地址,所以后面直接执行kubectl 命令就可以正常连接到API Server

 

查看node节点和组件

[root@docker-server1 ~]# kubectl get cs

NAME                 STATUS    MESSAGE             ERROR

controller-manager   Healthy   ok                 

scheduler            Healthy   ok                 

etcd-0               Healthy   {"health":"true"}

 

[root@docker-server1 ~]# kubectl get nodes

NAME              STATUS     ROLES    AGE     VERSION

192.168.200.111   NotReady   master   7m52s   v1.17.3

 

[root@docker-server1 ~]# kubectl get pods -n kube-system

NAME                                      READY   STATUS    RESTARTS   AGE

coredns-7f9c544f75-wx6q9                  0/1     Pending   0          52m

coredns-7f9c544f75-x5nff                  0/1     Pending   0          52m

etcd-192.168.200.111                      1/1     Running   0          52m

kube-apiserver-192.168.200.111            1/1     Running   0          52m

kube-controller-manager-192.168.200.111   1/1     Running   0          52m

kube-proxy-pfz6z                          1/1     Running   0          52m

kube-scheduler-192.168.200.111            1/1     Running   0          52m

 

发现两个问题:

1)         core节点为pending:是因为需要node节点,但是还没有安装node节点,所以是pending

2)         statusnotready状态:是因为还没有安装网络插件

2.8、安装flannel

Master 节点NotReady 的原因就是因为没有使用任何的网络插件,此时Node Master的连接还不正常。目前最流行的Kubernetes 网络插件有FlannelCalicoCanalWeave 这里选择使用flannel

 

master 节点上执行,执行完成后需要等flannel pods 运行起来,这需要点时间:

[root@docker-server1 ~]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

[root@docker-server1 ~]# sed -i 's@quay.io@quay.azk8s.cn@g' kube-flannel.yml

 

[root@docker-server1 ~]# kubectl apply -f kube-flannel.yml

podsecuritypolicy.policy/psp.flannel.unprivileged created

clusterrole.rbac.authorization.k8s.io/flannel created

clusterrolebinding.rbac.authorization.k8s.io/flannel created

serviceaccount/flannel created

configmap/kube-flannel-cfg created

daemonset.apps/kube-flannel-ds-amd64 created

daemonset.apps/kube-flannel-ds-arm64 created

daemonset.apps/kube-flannel-ds-arm created

daemonset.apps/kube-flannel-ds-ppc64le created

daemonset.apps/kube-flannel-ds-s390x created

 

[root@docker-server1 ~]# kubectl get nodes

NAME              STATUS   ROLES    AGE     VERSION

192.168.200.111   Ready    master   9m24s   v1.17.3

已经是ready状态

2.9、安装node节点

安装node方式一:

可以根据master安装时的提示信息

kubeadm join 192.168.200.111:6443 --token abcdef.0123456789abcdef \

    --discovery-token-ca-cert-hash sha256:c2d6067d5c3b12118275958dee222226d09a89fc5fb559687dc989d2508d5a50

 

安装node方式二:

master节点查看token信息

[root@docker-server1 ~]# cat kubeadm-config.yaml |grep token

  - system:bootstrappers:kubeadm:default-node-token

  token: abcdef.0123456789abcdef

 

docker-server2主机:

[root@docker-server2 ~]# kubeadm config print join-defaults > kubeadm-config.yaml

[root@docker-server2 ~]# vim kubeadm-config.yaml

apiVersion: kubeadm.k8s.io/v1beta2

caCertPath: /etc/kubernetes/pki/ca.crt

discovery:

  bootstrapToken:

    apiServerEndpoint: 192.168.200.111:6443

    token: abcdef.0123456789abcdef

    unsafeSkipCAVerification: true

  timeout: 5m0s

  tlsBootstrapToken: abcdef.0123456789abcdef

kind: JoinConfiguration

nodeRegistration:

  criSocket: /var/run/dockershim.sock

  name: 192.168.200.112

  taints: null

 

[root@docker-server2 ~]# kubeadm join --config kubeadm-config.yaml

W0212 22:13:36.627811    3819 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when

control-plane flag is not set.[preflight] Running pre-flight checks

         [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd

". Please follow the guide at https://kubernetes.io/docs/setup/cri/[preflight] Reading configuration from the cluster...

[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system names

pace[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

[kubelet-start] Starting the kubelet

[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

 

This node has joined the cluster:

* Certificate signing request was sent to apiserver and a response was received.

* The Kubelet was informed of the new secure connection details.

 

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

 

docker-server3主机:

[root@docker-server3 ~]# kubeadm config print join-defaults > kubeadm-config.yaml

[root@docker-server3 ~]# vim kubeadm-config.yaml

apiVersion: kubeadm.k8s.io/v1beta2

caCertPath: /etc/kubernetes/pki/ca.crt

discovery:

  bootstrapToken:

    apiServerEndpoint: 192.168.200.111:6443

    token: abcdef.0123456789abcdef

    unsafeSkipCAVerification: true

  timeout: 5m0s

  tlsBootstrapToken: abcdef.0123456789abcdef

kind: JoinConfiguration

nodeRegistration:

  criSocket: /var/run/dockershim.sock

  name: 192.168.200.112

  taints: null

 

[root@docker-server3 ~]# kubeadm join --config kubeadm-config.yaml

W0212 22:13:38.565506    3838 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when

control-plane flag is not set.[preflight] Running pre-flight checks

         [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd

". Please follow the guide at https://kubernetes.io/docs/setup/cri/[preflight] Reading configuration from the cluster...

[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.17" ConfigMap in the kube-system names

pace[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"

[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"

[kubelet-start] Starting the kubelet

[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

 

This node has joined the cluster:

* Certificate signing request was sent to apiserver and a response was received.

* The Kubelet was informed of the new secure connection details.

 

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

 

master查看node节点信息

[root@docker-server1 ~]# kubectl get nodes

NAME              STATUS   ROLES    AGE    VERSION

192.168.200.111   Ready    master   17m    v1.17.3

192.168.200.112   Ready    <none>   111s   v1.17.3

192.168.200.113   Ready    <none>   109s   v1.17.3

 

master查看Pod信息

[root@docker-server1 ~]# kubectl get pods -n kube-system

NAME                                      READY   STATUS    RESTARTS   AGE

coredns-7f9c544f75-6b8gq                  1/1     Running   0          17m

coredns-7f9c544f75-tjg2l                  1/1     Running   0          17m

etcd-192.168.200.111                      1/1     Running   0          17m

kube-apiserver-192.168.200.111            1/1     Running   0          17m

kube-controller-manager-192.168.200.111   1/1     Running   0          17m

kube-flannel-ds-amd64-bl49r               1/1     Running   3          2m24s

kube-flannel-ds-amd64-dfkgr               1/1     Running   0          9m14s

kube-flannel-ds-amd64-j74w7               1/1     Running   0          2m26s

kube-proxy-442vz                          1/1     Running   0          2m26s

kube-proxy-trrsg                          1/1     Running   0          17m

kube-proxy-xnn74                          1/1     Running   0          2m24s

kube-scheduler-192.168.200.111            1/1     Running   0          17m

2.10 节点管理命令

以下命令无需执行,仅作为了解

重置master配置

[root@docker-server1 ~]# kubeadm reset

删除node配置

[root@docker-server2 ~]# docker ps -aq|xargs  docker rm -f

[root@docker-server2 ~]# systemctl stop kubelet

[root@docker-server2 ~]# rm -rf /etc/kubernetes/*

[root@docker-server2 ~]# rm -rf /var/lib/kubelet/*

三、安装Dashboard UI

3.1部署Dashboard

dashboardgithub仓库地址:https://github.com/kubernetes/dashboard

代码仓库当中,有给出安装示例的相关部署文件,我们可以直接获取之后,直接部署即可

[root@docker-server1 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended.yaml

默认这个部署文件当中,会单独创建一个名为kubernetes-dashboard的命名空间,并将kubernetes-dashboard部署在该命名空间下。dashboard的镜像来自docker hub官方,所以可不用修改镜像地址,直接从官方获取即可。

3.2、开放端口设置

在默认情况下,dashboard并不对外开放访问端口,这里简化操作,直接使用nodePort的方式将其端口暴露出来,修改serivce部分的定义:

[root@docker-server1 ~]# vim recommended.yaml

kind: Service

apiVersion: v1

metadata:

  labels:

    k8s-app: kubernetes-dashboard

  name: kubernetes-dashboard

  namespace: kubernetes-dashboard

spec:

  type: NodePort            #添加

  ports:

    - port: 443

      targetPort: 8443

      nodePort: 32443          #添加

  selector:

    k8s-app: kubernetes-dashboard

3.3、权限配置

由于这个权限太小,修改一个超级管理员权限

[root@docker-server1 ~]# vim recommended.yaml

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: kubernetes-dashboard

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: cluster-admin

subjects:

  - kind: ServiceAccount

    name: kubernetes-dashboard

    namespace: kubernetes-dashboard

 

[root@docker-server1 ~]# kubectl apply -f recommended.yaml

namespace/kubernetes-dashboard created

serviceaccount/kubernetes-dashboard created

service/kubernetes-dashboard created

secret/kubernetes-dashboard-certs created

secret/kubernetes-dashboard-csrf created

secret/kubernetes-dashboard-key-holder created

configmap/kubernetes-dashboard-settings created

role.rbac.authorization.k8s.io/kubernetes-dashboard created

clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created

rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

deployment.apps/kubernetes-dashboard created

service/dashboard-metrics-scraper created

deployment.apps/dashboard-metrics-scraper created

 

获取token

[root@docker-server1 ~]# kubectl describe secret -n kubernetes-dashboard $(kubectl get secret -n kubernetes-dashboard |grep kubernetes-dashboard-token | awk '{print $1}') |grep token | awk '{print $2}'

kubernetes-dashboard-token-fk762

kubernetes.io/service-account-token

eyJhbGciOiJSUzI1NiIsImtpZCI6Ik5aYmhQMDA4aktaeUVyQVpBd3Y5VUNsTXFQV1VBeTRhSml4ZWlmNUV2NzAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi1mazc2MiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjZmYTVmZDM2LWIyOTItNDc3NS1hMWU0LThiOGE5MTY1NmI3ZiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.J_XYUsmSB1wWApYQkSebgd3BvEHoZe5pBgayw8N0xG6TYBsPhMEBVyhE6pR-P-R2eZKPAK9xkajMIwxtwxnIi2NTPv--FiecLINj2_XV7pegkEmd7AREXEPQmjGqM3Fulc7VkVFaG1YIdRmgi069GImpqFuTF0t19wOaloetUHY6LMRJsyHyesjvc2V82a_qgrFNcVtw9l0b8HhxebRIH6crhCMXKRpsjeF8zUg-Aq4ZfJxxEcc6wM2bOzAh00vJECHKBc7sTH2va8xic7GL_hMyE5SZzSOVeaulODWCc5hQdSc2BxeY4TVFz6GJXDC6ZgVj8gnNgUXxw3NVSiDmyg

 

使用token登录系统

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_Kubernetes_02

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_1.17.3_03

到此K8S集群安装全部完成

四、应用部署测试

下面我们部署一个简单的Nginx WEB服务,该容器运行时会监听80端口,同时访问/info路径会显示容器的主机名。服务由3个容器实例构成,并且通过Nodeport方式暴露给用户。

[root@docker-server1 ~]# kubectl run nginxweb --image=nginx  --port=80 --replicas=3

kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.deployment.apps/nginxweb created

 

查看创建的对象,可以看到已经有3pod在运行了

[root@docker-server1 ~]# kubectl get deployment

NAME       READY   UP-TO-DATE   AVAILABLE   AGE

nginxweb   0/3     3            0           14s

[root@docker-server1 ~]# kubectl get po

NAME                        READY   STATUS              RESTARTS   AGE

nginxweb-6d7457b898-5qcbs   0/1     ContainerCreating   0          31s

nginxweb-6d7457b898-m5tvh   0/1     ContainerCreating   0          31s

nginxweb-6d7457b898-v58bj   0/1     ContainerCreating   0          31s

 

创建svc,通过Nodeport方式暴露服务

[root@docker-server1 ~]# kubectl expose deployment nginxweb --name=nginxwebsvc --port=80  --target-port=80  --type=NodePort

service/nginxwebsvc exposed

 

查看svc,可以看到NodePort随机分配的端口为30715

[root@docker-server1 ~]# kubectl get svc

NAME          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE

kubernetes    ClusterIP   10.96.0.1     <none>        443/TCP        147m

nginxwebsvc   NodePort    10.96.63.33   <none>        80:30715/TCP   52s

接下来,在用户操作系统就可以通过master主机的ip地址 http://192.168.200.111:30715/ 来访问这个nginxwebsvc了,nginxwebsvc 会把80口的请求再负载均衡到实际的nginxweb pod

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_Kubernetes_04

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_1.17.3_05

全网最新:基于Kubeadm构建Kubernetes集群(1.17.3)_docker_06