etcd集群架构原理

转载

IT剑客行 2024-10-14 11:07:15

文章标签 etcd集群架构原理 etcd kubernetes 数据库 json 文章分类 架构后端开发

文章目录

部署ETCD集群

1、概述
2、服务器配置

2.1 服务器配置信息
2.2 服务器参数配置

3、准备cfssl证书生成工具
4、创建etcd相关目录
5、创建证书

5.1、创建ca证书json文件
5.2、生成ca证书
5.3、使用自签ca签发etcd证书

6、下载etcd二进制文件
7、部署ETCD集群

7.1、复制二进制文件到指定文件
7.2、创建etcd配置文件
7.3、创建systemd文件

8、分发etcd文件
9、分别启动etcd服务
10、查看集群状态
11、测试
12、数据备份与恢复

部署ETCD集群

1、概述

- 大意：
	etcd 是兼具一致性和高可用性的键值数据库，可以作为保存 Kubernetes 所有集群数据的后台数据库。
- 官方网址：
	https://etcd.io/docs/

2、服务器配置

2.1 服务器配置信息

主机名	cpu	内存	存储	操作系统
k8s01	4	4	100g	Ubuntu 18.04.5 LTS
k8s02	4	4	100g	Ubuntu 18.04.5 LTS
k8s03	4	4	100g	Ubuntu 18.04.5 LTS

2.2 服务器参数配置

$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

3、准备cfssl证书生成工具

cfssl是一个开源的证书管理工具，使用json文件生成证书.

在任意一台服务器上操作，这里选择k8s01

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

4、创建etcd相关目录

$ mkdir -pv /opt/kubernetes/etcd/{bin,cfg,ssl,data}

5、创建证书

5.1、创建ca证书json文件

$ cd /opt/kubernetes/etcd/ssl
# 创建ca-config
$ vim ca-config.json
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
# 创建ca-csr
$ vim ca-csr.json 
{
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
}

5.2、生成ca证书

$ cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

5.3、使用自签ca签发etcd证书

创建证书申请文件：

$ cd /opt/kubernetes/etcd/ssl
# 注意hosts内容，etcd集群内的ip都要写上，可以预留几个，为以后扩容使用
$ vim server-csr.json
{
    "CN": "etcd",
    "hosts": [
    "192.168.1.241",
    "192.168.1.242",
    "192.168.1.243"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
}

生成证书

$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
# 会生成server.pem和server-key.pem文件 
$ ll |grep server
-rw-r--r-- 1 root root 1013 Sep 14 15:06 server.csr
-rw-r--r-- 1 root root  290 Sep 14 15:05 server-csr.json
-rw------- 1 root root 1679 Sep 14 15:06 server-key.pem
-rw-r--r-- 1 root root 1338 Sep 14 15:06 server.pem

6、下载etcd二进制文件

下载地址
https://github.com/etcd-io/etcd/releases/download/v3.5.0/etcd-v3.5.0-linux-amd64.tar.gz

7、部署ETCD集群

7.1、复制二进制文件到指定文件

$ tar xf etcd-v3.5.0-linux-amd64.tar.gz
$ cp etcd-v3.5.0-linux-amd64/{etcd,etcdctl,etcdutl} /opt/kubernetes/etcd/bin

7.2、创建etcd配置文件

$ vim /opt/kubernetes/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME="etcd-1"											# k8s01为etcd-1，k8s02为etcd-2。。。每个节点唯一标识符
ETCD_DATA_DIR="/opt/kubernetes/etcd/data/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.1.241:2380"			# 修改对应ip，k8s01为241，k8s02为242...
ETCD_LISTEN_CLIENT_URLS="https://192.168.1.241:2379"		# 修改对应ip，k8s01为241，k8s02为242...

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.1.241:2380"		# 修改对应ip，k8s01为241，k8s02为242...
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.1.241:2379"				# 修改对应ip，k8s01为241，k8s02为242...
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.1.241:2380,etcd-2=https://192.168.1.242:2380,etcd-3=https://192.168.1.243:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

注释

ETCD_NAME：节点名称，集群中唯一
ETCD_DATA_DIR：数据目录
ETCD_LISTEN_PEER_URLS：集群通信监听地址
ETCD_LISTEN_CLIENT_URLS：客户端访问监听地址
ETCD_INITIAL_ADVERTISE_PEERURLS：集群通告地址
ETCD_ADVERTISE_CLIENT_URLS：客户端通告地址
ETCD_INITIAL_CLUSTER：集群节点地址
ETCD_INITIALCLUSTER_TOKEN：集群Token
ETCD_INITIALCLUSTER_STATE：加入集群的当前状态，new是新集群，existing表示加入已有集群

7.3、创建systemd文件

$ vim /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/opt/kubernetes/etcd/cfg/etcd.conf
ExecStart=/opt/kubernetes/etcd/bin/etcd \
--cert-file=/opt/kubernetes/etcd/ssl/server.pem \
--key-file=/opt/kubernetes/etcd/ssl/server-key.pem \
--peer-cert-file=/opt/kubernetes/etcd/ssl/server.pem \
--peer-key-file=/opt/kubernetes/etcd/ssl/server-key.pem \
--trusted-ca-file=/opt/kubernetes/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/opt/kubernetes/etcd/ssl/ca.pem \
--logger=zap
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

8、分发etcd文件

k8s02

$ scp -r /opt/kubernetes k8s02:/opt/
$ scp /etc/systemd/system/etcd.service k8s02:/etc/systemd/system/
# 记得修改etcd配置文件

k8s03

$ scp -r /opt/kubernetes k8s03:/opt/
$ scp /etc/systemd/system/etcd.service k8s03:/etc/systemd/system/
# 记得修改etcd配置文件

9、分别启动etcd服务

k8s01

$ systemctl start etcd.service

k8s02

$ systemctl start etcd.service

k8s03

$ systemctl start etcd.service

10、查看集群状态

$ cd /opt/kubernetes/etcd
$ ETCDCTL_API=3 ./bin/etcdctl --cacert=/opt/kubernetes/etcd/ssl/ca.pem --cert=/opt/kubernetes/etcd/ssl/server.pem --key=/opt/kubernetes/etcd/ssl/server-key.pem --endpoints="https://192.168.1.241:2379,https://192.168.1.242:2379,https://192.168.1.243:2379" endpoint health --write-out=table
+----------------------------+--------+-------------+-------+
|          ENDPOINT          | HEALTH |    TOOK     | ERROR |
+----------------------------+--------+-------------+-------+
| https://192.168.1.241:2379 |   true |  8.913068ms |       |
| https://192.168.1.242:2379 |   true |  9.757387ms |       |
| https://192.168.1.243:2379 |   true | 12.405075ms |       |
+----------------------------+--------+-------------+-------+

11、测试

写入数据

[root@etcd01 bin]# etcdctl --cacert=/opt/software/etcd/ssl/ca.pem --cert=/opt/software/etcd/ssl/server.pem --key=/opt/software/etcd/ssl/server-key.pem --endpoints="https://192.168.1.241:2379,https://192.168.1.242:2379,https://192.168.1.243:2379" put foo "Hello World"
OK

读取数据

[root@etcd01 bin]# etcdctl --cacert=/opt/software/etcd/ssl/ca.pem --cert=/opt/software/etcd/ssl/server.pem --key=/opt/software/etcd/ssl/server-key.pem --endpoints="https://192.168.1.241:2379,https://192.168.1.242:2379,https://192.168.1.243:2379" get foo
foo
Hello World

12、数据备份与恢复

备份

#集群多个节点，只需一个节点即可
etcdctl --cacert=/opt/software/etcd/ssl/ca.pem --cert=/opt/software/etcd/ssl/server.pem --key=/opt/software/etcd/ssl/server-key.pem --endpoints="https://192.168.1.241:2379" snapshot save snapshot.db

恢复

etcdctl --cacert=/opt/software/etcd/ssl/ca.pem --cert=/opt/software/etcd/ssl/server.pem --key=/opt/software/etcd/ssl/server-key.pem --endpoints="https://192.168.1.241:2379" snapshot restore snapshot.db --data-dir=/root/data

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。