vault 集群搭建（active standby 模式)

 
 

参考架构图：
vault 集群搭建（active standby 模式)_分享

consul server cluster 搭建

  • consul 基本配置格式
{
  "server": true,
  "node_name": "$NODE_NAME",
  "datacenter": "dc1",
  "data_dir": "$CONSUL_DATA_PATH",
  "bind_addr": "0.0.0.0",
  "client_addr": "0.0.0.0",
  "advertise_addr": "$ADVERTISE_ADDR",
  "bootstrap_expect": 3,
  "retry_join": ["$JOIN1", "$JOIN2", "$JOIN3"],
  "ui": true,
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}

参数说明

  • $NODE_NAME this is a unique label for the node; in our case, this will be consul_s1, consul_s2, and consul_s3 respectively.
  • $CONSUL_DATA_PATH: absolute path to Consul data directory; ensure that this directory is writable by the Consul process user.
  • $ADVERTISE_ADDR: set to address that you prefer the Consul servers advertise to the other servers in the cluster and should not be set to 0.0.0.0; for this guide, it should be set to the Consul server’s IP address in each instance of the configuration file, or 10.1.42.101,10.1.42.102, and 10.1.42.103 respectively.
  • JOIN1,JOIN2, $JOIN3: This example uses the retry_join method of joining the server agents to form a cluster; as such, the values for this guide would be 10.1.42.101, 10.1.42.102, and 10.1.42.103 respectively.
  • 参考配置
consul server 1
{
  "server": true,
  "node_name": "consul_s1",
  "datacenter": "dc1",
  "data_dir": "/var/consul/data",
  "bind_addr": "0.0.0.0",
  "client_addr": "0.0.0.0",
  "advertise_addr": "10.1.42.101",
  "bootstrap_expect": 3,
  "retry_join": ["10.1.42.101", "10.1.42.102", "10.1.42.103"],
  "ui": true,
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}
consul server 2
{
  "server": true,
  "node_name": "consul_s2",
  "datacenter": "dc1",
  "data_dir": "/var/consul/data",
  "bind_addr": "0.0.0.0",
  "client_addr": "0.0.0.0",
  "advertise_addr": "10.1.42.102",
  "bootstrap_expect": 3,
  "retry_join": ["10.1.42.101", "10.1.42.102", "10.1.42.103"],
  "ui": true,
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}
consul server 3
{
  "server": true,
  "node_name": "consul_s3",
  "datacenter": "dc1",
  "data_dir": "/var/consul/data",
  "bind_addr": "0.0.0.0",
  "client_addr": "0.0.0.0",
  "advertise_addr": "10.1.42.103",
  "bootstrap_expect": 3,
  "retry_join": ["10.1.42.101", "10.1.42.102", "10.1.42.103"],
  "ui": true,
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}
  • systemd 配置
### BEGIN INIT INFO
# Provides: consul
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Consul agent
# Description: Consul service discovery framework
### END INIT INFO

[Unit]
Description=Consul server agent
Requires=network-online.target
After=network-online.target

[Service]
User=consul
Group=consul
PIDFile=/var/run/consul/consul.pid
PermissionsStartOnly=true
ExecStartPre=-/bin/mkdir -p /var/run/consul
ExecStartPre=/bin/chown -R consul:consul /var/run/consul
ExecStart=/usr/local/bin/consul agent \
    -config-file=/usr/local/etc/consul/server_agent.json \
    -pid-file=/var/run/consul/consul.pid
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

consul agent 配置

  • 格式说明
{
  "server": false,
  "datacenter": "dc1",
  "node_name": "$NODE_NAME",
  "data_dir": "$CONSUL_DATA_PATH",
  "bind_addr": "$BIND_ADDR",
  "client_addr": "127.0.0.1",
  "retry_join": ["$JOIN1", "$JOIN2", "$JOIN3"],
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}

参数说明

  • $NODE_NAME this is a unique label for the node; in our case, this will be consul_c1 and consul_c2 respectively.
  • $CONSUL_DATA_PATH: absolute path to Consul data directory; ensure that this directory is writable by the Consul process user.
  • $BIND_ADDR: this should be set to address that you prefer the Consul servers advertise to the other servers in the cluster and should not be set to 0.0.0.0; for this guide, it should be set to the Vault server’s IP address in each instance of the configuration file, or 10.1.42.201 and 10.1.42.202 respectively.
  • JOIN1,JOIN2, $JOIN3: This example uses the retry_join method of joining the server agents to form a cluster; as such, the values for this guide would be 10.1.42.101, 10.1.42.102, and 10.1.42.103 respectively.
  • 参考
agent1
{
  "server": false,
  "datacenter": "dc1",
  "node_name": "consul_c1",
  "data_dir": "/var/consul/data",
  "bind_addr": "10.1.42.201",
  "client_addr": "127.0.0.1",
  "retry_join": ["10.1.42.101", "10.1.42.102", "10.1.42.103"],
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}
agent2
{
  "server": false,
  "datacenter": "dc1",
  "node_name": "consul_c2",
  "data_dir": "/var/consul/data",
  "bind_addr": "10.1.42.202",
  "client_addr": "127.0.0.1",
  "retry_join": ["10.1.42.101", "10.1.42.102", "10.1.42.103"],
  "log_level": "DEBUG",
  "enable_syslog": true,
  "acl_enforce_version_8": false
}
  • systemd
### BEGIN INIT INFO
# Provides: consul
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Consul agent
# Description: Consul service discovery framework
### END INIT INFO

[Unit]
Description=Consul client agent
Requires=network-online.target
After=network-online.target

[Service]
User=consul
Group=consul
PIDFile=/var/run/consul/consul.pid
PermissionsStartOnly=true
ExecStartPre=-/bin/mkdir -p /var/run/consul
ExecStartPre=/bin/chown -R consul:consul /var/run/consul
ExecStart=/usr/local/bin/consul agent \
    -config-file=/usr/local/etc/consul/client_agent.json \
    -pid-file=/var/run/consul/consul.pid
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

vault 配置

主要配置参数
api_addr ， cluster_addr

  • vault active
listener "tcp" {
  address = "0.0.0.0:8200"
  cluster_address = "10.1.42.201:8201"
  tls_disable = "true"
}

storage "consul" {
  address = "127.0.0.1:8500"
  path = "vault/"
}

api_addr = "http://10.1.42.201:8200"
cluster_addr = "https://10.1.42.201:8201"
  • vault standby
listener "tcp" {
  address = "0.0.0.0:8200"
  cluster_address = "10.1.42.202:8201"
  tls_disable = "true"
}

storage "consul" {
  address = "127.0.0.1:8500"
  path = "vault/"
}

api_addr = "http://10.1.42.202:8200"
cluster_addr = "https://10.1.42.202:8201"
  • systemd 配置
### BEGIN INIT INFO
# Provides: vault
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Vault server
# Description: Vault secret management tool
### END INIT INFO

[Unit]
Description=Vault secret management tool
Requires=network-online.target
After=network-online.target

[Service]
User=vault
Group=vault
PIDFile=/var/run/vault/vault.pid
ExecStart=/usr/local/bin/vault server -config=/etc/vault/vault_server.hcl -log-level=debug
ExecReload=/bin/kill -HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
RestartSec=42s
LimitMEMLOCK=infinity

[Install]
WantedBy=multi-user.target

loadbalance 说明

  • 参考图
    vault 集群搭建（active standby 模式)_分享_02
  • haproxy 配置
listen vault
    bind 0.0.0.0:80
    balance roundrobin
    option httpchk GET /v1/sys/health
    server vault1 192.168.33.10:8200 check
    server vault2 192.168.33.11:8200 check
    server vault3 192.168.33.12:8200 check

参考配置资料

https://www.vaultproject.io/docs/concepts/ha.html
https://www.vaultproject.io/guides/operations/vault-ha-consul.html
https://www.vaultproject.io/guides/operations/reference-architecture.html
https://github.com/rongfengliang/vault-consul-ha