Swarm是Docker公司推出的用来管docker的一个集群管理软件。
Swarm 可以在多个服务器或主机上创建容器集群服务。
swarm 的架构图
Swarm是典型的master-slave结构,通过发现服务来选举manager。
Swarm的几个关键概念
swarm :一个 swarm 由多个 Docker 主机组成,这些主机在 Warm 模式下运行,充当管理器(管理成员资格和委派)和工作线程(运行 swarm 服务)。Docker 引擎中嵌入的集群管理和编排功能是使用 swarmkit 构建的。Swarmkit 是一个单独的项目,它实现了 Docker 的编排层,并直接在 Docker 中使用。
node :node是参与swarm的 Docker 引擎的实例。,我们可以在单台服务器上运行一个或多个节点。
service :是要在管理器或工作器节点上执行的任务的定义。它是 swarm系统的中心结构和用户主要与 swarm的互动。创建服务时,您可以指定要使用的容器映像以及哪个容器映像 要在正在运行的容器中执行的命令。
task :任务是一个正在运行的容器,它是 swarm服务的一部分,由 swarm管理,而不是独立的容器。
1.节点的工作原理
2.服务的工作原理
3.swarm 模式如何接受服务创建请求和将任务计划到工作器节点。
4.复制服务和全局服务
swarm的调度策略
swarm在调度节点运行容器的时候,会根据指定的策略来计算最适合运行容器的节点,目前支持的策略有:spread,binpack,random。
1.Random
随机选择一个Node来运行容器,一般用作调试。
2.Spread
Spread策略会选择运行容器最少的那台节点来运行新的容器,会使得容器均衡的分布在集群中的各个节点上运行,一旦一个节点挂掉了只会损失少部分的容器。
3.Binpack
Binpack策略最大化的避免容器碎片化,就是说binpack策略尽可能的把还未使用的节点留给需要更大空间的容器运行,尽可能的把容器运行在一个节点上面。
Swarm集群部署
ip、hostname、role规划
ip | hostname | role |
192.168.1.140 | manger | manger |
192.168.1.123 | node1 | node |
192.168.1.125 | node2 | node |
第一步:前期的准备工作
1.修改主机名
[root@manager ~]# hostnamectl set-hostname manager
[root@node1 ~]# hostnamectl set-hostname node1
[root@node2 ~]# hostnamectl set-hostname node2
2.配置hosts文件
cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.140 manager
192.168.1.123 node1
192.168.1.125 node2
3.关闭防火墙
systemctl stop firewalld && systemctl disable firewalld
4.关闭seLinux
vim /etc/selinux/config
SELINUX=disabled
5.已经安装docker环境
[root@manger ~]# docker version
Client: Docker Engine - Community
Version: 24.0.4
API version: 1.43
Go version: go1.20.5
Git commit: 3713ee1
Built: Fri Jul 7 14:54:21 2023
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 24.0.4
API version: 1.43 (minimum version 1.12)
Go version: go1.20.5
Git commit: 4ffc614
Built: Fri Jul 7 14:53:26 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.21
GitCommit: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc:
Version: 1.1.7
GitCommit: v1.1.7-0-g860f061
docker-init:
Version: 0.19.0
GitCommit: de40ad0
第二步:创建Swarm集群并且添加节点
[root@manger ~]# docker swarm init --advertise-addr 192.168.1.140
Swarm initialized: current node (b5xsdqqky3kzuav4x9i3ks1y3) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-3lsbxnxitex8rvuebocuyvsexu0rp3x14njplny0bjxmdj203e-5zi94bk6j2qr4fiy3qnq33xfs 192.168.1.140:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
[root@node1 ~]# docker swarm join --token SWMTKN-1-3lsbxnxitex8rvuebocuyvsexu0rp3x14njplny0bjxmdj203e-5zi94bk6j2qr4fiy3qnq33xfs 192.168.1.140:2377
This node joined a swarm as a worker.
[root@node2 ~]# docker swarm join --token SWMTKN-1-3lsbxnxitex8rvuebocuyvsexu0rp3x14njplny0bjxmdj203e-5zi94bk6j2qr4fiy3qnq33xfs 192.168.1.140:2377
This node joined a swarm as a worker.
第三步:查看集群的相关信息
[root@manger ~]# docker info
Client: Docker Engine - Community
Version: 24.0.4
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.19.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 24.0.4
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: b5xsdqqky3kzuav4x9i3ks1y3
Is Manager: true
ClusterID: pje9ziqpdyl2mwftvvorshltl
Managers: 1
Nodes: 3
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.1.140
Manager Addresses:
192.168.1.140:2377
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc version: v1.1.7-0-g860f061
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 3.10.0-1160.88.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.776GiB
Name: manger
ID: baadf173-c534-4bde-9165-a227e8772f24
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
[root@manger ~]# docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
b5xsdqqky3kzuav4x9i3ks1y3 * manger Ready Active Leader 24.0.4
s080ygbxrkhz2gzc4ubl73x1j node1 Ready Active 24.0.4
kq1xwrkvhukqjtgpa17jw935t node2 Ready Active 24.0.4
swarm集群中node的availability状态可以为 active或者drain
active状态下,node可以接受来自manager节点的任务分派;
drain状态下,node节点会结束task,且不再接受来自manager节点的任务分派。下线节点
docker node update --availability drain node1
第四步:在swarm中部署nginx服务
1.创建overlay网络类型
[root@manger ~]# docker network create -d overlay nginx_net
u89gicrtx64t26doogi213vb2
[root@manger ~]# docker network ls
NETWORK ID NAME DRIVER SCOPE
0f9713843296 bridge bridge local
0ac26c162b96 docker_gwbridge bridge local
c1e5931ccec6 host host local
7h617gwnxo9x ingress overlay swarm
u89gicrtx64t nginx_net overlay swarm
23706824b4f2 none null local
2.部署服务
[root@manger ~]# docker service create --replicas 1 --network nginx_net --name nginx-1 -p 8080:80 nginx
sql7arzw1jk9rcpr25mrl2a9d
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged
在manager节点上使用上面这个覆盖网络创建nginx服务:
其中,--replicas 参数指定服务由几个实例组成。
注意:不需要提前在节点上下载nginx镜像,这个命令执行后会自动下载这个容器镜像。
[root@manger ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
sql7arzw1jk9 nginx-1 replicated 1/1 nginx:latest *:8080->80/tcp
3.查看swarm中服务的详细信息
[root@manger ~]# docker service inspect --pretty nginx-1
ID: sql7arzw1jk9rcpr25mrl2a9d
Name: nginx-1
Service Mode: Replicated
Replicas: 1
Placement:
UpdateConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Update order: stop-first
RollbackConfig:
Parallelism: 1
On failure: pause
Monitoring Period: 5s
Max failure ratio: 0
Rollback order: stop-first
ContainerSpec:
Image: nginx:latest@sha256:08bc36ad52474e528cc1ea3426b5e3f4bad8a130318e3140d6cfe29c8892c7ef
Init: false
Resources:
Networks: nginx_net
Endpoint Mode: vip
Ports:
PublishedPort = 8080
Protocol = tcp
TargetPort = 80
PublishMode = ingress
--pretty 是格式化输出
4.查看服务调度到那个节点
[root@manger ~]# docker service ps nginx-1
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
fbuuzgg30vhf nginx-1.1 nginx:latest manger Running Running 6 minutes ago
[root@manger ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d8fcb7f18b00 nginx:latest "/docker-entrypoint.…" 8 minutes ago Up 8 minutes 80/tcp nginx-1.1.fbuuzgg30vhf4l6zu7ndtbp6e
第五步:在Swarm中动态扩展服务
将nginx-1容器动态扩展到3个
[root@manger ~]# docker service scale nginx-1=3
nginx-1 scaled to 3
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
[root@manger ~]# docker service ps nginx-1
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
fbuuzgg30vhf nginx-1.1 nginx:latest manger Running Running 18 minutes ago
d7nhfm0865r4 nginx-1.2 nginx:latest node1 Running Running 5 minutes ago
zm9zh239vrz4 nginx-1.3 nginx:latest node2 Running Running 6 minutes ago
将nginx-1容器动态缩小到1个
[root@manger ~]# docker service scale nginx-1=1
nginx-1 scaled to 1
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged
[root@manger ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
sql7arzw1jk9 nginx-1 replicated 1/1 nginx:latest *:8080->80/tcp
[root@manger ~]# docker service ps nginx-1
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
fbuuzgg30vhf nginx-1.1 nginx:latest manger Running Running 21 minutes ago
[root@manger ~]# docker service update --replicas 3 nginx-1
nginx-1
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
[root@manger ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
sql7arzw1jk9 nginx-1 replicated 3/3 nginx:latest *:8080->80/tcp
[root@manger ~]# docker service ps nginx-1
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
fbuuzgg30vhf nginx-1.1 nginx:latest manger Running Running 23 minutes ago
94k1s7oh5ghg nginx-1.2 nginx:latest node1 Running Running 20 seconds ago
jqx9v4y78hzi nginx-1.3 nginx:latest node2 Running Running 20 seconds ago
docker service update 也可用于直接升级镜像
docker service update --image nginx:new nginx-1
删除nginx-1服务
[root@manger ~]# docker service rm nginx-1
nginx-1
[root@manger ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
注意:如果一个节点宕机了(即该节点就会从swarm集群中被踢出),则Docker会将在该节点运行的容器,调度到其他节点,以满足指定数量的副本保持运行状态。
第六步:Swarm中使用Volume
1.创建卷
[root@manger ~]# docker volume create test-vol
test-vol
[root@manger ~]# docker volume ls
DRIVER VOLUME NAME
local test-vol
2.查看卷的详细信息
[root@manger ~]# docker volume inspect test-vol
[
{
"CreatedAt": "2023-07-11T01:53:04+08:00",
"Driver": "local",
"Labels": null,
"Mountpoint": "/var/lib/docker/volumes/test-vol/_data",
"Name": "test-vol",
"Options": null,
"Scope": "local"
}
]
创建新的服务并挂载test-vol
[root@manger ~]# docker service create --replicas 3 --mount type=volume,src=test-vol,dst=/data --name test-nginx nginx
50fdr2dygc0d6xqnzd9tdtnoz
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
[root@manger ~]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
50fdr2dygc0d test-nginx replicated 3/3 nginx:latest
[root@manger ~]# docker service ps test-nginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
g7sfnacn7rd5 test-nginx.1 nginx:latest node1 Running Running 24 seconds ago
04n0m74ir7lr test-nginx.2 nginx:latest node2 Running Running 24 seconds ago
k5fm278i2war test-nginx.3 nginx:latest manger Running Running 24 seconds ago
进入容器创建test.txt
[root@manger ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a61dcb356e84 nginx:latest "/docker-entrypoint.…" About a minute ago Up About a minute 80/tcp test-nginx.3.k5fm278i2war26y2itqg33me5
[root@manger ~]# docker exec -it a61dcb356e84 bash
root@a61dcb356e84:/# cd /data
root@a61dcb356e84:/data# ls
root@a61dcb356e84:/data# echo "welcome to changsha" > test.txt
root@a61dcb356e84:/data# ls
test.txt
root@a61dcb356e84:/data# exit
exit
[root@manger ~]# cd /var/lib/docker/volumes/test-vol/_data
[root@manger _data]# ls
test.txt
[root@manger _data]# cat test.txt
welcome to changsha
还可以将node节点机上的volume数据目录做成软链接
[root@manager _data]# ln -s /var/lib/docker/volumes/testvolume/_data /data
[root@manager _data]# cd /data
[root@manager data]# ls
test.txt
[root@manager data]# echo "123" > a.txt
[root@manager data]# ll
总用量 8
-rw-r--r-- 1 root root 4 7月 21 11:04 a.txt
-rw-r--r-- 1 root root 18 7月 21 11:00 test.txt
另一种种挂载方式
命令格式:
docker service create --mount type=bind,target=/container_data/,source=/host_data/
其中,参数target表示容器里面的路径,source表示本地硬盘路径
docker service create --replicas 1 --mount type=bind,target=/usr/share/nginx/html/,source=/opt/web/ --network nginx_net --name web_nginx -p 8880:80 nginx
第七步:多服务Swarm集群部署
通过docker-compose.yml的文件,结合compose和swarm进行多服务的编排。
1.编写docker-compose.yml文件
[root@manager ~]# mkdir testswarm
[root@manager ~]# cd testswarm/
[root@manager testswarm]# cat docker-compose.yml
version: "3"
services:
nginx:
image: nginx
ports:
- 8888:80
deploy:
mode: replicated
replocas: 3
visualizer:
image: dockersamples/visualizer
ports:
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
replicas: 1
placement:
constraints: [node.role == manager]
portainer:
image: portainer/portainer
ports:
- "9000:9000"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
replicas: 1
placement:
constraints: [node.role == manager]
2.通过这个yml文件部署服务
[root@manager testswarm]# docker stack deploy -c docker-compose.yml deploy_deamon
Creating network deploy_deamon_default
Creating service deploy_deamon_portainer
Creating service deploy_deamon_nginx
Creating service deploy_deamon_visualizer
3.查看服务
[root@manager testswarm]# docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
xj2f1t5ax3nm deploy_deamon_nginx replicated 3/3 nginx:latest *:8888->80/tcp
ky9qpldr5abb deploy_deamon_portainer replicated 1/1 portainer/portainer:latest *:9000->9000/tcp
r47ff177x1ir deploy_deamon_visualizer replicated 1/1 dockersamples/visualizer:latest *:8080->8080/tcp
[root@manager43 testswarm]# docker service ps deploy_deamon_nginx
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
z3v4uc1ujsnq deploy_deamon_nginx.1 nginx:latest node1 Running Running about a minute ago
jhg3ups0cko5 deploy_deamon_nginx.2 nginx:latest manager Running Running about a minute ago
3e6guv791x21 deploy_deamon_nginx.3 nginx:latest node2 Running Running about a minute ago
[root@manager43 testswarm]# docker service ps deploy_deamon_portainer
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
whyuvy82cvvw deploy_deamon_portainer.1 portainer/portainer:latest manager Running Running about a minute ago
[root@manager43 testswarm]# docker service ps deploy_deamon_visualizer
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
wge5w1eqykg3 deploy_deamon_visualizer.1 dockersamples/visualizer:latest manager Running Starting 7 seconds ago