前提.安装配置docker
[root@test2 ~]# docker -v
Docker version 19.03.9, build 9d988398e7一 overlay网络简单配置
1.overlay需求
配置overlay 网路需要一个key-value数据库，用来保存网路状态信息，network， ip等。这里使用consul作为键值服务，通过启动一个consul容器配置
docker run -d --restart="always" --publish="8500:8500" --hostname="consul" --name="consul" index.alauda.cn/sequenceiq/consul:v0.5.0-v6 -server -bootstrap3.修改docker服务配置文件
[root@test1 ~]# cat /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --cluster-store=consul://192.168.137.131:8500 --cluster-advertise=ens33:2375
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
# Note that StartLimit* options were moved from "Service" to "Unit" in systemd 229.
# Both the old, and new location are accepted by systemd 229 and up, so using the old location
# to make them work for either version of systemd.
StartLimitBurst=3
# Note that StartLimitInterval was renamed to StartLimitIntervalSec in systemd 230.
# Both the old, and new name are accepted by systemd 230 and up, so using the old name to make
# this option work for either version of systemd.
StartLimitInterval=60s
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Comment TasksMax if your systemd version does not support it.
# Only systemd 226 and above support this option.
TasksMax=infinity
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process
[Install]
WantedBy=multi-user.target
[root@test1 ~]#注释：
1).--cluster-store=consul://10.19.137.180:8500 #启动consul的主机IP地址加上consul的端口，提供给要加入跨网络通信的服务地址
2).--cluster-advertise=eth0:2375 #网卡信息
将同样的配置，在其他节点上完成配置，重启docker
systemctl daemon-reload
systemctl restart docker此时登录到consul可以查看节点
4.创建docker 中的overlay网络
[root@test1 ~]# docker network create -d overlay overlay_test
ce7588820b8d35ec7c649b054e2fa28690e9687132d45afac6eb0d8f99cedd9a
[root@test1 ~]# docker network list
NETWORK ID NAME DRIVER SCOPE
bb3c1942b560 bridge bridge local
e0befb2cefb2 docker_gwbridge bridge local
81d47d5fad9f host host local
ab1d0da9e9a3 none null local
ce7588820b8d overlay_test overlay global执行一次网络创建，其他节点也会自动存在相同的网络
[root@test2 ~]# docker network list
NETWORK ID NAME DRIVER SCOPE
a83101a46630 bridge bridge local
e5df93642271 docker_gwbridge bridge local
c789dc426cd2 host host local
7784b5b2d498 leo overlay global
80a53560af4d none null local
ce7588820b8d overlay_test overlay global5.测试跨主机容器的连通性
分别在两台主机上创建一个网路测试容器busybox1和busybox2
节点1
[root@test1 ~]# docker run -itd --name=testoverlay1 --net=overlay_test busybox
[root@test1 ~]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bdee2223d882 busybox "sh" 3 minutes ago Up 3 minutes testoverlay1节点2
[root@test2 ~]# docker run -itd --name=testoverlay2 --net=overlay_test busybox
[root@test2 ~]# docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
691fea201008 busybox "sh" 5 seconds ago Up 4 seconds testoverlay2测试连通性
[root@test1 ~]# docker exec -it testoverlay1 ping testoverlay2
PING testoverlay2 (10.0.0.3): 56 data bytes
64 bytes from 10.0.0.3: seq=0 ttl=64 time=9.165 ms
64 bytes from 10.0.0.3: seq=1 ttl=64 time=1.516 ms
64 bytes from 10.0.0.3: seq=2 ttl=64 time=0.663 ms
^C
--- testoverlay2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.663/3.781/9.165 ms网络可达，至此 overlay 网络配置完成。
二、macvlan网路简单配置
1.简介： macvlan是一种网卡虚拟化技术，是linux 内核的新功能
查看是否支持，有输出不报错基本可判断支持该技术
[root@test1 ~]# modprobe macvlan
[root@test1 ~]# lsmod | grep macvlan
macvlan 19239 0模式
- private 模式：同一主接口下的子接口彼此隔离，即使从外部交换机导流数据包也会被丢掉
- vepa(virtual ethernet port aggregator) 模式：子接口通信需要导到外部支持
802.1Qbg/VPEA交换机设备（或虚机），经外部交换设备转发后实现通信 - bridge 模式：子接口可以直接通信，且mac 地址已知不需要学习
- passthru 模式：只允许单个子接口连接主接口，必须设置混合模式
2.配置
2.1相同macvlan网络通信配置
网卡开启/关闭混杂模式
开启
ip link set ens33 promisc on
关闭
ip link set ens33 promisc off分别在两个主机上创建macvlan网络，执行相同的命令
docker network create -d macvlan --subnet=172.16.10.0/24 --gateway=172.16.10.1 -o parent=ens33 mac1[root@test1 ~]# docker network list
NETWORK ID NAME DRIVER SCOPE
bb3c1942b560 bridge bridge local
e0befb2cefb2 docker_gwbridge bridge local
81d47d5fad9f host host local
7784b5b2d498 leo overlay global
9a35c8ff4b53 mac1 macvlan local
af556e03b494 mytest bridge local
ab1d0da9e9a3 none null local
ce7588820b8d overlay_test overlay global
[root@test2 ~]# docker network list
NETWORK ID NAME DRIVER SCOPE
a83101a46630 bridge bridge local
e5df93642271 docker_gwbridge bridge local
c789dc426cd2 host host local
7784b5b2d498 leo overlay global
8afd12bb9de2 mac1 macvlan local
80a53560af4d none null local
ce7588820b8d overlay_test overlay global分别在两个主机创建容器
[root@test1 ~]# docker run -itd --name c1 --ip=172.16.10.2 --network mac1 busybox
[root@test2 ~]# docker run -itd --name c2 --ip=172.16.10.3 --network mac1 busybox测试连通性
[root@test1 ~]# docker exec c1 ping -c 2 172.16.10.3
PING 172.16.10.3 (172.16.10.3): 56 data bytes
64 bytes from 172.16.10.3: seq=0 ttl=64 time=0.800 ms
64 bytes from 172.16.10.3: seq=1 ttl=64 time=1.324 ms
--- 172.16.10.3 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.800/1.062/1.324 ms2.2不同macvlan网络之间的通信
由于 macvlan 网络会独占物理网卡，也就是说一张物理网卡只能创建一个 macvlan 网络，如果我们想创建多个 macvlan 网络就得用多张网卡，但主机的物理网卡是有限的,此时，可以通过 VLAN 技术将一个网口划分出多个子网口，这样就可以基于子网口来创建 macvlan 网络了：
lsmod|grep 8021q 确认内核是够载入了802.1q模组
modprobe -a 8021q 如果没载入使用这个命令载入模组注意：在仅将ens33开启混杂模式的情况下，一直无法跨主机ping通，ens33.100和ens33.200开启后可以ping通，不确定是否因为ens33.100和ens33.200开启混杂模式影响到了通信
ip link set ens33.100 promisc on
ip link set ens33.200 promisc on安装vconfig 工具
配置yum源并安装
配置epel源：
yum install epel-release -y
安装vconfig：
yum install vconfig -y创建两个vlan
[root@test2 ~]# vconfig add ens33 100
Added VLAN with VID == 100 to IF -:ens33:-
[root@test2 ~]# vconfig add ens33 200
Added VLAN with VID == 200 to IF -:ens33:-
通过ifconfig a 或者ip a 查看配置
20: ens33.100@ens33: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:cf:16:d7 brd ff:ff:ff:ff:ff:ff
21: ens33.200@ens33: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 00:0c:29:cf:16:d7 brd ff:ff:ff:ff:ff:ff设置reorder 参数
[root@test1 ~]# vconfig set_flag ens33.100 1 1
Set flag on device -:ens33.100:- Should be visible in /proc/net/vlan/ens33.100
[root@test1 ~]# vconfig set_flag ens33.200 1 1
Set flag on device -:ens33.200:- Should be visible in /proc/net/vlan/ens33.200启动接口
[root@test1 ~]# ifconfig ens33.100 up
[root@test1 ~]# ifconfig ens33.200 up在 host1 和 host2 上基于两个 VLAN 子接口创建 2 个 macvlan 网络，mac10 和 mac20。
[root@test1 ~]# docker network create -d macvlan --subnet=172.16.21.0/24 --gateway=172.16.21.1 -o parent=ens33.100 mac100
[root@test1 ~]# docker network create -d macvlan --subnet=172.16.22.0/24 --gateway=172.16.22.1 -o parent=ens33.200 mac200
[root@test1 ~]# docker network list
NETWORK ID NAME DRIVER SCOPE
bb3c1942b560 bridge bridge local
e0befb2cefb2 docker_gwbridge bridge local
81d47d5fad9f host host local
7784b5b2d498 leo overlay global
9a35c8ff4b53 mac1 macvlan local
27008bacbae5 mac100 macvlan local
258c9cd94d4a mac200 macvlan local
af556e03b494 mytest bridge local
ab1d0da9e9a3 none null local
ce7588820b8d overlay_test overlay global主机两端都添加了相同的网络后创建容器测试连通性
主机1
[root@test1 ~]# docker run -itd --name test3 --ip=172.16.21.11 --network mac100 busybox
59aa5dea7c3dacfedd1dda8e5c695a3d9724054a81046639ace5fc87a2c044a1
[root@test1 ~]# docker run -itd --name test4 --ip=172.16.22.11 --network mac200 busybox
2c70c6e8576c201ac53e66c3b9494d09d654c954af8b92d580ff2930d2812234
主机2
[root@test2 ~]# docker run -itd --name test1 --ip=172.16.21.10 --network=mac100 busybox
27364d969fbb5d09d0825bc1640990600df479d3d1937912442a210f78600e75
[root@test2 ~]# docker run -itd --name test2 --ip=172.16.22.10 --network=mac200 busybox
bd7aa08fb7ca62515e8a71f0ac50bf3358bdb2ba52572c12c164997615070da2目前是无法实现跨网段访问的，需要通过增加主机，配置路由转发实现三层路由功能
新建host3，配置软路由功能实现不同macvlan 路由转发
开启路由转发功能
sysctl -w net.ipv4.ip_forward=1
sysctl -p或者echo "1" > /proc/sys/net/ipv4/ip_forward
sysctl -p在host3上创建host1host2的子接口网络
[root@test3 ~]# vconfig add ens33 100
[root@test3 ~]# vconfig add ens33 200
[root@test3 ~]# vconfig set_flag ens33.100 1 1
[root@test3 ~]# vconfig set_flag ens33.200 1 1为不同接口配置对应的网关地址
# 对 vlan 子接口配置网关 IP 并启用
[root@test3 ~]# ifconfig ens33.100 172.16.10.1 netmask 255.255.255.0 up
[root@test3 ~]# ifconfig ens33.200 172.16.20.1 netmask 255.255.255.0 uphosts3路由信息
[root@test3 ~]# ip route
default via 192.168.137.2 dev ens33 proto static metric 100
172.16.10.0/24 dev ens33.100 proto kernel scope link src 172.16.10.1
172.16.20.0/24 dev ens33.200 proto kernel scope link src 172.16.20.1
192.168.137.0/24 dev ens33 proto kernel scope link src 192.168.137.135 metric 100容器路由信息
[root@test2 ~]# docker exec d3 ip route
default via 172.16.10.1 dev eth0
172.16.10.0/24 dev eth0 scope link src 172.16.10.11测试跨主机，跨网段容器连通性
[root@test1 ~]# docker exec d1 ping -c 2 172.16.10.1
PING 172.16.10.1 (172.16.10.1): 56 data bytes
64 bytes from 172.16.10.1: seq=0 ttl=64 time=0.797 ms
64 bytes from 172.16.10.1: seq=1 ttl=64 time=1.501 ms
--- 172.16.10.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.797/1.149/1.501 ms
[root@test1 ~]# docker exec d1 ping -c 2 172.16.10.10
PING 172.16.10.10 (172.16.10.10): 56 data bytes
64 bytes from 172.16.10.10: seq=0 ttl=64 time=0.059 ms
64 bytes from 172.16.10.10: seq=1 ttl=64 time=0.151 ms
--- 172.16.10.10 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.059/0.105/0.151 ms
[root@test1 ~]# docker exec d1 ping -c 2 172.16.10.11
PING 172.16.10.11 (172.16.10.11): 56 data bytes
64 bytes from 172.16.10.11: seq=0 ttl=64 time=0.377 ms
64 bytes from 172.16.10.11: seq=1 ttl=64 time=1.651 ms
--- 172.16.10.11 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.377/1.014/1.651 ms
[root@test1 ~]# docker exec d1 ping -c 2 172.16.20.10
PING 172.16.20.10 (172.16.20.10): 56 data bytes
64 bytes from 172.16.20.10: seq=0 ttl=63 time=1.101 ms
64 bytes from 172.16.20.10: seq=1 ttl=63 time=2.388 ms
--- 172.16.20.10 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.101/1.744/2.388 ms
[root@test1 ~]# docker exec d1 ping -c 2 172.16.20.11
PING 172.16.20.11 (172.16.20.11): 56 data bytes
64 bytes from 172.16.20.11: seq=0 ttl=63 time=0.512 ms
64 bytes from 172.16.20.11: seq=1 ttl=63 time=3.122 ms
--- 172.16.20.11 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.512/1.817/3.122 ms
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
