docker私有镜像仓库harbor安装部署

一、harbor介绍:

Docker容器应用的开发和运行离不开可靠的镜像管理,虽然Docker官方也提供了公共的镜像仓库, 但是从安全和效率等方面考虑,部署我们私有环境内的Registry也是非常必要的。Harbor是由VMware 公司开源的企业级的Docker Registry管理项目,它包括权限管理(RBAC)、LDAP、日志审核、管理界面、 自我注册、镜像复制和中文支持等功能。

官网地址:https://github.com/goharbor/harbor

二、硬件准备:

  • 4C4G100G
  • IP地址:192.168.225.152
  • 主机名:harbor
  • master1节点:192.168.225.138(仓库功能测试)

三、部署docker harbor:

3.1 为harbor自签发证书:

  • 创建证书目录:
[root@harbor ~]# mkdir /data/ssl -p
[root@harbor ~]# cd /data/ssl/
  • 生成ca根证书key和ca根证书:
#生成ca-key:
[root@harbor ssl]# openssl genrsa -out ca.key 3072
#生成ca证书,有效期为3年,回车后前三个选型依次填CH、BJ、BJ,然后一路回车生成ca证书:
[root@harbor ssl]# openssl req -new -x509 -days 3650 -key ca.key -out ca.pem

1.png

  • 生成为harbor域名颁发证书的key和ca证书颁发请求:
#生成域名证书私钥:
[root@harbor ssl]# openssl genrsa -out harbor.key 3072
#生成一个ca证书请求,一会用于签发证书时使用:
[root@harbor ssl]# openssl req -new -key harbor.key -out harbor.csr

2.png

  • 生成harbor域名证书:
#签发证书:
[root@harbor ssl]# openssl x509 -req -in harbor.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out harbor.pem -days 3650

3.png
4.png

3.2 安装harbor前提环境准备:

#关闭防火墙:
[root@harbor ssl]# systemctl stop firewalld && systemctl disable firewalld
#关闭iptables防火墙,清空规则:
[root@harbor ssl]# systemctl stop iptables  && systemctl disable iptables && iptables -F
#关闭selinux,修改selinux配置文件之后,重启机器,selinux才能永久生效:
[root@harbor ssl]# setenforce 0
[root@harbor ssl]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
#配置时间同步:
[root@harbor ssl]# yum install -y ntp ntpdate
[root@harbor ssl]# ntpdate cn.pool.ntp.org
#编写同步时间计划任务:
[root@harbor ssl]# crontab -e
* */1 * * * /usr/sbin/ntpdate  cn.pool.ntp.org
[root@harbor ssl]# systemctl restart crond
#配置k8s master1的hosts文件:
[root@master1 ~]# cat /etc/hosts
192.168.225.138   master1
192.168.225.139   master2
192.168.225.140   master3
192.168.225.141   work1
192.168.225.142   work2
192.168.225.143   work3
192.168.225.152 shibosen.harbor01.com
#安装基础软件包:
[root@harbor ~]# yum install -y wget net-tools nfs-utils lrzsz gcc gcc-c++ make cmake libxml2-devel openssl-devel curl curl-devel unzip sudo ntp libaio-devel wget vim ncurses-devel autoconf automake zlib-devel python-devel epel-release openssh-server socat ipvsadm conntrack
#配置docker-ce国内yum源(阿里云):
[root@harbor ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
#安装docker-ce:
[root@harbor ~]# yum install docker-ce -y
#启动docker服务:
[root@harbor ~]# systemctl start docker && systemctl enable docker
#查看docker版本信息:
[root@harbor ~]# docker version
#开启包转发功能和修改内核参数:
[root@harbor ~]# modprobe br_netfilter
[root@harbor ~]# cat > /etc/sysctl.d/docker.conf <<EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
[root@harbor ~]# sysctl -p /etc/sysctl.d/docker.conf
#重启后模块失效,下面是开机自动加载模块的脚本
[root@harbor ~]# cat /etc/rc.sysinit
#!/bin/bash
for file in /etc/sysconfig/modules/*.modules ; do
[ -x $file ] && $file
done
#在/etc/sysconfig/modules/目录下新建文件如下
[root@harbor ~]# cat /etc/sysconfig/modules/br_netfilter.modules
modprobe br_netfilter
#增加权限
[root@harbor ~]# chmod 755 /etc/sysconfig/modules/br_netfilter.modules
#重启机器模块也会自动加载
[root@harbor ~]# lsmod |grep br_netfilter
br_netfilter 22209 0
bridge 136173 1 br_netfilter
#注意:
Docker 安装后出现:WARNING: bridge-nf-call-iptables is disabled 的解决办法:
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1

将Linux系统作为路由或者VPN服务就必须要开启IP转发功能。当linux主机有多个网卡时一个网卡收到的信息是否能够传递给其他的网卡 ,如果设置成1 的话 可以进行数据包转发,可以实现VxLAN 等功能。不开启会导致docker部署应用无法访问。
#重启docker
[root@harbor ~]# systemctl restart docker
#配置docker镜像加速器:
[root@docker ~]# sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": ["https://0210mxpv.mirror.aliyuncs.com"]
}
EOF
[root@harbor ~]# sudo systemctl daemon-reload
[root@harbor ~]# sudo systemctl restart docker

3.3 安装harbor:

#创建安装目录:
[root@harbor ssl]# mkdir /data/install -p
[root@harbor ssl]# cd /data/install/
#下载最新版harbor安装包:
[root@harbor ssl]#  wget https://github.com/goharbor/harbor/releases/download/v2.3.0-rc3/harbor-offline-installer-v2.3.0-rc3.tgz
#解压安装包,并修改harbor.yml文件:
-- 修改hostname,hostname是harbor的域名,不是宿主机的主机名;
-- 协议用https,如果有需要可以修改端口号;
-- 邮件和ldap不需要配置,在harbor的web界面可以配置;
-- 其他的配置使用默认即可;
-- 修改之后退出保存;
-- harbor默认的账号密码:admin/Harbor12345,建议修改默认密码;
[root@harbor install]# tar zxvf harbor-offline-installer-v2.3.0-rc3.tgz
[root@harbor install]# cd harbor/
[root@harbor harbor]# cp harbor.yml.tmpl harbor.yml
[root@harbor harbor]# vim harbor.yml

5.png

  • 安装docker-compose
[root@harbor-01 install]# ll
total 2527680
-rw-r--r-- 1 root root  216535040 Mar 21 09:22 centos.tar.gz
-rw-r--r-- 1 root root   12254160 Mar 21 09:19 docker-compose-Linux-x86_64.64
-rw-r--r-- 1 root root 1730312704 Mar 21 09:22 docker-harbor-2-3-0.tar.gz
drwxr-xr-x 2 root root        140 Mar 21 09:36 harbor
-rw-r--r-- 1 root root  629238614 Mar 21 09:24 harbor-offline-installer-v2.3.0-rc3.tgz

[root@harbor harbor]# mv docker-compose-Linux-x86_64.64 /usr/bin/docker-compose
[root@harbor harbor]# chmod +x /usr/bin/docker-compose

注: docker-compose项目是Docker官方的开源项目,负责实现对Docker容器集群的快速编排。Docker-Compose的工程配置文件默认为docker-compose.yml,Docker-Compose运行目录下的必要有一个docker-compose.yml。docker-compose可以管理多个docker实例。

  • 部署harbor:
[root@harbor harbor]# cd /data/install/harbor/
[root@harbor-01 harbor]# ./install.sh 
[Step 0]: checking if docker is installed ...
[Step 1]: checking docker-compose is installed ...
[Step 2]: loading Harbor images ...
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating harbor-db     ... done
Creating redis         ... done
Creating harbor-portal ... done
Creating registryctl   ... done
Creating registry      ... done
Creating harbor-core   ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----
#看到上面内容,说明安装成功部署成功;
  • 启动和关闭harbor:
#停止harbor:
[root@harbor harbor]# cd /data/install/harbor
[root@harbor harbor]# docker-compose stop
#启动harbor:
[root@harbor harbor]# cd /data/install/harbor
[root@harbor harbor]# docker-compose start

#注:如果docker-compose start启动harbor之后,还是访问不了,那就需要宿主机尝试;

四、修改DNS hosts文件:

#注:如果内网有dns服务器,直接添加A记录将harbor的域名指向harbor宿主机地址,如果没有,则直接修改客户端的hosts文件即可:
root@OpenWrt:~# cat /etc/hosts | grep 192.168.225.152
192.168.225.152 shibosen.harbor01.com

root@OpenWrt:~# /etc/init.d/dnsmasq restart
udhcpc: started, v1.30.1
udhcpc: sending discover
udhcpc: no lease, failing

#使用客户端解析,查看解析记录:
[c:\~]$ ping shibosen.harbor01.com
正在 Ping shibosen.harbor01.com [192.168.225.152] 具有 32 字节的数据:
来自 192.168.225.152 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.225.152 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.225.152 的回复: 字节=32 时间<1ms TTL=64
来自 192.168.225.152 的回复: 字节=32 时间<1ms TTL=64

192.168.225.152 的 Ping 统计信息:
    数据包: 已发送 = 4,已接收 = 4,丢失 = 0 (0% 丢失),
往返行程的估计时间(以毫秒为单位):
    最短 = 0ms,最长 = 0ms,平均 = 0ms

五、登录harbor,创建仓库:

6.png7.png8.png9.png

六、修改master1 docker daemon.json和hosts配置:

#修改hosts解析记录:
[root@master1 ~]# cat /etc/hosts | grep 192.168.225.152
192.168.225.152 shibosen.harbor01.com
#修改docker daemon.json配置文件:
[root@master1 ~]# cat /etc/docker/daemon.json 
{
 "registry-mirrors":["https://rsbud4vc.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com","http://qtid6917.mirror.aliyuncs.com", "https://rncxm540.mirror.aliyuncs.com"],

               "insecure-registries": ["shibosen.harbor01.com:4444"], 

  "exec-opts": ["native.cgroupdriver=systemd"]
} 
#重新加载配置,重启docker服务:
[root@master1 ~]# systemctl daemon-reload  && systemctl restart docker

docker客户端如果配置中添加了insecure-registary配置,就不需要在docker 客户端配置上对应证书。如果不配置就需要在/etc/docker/certs.d/目录中添加对应证书才能正常登录。不能同时修改,二选一即可。

目前添加该配置有2种常用方式:

  • 修改daemon.json文件:

[root@master1 ~]# cat /etc/docker/daemon.json
{
"registry-mirrors":["https://rsbud4vc.mirror.aliyuncs.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn","https://dockerhub.azk8s.cn","http://hub-mirror.c.163.com","http://qtid6917.mirror.aliyuncs.com", "https://rncxm540.mirror.aliyuncs.com"],

​ "insecure-registries": ["shibosen.harbor01.com:4444"],

"exec-opts": ["native.cgroupdriver=systemd"]
}

  • 修改docker.service启动文件:

[root@master1 ~]# cat /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service

[Service]
Type=notify

ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry shibosen.harbor01.com:4444
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always

七、从私有镜像仓库推送(push)拉取(pull)镜像测试:

  • 使用docker pull拉去官方nginx镜像包,作为我们后续的功能测试:
[root@master1 ~]# docker pull nginx
Using default tag: latest
latest: Pulling from library/nginx
a2abf6c4d29d: Pull complete 
a9edb18cadd1: Pull complete 
589b7251471a: Pull complete 
186b1aaa4aa6: Pull complete 
b4df32aa5a72: Pull complete 
a0bcbecc962e: Pull complete 
Digest: sha256:0d17b565c37bcbd895e9d92315a05c1c3c9a29f762b011a10c54a66cd53c9b31
Status: Downloaded newer image for nginx:latest
docker.io/library/nginx:latest
  • 使用docker tag命令修改镜像名称为对饮仓库路径名称,并登录仓库,推送nginx镜像至test仓库:
[root@master1 ~]# docker tag nginx:latest  shibosen.harbor01.com:4444/test/nginx:latest 
[root@master1 ~]# docker images shibosen.harbor01.com:4444/test/nginx
REPOSITORY                              TAG       IMAGE ID       CREATED        SIZE
shibosen.harbor01.com:4444/test/nginx   latest    605c77e624dd   2 months ago   141MB
  • 使用docker login 命令登录harbor仓库:
[root@master1 ~]# docker login shibosen.harbor01.com:4444 -u admin -p sbs285329013
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
  • 使用docker push命令将本地的nginx镜像推送至harbor中的test仓库中:
[root@master1 ~]# docker push  shibosen.harbor01.com:4444/test/nginx:latest 
The push refers to repository [shibosen.harbor01.com:4444/test/nginx]
d874fd2bc83b: Pushed 
32ce5f6a5106: Pushed 
f1db227348d0: Pushed 
b8d6e692a25e: Pushed 
e379e8aedd4d: Pushed 
2edcec3590a4: Pushed 
latest: digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3 size: 1570

10.png

  • 删除本地nginx镜像,从harbor拉取镜像测试:
[root@master1 ~]# docker rmi -f shibosen.harbor01.com:4444/test/nginx:latest 
Untagged: shibosen.harbor01.com:4444/test/nginx:latest
Untagged: shibosen.harbor01.com:4444/test/nginx@sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3

[root@master1 ~]# docker images shibosen.harbor01.com:4444/test/nginx:latest
REPOSITORY   TAG       IMAGE ID   CREATED   SIZE

[root@master1 ~]# docker pull shibosen.harbor01.com:4444/test/nginx:latest
latest: Pulling from test/nginx
Digest: sha256:ee89b00528ff4f02f2405e4ee221743ebc3f8e8dd0bfd5c4c20a2fa2aaa7ede3
Status: Downloaded newer image for shibosen.harbor01.com:4444/test/nginx:latest
shibosen.harbor01.com:4444/test/nginx:latest

[root@master1 ~]# docker images shibosen.harbor01.com:4444/test/nginx:latest
REPOSITORY                              TAG       IMAGE ID       CREATED        SIZE
shibosen.harbor01.com:4444/test/nginx   latest    605c77e624dd   2 months ago   141MB

本文正在参加云原生有奖征文活动】,活动链接:https://ost.51cto.com/posts/12598