目录
一、仓库类型
二、Harbor组件说明
三、harbor单机版部署
1、添加主机名解析
2、安装docker
3、安装docker-compose
4、安装harbor
5、配置CA为harbor签发证书/生成V3证书
1) 创建CA证书
2) 为harbor服务器创建证书
3)创建v3.ext文件
6、编辑harbor配置文件
7、启动harbor
四、访问harbor
1、上传镜像
1) 登录仓库
2) 上传镜像
五、部署Harbor高可用集群
1) 环境描述
2) 在两台Harbor服务器分别安装docker、docker-compose、harbor
3) 配置NFS服务器
3) 配置harbor服务挂载NFS存储
4) 部署redis服务
5) 部署postgreSQL
a. 安装postgreSQL
b. 初始化postgreSQL
c. 编辑postgreSQL配置文件
d. 指定允许远程连接数据库的客户端
e. 启动数据库
f. 创建harbor需要的数据库及远程连接用户
g. 远程主机测试连接数据库
6、安装harbor
7、编辑harbor配置文件
8、启动harbor
9、测试harbor集群
Harbor(Harbor Container Registry)是一个开源的企业级容器镜像注册中心(Container Registry),旨在提供安全、可靠、高度可扩展的容器镜像管理和分发解决方案。Harbor支持Docker镜像,并提供了多种功能,适用于容器化应用的构建、部署和管理。
看到这里有小伙伴不明白有docker的,可以去看看这篇文章,对小白非常的友好,内容非常详细。
在构建镜像仓库之前,我们先了解一下仓库的类型
一、仓库类型
- 公有仓库
- https://hub.docker.com/
- DockerHub
- 私有仓库
- registry
- harbor
市面上还有很多很多仓库类型,今天我们就用开源的最经典的harbor仓库,为大家讲讲如何构建以及如何使用harbor仓库
二、Harbor组件说明
在使用之前呢,我们先来了解一下harbor的原理,以及各个组件之间的关系
- redis
- 负责存储会话信息
- harbor-db
- 负责存储仓库数据(项目名称、镜像名称)
- registry
- 镜像的上传、下载
- Log collector
- 记录日志
- harbor-core
- 提供web UI界面
- 颁发令牌token
- webhook
- 负责搜集镜像变化的信息,通知给UI进行展示
- proxy
- 提供反向代理
- Job service
- 提供镜像复制功能
三、harbor单机版部署
1、添加主机名解析
[root@harbor ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.140.11 harbor.linux.com
2、安装docker
[root@harbor ~]# rpm -q docker-ce
docker-ce-20.10.7-3.el7.x86_64
修改镜像下载路径
[root@harbor ~]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["http://f1361db2.m.daocloud.io"]
}
3、安装docker-compose
这里已经提前下好了docker-compose,直接移动到bin目录下生效
[root@harbor ~]# mv docker-compose /usr/local/bin/
[root@harbor ~]# chmod a+x /usr/local/bin/docker-compose
4、安装harbor
harbor的下载地址:https://github.com/goharbor/harbor/tags
这里也已经提前下载好了harbor压缩包
[root@harbor ~]# tar xf harbor-offline-installer-v2.2.2.tgz
[root@harbor ~]# cp harbor/harbor.yml.tmpl harbor/harbor.yml
5、配置CA为harbor签发证书/生成V3证书
1) 创建CA证书
mkdir /opt/ssl
cd /opt/ssl
openssl genrsa -out ca.key 4096
ls
openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/CN=harbor.linux.com" -key ca.key -out ca.crt
2) 为harbor服务器创建证书
openssl genrsa -out server.key 4096
openssl req -new -sha512 -subj "/CN=harbor.linux.com" -key server.key -out server.csr
3)创建v3.ext文件
[root@localhost ssl]# cat v3.ext
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.linux.com
[root@localhost ssl]# openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key -CAcreateserial -in server.csr -out server.crt
6、编辑harbor配置文件
[root@harbor harbor]# vim harbor.yml
hostname: harbor.linux.com
# http related config
http:
# port for http, default is 80. If https enabled, this port will redirect to https port
port: 80
# https related config
https:
# https port for harbor, default is 443
port: 443
# The path of cert and key files for nginx
certificate: /opt/ssl/server.crt
private_key: /opt/ssl/server.key
7、启动harbor
[root@harbor harbor]# ./prepare
prepare base dir is set to /root/harbor
Unable to find image 'goharbor/prepare:v2.2.2' locally
v2.2.2: Pulling from goharbor/prepare
b31150c04016: Pull complete
d504272addf9: Pull complete
a9c2d9be0ec7: Pull complete
ba14108b237f: Pull complete
888a2dd12a77: Pull complete
08591f736052: Pull complete
e9a06c50605c: Pull complete
fcc257111f80: Pull complete
Digest: sha256:d12185f2c925416fa260d2af8764d8c27d35b4f66d9bcff67bf7e35d9409789e
Status: Downloaded newer image for goharbor/prepare:v2.2.2
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Successfully called func: create_root_cert
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@harbor harbor]# ./install.sh
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating harbor-db ... done
Creating harbor-portal ... done
Creating redis ... done
Creating registryctl ... done
Creating registry ... done
Creating harbor-core ... done
Creating nginx ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----
[root@harbor harbor]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
goharbor/harbor-exporter v2.2.2 d63334489be4 2 months ago 76.4MB
goharbor/chartmuseum-photon v2.2.2 c3c6b2f81c7c 2 months ago 165MB
goharbor/redis-photon v2.2.2 11a777ead643 2 months ago 69MB
goharbor/trivy-adapter-photon v2.2.2 a0cd0b967c47 2 months ago 120MB
goharbor/notary-server-photon v2.2.2 f963e34d9130 2 months ago 102MB
goharbor/notary-signer-photon v2.2.2 c3ce4df1b104 2 months ago 98.5MB
goharbor/harbor-registryctl v2.2.2 461e67c4ec3d 2 months ago 128MB
goharbor/registry-photon v2.2.2 fb9295e771e6 2 months ago 77.3MB
goharbor/nginx-photon v2.2.2 6744b15891f6 2 months ago 40.4MB
goharbor/harbor-log v2.2.2 3837bbad1328 2 months ago 108MB
goharbor/harbor-jobservice v2.2.2 c9db96b1363f 2 months ago 163MB
goharbor/harbor-core v2.2.2 2e6b040afa40 2 months ago 148MB
goharbor/harbor-portal v2.2.2 c240f7374709 2 months ago 51.1MB
goharbor/harbor-db v2.2.2 e6b67be16b5b 2 months ago 177MB
goharbor/prepare v2.2.2 eb938b7f85aa 2 months ago 165MB
[root@harbor harbor]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2c20abb4d255 goharbor/harbor-jobservice:v2.2.2 "/harbor/entrypoint.…" 2 minutes ago Up About a minute (healthy) harbor-jobservice
b5393b959230 goharbor/nginx-photon:v2.2.2 "nginx -g 'daemon of…" 2 minutes ago Up About a minute (healthy) 0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp nginx
64f26bafbd90 goharbor/harbor-core:v2.2.2 "/harbor/entrypoint.…" 2 minutes ago Up 2 minutes (healthy) harbor-core
6c8a836e6c10 goharbor/registry-photon:v2.2.2 "/home/harbor/entryp…" 2 minutes ago Up 2 minutes (healthy) registry
048bb45ebb42 goharbor/redis-photon:v2.2.2 "redis-server /etc/r…" 2 minutes ago Up 2 minutes (healthy) redis
bf345e4dbce7 goharbor/harbor-registryctl:v2.2.2 "/home/harbor/start.…" 2 minutes ago Up 2 minutes (healthy) registryctl
26eaff3ead77 goharbor/harbor-portal:v2.2.2 "nginx -g 'daemon of…" 2 minutes ago Up 2 minutes (healthy) harbor-portal
b57fdf68b23a goharbor/harbor-db:v2.2.2 "/docker-entrypoint.…" 2 minutes ago Up 2 minutes (healthy) harbor-db
61261e5bb47c goharbor/harbor-log:v2.2.2 "/bin/sh -c /usr/loc…" 2 minutes ago Up 2 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log
四、访问harbor
- 创建项目
- 创建用户
- 用户授权
1、上传镜像
1) 登录仓库
Harbor仓库启用了https后,docker服务器要能正常登录访问仓库,需要将harbor的证书拷贝到docker服务器
[root@test ~]# mkdir /etc/docker/certs.d/harbor.linux.com -p
[root@harbor ~]# scp -a /opt/ssl/harbor.crt root@192.168.140.10:/etc/docker/certs.d/harbor.linux.com
[root@localhost ~]# docker login harbor.linux.com
Username: sam
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
2) 上传镜像
[root@localhost ~]# docker tag nginx:1.14 harbor.linux.com/project_yw/nginx:1.14
[root@localhost ~]# docker push harbor.linux.com/project_yw/nginx:1.14
3)查看上传的镜像
五、部署Harbor高可用集群
1、高可用方案介绍
- 利用Harbor自带的复制功能
- 两个harbor节点在进行镜像复制时,如果其中一个节点故障, 此时有新镜像的话,并不会复制到故障节点,而且故障的harbor修复后也不会自动复制数据,会导致数据不一致问题
- 基于共享存储的方式实现
- 将数据库、redis独立出来
- harbor 1.6版本后,数据库仅支持postgreSQL
- 利用共享存储(nfs, ipsan, glusterfs)存储镜像
2、部署Harbor高可用集群
1) 环境描述
192.168.140.11 harbor服务器
192.168.140.13 harbor服务器
192.168.140.12 NFS、Redis、PostgreSQL
2) 在两台Harbor服务器分别安装docker、docker-compose、harbor
3) 配置NFS服务器
[root@share_storage ~]# mkdir /data/harbor -p
[root@share_storage ~]# chmod o+w /data/harbor/
[root@share_storage ~]# yum install -y nfs-utils rpcbind
[root@share_storage ~]# vim /etc/exports
[root@share_storage ~]# cat /etc/exports
/data/harbor 192.168.140.11(rw,no_root_squash) 192.168.140.13(rw,no_root_squash)
[root@share_storage ~]# systemctl start nfs-server
[root@share_storage ~]# systemctl enable nfs-server.service
Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
[root@share_storage ~]# showmount -e localhost
Export list for localhost:
/data/harbor 192.168.140.13,192.168.140.11
3) 配置harbor服务挂载NFS存储
[root@harbor_01 ~]# yum install -y nfs-utils
[root@harbor_01 ~]# vim /etc/fstab
192.168.140.12:/harbordata /data nfs defaults 0 0
[root@harbor_01 ~]# df -hT | grep nfs
192.168.140.12:/harbordata nfs4 18G 3.0G 15G 18% /data
4) 部署redis服务
[root@share_storage ~]# yum install -y redis
[root@share_storage ~]# vim /etc/redis.conf
bind 192.168.140.12
daemonize yes
[root@share_storage ~]# systemctl start redis
[root@share_storage ~]# systemctl enable redis
[root@share_storage ~]# netstat -antp | grep redis
tcp 0 0 192.168.140.12:6379 0.0.0.0:* LISTEN 17493/redis-server
5) 部署postgreSQL
a. 安装postgreSQL
[root@storage ~]# yum install -y cmake gcc gcc-c++ perl readline readline-devel openssl openssl-devel zlib zlib-devel ncurses-devel readline readline-devel zlib zlib-devel
[root@storage ~]# wget https://ftp.postgresql.org/pub/source/v12.2/postgresql-12.2.tar.gz
[root@storage ~]# tar xf postgresql-12.2.tar.gz
[root@storage ~]# cd postgresql-12.2/
[root@storage ~]#./configure --prefix=/usr/local/postsql
[root@storage ~]# make && make install
b. 初始化postgreSQL
[root@storage ~]# useradd postgres
[root@storage ~]# mkdir -p /work/harbor-db/{data,temp,log}
[root@storage ~]# chown -R postgres.postgres /work/harbor-db
[root@storage ~]# /usr/local/postgresql/bin/initdb --username=postgres -D /work/harbor-db/data/
c. 编辑postgreSQL配置文件
[root@storage ~]# vim /work/harbor-db/data/postgresql.conf
data_directory = '/work/harbor-db/data'
listen_addresses = '*'
port = 5432
max_connections = 100
unix_socket_directories = '/work/harbor-db/temp'
unix_socket_group = ''
unix_socket_permissions = 0777
shared_buffers = 128MB
timezone = 'Asia/Shanghai'
logging_collector = on
log_directory = '/work/harbor-db/log'
log_rotation_size = 1GB
log_timezone = 'Asia/Shanghai'
log_min_duration_statement = 100
d. 指定允许远程连接数据库的客户端
[root@storage ~]# tail -n 2 /work/harbor-db/data/pg_hba.conf
host all harbor 192.168.140.11/24 trust
host all harbor 192.168.140.13/24 trust
e. 启动数据库
[root@storage ~]# su - postgres
[root@storage ~]# /usr/local/postgresql/bin/pg_ctl -D /work/harbor-db/data/ -l /work/harbor-db/log/start.log start
[postgres@storage ~]$ netstat -antp | grep 5432
tcp 0 0 0.0.0.0:5432 0.0.0.0:* LISTEN 32215/postgres
f. 创建harbor需要的数据库及远程连接用户
[root@storage ~]# /usr/local/postgresql/bin/psql -h 127.0.0.1 -p 5432 -U postgres
postgres=# \password postgres
Enter new password:
Enter it again:
postgres=# create user harbor with password 'redhat';
CREATE ROLE
postgres=# CREATE DATABASE harbor;
CREATE DATABASE
postgres=# create database harbor_clair;
CREATE DATABASE
postgres=# create database harbor_notary_server;
CREATE DATABASE
postgres=# create database harbor_notary_signer;
CREATE DATABASE
harbor
postgres=# GRANT ALL ON DATABASE harbor to harbor;
GRANT
postgres=# GRANT ALL ON DATABASE harbor_clair to harbor;
GRANT
postgres=# GRANT ALL ON DATABASE harbor_notary_server to harbor;
GRANT
postgres=# GRANT ALL ON DATABASE harbor_notary_signer to harbor;
GRANT
g. 远程主机测试连接数据库
[root@harbor_02 ~]# psql -h 192.168.140.12 -p 5432 -U harbor -W
6、安装harbor
[root@harbor_01 ~]# tar xf harbor-offline-installer-v2.2.1.tgz
7、编辑harbor配置文件
[root@harbor_02 ~]# vim harbor/harbor.yml
hostname: harbor.linux.com
注释https相关配置,本案例中没有配置证书
#https:
# port: 443
# certificate: /usr/local/harbor/ssl/harbor.ssl
# private_key: /usr/local/harbor/ssl/harbor.key
data_volume: /data
harbor_admin_password: Harbor12345
注释或删除harbor自带的数据库配置,后续配置外部数据库连接
#database:
# password: root123
# max_idle_conns: 50
# max_open_conns: 1000
配置harbor连接外部postgresql数据库
external_database:
harbor:
host: 192.168.140.12
port: 5432
db_name: harbor
username: harbor
password: redhat
ssl_mode: disable
max_idle_conns: 2
max_open_conns: 0
notary_signer:
host: 192.168.140.12
port: 5432
db_name: harbor_notary_signer
username: harbor
password: redhat
ssl_mode: disable
notary_server:
host: 192.168.140.12
port: 5432
db_name: harbor_notary_server
username: harbor
password: redhat
ssl_mode: disable
配置连接外部redis存储会话信息
external_redis:
host: 192.168.140.12:6379
password:
registry_db_index: 1
jobservice_db_index: 2
chartmuseum_db_index: 3
trivy_db_index: 5
idle_timeout_seconds: 30
8、启动harbor
[root@harbor_01 harbor]# ./prepare
[root@harbor_01 harbor]# ./install.sh
[root@harbor_02 harbor]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7839506e6909 goharbor/nginx-photon:v2.2.2 "nginx -g 'daemon of…" 19 seconds ago Up 16 seconds (health: starting) 0.0.0.0:80->8080/tcp, :::80->8080/tcp, 0.0.0.0:443->8443/tcp, :::443->8443/tcp nginx
0685bef08838 goharbor/harbor-jobservice:v2.2.2 "/harbor/entrypoint.…" 19 seconds ago Up 16 seconds (health: starting) harbor-jobservice
0745cdc06650 goharbor/harbor-core:v2.2.2 "/harbor/entrypoint.…" 19 seconds ago Up 18 seconds (health: starting) harbor-core
2dca99a9b8d9 goharbor/harbor-portal:v2.2.2 "nginx -g 'daemon of…" 20 seconds ago Up 19 seconds (health: starting) harbor-portal
3b3f3a4d4192 goharbor/registry-photon:v2.2.2 "/home/harbor/entryp…" 20 seconds ago Up 19 seconds (health: starting) registry
db630535b7e8 goharbor/harbor-registryctl:v2.2.2 "/home/harbor/start.…" 20 seconds ago Up 19 seconds (health: starting) registryctl
27103477a551 goharbor/harbor-log:v2.2.2 "/bin/sh -c /usr/loc…" 21 seconds ago Up 20 seconds (health: starting) 127.0.0.1:1514->10514/tcp
注意:所有Harbor节点配置保持一致
9、测试harbor集群
连接任意一台harbor上传镜像,上传镜像完毕后。修改本地hosts文件解析到其他harbor服务器,再次访问harbor web界面。第一会发现不需要登录直接可以访问数据,第二会看到同样的项目及镜像