Proxmox VE6 集群记录

3台服务器

pve01 192.168.0.100/24 172.16.0.100/24(ceph 存储网络) 
pve02 192.168.0.110/24 172.16.0.110/24(ceph 存储网络)
pve03 192.168.0.120/24 172.16.0.120/24(ceph 存储网络)

镜像下载
地址:https://www.proxmox.com/en/downloads

安装超简单,略过。

修改主机名并配置hosts

vi /etc/hostname

vim /etc/hosts

#public-network
192.168.0.100 pve01
192.168.0.110 pve02
192.168.0.120 pve03

#cluster-network
172.16.0.100 pve01
172.16.0.110 pve02
172.16.0.120 pve03

安装NTP服务,设置时钟同步服务器


sudo apt-get install -y ntp
vim /etc/ntp.conf

pool ntp1.aliyun.com iburst
pool ntp2.aliyun.com iburst
pool ntp3.aliyun.com iburst
pool ntp4.aliyun.com iburst

设置ssh互信key登录

ssh-keygen
ssh-copy-id root@pve01
ssh-copy-id root@pve02
ssh-copy-id root@pve03

修改pve源

SSH登陆各节点

将/etc/apt/sources.list.d/pve-enterprise.list  文件内的唯一一条记录注释掉:
#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise
echo "#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise" > /etc/apt/sources.list.d/pve-enterprise.list
wget -q -O- 'http://download.proxmox.com/debian/proxmox-ve-release-6.x.gpg' | apt-key add -
echo "deb http://download.proxmox.com/debian/pve buster pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
apt update && apt dist-upgrade
使用国内源:

分两部份:

Debian系统:
编辑/etc/apt/sources.list

[163.com]
 deb http://mirrors.163.com/debian/ buster  main non-free contrib
 deb http://mirrors.163.com/debian/ buster-updates  main non-free contrib
 deb http://mirrors.163.com/debian/ buster-backports main non-free contrib
 deb-src  http://mirrors.163.com/debian/ buster main non-free contrib
 deb-src  http://mirrors.163.com/debian/ buster-updates main non-free contrib
 deb-src  http://mirrors.163.com/debian/ buster-backports main non-free contrib
 deb http://mirrors.163.com/debian-security/ buster/updates main non-free contrib
 deb-src http://mirrors.163.com/debian-security/ buster/updates main non-free contrib

 [aliyun.com]
deb http://mirrors.aliyun.com/debian/ buster main non-free contrib
deb-src http://mirrors.aliyun.com/debian/ buster main non-free contrib
deb http://mirrors.aliyun.com/debian-security buster/updates main
deb-src http://mirrors.aliyun.com/debian-security buster/updates main
deb http://mirrors.aliyun.com/debian/ buster-updates main non-free contrib
deb-src http://mirrors.aliyun.com/debian/ buster-updates main non-free contrib
deb http://mirrors.aliyun.com/debian/ buster-backports main non-free contrib
deb-src http://mirrors.aliyun.com/debian/ buster-backports main non-free contrib

proxmox VE:
编辑/etc/apt/surces.list.d/pve-no-subscription.list 使用中科大的源

#deb http://download.proxmox.com/debian/pve buster pve-no-subscription
deb https://mirrors.ustc.edu.cn/proxmox/debian/pve buster pve-no-subscription

创建pve集群

pve01 上执行:
pvecm create my-data-center

pve02 、pve03 上执行:
pvecm add 192.168.0.100

加入pve01 的群集

CEPH搭建及挂载

在每个节点安装ceph , 可以使用参数 --version 指定安装版本如: pveceph install --version luminous
pveceph install

安装完成后,配置ceph集群存储网络
pveceph init --network 172.16.0.0/24

创建ceph集群Mon监控 、ceph mgr、OSDs - 每台执行

pveceph createmon
pveceph createmgr
pveceph createosd /dev/sdb
pveceph createosd /dev/sdc

创建集群存储资源池ceph osd pool create 资源池名称 128 128 (这里只要一台执行)
ceph osd pool create pvepool1 128 128

Pool 对应 PG PGP数量的计算公式:
Total PGs = ((Total_number_of_OSD * Target PGs per OSD) / max_replication_count) / pool_count
Target PGs per OSD 通常被设置为 100
最终的结果取2的幂次方最靠近的数,这里为128

激活pvepool1为rbd存储池,用于存储pve的磁盘映像跟容器
ceph osd pool application enable pvepool1 rbd

挂载ceph rbd 磁盘

#在数据中心-存储-添加RBD,填写ID,设置存储对象,这样集群节点就都会挂载上这个磁盘