minio分布式集群的详细搭建
1，准备工作
4台服务器 ，每台服务器 ，挂载一个硬盘
以下 不重启识别硬盘
echo "- - -" > /sys/class/scsi_host/host0/scan
echo "- - -" > /sys/class/scsi_host/host1/scan
echo "- - -" > /sys/class/scsi_host/host2/scan
1、 基础准备
192.168.40.180挂载磁盘路径：/app/minio/miniodata
192.168.40.181挂载磁盘路径：/app/minio/miniodata
192.168.40.182挂载磁盘路径：/app/minio/miniodata
192.168.40.61挂载磁盘路径：/app/minio/miniodata
生产环境强烈建议至少四台机器，这也是官方的建议要求，这样的话就可以做到挂掉一台机器集群依然可以读写，挂掉两台机器集群依然可读，本文仅以四台机器为例子说明如何搭建集群
2 下载软件包
官网
https://min.io/download#/linux
wget https://dl.min.io/server/minio/release/linux-amd64/minio
3 创建minio用户（密码根据实际情况设置）并增加sudo权限
groupadd -g 10002 minio;useradd -d /home/minio -g 10002 -u 10002 -m minio;echo 'minio:Huawei12#$' | chpasswd;chage -M 99999 minio;usermod -G wheel minio
1、 创建minio的data路径
mkdir -p /app/minio/miniodata
chown -R minio:minio /app/minio
1、 上传minio.tar.gz到/home/minio下，
解压并编辑编写集群启动脚本run.sh,
其中MINIO_ACCESS_KEY、MINIO_SECRET_KEY根据实际情况设置
Minio用户编写集群启动脚本（所有节点配置文件相同）
其中，“MINIO_ACCESS_KEY”为用户名，“MINIO_SECRET_KEY”为密码，密码不能设置过于简单，不然minio会启动失败
[root@master minio]# cat run.sh
#!/bin/bash
export MINIO_ACCESS_KEY=yR3eJ5mF5bK5vE2oA7aE
export MINIO_SECRET_KEY=yH4cH8zI7gL6wG4zE0fB6xS1xN7dN2
/home/minio/minio server \
http://192.168.40.180/app/minio/miniodata http://192.168.40.181/app/minio/miniodata \
http://192.168.40.182/app/minio/miniodata http://192.168.40.61/app/minio/miniodata
[root@master minio]# pwd
/home/minio
=========================如果修改端口 使用下面方法，各节点修改======================
[root@master ~]# cat /home/minio/run.sh
#!/bin/bash
export MINIO_ACCESS_KEY=yR3eJ5mF5bK5vE2oA7aE
export MINIO_SECRET_KEY=yH4cH8zI7gL6wG4zE0fB6xS1xN7dN2
/home/minio/minio server --console-address "192.168.40.180:9001" --address "192.168.40.180:9000" \
http://192.168.40.180/app/minio/miniodata http://192.168.40.181/app/minio/miniodata \
http://192.168.40.182/app/minio/miniodata http://192.168.40.61/app/minio/miniodata
==========================
1、 编写服务脚本（root用户），如需要修改的话请编辑，否则保持默认值即可
其中，“WorkingDirectory”为启动脚本目录，“ExecStart”为指定集群启动脚本
[root@master minio]# cat minio.service
[Unit]
Description=Minio service
Documentation=https://docs.minio.io/
[Service]
User=minio
WorkingDirectory=/home/minio/
ExecStart=/home/minio/run.sh
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
将修改好的内容复制到其他节点
[minio@master ~]$ pwd
/home/minio
scp -rq * minio@192.168.40.181:/home/minio/
scp -rq * minio@192.168.40.182:/home/minio/
scp -rq * minio@192.168.40.61:/home/minio/
6、 启动测试
1）将下载好的minio文件赋予权限
sudo chmod +x /home/minio/minio
2）赋予启动脚本文件权限
sudo chmod +x /home/minio/run.sh
3）依次启动每个服务器的minio
sudo cp /home/minio/minio.service /usr/lib/systemd/system/
sudo systemctl daemon-reload
sudo systemctl start minio
sudo systemctl enable minio
4）浏览器输入集群任意节点地址+9000端口，即可访问minio，用户名密码为前面设置的“MINIO_ACCESS_KEY”和“MINIO_SECRET_KEY”，可创建“bucket”并上传文件测试
基于nginx实现loadbalancer
官网下载最新版本
https://nginx.org/en/download.html
将nginx-1.23.1.tar.gz上传到与NGINX规划所在主机上
新建目录：mkdir -p /app/nginx,然后上传
解压：cd /app/nginx ; tar -zxvf nginx-1.23.1.tar.gz
编译安装前，安装依赖包：
yum -y install gcc gcc-c++ autoconf automake make openssl openssl-devel pcre-devel
编译nginx：
cd /app/nginx/nginx-1.23.1
./configure --prefix=/app/nginx --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module
(如果报找不到文件的错，则看下/app/nginx目录下的文件)
安装：
make && make install
先测试下Nginx，然后如果是少文件，则创建。然后启动Nginx
ln -s /app/nginx/sbin/nginx /usr/local/sbin/
测试命令：nginx -t -c ./conf/nginx.conf
启动命令：nginx -c ./conf/nginx.conf
nginx 配置如下
[root@master nginx]# cat conf/nginx.conf
user nginx;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 1024;
}
include /usr/share/nginx/modules/*.conf;
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream minio {
server 192.168.40.180:9000;
server 192.168.40.181:9000;
server 192.168.40.182:9000;
server 192.168.40.61:9000;
}
upstream console {
ip_hash;
server 192.168.40.180:9001;
server 192.168.40.181:9001;
server 192.168.40.182:9001;
server 192.168.40.61:9001;
}
server {
listen 80;
listen [::]:80;
server_name minio.sfere.local;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300;
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1
proxy_http_version 1.1;
proxy_set_header Connection "";
chunked_transfer_encoding off;
proxy_pass http://minio;
}
}
server {
listen 19001;
listen [::]:19001;
server_name minio.sfere.local;
# To allow special characters in headers
ignore_invalid_headers off;
# Allow any size file to be uploaded.
# Set to a value such as 1000m; to restrict file size to a specific value
client_max_body_size 0;
# To disable buffering
proxy_buffering off;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-NginX-Proxy true;
# This is necessary to pass the correct IP to be hashed
real_ip_header X-Real-IP;
proxy_connect_timeout 300;
# To support websocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
chunked_transfer_encoding off;
proxy_pass http://console;
}
}
}
启动测试ok
http://192.168.40.180:19001/
minio挂载在本地
minio挂载在本地
1.安装s3fs: sudo apt install s3fs
2.获取Access Key和Secret Key，修改${HOME}/.passwd-s3fs，写入：
Access Key:Secret Key
chmod 600 ${HOME}/.passwd-s3fs
sudo vim /etc/fuse.conf，取消user_allow_other前的注释
3.
s3fs -o passwd_file=${HOME}/.passwd-s3fs -o url=http://minio服务器地址:9000 -o allow_other -o nonempty -o no_check_certificate -o use_path_request_style -o umask=022 桶名称 需要mount的文件夹地址
黄世宇/Shiyu Huang's Personal Page：https://huangshiyu13.github.io/
