Technology技术

Minio分布式集群部署+nginx负载均衡

2019年6月24日
0

一、Minio分布式集群搭建（多机器多硬盘）

节点 数据目录（不能是/root下的） 进程/脚本目录

10.34.252.87	/home/minio/{data1.data2}	/home/minio/{run}
10.34.252.88	/home/minio/{data1.data2}	/home/minio/{run}
10.34.252.89	/home/minio/{data1.data2}	/home/minio/{run}
10.34.252.93	/home/minio/{data1.data2}	/home/minio/{run}

2、从官网获取Minio二进制文件并上传至4台集群主机：

wget https://dl.min.io/server/minio/release/linux-amd64/minio

3、创建目录(4台集群主机都执行)

mkdir -p /home/minio/{run,data1,data2} && mkdir -p /etc/minio

4、集群启动文件配置(4台集群主机都执行)

Minio默认9000端口，在配置文件中加入–address “127.0.0.1:9029” 可更改端口
注意 ：
MINIO_ROOT_USER：用户名，长度最小是5个字符
MINIO_ROOT_PASSWORD：密码，密码不能设置过于简单，不然minio会启动失败，长度最小是8个字符
–config-dir：指定集群配置文件目录
vim /home/minio/run/minio-run.sh

（1）集群节点10.34.252.87内容为:

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin123
/home/minio/run/minio server --config-dir /etc/minio \
--address "10.34.252.87:9000" \
http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 \
http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 \
http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 \
http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

（2）集群节点10.34.252.88内容为:

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin123
/home/minio/run/minio server --config-dir /etc/minio \
--address "10.34.252.88:9000" \
http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 \
http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 \
http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 \
http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

（3）集群节点10.34.252.89内容为:

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin123
/home/minio/run/minio server --config-dir /etc/minio \
--address "10.34.252.89:9000" \
http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 \
http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 \
http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 \
http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

（4）集群节点10.34.252.93内容为:

#!/bin/bash
export MINIO_ROOT_USER=admin
export MINIO_ROOT_PASSWORD=admin123
/home/minio/run/minio server --config-dir /etc/minio \
--address "10.34.252.93:9000" \
http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 \
http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 \
http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 \
http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

5、创建Minio.server,将minio加入系统服务（4台集群主机都执行）

vim /usr/lib/systemd/system/minio.service
[Unit]
Description=Minio service
Documentation=https://docs.minio.io/

[Service]
WorkingDirectory=/home/minio/run/
ExecStart=/home/minio/run/minio-run.sh

Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

6、权限修改（4台集群主机都执行）

chmod +x /usr/lib/systemd/system/minio.service && chmod +x /home/minio/run/minio && chmod +x /home/minio/run/minio-run.sh

7、启动集群（4台集群主机都执行）

systemctl daemon-reload
systemctl start minio
systemctl enable minio

查看集群状态

systemctl status minio.service -l

日志类似以下内容，则启动成功:

[root@cn-gri-gicar-bdms-05 csz]# systemctl status minio.service -l
● minio.service - Minio service
   Loaded: loaded (/usr/lib/systemd/system/minio.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-01-22 10:44:29 CST; 43min ago
     Docs: https://docs.minio.io/
 Main PID: 3663052 (minio-run.sh)
    Tasks: 41
   Memory: 89.2M
   CGroup: /system.slice/minio.service
           ├─3663052 /bin/bash /root/docker/minio/run/minio-run.sh
           └─3663053 /root/docker/minio/run/minio server --config-dir /etc/minio --address 10.34.252.87:9000 http://10.34.252.87/home/minio/data1 http://10.34.252.87/home/minio/data2 http://10.34.252.88/home/minio/data1 http://10.34.252.88/home/minio/data2 http://10.34.252.89/home/minio/data1 http://10.34.252.89/home/minio/data2 http://10.34.252.93/home/minio/data1 http://10.34.252.93/home/minio/data2

Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Endpoint: http://10.34.252.87:9000
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Browser Access:
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: http://10.34.252.87:9000
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Object API (Amazon S3 compatible):
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Go:         https://docs.min.io/docs/golang-client-quickstart-guide
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Java:       https://docs.min.io/docs/java-client-quickstart-guide
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Python:     https://docs.min.io/docs/python-client-quickstart-guide
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
Jan 22 10:44:32 cn-gri-gicar-bdms-05 minio-run.sh[3663052]: Waiting for all MinIO IAM sub-system to be initialized.. lock acquired

8、登录页面测试

http://10.34.252.87:9000
http://10.34.252.88:9000
http://10.34.252.89:9000
http://10.34.252.93:9000

9：负载均衡配置

ser  nginx;
worker_processes  auto;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  4096;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;

    # include /etc/nginx/conf.d/*.conf;

    upstream minio {
     server 155.159.254.195:9000; #负载轮询算法
     server 103.135.147.35:9000;
     server 155.159.254.229:9000;
     server 155.159.254.24:9000;

    }

  #  upstream console {
  #      ip_hash;
  #   server 155.159.254.195:9000; #负载轮询算法
  #   server 103.135.147.35:9000;
  #   server 155.159.254.229:9000;
  #   server 155.159.254.24:9000;
 #  }

    server {
        listen       80;
        listen  [::]:80;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 300;
            # Default is HTTP/1, keepalive is only enabled in HTTP/1.1
            proxy_http_version 1.1;
            proxy_set_header Connection "";
            chunked_transfer_encoding off;

            proxy_pass http://minio; #反向代理 指向minio参数地址
        }
    }

    server {
        listen       80;
        listen  [::]:80;
        server_name  localhost;

        # To allow special characters in headers
        ignore_invalid_headers off;
        # Allow any size file to be uploaded.
        # Set to a value such as 1000m; to restrict file size to a specific value
        client_max_body_size 0;
        # To disable buffering
        proxy_buffering off;

        location / {
            proxy_set_header Host $http_host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
            proxy_set_header X-NginX-Proxy true;

            # This is necessary to pass the correct IP to be hashed
            real_ip_header X-Real-IP;

            proxy_connect_timeout 300;
            
            # To support websocket
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            
            chunked_transfer_encoding off;

           # proxy_pass http://console; #反向代理console指向参数地址
        }
    }
}

我完成minio测试地址

www.wsnem.icu
测试账户wusunan123
测试密码wusunan123
Next
No Comments

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注