springboot通过ssl链接mongodb

转载

技术博客领航者 2024-09-14 23:43:38

文章标签 mongodb nosql 数据库 java linux 文章分类 MongoDB 数据库

前言

业务场景

用户在拉勾网投递简历时，我们会为每次投递的简历生成一份快照，将快照信息存储到 MongoDB中。功能需求：搭建MongoDB分片集群，模拟简历快照数据进行操作，具体要求如下：

(1) 如图搭建一个分片集群，要求每个分片节点中的复制集含有一个仲裁节点

(2) 使用权限控制，建立要访问的数据库lg_resume，这个账号名字是lagou_gx、密码是abc321 这个账号对数据库有读写权限

(3) 使用SpringBoot 进行访问分片集群，对lg_resume 库中的lg_resume_datas 进行增加和查询操作

网络拓扑如下：

springboot通过ssl链接mongodb_数据库

思路分析

1 搭建复制集群的环境

由网络拓扑结构可知，该MongoDB分片集群中总共3类角色

1 路由节点（1个）；

2 配置节点集群（3个）；

3 复制节点集群，每个集群中4个MongoDB实例（4个）

共1+3+4*4=20个MongDB实例.

2 集群添加权限控制

路由节点生成testKeyFile.file并发送到集群中所有的节点。并创建一个账户授予读写权限。

3 springboot访问

编写 Repository 接口继承 MongoRepository

一搭建MongoDB复制分片集群

1.1 搭建单机版monogoDB

tar -xvf  mongodb-linux-x86_64-4.1.3.tgz 


 mv mongodb-linux-x86_64-4.1.3 mongodb

 mv mongodb /

cd /mongodb
mkdir single

我们在single文件下创建配置文件mongo.conf，内容如下：

dbpath=/mongodb/single/data/
port=27017
bind_ip=0.0.0.0
fork=true
logpath = /mongodb/single/data/MongoDB.log
logappend = true
auth=false

启动

/mongodb/bin/mongod -f /mongodb/single/mongo.conf

springboot通过ssl链接mongodb_mongodb_02

我们使用自带客户端进行登录

/mongodb/bin/mongo

mongo的标准格式为

mongo --host=主机IP --port=端口

springboot通过ssl链接mongodb_java_03

1.2 搭建mongoDB复制集群

1.2.1 安装psmisc

因为mongoDB集群可以使用一台机器实现，或者对等数量的虚拟机，我们采用第一种方式方便快捷（主要是因为穷，囧！）。故我们安装psmisc工具帮助我们批量的杀程序；

如杀掉所有的java进程

killall java

我们使用yum进行安装

yum install -y psmisc

1.2.2 搭建配置节点集群

我们创建一个目录用于存放配置节点集群

mkdir -p /mongodb/config/config1

参照单机的搭建方式，创建配置文件，配置如下：

# 数据库文件位置
dbpath=/mongodb/config/config1
#日志文件位置
logpath=/mongodb/config/config1/config1.log
# 以追加方式写入日志
logappend=true
# 是否以守护进程方式运行
fork = true
bind_ip=0.0.0.0
port = 17011
# 表示是一个配置服务器
configsvr=true
#配置服务器副本集名称
replSet=configsvr

重复1.2.2的步骤，创建复制集群另外两个节点17013和17015

1.2.3 搭建配置分片集群

创建4个分片集群文件夹

mkdir -p /mongodb/shard/shard1/shard1-37011
 mkdir -p /mongodb/shard/shard1/shard1-37013
 mkdir -p /mongodb/shard/shard1/shard1-37015
 mkdir -p /mongodb/shard/shard1/shard1-37017


 mkdir -p /mongodb/shard/shard2/shard2-47011
 mkdir -p /mongodb/shard/shard2/shard2-47013
 mkdir -p /mongodb/shard/shard2/shard2-47015
 mkdir -p /mongodb/shard/shard2/shard2-47017


mkdir -p /mongodb/shard/shard3/shard3-57011
 mkdir -p /mongodb/shard/shard3/shard3-57013
 mkdir -p /mongodb/shard/shard3/shard3-57015
 mkdir -p /mongodb/shard/shard3/shard3-57017


 mkdir -p /mongodb/shard/shard4/shard4-58011
 mkdir -p /mongodb/shard/shard4/shard4-58013
 mkdir -p /mongodb/shard/shard4/shard4-58015
 mkdir -p /mongodb/shard/shard4/shard4-58017

以第一个分片集群shard1为例

其配置文件如下

dbpath=/mongodb/shard/shard1/shard1-37011
bind_ip=0.0.0.0
port=37011
fork=true
logpath=shard/shard1/shard1-37011.log
replSet=shard1
shardsvr=true



dbpath=/mongodb/shard/shard1/shard1-37013
bind_ip=0.0.0.0
port=37013
fork=true
logpath=shard/shard1/logs/shard1-37013.log
replSet=shard1
shardsvr=true



dbpath=/mongodb/shard/shard1/shard1-37015
bind_ip=0.0.0.0
port=37015
fork=true
logpath=shard/shard1/logs/shard1-37015.log
replSet=shard1
shardsvr=true


dbpath=/mongodb/shard/shard1/shard1-37017
bind_ip=0.0.0.0
port=37017
fork=true
logpath=shard/shard1/logs/shard1-37017.log
replSet=shard1
shardsvr=true

shard2配置文件

dbpath=/mongodb/shard/shard2/shard2-47011
bind_ip=0.0.0.0
port=47011
fork=true
logpath=shard/shard2/shard2-47011.log
replSet=shard2
shardsvr=true



dbpath=/mongodb/shard/shard2/shard2-47013
bind_ip=0.0.0.0
port=47013
fork=true
logpath=shard/shard2/logs/shard2-47013.log
replSet=shard2
shardsvr=true



dbpath=/mongodb/shard/shard2/shard2-47015
bind_ip=0.0.0.0
port=47015
fork=true
logpath=shard/shard2/logs/shard2-47015.log
replSet=shard2
shardsvr=true


dbpath=/mongodb/shard/shard2/shard2-47017
bind_ip=0.0.0.0
port=47017
fork=true
logpath=shard/shard2/logs/shard2-47017.log
replSet=shard2
shardsvr=true

shard3配置文件

dbpath=/mongodb/shard/shard3/shard3-57011
bind_ip=0.0.0.0
port=57011
fork=true
logpath=shard/shard3/shard3-57011.log
replSet=shard3
shardsvr=true



dbpath=/mongodb/shard/shard3/shard3-57013
bind_ip=0.0.0.0
port=57013
fork=true
logpath=shard/shard3/logs/shard3-57013.log
replSet=shard3
shardsvr=true



dbpath=/mongodb/shard/shard3/shard3-57015
bind_ip=0.0.0.0
port=57015
fork=true
logpath=shard/shard3/logs/shard3-57015.log
replSet=shard3
shardsvr=true


dbpath=/mongodb/shard/shard3/shard3-57017
bind_ip=0.0.0.0
port=57017
fork=true
logpath=shard/shard3/logs/shard3-57017.log
replSet=shard3
shardsvr=true

shard4配置文件

dbpath=/mongodb/shard/shard4/shard4-58011
bind_ip=0.0.0.0
port=58011
fork=true
logpath=shard/shard4/shard4-58011.log
replSet=shard4
shardsvr=true



dbpath=/mongodb/shard/shard4/shard4-58013
bind_ip=0.0.0.0
port=58013
fork=true
logpath=shard/shard4/logs/shard4-58013.log
replSet=shard4
shardsvr=true



dbpath=/mongodb/shard/shard4/shard4-58015
bind_ip=0.0.0.0
port=58015
fork=true
logpath=shard/shard4/logs/shard4-58015.log
replSet=shard4
shardsvr=true


dbpath=/mongodb/shard/shard4/shard4-58017
bind_ip=0.0.0.0
port=58017
fork=true
logpath=shard/shard4/logs/shard4-58017.log
replSet=shard4
shardsvr=true

1.2.4 搭建mongoDB路由节点

创建路由文件夹router

mkdir -p /mongodb/router

创建路由配置文件，内容如下

vim /mongodb/router/router27017.conf

# 数据库文件位置
dbpath=/mongodb/router
#日志文件位置
logpath=/mongodb/router/config1.log
# 以追加方式写入日志
logappend=true
# 是否以守护进程方式运行
fork = true
bind_ip=0.0.0.0
port = 27017
# 表示是一个配置服务器
configsvr=true
#配置服务器副本集名称
replSet=configsvr

1.2.5 启动集群

启动配置集群

进入任意配置节点，添加配置集群

/mongodb/bin/mongo --port 17017
use admin
var cfg ={"_id":"configsvr",
"members":[
{"_id":1,"host":"192.168.126.128:17011"},
{"_id":2,"host":"192.168.126.128:17013"},
{"_id":3,"host":"192.168.126.128:17015"}]
};
rs.initiate(cfg)

启动分片集群1

进入shard1中任意节点，添加分片集群shard1

var cfg ={"_id":"shard1",
"protocolVersion" : 1,
"members":[
{"_id":1,"host":"192.168.126.128:37011","priority":10},
{"_id":2,"host":"192.168.126.128:37013","priority":5},
{"_id":3,"host":"192.168.126.128:37015","priority":0},
{"_id":4,"host":"192.168.126.128:37017","arbiterOnly":true}
]
};
rs.initiate(cfg)
rs.status()

启动分片集群2

进入shard2中任意节点，添加分片集群shard2

var cfg ={"_id":"shard2",
"protocolVersion" : 1,
"members":[
{"_id":1,"host":"192.168.126.128:47011","priority":10},
{"_id":2,"host":"192.168.126.128:47013","priority":5},
{"_id":3,"host":"192.168.126.128:47015","priority":0},
{"_id":4,"host":"192.168.126.128:47017","arbiterOnly":true}
]
};
rs.initiate(cfg)
rs.status()

启动分片集群3

进入shard3中任意节点，添加分片集群shard3

var cfg ={"_id":"shard3",
"protocolVersion" : 1,
"members":[
{"_id":1,"host":"192.168.126.128:57011","priority":10},
{"_id":2,"host":"192.168.126.128:57013","priority":5},
{"_id":3,"host":"192.168.126.128:57015","priority":0},
{"_id":4,"host":"192.168.126.128:57017","arbiterOnly":true}
]
};
rs.initiate(cfg)
rs.status()

启动分片集群4

进入shard4中任意节点，添加分片集群shard4

var cfg ={"_id":"shard4",
"protocolVersion" : 1,
"members":[
{"_id":1,"host":"192.168.126.128:58011","priority":10},
{"_id":2,"host":"192.168.126.128:58013","priority":5},
{"_id":3,"host":"192.168.126.128:58015","priority":0},
{"_id":4,"host":"192.168.126.128:58017","arbiterOnly":true}
]
};
rs.initiate(cfg)
rs.status()

启动路由节点

进入路由节点，添加分片集群

/mongodb/bin/mongo --port 27017
sh.status()
sh.addShard("shard1/192.168.126.128:37011,192.168.126.128:37013,192.168.126.128:37015,192.168.126.128:37017");
sh.addShard("shard2/192.168.126.128:47011,192.168.126.128:47013,192.168.126.128:47015,192.168.126.128:47017");
sh.addShard("shard3/192.168.126.128:57011,192.168.126.128:57013,192.168.126.128:57015,192.168.126.128:57017");
sh.addShard("shard4/192.168.126.128:58011,192.168.126.128:58013,192.168.126.128:58015,192.168.126.128:58017");
sh.status()

1.2.6 编写一键启动脚本

vim /mongodb/bin/startAll.sh




echo "  =============================================    开始启动mongoDB集群   ================================================"
echo  "===================================================      开始启动配置节点          ==========================================================="





/mongodb/bin/mongod -f /mongodb/config/config1/17011conf.conf
/mongodb/bin/mongod -f /mongodb/config/config2/17013conf.conf 
/mongodb/bin/mongod -f /mongodb/config/config3/17015conf.conf
echo  "========================================================   配置节点启动完毕              ============================================================================="

echo  "========================================================   shard1集群开始启动              ============================================================================="

/mongodb/bin/mongod -f /mongodb/shard/shard1/shard1-37011/37011.conf
/mongodb/bin/mongod -f /mongodb/shard/shard1/shard1-37013/37013.conf
/mongodb/bin/mongod -f /mongodb/shard/shard1/shard1-37015/37015.conf
/mongodb/bin/mongod -f /mongodb/shard/shard1/shard1-37017/37017.conf

echo  "========================================================   shard1集群启动完毕              ============================================================================="
echo  "========================================================   shard2集群开始启动              ============================================================================="
/mongodb/bin/mongod -f /mongodb/shard/shard1/shard1-37011/37011.conf
/mongodb/bin/mongod -f /mongodb/shard/shard2/shard2-47011/47011.conf
/mongodb/bin/mongod -f /mongodb/shard/shard2/shard2-47013/47013.conf
/mongodb/bin/mongod -f /mongodb/shard/shard2/shard2-47015/47015.conf
/mongodb/bin/mongod -f /mongodb/shard/shard2/shard2-47017/47017.conf

echo  "========================================================   shard2集群启动完毕              ============================================================================="

echo  "========================================================   shard3集群开始启动              ============================================================================="

/mongodb/bin/mongod -f /mongodb/shard/shard3/shard3-57011/57011.conf
/mongodb/bin/mongod -f /mongodb/shard/shard3/shard3-57013/57013.conf
/mongodb/bin/mongod -f /mongodb/shard/shard3/shard3-57015/57015.conf
/mongodb/bin/mongod -f /mongodb/shard/shard3/shard3-57017/57017.conf


echo  "========================================================   shard3集群启动完毕              ============================================================================="
echo  "========================================================   shard4集群开始启动              ============================================================================="

/mongodb/bin/mongod -f /mongodb/shard/shard4/shard4-58011/58011.conf 
/mongodb/bin/mongod -f /mongodb/shard/shard4/shard4-58013/58013.conf 
/mongodb/bin/mongod -f /mongodb/shard/shard4/shard4-58015/58015.conf 
/mongodb/bin/mongod -f /mongodb/shard/shard4/shard4-58017/58017.conf 


echo  "========================================================   shard4集群启动完毕              ============================================================================="
echo  "========================================================   路由节点开始启动              ============================================================================="


/mongodb/bin/mongos -f /mongodb/router/router27017.conf

echo  "========================================================   路由节点启动完毕              ============================================================================="
echo  "========================================================   mongodb集群启动完毕              ============================================================================="

授予执行权限

chmod +x /mongodb/bin/startAll.sh

1.2.7 集群开启数据库和集合分片

登录路由节点

/mongodb/bin/mongo --port 27017

sh.enableSharding("lg_resume")

sh.shardCollection("lg_resume.lagou_resume_datas",{"name":"hashed"})

1.2.8 创建集群用户

/mongodb/bin/momgo --port

use admin



db.createUser(
... {
... user:"root",
... pwd:"123456",
... roles:[{role:"root",db:"admin"}]
... })


 use lg_resume;


db.createUser({
... user:"lagou_gx",
... pwd:"123456",
... roles:[{role:"readWrite",db:"lg_resume"}]
... })



db.createUser({
... user:"lagou_gx",
... pwd:"123456",
... roles:[{role:"read",db:"lg_resume"}]
... })

1.2.9 分片集群安全认证

首先我们需要将所有的mongo关闭

killall mongod
killall mongos

生成密钥文件并修改权限

openssl rand -base64 756 > /mongodb/bin/testKeyFile.file
chmod 600 /mongodb/bin/testKeyFile.file

配置节点集群和分片节点集群开启安全认证和指定密钥文件

auth=true
keyFile=/mongodb/bin/testKeyFile.file

在路由配置文件中设置密钥文件

keyFile/mongodb/bin/testKeyFile.file

1.3 代码实例

新建springboot,pom.xml文件内容如下:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.lagou</groupId>
    <artifactId>mongo_springboot_repository</artifactId>
    <version>1.0-SNAPSHOT</version>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
            <version>2.2.2.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>2.2.2.RELEASE</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>11</source>
                    <target>11</target>
                    <testSource>11</testSource>
                    <testTarget>11</testTarget>
                </configuration>
                <version>3.8.1</version>
            </plugin>
        </plugins>
    </build>
</project>

application.properties

spring.data.mongodb.host=192.168.126.128
spring.data.mongodb.port=27017
spring.data.mongodb.database=lg_resume
spring.data.mongodb.username=lagou_gx
spring.data.mongodb.password=123456

建立resume实体类

package com.lagou.bean;

import org.springframework.data.mongodb.core.mapping.Document;

import java.util.Date;

@Document("lg_resume_datas")
public class Resume {
    @Override
    public String toString() {
        return "Resume{" +
                "id='" + id + '\'' +
                ", name='" + name + '\'' +
                ", city='" + city + '\'' +
                ", birthday=" + birthday +
                ", expectSalary=" + expectSalary +
                '}';
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getCity() {
        return city;
    }

    public void setCity(String city) {
        this.city = city;
    }

    public Date getBirthday() {
        return birthday;
    }

    public void setBirthday(Date birthday) {
        this.birthday = birthday;
    }

    public double getExpectSalary() {
        return expectSalary;
    }

    public void setExpectSalary(double expectSalary) {
        this.expectSalary = expectSalary;
    }

    public Resume() {
    }

    public Resume(String id, String name, String city, Date birthday, double expectSalary) {
        this.id = id;
        this.name = name;
        this.city = city;
        this.birthday = birthday;
        this.expectSalary = expectSalary;
    }

    private String id;
    private String name;
    private String city;
    private Date birthday;
    private  double  expectSalary;
}

建立ResumeRepository

package com.lagou.repository;

import com.lagou.bean.Resume;
import org.springframework.data.mongodb.repository.MongoRepository;

import java.util.List;

public interface ResumeRepository extends MongoRepository<Resume,String> {
    List<Resume>  findByNameEquals(String name);
    List<Resume>  findByNameAndExpectSalary(String name,double expectSalary);
}

建立测试类

package com.lagou;

import com.lagou.bean.Resume;
import com.lagou.repository.ResumeRepository;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.List;

/**
 * <p>Title: mongo Repository测试</p>
 * <p>Description: </p>
 * <p>Copyright: Copyright (c) 2020</p>
 * <p>Company: http://www.ubisys.com.cn/</p>
 *
 * @Auther: cw
 * @Date: 2020/8/29 00:29
 */
@RunWith(SpringRunner.class)
@SpringBootTest(classes = MongoRepositoryMain.class)
public class TestMongoDB {


    @Autowired
    private ResumeRepository resumeRepository;


    /**
     * 测试 写
     */
    @Test
    public void testMongoWrite() {
        Resume resume = new Resume();
        resume.setName("chengdaotest");
        resume.setExpectSalary(1);
        resume.setCity("bj");
        resumeRepository.save(resume);
    }


    /**
     * 测试读
     */
    @Test
    public void testMongoRead() {

        List<Resume> list = resumeRepository.findByNameEquals("chengdaotest");
        list.forEach(resume -> {
            System.out.println(resume.toString());
        });

    }


    /**
     * 测试分片
     */
    @Test
    public void testMongoShard() {
        for (int i = 1; i < 1000; i++) {
            Resume resume = new Resume();
            resume.setName("chengdaotest1" + i);
            resume.setExpectSalary(1);
            resume.setCity("bj");
            resumeRepository.save(resume);
        }

    }


}

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。