Mongodb

Mongodb

基础

端口

默认端口

介绍

27017

mongod 和 mongos 实例的默认端口。你可以通过 port 或 –port 改变该端口。

27018

设置 –shardsvr 运行变量或在配置文件里设置 clusterRole 为 shardsvr 时的默认端口。

27019

设置 –configsvr 运行变量或在配置文件中将 clusterRole 设置为 configsvr 时的默认端口。

28017

系统状态网页的默认端口。系统状态网络页面永远可以在比 port 大 1000 的端口反问。

Mongodb 概念

SQL术语/概念

MongoDB术语/概念

解释/说明

database

database

数据库

table

collection

数据库表/集合

row

document

数据记录行/文档

column

field

数据字段/域

index

index

索引

table joins

表连接,MongoDB不支持

primary key

primary key

主键,MongoDB自动将_id字段设置为主键

数据类型

数据类型

描述

String

字符串。存储数据常用的数据类型。在 MongoDB 中，UTF-8 编码的字符串才是合法的。

Integer

整型数值。用于存储数值。根据你所采用的服务器，可分为 32 位或 64 位。

Boolean

布尔值。用于存储布尔值（真/假）。

Double

双精度浮点值。用于存储浮点值。

Min/Max keys

将一个值与 BSON（二进制的 JSON）元素的最低值和最高值相对比。

Array

用于将数组或列表或多个值存储为一个键。

Timestamp

时间戳。记录文档修改或添加的具体时间。

Object

用于内嵌文档。

Null

用于创建空值。

Symbol

符号。该数据类型基本上等同于字符串类型，但不同的是，它一般用于采用特殊符号类型的语言。

Date

日期时间。用 UNIX 时间格式来存储当前日期或时间。你可以指定自己的日期时间：创建 Date 对象，传入年月日信息。

Object ID

对象 ID。用于创建文档的 ID。

Binary Data

二进制数据。用于存储二进制数据。

Code

代码类型。用于在文档中存储 JavaScript 代码。

Regular expression

正则表达式类型。用于存储正则表达式。

Mongodb 连接
mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]
mongodb:// 这是固定的格式，必须要指定。

username:password@ 可选项，如果设置，在连接数据库服务器之后，驱动都会尝试登录这个数据库

host1 必须的指定至少一个host, host1 是这个URI唯一要填写的。它指定了要连接服务器的地址。如果要连接复制集，请指定多个主机地址。

portX 可选的指定端口，如果不填，默认为27017

/database 如果指定username:password@，连接并验证登录指定数据库。若不指定，默认打开 test 数据库。

?options 是连接选项。如果不使用/database，则前面需要加上/。所有连接选项都是键值对name=value，键值对之间通过&或;（分号）隔开

shell

database

创建database、没有则创建、已存在则切换

use DATABASE_NAME
show dbs

查询所有数据库

show dbs

可以看到，我们刚创建的数据库 runoob 并不在数据库的列表中， 要显示它，我们需要向 runoob 数据库插入一些数据。

> db.runoob.insert({"name":"菜鸟教程"})
WriteResult({ "nInserted" : 1 })
> show dbs
admin   0.000GB
config  0.000GB
local   0.000GB
runoob  0.000GB
db.orders.insertMany([
  {
    order_id: 101,
    order_date: ISODate("2020-07-30T10:08:22.001Z"),
    customer_id: 1001,
    price: NumberDecimal("50.50"),
    product: {
      name: 'scooter',
      description: 'Small 2-wheel scooter'
    },
    order_status: false
  },
  {
    order_id: 102, 
    order_date: ISODate("2020-07-30T10:11:09.001Z"),
    customer_id: 1002,
    price: NumberDecimal("15.00"),
    product: {
      name: 'car battery',
      description: '12V car battery'
    },
    order_status: false
  },
  {
    order_id: 103,
    order_date: ISODate("2020-07-30T12:00:30.001Z"),
    customer_id: 1003,
    price: NumberDecimal("25.25"),
    product: {
      name: 'hammer',
      description: '16oz carpenter hammer'
    },
    order_status: false
  }
]);
> use admin
switched to db admin
> db.createUser({user:"root",pwd:"root",roles:[{role:'root',db:'admin'}]})
Successfully added user: {
        "user" : "root",
        "roles" : [
                {
                        "role" : "root",
                        "db" : "admin"
                }
        ]
}
> db.auth("root","123456");
Error: Authentication failed.
0
> db.auth("root","root");
1
>
db 查看当前处于哪个database
集合（类似关系型数据库的表）
> db.createCollection("runoob")
{ "ok" : 1 }
> show tables;
runoob
user
> db.createCollection("mycol", { capped : true, autoIndexId : true, size :
...    6142800, max : 10000 } )
{
        "note" : "The autoIndexId option is deprecated and will be removed in a future release",
        "ok" : 1
}
> show tables;
mycol
runoob
user
> db.user.mycol.find
db.user.mycol.find(               db.user.mycol.findOne(            db.user.mycol.findOneAndReplace(
db.user.mycol.findAndModify(      db.user.mycol.findOneAndDelete(   db.user.mycol.findOneAndUpdate(
> db.user.mycol.find()
> db.user.mycol.insert({"name" : "菜鸟教程"})
WriteResult({ "nInserted" : 1 })
> db.user.mycol.find()
{ "_id" : ObjectId("621093c0a42e17048e8f17e8"), "name" : "菜鸟教程" }
>
> use flink-mongodb-databases
switched to db flink-mongodb-databases
> show tables
> 
> db.dropDatabase()
{ "ok" : 1 }
> use user
switched to db user
> db.user.insert({id:1,name:'zhangsan'})
WriteResult({ "nInserted" : 1 })
> db.user.save({id:1,name:'zhangsan'})
WriteResult({ "nInserted" : 1 })
> db.user.update({id:1},{$set:{age:22}},true)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
> show tables;
user
> db.user.find()
{ "_id" : ObjectId("62109146a42e17048e8f17e5"), "id" : 1, "name" : "zhangsan", "age" : 22 }
{ "_id" : ObjectId("62109157a42e17048e8f17e6"), "id" : 1, "name" : "zhangsan" }
>
### 删除满足条件的 document ###
> db.user.remove({age:22})
WriteResult({ "nRemoved" : 1 })
> db.user.find()
{ "_id" : ObjectId("62109157a42e17048e8f17e6"), "id" : 1, "name" : "zhangsan" }
>
> db.user.remove({age:22},true)	#删除1条数据
> db.user.remove			   #删除所有数据
> db.user.find()  #查询全部数据
> db.user.find({},{id:1,username:1})  #只查询id与username字段
> db.user.find().count()  #查询数据条数
> db.user.save({id:1,name:'zhangsan', age:18})
WriteResult({ "nInserted" : 1 })
> db.user.find().count()
2


> db.user.find({id:1}) #查询id为1的数据
> db.user.find({age:{$lte:21}}) #查询小于等于21的数据
> db.user.find({age:{$lte:21}, id:{$gte:2}}) #and查询，age小于等于21并且id大于等于2
> db.user.find({$or:[{id:1},{id:2}]}) #查询id=1 or id=2
#查看索引
> db.user.getIndexes()
#创建索引
> db.user.createIndex({'age':1})
#删除索引
> db.user.dropIndex("age_1")
#删除除了_id之外的索引
> db.user.dropIndexes()
#创建联合索引
> db.user.createIndex({'age':1, 'id':-1})
#查看索引大小，单位：字节
> db.user.totalIndexSize()
添加用户
mongo-rs:PRIMARY> use admin
switched to db admin
mongo-rs:PRIMARY> db.auth("root","123456")
Error: Authentication failed.
0
mongo-rs:PRIMARY>
mongo-rs:PRIMARY> use admin
switched to db admin
mongo-rs:PRIMARY> db.auth("root","123456")
Error: Authentication failed.
0
mongo-rs:PRIMARY> db.createUser(
  {
    user: "root",
    pwd: "123456",
    roles: [ { role: "userAdminAnyDatabase", db: "admin" } ]
  }
)
Successfully added user: {
        "user" : "root",
        "roles" : [
                {
                        "role" : "userAdminAnyDatabase",
                        "db" : "admin"
                }
        ]
}
mongo-rs:PRIMARY> db.auth("root","123456")
1
mongo-rs:PRIMARY>
集群状态
rs.status()
删除节点
rs.remove("hadoop02:27017")
查询
db.col.find().pretty()
and
> db.user.mycol.find({"company":"nufront", "name":"菜鸟教程"})
{ "_id" : ObjectId("621094dca42e17048e8f17e9"), "name" : "菜鸟教程", "company" : "nufront", "home" : "panyu" }
>
or
> db.user.mycol.find({$or:[{"company":"nufront", "name":"菜鸟教程1"},{"home":"panyu"}]})
{ "_id" : ObjectId("621094dca42e17048e8f17e9"), "name" : "菜鸟教程", "company" : "nufront", "home" : "panyu" }
>

and 与 or 结合使用

db.col.find({"likes": {$gt:50}, $or: [{"by": "菜鸟教程"},{"title": "MongoDB 教程"}]}).pretty()
watch
> db.user.mycol.watch()
uncaught exception: Error: command failed: {
        "ok" : 0,
        "errmsg" : "The $changeStream stage is only supported on replica sets",
        "code" : 40573,
        "codeName" : "Location40573"
} with original command request: {
        "aggregate" : "user.mycol",
        "pipeline" : [
                {
                        "$changeStream" : {
                                "fullDocument" : "default"
                        }
                }
        ],
        "cursor" : {

        },
        "lsid" : {
                "id" : UUID("93b180b0-bb94-415e-a089-f95f27c726ce")
        }
} on connection: connection to 127.0.0.1:27017 : aggregate failed :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
doassert@src/mongo/shell/assert.js:18:14
_assertCommandWorked@src/mongo/shell/assert.js:737:17
assert.commandWorked@src/mongo/shell/assert.js:829:16
DB.prototype._runAggregate@src/mongo/shell/db.js:281:5
DBCollection.prototype.aggregate@src/mongo/shell/collection.js:1058:12
DBCollection.prototype.watch@src/mongo/shell/collection.js:1554:12
@(shell):1:1
>
update 语句
mongo-rs:PRIMARY> db.student.update({"name":'KD'},{$set:{age:27}},true)
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
删除数据
### 删除多条数据
mongo-rs:PRIMARY> db.student.deleteMany({"name":"KD"})
{ "acknowledged" : true, "deletedCount" : 1 }
### 删除单条记录
mongo-rs:PRIMARY> db.student.deleteOne({"name":"KD"})
{ "acknowledged" : true, "deletedCount" : 0 }
### 删除 student 表所有数据
mongo-rs:PRIMARY> db.student.deleteMany()
副本相关 rs
读写权限相关

mongodb默认读写都是在Primary上进行的，副本节点不允许读写，可以使用如下命令来允许副本读

db.getMongo().setSlaveOk()

SECONDARY 新增失败、不允许写入；

SECONDARY 读取通过 db.getMongo().setSlaveOk() 设置可读；

mongo-rs:SECONDARY> use school
switched to db school
mongo-rs:SECONDARY> db.student.find()
Error: error: {
        "topologyVersion" : {
                "processId" : ObjectId("6212135aeaf0477bb419259f"),
                "counter" : NumberLong(3)
        },
        "ok" : 0,
        "errmsg" : "not master and slaveOk=false",
        "code" : 13435,
        "codeName" : "NotPrimaryNoSecondaryOk",
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645351808, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645351808, 1)
}
mongo-rs:SECONDARY> db.getMongo().setSecondaryOk()
mongo-rs:SECONDARY> db.student.find()
{ "_id" : ObjectId("62120e82b89a69719f90735a"), "name" : "Tom", "age" : 16 }
{ "_id" : ObjectId("62120e82b89a69719f90735b"), "name" : "Jerry", "age" : 15 }
{ "_id" : ObjectId("62120e84b89a69719f90735c"), "name" : "Mary", "age" : 9 }
{ "_id" : ObjectId("62121136b89a69719f90735d"), "name" : "Tom", "age" : 16 }
mongo-rs:SECONDARY> db.student.insert({name:"lancelot", age:10})
WriteCommandError({
        "topologyVersion" : {
                "processId" : ObjectId("6212135aeaf0477bb419259f"),
                "counter" : NumberLong(3)
        },
        "ok" : 0,
        "errmsg" : "not master",
        "code" : 10107,
        "codeName" : "NotWritablePrimary",
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645351848, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645351848, 1)
})
查询同步状态
  • source——从库的IP及端口
  • syncedTo——当前的同步情况，延迟了多久等信息
mongo-rs:PRIMARY> db.printSlaveReplicationInfo()
WARNING: printSlaveReplicationInfo is deprecated and may be removed in the next major release. Please use printSecondaryReplicationInfo instead.
source: 172.17.0.3:27017
        syncedTo: Mon Feb 21 2022 15:42:54 GMT+0800 (CST)
        0 secs (0 hrs) behind the primary
source: 172.17.0.4:27017
        syncedTo: Mon Feb 21 2022 15:42:54 GMT+0800 (CST)
        0 secs (0 hrs) behind the primary
mongo-rs:PRIMARY>

扩展

ChangeStreams
基于Docker的MongoDB replica set（副本集）搭建
Mongodb oplog解析

oplog是local库下的一个固定集合，Secondary就是通过查看Primary 的oplog这个集合来进行复制的。每个节点都有oplog，记录这从主节点复制过来的信息，这样每个成员都可以作为同步源给其他节点。

Oplog 可以说是Mongodb Replication的纽带了。

mongo-rs:SECONDARY> use local
switched to db local
mongo-rs:SECONDARY> db.oplog.rs.find()
{ "op" : "n", "ns" : "", "o" : { "msg" : "initiating set" }, "ts" : Timestamp(1645413861, 1), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T03:24:21.655Z") }
{ "op" : "c", "ns" : "config.$cmd", "ui" : UUID("ee940e9f-6ee5-4feb-bd0d-9ac1aa11078a"), "o" : { "create" : "transactions", "idIndex" : { "v" : 2, "key" : { "_id" : 1 }, "name" : "_id_" } }, "ts" : Timestamp(1645413873, 2), "t" : NumberLong(1), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T03:24:33.181Z") }
...
mongo-rs:SECONDARY>
mongo-rs:SECONDARY> use local
switched to db local
mongo-rs:SECONDARY> db.oplog.rs.find({'ns':'school.student'})
{ "op" : "i", "ns" : "school.student", "ui" : UUID("3faa4550-97ef-4684-8651-ba7a562689d9"), "o" : { "_id" : ObjectId("621324818dadd47ef029bb2e"), "name" : "MJ", "age" : 21 }, "ts" : Timestamp(1645421697, 2), "t" : NumberLong(6), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T05:34:57.768Z") }
...
{ "op" : "u", "ns" : "school.student", "ui" : UUID("3faa4550-97ef-4684-8651-ba7a562689d9"), "o" : { "$v" : 2, "diff" : { "u" : { "age" : 28 } } }, "o2" : { "_id" : ObjectId("62133d5dd44bc191fff5f66c") }, "ts" : Timestamp(1645431111, 1), "t" : NumberLong(9), "v" : NumberLong(2), "wall" : ISODate("2022-02-21T08:11:51.250Z") }
mongo-rs:SECONDARY>
副本集信息
mongo-rs:PRIMARY> db.getReplicationInfo()
{
        "logSizeMB" : 990,
        "usedMB" : 0.23,
        "timeDiff" : 23018,
        "timeDiffHours" : 6.39,
        "tFirst" : "Mon Feb 21 2022 11:24:21 GMT+0800 (CST)",
        "tLast" : "Mon Feb 21 2022 17:47:59 GMT+0800 (CST)",
        "now" : "Mon Feb 21 2022 17:48:02 GMT+0800 (CST)"
}

问题记录

The $changeStream stage is only supported on replica sets

docker 服务访问不到？

mongodb

hadoop03（docker 宿主机）却可以，网桥正常、端口映射正常

### 清除nat表 ###
[root@hadoop03 ~]# iptables --flush&iptables -tnat --flush & iptables -P FORWARD ACCEPT
[1] 90831
[2] 90832
Another app is currently holding the xtables lock. Perhaps you want to use the -w option?
[1]-  Exit 4                  iptables --flush
[2]+  Done                    iptables -tnat --flush
[root@hadoop03 ~]#
### 重启docker ###
[root@hadoop03 ~]# service docker restart
Redirecting to /bin/systemctl restart  docker.service
[root@hadoop03 ~]# 
### 启动容器 ###
[root@hadoop03 ~]# docker start my-mongo
my-mongo

### hadoop02 ###
[hadoop@hadoop02 ~]$ telnet hadoop03 27017
Trying 192.168.153.103...
Connected to hadoop03.
Escape character is '^]'.
^[[A

副本集模式下 SECONDARY读写权限问题 no master | not master and slaveOk=false

向上翻、见 shell ——>>>> 读写权限相关

Authentication failed 权限问题

Command failed with error 18 (AuthenticationFailed): 'Authentication failed.' on server hadoop03:30012. The full response is {"ok": 0.0, "errmsg": "Authentication failed.", "code": 18, "codeName": "AuthenticationFailed", "$clusterTime": {"clusterTime": {"$timestamp": {"t": 1645400170, "i": 1}}, "signature": {"hash": {"$binary": {"base64": "AAAAAAAAAAAAAAAAAAAAAAAAAAA=", "subType": "00"}}, "keyId": 0}}, "operationTime": {"$timestamp": {"t": 1645400170, "i": 1}}}

node is not in primary or recovering state

副本集健康问题

mongo-rs:SECONDARY> rs.status()
{
        "set" : "mongo-rs",
        "date" : ISODate("2022-02-21T02:48:20.906Z"),
        "myState" : 2,
        "term" : NumberLong(6),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 3,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastCommittedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1645400170, 1),
                        "t" : NumberLong(6)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1645400170, 1),
                        "t" : NumberLong(6)
                },
                "lastAppliedWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                "lastDurableWallTime" : ISODate("2022-02-20T23:36:10.501Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1645400130, 1),
        "members" : [
                {
                        "_id" : 0,
                        "name" : "172.17.0.4:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 3911,
                        "optime" : {
                                "ts" : Timestamp(1645400170, 1),
                                "t" : NumberLong(6)
                        },
                        "optimeDate" : ISODate("2022-02-20T23:36:10Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T23:36:10.501Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 6,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "172.17.0.5:27017",
                        ### ???
                        "health" : 0,
                        "state" : 8,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastAppliedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastDurableWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2022-02-21T02:48:16.866Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : NumberLong(0),
                        ### ???
                        "lastHeartbeatMessage" : "Error connecting to 172.17.0.5:27017 :: caused by :: No route to host",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1,
                        "configTerm" : -1
                },
                {
                        "_id" : 2,
                        "name" : "172.17.0.6:27017",
                        "health" : 0,
                        "state" : 8,
                        "stateStr" : "(not reachable/healthy)",
                        "uptime" : 0,
                        "optime" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(0, 0),
                                "t" : NumberLong(-1)
                        },
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "optimeDurableDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastAppliedWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastDurableWallTime" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2022-02-21T02:48:16.866Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "Error connecting to 172.17.0.6:27017 :: caused by :: No route to host",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : -1,
                        "configTerm" : -1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645400170, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645400170, 1)
}
mongo-rs:SECONDARY>

ReadConcernMajorityNotAvailableYet: Read concern majority reads are currently not possible.

Could not find member to sync from

rs.status()

"members": [{
...
"infoMessage": "Could not find member to sync from "
...

}]

replica set副本集

mongodb compass没有Language mongodb show dbs_mongodb

[root@hadoop03 ~]#  docker network create --subnet=172.30.0.0/24 mongodbnetwork
[root@hadoop03 ~]#  
[root@hadoop03 ~]#  
[root@hadoop03 ~]#  sudo docker run -d --name mongo1 -p 27016:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep
[root@hadoop03 ~]#  sudo docker run -d --name mongo2 -p 27017:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep
[root@hadoop03 ~]#  sudo docker run -d --name mongo3 -p 27018:27017 --net=mongodbnetwork mongo:5.0.6 --replSet test-rep



sudo docker run --name m0 -idt -p 28017:28017  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'
sudo docker run --name m1 -idt  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'
sudo docker run --name m2 -idt  mongo:5.0.6 /bin/bash -c 'mongod --replSet replset0'
config = { _id:"replset0", members:[{_id:0,host:"172.17.0.4:27017"},{_id:1,host:"172.17.0.5:27017"},{_id:2,host:"172.17.0.6:27017"}]}
docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

docker run --name mongo-rs02 -p 30011:27017 -v /apps/mongo-rs/data02:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

docker run --name mongo-rs03 -p 30012:27017 -v /apps/mongo-rs/data03:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -e MONGO_INITDB_ROOT_USERNAME=admin -e MONGO_INITDB_ROOT_PASSWORD=PWD2020 -d mongo:5.0.6 --replSet mongo-rs --auth --keyFile /data/key/mongo-rs.key --bind_ip_all

TMD副本集终于成功了

docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

docker run --name mongo-rs02 -p 30011:27017 -v /apps/mongo-rs/data02:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

docker run --name mongo-rs03 -p 30012:27017 -v /apps/mongo-rs/data03:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all

root@614b75e20c92:/# mongo
MongoDB shell version v5.0.6
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("4bfc114e-5e21-485c-b20b-b897f8d5fd5e") }
MongoDB server version: 5.0.6
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
        https://community.mongodb.com
---
The server generated these startup warnings when booting:
        2022-02-20T16:54:33.971+08:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
        2022-02-20T16:54:33.971+08:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
        2022-02-20T16:54:33.971+08:00: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. We suggest setting it to 'never'
---
---
        Enable MongoDB's free cloud-based monitoring service, which will then receive and display
        metrics about your deployment (disk utilization, CPU, operation statistics, etc).

        The monitoring data will be available on a MongoDB website with a unique URL accessible to you
        and anyone you share the URL with. MongoDB may use this information to make product
        improvements and to suggest MongoDB products and deployment options to you.

        To enable free monitoring, run the following command: db.enableFreeMonitoring()
        To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---
>
> config = { _id:"mongo-rs", members:[{_id:0,host:"172.17.0.4:27017"},{_id:1,host:"172.17.0.5:27017"},{_id:2,host:"172.17.0.6:27017"}]}
{
        "_id" : "mongo-rs",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "172.17.0.4:27017"
                },
                {
                        "_id" : 1,
                        "host" : "172.17.0.5:27017"
                },
                {
                        "_id" : 2,
                        "host" : "172.17.0.6:27017"
                }
        ]
}
> rs.initiate(config)
{ "ok" : 1 }
mongo-rs:SECONDARY> rs.status()
{
        "set" : "mongo-rs",
        "date" : ISODate("2022-02-20T09:43:56.591Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 3,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "lastCommittedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1645350235, 1),
                        "t" : NumberLong(1)
                },
                "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1645350225, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2022-02-20T09:40:05.507Z"),
                "electionTerm" : NumberLong(1),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(1645349994, 1),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1645349994, 1),
                        "t" : NumberLong(-1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2022-02-20T09:40:05.598Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2022-02-20T09:40:05.694Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "172.17.0.4:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 2967,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1645350005, 1),
                        "electionDate" : ISODate("2022-02-20T09:40:05Z"),
                        "configVersion" : 1,
                        "configTerm" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "172.17.0.5:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 241,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "optimeDurableDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastHeartbeat" : ISODate("2022-02-20T09:43:55.692Z"),
                        "lastHeartbeatRecv" : ISODate("2022-02-20T09:43:55.282Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "172.17.0.6:27017",
                        "syncSourceId" : 2,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 1
                },
                {
                        "_id" : 2,
                        "name" : "172.17.0.6:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 241,
                        "optime" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1645350235, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2022-02-20T09:43:55Z"),
                        "optimeDurableDate" : ISODate("2022-02-20T09:43:55Z"),
                        "lastAppliedWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastDurableWallTime" : ISODate("2022-02-20T09:43:55.636Z"),
                        "lastHeartbeat" : ISODate("2022-02-20T09:43:55.748Z"),
                        "lastHeartbeatRecv" : ISODate("2022-02-20T09:43:56.476Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "172.17.0.4:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 1,
                        "configTerm" : 1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1645350235, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1645350235, 1)
}
mongo-rs:PRIMARY>

error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.

rs.initiate({"_id": "test-rep", "members": [{"_id":0, "host":  "172.30.0.2:27016"}, {"_id": 1, "host": "172.30.0.3:27017"}, {"_id":2, "host":  "172.30.0.4:27018"}]})

No host described in new configuration with {version: 1, term: 0} for replica set test-rep maps to this node

mongodb启动报错：key are too open

docker run --name mongo-rs01 -p 30010:27017 -v /apps/mongo-rs/data01:/data/db -v /apps/mongo-rs/backup:/data/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime  -d mongo:5.0.6 --replSet mongo-rs  --bind_ip_all


62121136b89a69719f90735d
8262132356000000022B0229296E04

实时采集Mongodb

flinkCDC Mongodb demo

CREATE TABLE mongodb_sources_test ( 
 _id STRING, 
 name STRING, 
 age int, 
 PRIMARY KEY (_id) NOT ENFORCED
) WITH (
 'connector' = 'mongodb-cdc', 
 'hosts' = 'hadoop03:30010', 
 'username' = 'root',
 'password' = '123456', 
 'database' = 'school', 
 'collection' = 'student'
)

###

CREATE TABLE sink (
 _id STRING, 
 name STRING, 
 age int 
) WITH (
  'connector' = 'kafka',
  'topic' = 'cdc-test-1',
  'properties.bootstrap.servers' = 'hadoop01:9092',
  'format' = 'changelog-json' 
)

### 
insert into sink select * from mongodb_sources_test

实时处理 change stream方式实现对mongodb的数据实时监控代码

ata/backup -v /apps/mongo-rs/key:/data/key -v /etc/localtime:/etc/localtime -d mongo:5.0.6 --replSet mongo-rs --bind_ip_all

62121136b89a69719f90735d
8262132356000000022B0229296E04

# 实时采集Mongodb

## flinkCDC Mongodb demo

```shell
CREATE TABLE mongodb_sources_test ( 
 _id STRING, 
 name STRING, 
 age int, 
 PRIMARY KEY (_id) NOT ENFORCED
) WITH (
 'connector' = 'mongodb-cdc', 
 'hosts' = 'hadoop03:30010', 
 'username' = 'root',
 'password' = '123456', 
 'database' = 'school', 
 'collection' = 'student'
)

###

CREATE TABLE sink (
 _id STRING, 
 name STRING, 
 age int 
) WITH (
  'connector' = 'kafka',
  'topic' = 'cdc-test-1',
  'properties.bootstrap.servers' = 'hadoop01:9092',
  'format' = 'changelog-json' 
)

### 
insert into sink select * from mongodb_sources_test

实时处理 change stream方式实现对mongodb的数据实时监控代码