目录
taosc方式见 使用Nginx实现TDengine的负载均衡-基于taosc
实验环境服务器：
172.16.216.2 test1 ##TDengine Server 1 & Arbitrator
172.16.216.3 test2 ##TDengine Server 2
172.16.216.4 test3 ##Nginx 服务器
**操作系统：**CentOS Linux release 7.9
软件版本：
nginx 1.20.0
TDengine 2.0.20.0
1.1.安装依赖包
yum install gcc gcc-c++ autoconf automake
yum install pcre-devel
yum install zlib-devel
1.2.安装Nginx
Naginx在1.9.0以后支持TCP负载均衡，如果要Naginx支持TCP/UDP，需要安装stream模块。
./configure --with-stream
make
make install
1.3.配置负载均衡
TDengine的RESTful接口为6041
TCP方式
vi /usr/local/nginx/conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
stream {
server {
listen 6041;
proxy_pass app;
}
upstream app {
server 172.16.216.2:6041;
server 172.16.216.3:6041;
}
}
HTTP方式
vi /usr/local/nginx/conf/nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
server {
listen 6041;
location /{
proxy_pass http://app;
}
}
upstream app {
server 172.16.216.2:6041;
server 172.16.216.3:6041;
}
}
1.4.启动Nginx
/usr/local/nginx/sbin/nginx
[root@test3 ~]# netstat -ant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:6041 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 172.16.216.4:22 172.16.216.1:62439 ESTABLISHED
tcp6 0 0 :::22 :::* LISTEN
tcp6 0 0 ::1:25 :::* LISTEN
具体安装步骤可见 TDengine学习笔记-集群安装
2.1. 安装TDengine
tar xvzf TDengine-server-2.0.20.0-Linux-x64.tar.gz
cd TDengine-server-2.0.20.0
./install
2.2. 安装Arbitrator
TDengine中的Arbitrator 类似于PostgreSQL中的witness服务，防止集群出现脑裂。因为实验只有2个节点，为防止停止一个节点后，无法通过无法，需要安装Arbitrator服务。
官方文档-Arbitrator的使用
tar xvzf TDengine-arbitrator-2.0.20.0-Linux-x64.tar.gz
cd TDengine-arbitrator-2.0.20.0
./install_arbi.sh
systemctl start tarbitratord
2.3. 配置文件taos.cfg
##test1
firstEp test1:6030
secondEp test2:6030
arbitrator test1:6042
fqdn test1
##test2
firstEp test1:6030
secondEp test2:6030
arbitrator test1:6042
fqdn test2
2.4.启动集群
systemctl start taosd
添加节点
taos> create dnode test2:6030;
taos> show dnodes;
id | end_point | vnodes | cores | status | role | create_time | offline reason |
======================================================================================================================================
1 | test1:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:15.802 | |
2 | test2:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:30.431 | |
0 | test1:6042 | 0 | 0 | ready | arb | 1970-01-01 08:00:00.000 | - |
Query OK, 3 row(s) in set (0.004701s)
TDengine提供了JDBC的测试工具，可以支持RESTful连接。位置：tests/examples/JDBC/taosdemo，编译方法见文件夹下readme.md
3.1. 使用RESTful连接集群
java -jar target/taosdemo-2.0-jar-with-dependencies.jar -host 172.16.216.4 -database db01 -doCreateTable true -superTableSQL "create table weather(ts timestamp, f1 int) tags(t1 nchar(4))" -numOfTables 10 -numOfRowsPerTable 100 -numOfThreadsForInsert 10 -numOfTablesPerSQL 10 -numOfValuesPerSQL 100
查看结果
taos> show databases\G；
name: db01
created_time: 2021-05-05 17:49:33.391
ntables: 10
vgroups: 1
replica: 1
quorum: 1
days: 30
keep0,keep1,keep(D): 3650,3650,3650
cache(MB): 16
blocks: 6
minrows: 100
maxrows: 4096
wallevel: 1
fsync: 3000
comp: 2
cachelast: 0
precision: ms
update: 0
status: ready
3.2.负载均衡测试
关闭节点test1 上服务
systemctl stop taosd
测试连接
java -jar target/taosdemo-2.0-jar-with-dependencies.jar -host 172.16.216.4 -database db01 -doCreateTable true -superTableSQL "create table weather(ts timestamp, f1 int) tags(t1 nchar(4))" -numOfTables 10 -numOfRowsPerTable 100 -numOfThreadsForInsert 10 -numOfTablesPerSQL 10 -numOfValuesPerSQL 100
查看结果
taos> show dnodes;
id | end_point | vnodes | cores | status | role | create_time | offline reason |
======================================================================================================================================
1 | test1:6030 | 1 | 8 | offline | any | 2021-05-05 17:20:15.802 | status msg timeout |
2 | test2:6030 | 1 | 8 | ready | any | 2021-05-05 17:20:30.431 | |
0 | test1:6042 | 0 | 0 | ready | arb | 1970-01-01 08:00:00.000 | - |
Query OK, 3 row(s) in set (0.006592s)
taos> show databases\G;
name: db01
created_time: 2021-05-05 18:54:37.856
ntables: 10
vgroups: 1
replica: 1
quorum: 1
days: 30
keep0,keep1,keep(D): 3650,3650,3650
cache(MB): 16
blocks: 6
minrows: 100
maxrows: 4096
wallevel: 1
fsync: 3000
comp: 2
cachelast: 0
precision: ms
update: 0
status: ready
Nginx日志显示连接172.16.216.2失败后，连接被转移到下一台服务器。
TCP方式下的日志
[error] 9418#0: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.216.1, server: 0.0.0.0:6041, upstream: "172.16.216.2:6041", bytes from/to client:0/0, bytes from/to upstream:0/0
HTTP方式下的日志
[error] 9384#0: *598 connect() failed (111: Connection refused) while connecting to upstream, client: 172.16.216.1, server: , request: "POST /rest/sql HTTP/1.1", upstream: "http://172.16.216.2:6041/rest/sql", host: "172.16.216.4:6041"
