本片主要讲述使用docker安装mycat,直接上过程,亲测无误!

一、MyCat简介

1. 什么是MyCat

  • MyCat 是目前最流行的基于 java 语言编写的数据库中间件,是一个实现了 MySQL 协议的服务器,前端用户可以把它看作是一个数据库代理,用 MySQL 客户端工具和命令行访问,而其后端可以用 MySQL 原生协议与多个 MySQL 服务器通信,也可以用 JDBC 协议与大多数主流数据库服务器通信,其核心功能是分库分表。配合数据库的主从模式还可实现读写分离。
  • MyCat 是基于阿里开源的 Cobar 产品而研发,Cobar 的稳定性、可靠性、优秀的架构和性能以及众多成熟的使用案例使得 MyCat 变得非常的强大。
  • MyCat 发展到目前的版本,已经不是一个单纯的 MySQL 代理了,它的后端可以支持MySQL、SQL Server、Oracle、DB2、PostgreSQL 等主流数据库,也支持 MongoDB 这种新型NoSQL 方式的存储,未来还会支持更多类型的存储。而在最终用户看来,无论是那种存储方式,在 MyCat 里,都是一个传统的数据库表,支持标准的 SQL 语句进行数据的操作,这样一来,对前端业务系统来说,可以大幅降低开发难度,提升开发速度。
  • 简单来说MyCat就是实现数据库集群的,对海量数据的数据存储的一种解决方案,因为很多数据库不想Oracle一样自带集群的配置,那么在进行海量数据存储的时候就要使用到MyCat进行数据库的管理了。

  MyCat 官网: http://www.mycat.io/

2.Mycat的结构

docker 安装mysql初始化 docker安装mycat_中间件

二、MyCat安装及配置介绍

 1.安装JDK,因为MyCat是java开发的,所以需要java虚拟机环境,在Linux节点中安装JDK是必须的。

由于本文章基于Docker安装的,所以不需要在本机上装,可参考安装Docker Docker自动化部署安装(一)之安装Nginx

2.安装一个Mysql,可参考前面的文章 

Docker自动化部署安装(四)之安装Mysql

3.安装Mycat

1.创建文件夹
mkdir -p /work/docker/build/mycat
mkdir -p /work/docker/conf/mycat
 
 
2.新建Dockerfile文件
echo '
#FROM openjdk:8-jdk-stretch
FROM centos:7.7.1908
MAINTAINER zxk <zxk@qq.com>

RUN yum install -y net-tools

ENV JAVA_HOME /usr/local/java/jdk
ENV JRE_HOME ${JAVA_HOME}/jre
ENV CLASSPATH .:${JAVA_HOME}/lib:${JRE_HOME}/lib
ENV PATH ${JAVA_HOME}/bin:$PATH

WORKDIR ${JAVA_HOME}

#这里的jdk下载地址要换成自己的哦,或者放在自己的服务器下然后替换地址如(http://136.223.11.12:8080/tools/jdk-8u251-linux-x64.tar.gz) 
#附带一个百度放盘下载地址(链接:https://pan.baidu.com/s/1Bim4DHo-CaQm4Qfa1BGOjA 
提取码:x4zp 
复制这段内容后打开百度网盘手机App,操作更方便哦)
RUN set -x \
    && curl -Os "http://ip:port/tools/jdk-8u251-linux-x64.tar.gz" \
    && tar -xzvf jdk-8u251-linux-x64.tar.gz \
    && mv jdk1.8.0_251/* . \
    && rm -fr jdk1.8.0_251 jdk-8u251-linux-x64.tar.gz

ADD http://dl.mycat.org.cn/1.6.7.6/20201126013625/Mycat-server-1.6.7.6-release-20201126013625-linux.tar.gz /usr/local
RUN cd /usr/local && tar -zxvf Mycat-server-1.6.7.6-release-20201126013625-linux.tar.gz && ls -lna

ENV MYCAT_HOME=/usr/local/mycat
WORKDIR /usr/local/mycat

ENV TZ Asia/Shanghai

EXPOSE 8066 9066

CMD ["/usr/local/mycat/bin/mycat", "console","&"]
' > /work/docker/build/nginx/Dockerfile
 
3.在/work/docker/conf/mycat下准备Mycat的配置文件(文章末尾会介绍这些文件)
server.xml	MyCat 的配置文件,设置账号、参数等
schema.xml	MyCat 对应的物理数据库和数据库表的配置
rule.xml	MyCat 分片(分库分表)规则
wrapper.conf 目前Mycat的启动是经过warapper封装成启动脚本,所以日志也会有其相关的日志文件:${MYCAT_HOME}/logs/warapper.log,在启动时候如果系统环境配置错误或缺少配置时,导致Mycat无法启动,可以通过查看warrpper.log查看具体错误原因

4.新建docker-mycat.yml文件
echo '
version: '3'
services:
  mycat:
    build:
      context: ./build/mycat
      dockerfile: Dockerfile
    restart: always
    privileged: true
    image: mycat:1.6.7.6
    container_name: mycat
    hostname: mycat
    ports:
     - '8066:8066'
     - '9066:9066'
    volumes:
     - /etc/localtime:/etc/localtime
     - /work/docker/conf/mycat/schema.xml:/usr/local/mycat/conf/schema.xml
     - /work/docker/conf/mycat/rule.xml:/usr/local/mycat/conf/rule.xml
     - /work/docker/conf/mycat/server.xml:/usr/local/mycat/conf/server.xml
     #当服务器内存过大时需要此配置文件
     #- /work/docker/conf/mycat/wrapper.conf:/usr/local/mycat/conf/wrapper.conf
     - /work/docker/logs/mycat:/usr/local/mycat/logs
     #- /temp:/temp
' > /work/docker/docker-mycat.yml
 
 
5.启动mycat服务
docker-compose -f /work/docker/docker-mycat.yml up -d

4.检查Mycat

docker 安装mysql初始化 docker安装mycat_docker 安装mysql初始化_02

docker 安装mysql初始化 docker安装mycat_Mycat_03

 三、Mycat相关相关配置文件介绍

 1.server.xml

server.xml是用来配置账号,参数及相关操作权限的文件,下面是给文件的简要内容,去掉了相关的注释

<?xml version="1.0" encoding="UTF-8"?>
<!-- - - Licensed under the Apache License, Version 2.0 (the "License");
             - you may not use this file except in compliance with the License. - You
        may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0
        - - Unless required by applicable law or agreed to in writing, software -
        distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT
        WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the
        License for the specific language governing permissions and - limitations
        under the License. -->
<!DOCTYPE mycat:server SYSTEM "server.dtd">
<mycat:server xmlns:mycat="http://io.mycat/">
        <system>
        <property name="nonePasswordLogin">0</property> <!-- 0为需要密码登陆、1为不需要密码登陆 ,默认为0,设置为1则需要指定默认账户-->
        <property name="ignoreUnknownCommand">0</property><!-- 0遇上没有实现的报文(Unknown command:),就会报错、1为忽略该报文,返回ok报文。
        在某些mysql客户端存在客户端已经登录的时候还会继续发送登录报文,mycat会报错,该设置可以绕过这个错误-->
        <property name="useHandshakeV10">1</property>
    <property name="removeGraveAccent">1</property>
        <property name="useSqlStat">0</property>  <!-- 1为开启实时统计、0为关闭 -->
        <property name="useGlobleTableCheck">0</property>  <!-- 1为开启全加班一致性检测、0为关闭 -->
        <property name="sqlExecuteTimeout">300</property>  <!-- SQL 执行超时 单位:秒-->
                <property name="sequenceHandlerType">1</property>
        <!--<property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>
                     INSERT INTO `travelrecord` (`id`,user_id) VALUES ('next value for MYCATSEQ_GLOBAL',"xxx");
        -->
        <!--必须带有MYCATSEQ_或者 mycatseq_进入序列匹配流程 注意MYCATSEQ_有空格的情况-->
        <property name="sequnceHandlerPattern">(?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+</property>
        <property name="subqueryRelationshipCheck">false</property> <!-- 子查询中存在关联查询的情况下,检查关联字段中是否有分片字段 .默认 false -->
        <property name="sequenceHanlderClass">io.mycat.route.sequence.handler.HttpIncrSequenceHandler</property>
      <!--  <property name="useCompression">1</property>--> <!--1为开启mysql压缩协议-->
        <!--  <property name="fakeMySQLVersion">5.6.20</property>--> <!--设置模拟的MySQL版本号-->
        <!-- <property name="processorBufferChunk">40960</property> -->
        <!--
                     <property name="processors">1</property>
        <property name="processorExecutor">32</property>
         -->
        <!--默认为type 0: DirectByteBufferPool | type 1 ByteBufferArena | type 2 NettyBufferPool -->
                <property name="processorBufferPoolType">0</property>
                <!--默认是65535 64K 用于sql解析时最大文本长度 -->
                <!--<property name="maxStringLiteralLength">65535</property>-->
                <!--<property name="sequenceHandlerType">0</property>-->
                <!--<property name="backSocketNoDelay">1</property>-->
                <!--<property name="frontSocketNoDelay">1</property>-->
                <!--<property name="processorExecutor">16</property>-->
                <!--
                                             <property name="serverPort">8066</property> <property name="managerPort">9066</property>
                        <property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property>
                        <property name="dataNodeIdleCheckPeriod">300000</property> 5 * 60 * 1000L; //连接空闲检查
                        <property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
                <!--分布式事务开关,0为不过滤分布式事务,1为过滤分布式事务(如果分布式事务内只涉及全局表,则不过滤),2为不过滤分布式事务,但是记录分布式事务日志-->
                <property name="handleDistributedTransactions">0</property>

                        <!--
                                                     off heap for merge/order/group/limit      1开启   0关闭
                -->
                <property name="useOffHeapForMerge">0</property>

                <!--
                                             单位为m
                -->
        <property name="memoryPageSize">64k</property>

                <!--
                                             单位为k
                -->
                <property name="spillsFileBufferSize">1k</property>

                <property name="useStreamOutput">0</property>

                <!--
                                             单位为m
                -->
                <property name="systemReserveMemorySize">384m</property>


                <!--是否采用zookeeper协调切换  -->
                <property name="useZKSwitch">false</property>

                <!-- XA Recovery Log日志路径 -->
                <!--<property name="XARecoveryLogBaseDir">./</property>-->

                <!-- XA Recovery Log日志名称 -->
                <!--<property name="XARecoveryLogBaseName">tmlog</property>-->
                <!--如果为 true的话 严格遵守隔离级别,不会在仅仅只有select语句的时候在事务中切换连接-->
                <property name="strictTxIsolation">false</property>
                <!--如果为0的话,涉及多个DataNode的catlet任务不会跨线程执行-->
                <property name="parallExecute">0</property>
        </system>

        <!-- 全局SQL防火墙设置 -->
        <!--白名单可以使用通配符%或着*-->
        <!--例如<host host="127.0.0.*" user="root"/>-->
        <!--例如<host host="127.0.*" user="root"/>-->
        <!--例如<host host="127.*" user="root"/>-->
        <!--例如<host host="1*7.*" user="root"/>-->
        <!--这些配置情况下对于127.0.0.1都能以root账户登录-->
        <!--
                     <firewall>
           <whitehost>
              <host host="1*7.0.0.*" user="root"/>
           </whitehost>
       <blacklist check="false">
       </blacklist>
        </firewall>
        -->

        <user name="root" defaultAccount="true">
                <property name="password">Root@2021</property>
				 <!-- 可访问的逻辑库,可为多个逗号分开 -->
                <property name="schemas">demo</property>
				<property name="defaultSchema">demo</property>
                <!--No MyCAT Database selected 错误前会尝试使用该schema作为schema,不设置则为null,报错 -->
				<!-- 是否只读 -->
				<!-- <property name="readOnly">true</property>  -->
                <!-- 表级 DML 权限设置 -->
                <!--
                                     <privileges check="false">
                        <schema name="TESTDB" dml="0110" >
                                <table name="tb01" dml="0000"></table>
                                <table name="tb02" dml="1111"></table>
                        </schema>
                </privileges>
                 -->
        </user>

</mycat:server>

2.schema.xml

  schema.xml 是最主要的配置文件,首先看默认的配置文件

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">
<!--	
	<schema name="testData1" checkSQLschema="false" sqlMaxLimit="1000">
		<table name="testTable1" dataNode="dataNode2" subTables="testTable1_2021$01-12,testTable1_2022$01-05" rule="sharding-by-month" />
		<table name="testTable2" dataNode="dataNode2" ruleRequired="false"/>
    </schema>

	<dataNode name="dataNode1" dataHost="dataHost1" database="testData1" />

	<dataHost name="dataHost1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="writeHost1" url="172.18.0.3:3306" user="root" password="Root@2021">
		</writeHost>
	</dataHost>
	-->
	
	<schema name="demo" checkSQLschema="false" sqlMaxLimit="2000">
		<table name="t_user" dataNode="dataNode1" ruleRequired="false"/>
    </schema>
	
	<schema name="qianqian" checkSQLschema="false" sqlMaxLimit="2000">
		<table name="sys_user" dataNode="dataNode2" ruleRequired="false"/>
    </schema>

	<dataNode name="dataNode1" dataHost="dataHost1" database="demo" />
	<dataNode name="dataNode2" dataHost="dataHost1" database="qianqian" />

    <dataHost name="dataHost1" maxCon="1000" minCon="10" balance="0" writeType="0" dbType="mysql" dbDriver="native" switchType="1"  slaveThreshold="100">
		<heartbeat>select user()</heartbeat>
		<writeHost host="writeHost1" url="172.18.0.3:3306" user="root" password="Root@2021">
		</writeHost>
	 </dataHost> 
	
</mycat:schema>

3.rule.xml

分片(分库分表)规则

<?xml version="1.0" encoding="UTF-8"?>
<!-- - - Licensed under the Apache License, Version 2.0 (the "License");
             - you may not use this file except in compliance with the License. - You
        may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0
        - - Unless required by applicable law or agreed to in writing, software -
        distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT
        WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the
        License for the specific language governing permissions and - limitations
        under the License. -->
<!DOCTYPE mycat:rule SYSTEM "rule.dtd">
<mycat:rule xmlns:mycat="http://io.mycat/">
        <tableRule name="rule1">
                <rule>
                        <columns>id</columns>
                        <algorithm>func1</algorithm>
                </rule>
        </tableRule>

        <tableRule name="sharding-by-date">
                <rule>
                        <columns>createTime</columns>
                        <algorithm>partbyday</algorithm>
                </rule>
        </tableRule>

        <tableRule name="rule2">
                <rule>
                        <columns>user_id</columns>
                        <algorithm>func1</algorithm>
                </rule>
        </tableRule>

        <tableRule name="sharding-by-intfile">
                <rule>
                        <columns>sharding_id</columns>
                        <algorithm>hash-int</algorithm>
                </rule>
        </tableRule>
        <tableRule name="auto-sharding-long">
                <rule>
                        <columns>id</columns>
                        <algorithm>rang-long</algorithm>
                </rule>
        </tableRule>
        <tableRule name="mod-long">
                <rule>
                        <columns>id</columns>
                        <algorithm>mod-long</algorithm>
                </rule>
        </tableRule>
        <tableRule name="sharding-by-murmur">
                <rule>
                        <columns>id</columns>
                        <algorithm>murmur</algorithm>
                </rule>
        </tableRule>
        <tableRule name="crc32slot">
                <rule>
                        <columns>id</columns>
                        <algorithm>crc32slot</algorithm>
                </rule>
        </tableRule>
        <tableRule name="sharding-by-month">
                <rule>
                        <columns>create_time</columns>
                        <algorithm>partbymonth</algorithm>
                </rule>
        </tableRule>
        <tableRule name="latest-month-calldate">
                <rule>
                        <columns>calldate</columns>
                        <algorithm>latestMonth</algorithm>
                </rule>
        </tableRule>

        <tableRule name="auto-sharding-rang-mod">
                <rule>
                        <columns>id</columns>
                        <algorithm>rang-mod</algorithm>
                </rule>
        </tableRule>

        <tableRule name="jch">
                <rule>
                        <columns>id</columns>
                        <algorithm>jump-consistent-hash</algorithm>
                </rule>
        </tableRule>

        <function name="murmur"
                          class="io.mycat.route.function.PartitionByMurmurHash">
                <property name="seed">0</property><!-- 默认是0 -->
                <property name="count">2</property><!-- 要分片的数据库节点数量,必须指定,否则没法分片 -->
                <property name="virtualBucketTimes">160</property><!-- 一个实际的数据库节点被映射为这么多虚拟节点,默认是160倍,也就是虚拟节点数是物理节点数的160倍 -->
                <!-- <property name="weightMapFile">weightMapFile</property> 节点的权重,没有指定权重的节点默认是1。以properties文件的格式填写,以从0开始到count-1的整数值也就是节点索引为key,以节点权重值为值。所有权重值必须是正整数,否则以1代替 -->
                <!-- <property name="bucketMapPath">/etc/mycat/bucketMapPath</property>
                                             用于测试时观察各物理节点与虚拟节点的分布情况,如果指定了这个属性,会把虚拟节点的murmur hash值与物理节点的映射按行输出到这个文件,没有默认值,如果不指定,就不会输出任何东西 -->
        </function>

        <function name="crc32slot"
                          class="io.mycat.route.function.PartitionByCRC32PreSlot">
                <property name="count">2</property><!-- 要分片的数据库节点数量,必须指定,否则没法分片 -->
        </function>
        <function name="hash-int"
                          class="io.mycat.route.function.PartitionByFileMap">
                <property name="mapFile">partition-hash-int.txt</property>
        </function>
        <function name="rang-long"
                          class="io.mycat.route.function.AutoPartitionByLong">
                <property name="mapFile">autopartition-long.txt</property>
        </function>
        <function name="mod-long" class="io.mycat.route.function.PartitionByMod">
                <!-- how many data nodes -->
                <property name="count">3</property>
        </function>

        <function name="func1" class="io.mycat.route.function.PartitionByLong">
                <property name="partitionCount">8</property>
                <property name="partitionLength">128</property>
        </function>
        <function name="latestMonth"
                          class="io.mycat.route.function.LatestMonthPartion">
                <property name="splitOneDay">24</property>
        </function>
        <function name="partbymonth"
                          class="io.mycat.route.function.PartitionByMonth">
                <property name="dateFormat">yyyy-MM-dd</property>
                <property name="sBeginDate">2022-01-01</property>
        </function>


        <function name="partbyday"
                          class="io.mycat.route.function.PartitionByDate">
                <property name="dateFormat">yyyy-MM-dd</property>
                <property name="sNaturalDay">0</property>
                <property name="sBeginDate">2014-01-01</property>
                <property name="sEndDate">2014-01-31</property>
                <property name="sPartionDay">10</property>
        </function>

        <function name="rang-mod" class="io.mycat.route.function.PartitionByRangeMod">
                <property name="mapFile">partition-range-mod.txt</property>
        </function>

        <function name="jump-consistent-hash" class="io.mycat.route.function.PartitionByJumpConsistentHash">
                <property name="totalBuckets">3</property>
        </function>
</mycat:rule>

4.wrapper.conf

Mycat的启动是经过warapper封装成启动脚本,在服务器确实一些配置的时候可以通过wrapper.conf改变mycat的启动参数,保证mycat正常运行

#********************************************************************
# Wrapper Properties
#********************************************************************
# Java Application
wrapper.java.command=java
wrapper.working.dir=..

# Java Main class.  This class must implement the WrapperListener interface
#  or guarantee that the WrapperManager class is initialized.  Helper
#  classes are provided to do this for you.  See the Integration section
#  of the documentation for details.
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp
set.default.REPO_DIR=lib
set.APP_BASE=.

# Java Classpath (include wrapper.jar)  Add class path elements as
#  needed starting from 1
wrapper.java.classpath.1=lib/wrapper.jar
wrapper.java.classpath.2=conf
wrapper.java.classpath.3=%REPO_DIR%/*

# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=lib

# Java Additional Parameters
#wrapper.java.additional.1=
wrapper.java.additional.1=-DMYCAT_HOME=.
wrapper.java.additional.2=-server
wrapper.java.additional.3=-XX:+AggressiveOpts
wrapper.java.additional.4=-XX:MaxDirectMemorySize=4G
wrapper.java.additional.5=-Dcom.sun.management.jmxremote
wrapper.java.additional.6=-Dcom.sun.management.jmxremote.port=1984
wrapper.java.additional.7=-Dcom.sun.management.jmxremote.authenticate=false
wrapper.java.additional.8=-Dcom.sun.management.jmxremote.ssl=false
wrapper.java.additional.9=-Xmx4G
wrapper.java.additional.10=-Xms1G

# Initial Java Heap Size (in MB)
#wrapper.java.initmemory=3

# Maximum Java Heap Size (in MB)
#wrapper.java.maxmemory=64

# Application parameters.  Add parameters as needed starting from 1
wrapper.app.parameter.1=io.mycat.MycatStartup
wrapper.app.parameter.2=start

#********************************************************************
# Wrapper Logging Properties
#********************************************************************
# Format of output for the console.  (See docs for formats)
wrapper.console.format=PM

# Log Level for console output.  (See docs for log levels)
wrapper.console.loglevel=INFO

# Log file to use for wrapper output logging.
wrapper.logfile=logs/wrapper.log

# Format of output for the log file.  (See docs for formats)
wrapper.logfile.format=LPTM

# Log Level for log file output.  (See docs for log levels)
wrapper.logfile.loglevel=INFO

# Maximum size that the log file will be allowed to grow to before
#  the log is rolled. Size is specified in bytes.  The default value
#  of 0, disables log rolling.  May abbreviate with the 'k' (kb) or
#  'm' (mb) suffix.  For example: 10m = 10 megabytes.
wrapper.logfile.maxsize=512m

# Maximum number of rolled log files which will be allowed before old
#  files are deleted.  The default value of 0 implies no limit.
wrapper.logfile.maxfiles=30

# Log Level for sys/event log output.  (See docs for log levels)
wrapper.syslog.loglevel=NONE

#********************************************************************
# Wrapper Windows Properties
#********************************************************************
# Title to use when running as a console
wrapper.console.title=Mycat-server

#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.

# Name of the service
wrapper.ntservice.name=mycat

# Display name of the service
wrapper.ntservice.displayname=Mycat-server

# Description of the service
wrapper.ntservice.description=The project of Mycat-server

# Service dependencies.  Add dependencies as needed starting from 1
wrapper.ntservice.dependency.1=

# Mode in which the service is installed.  AUTO_START or DEMAND_START
wrapper.ntservice.starttype=AUTO_START

# Allow the service to interact with the desktop.
wrapper.ntservice.interactive=false

wrapper.ping.timeout=120
configuration.directory.in.classpath.first=conf

至此Mycat的安装和配置介绍完毕!