【前言】引出ER分片

在《Mycat之——设置全局自增id》一文中,我们为水平分片的数据表设置了全局自增的id值,解决了由于分片后各自数据节点的数据表自增,导致Mycat查询数据时,出现的id值重复的问题。

此时,我们继续在Mycat下执行如下SQL语句来关联查询order_master数据表和order_detail数据表的数据。

mysql> select * from order_master a join order_detail b on a.order_id = b.order_id;
ERROR 1064 (HY000): invalid route in sql, multi tables found but datanode has no intersection sql:select * from order_master a join order_detail b on a.order_id = b.order_id

结果显示MySQL抛出了错误,错误信息为:多个数据表没有在数据节点中找到。这是由于我们对order_master进行了数据分片操作,order_master的数据被分散在多个数据节点上,没有对order_detail数据表进行数据分片操作。所以,上述SQL语句对order_master数据表和order_detail数据表进行关联查询时,是一种跨分片的数据关联操作。

那么,我们如何解决这种跨分片的数据关联操作呢?一种方式是使用全局表;而order_detail数据表会随着业务进行频繁的增删改查操作,而且表中的数据量比较大,不太适合使用全局表的方式来解决跨分片的数据关联操作。另一种方式是使用数据冗余,如果在order_master数据表的分片节点上都冗余一份order_detail数据表也不太合理,这会造成极大的存储空间浪费。

此时,我们可以使用Mycat提供了另一种方式来解决这种跨分片的数据关联操作,那就是ER分片表。在Mycat中,ER分片表示通过ER关系来进行分片的,可以把需要关联的两个表根据它们的关联键,把这些关联的数据分布到同一个数据分片内,那么,就可以在同一个数据分片内来解决数据的关联问题。

【实现】如何实现ER分片

如果使用ER分片,就需要对order_detail数据表也进行数据分片操作,首先,需要在binghe155—binghe158服务器上的MySQL中的orderdb01—orderdb04数据库中创建order_detail数据表,创建order_detail数据表的SQL语句如下所示。

CREATE TABLE `order_detail` (
`order_detail_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增主键ID,订单详情表ID',
`order_id` int(10) unsigned NOT NULL COMMENT '订单表ID',
`product_id` int(10) unsigned NOT NULL COMMENT '订单商品ID',
`product_name` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '商品名称',
`product_cnt` int(11) NOT NULL DEFAULT '1' COMMENT '购买商品数量',
`product_price` decimal(8,2) NOT NULL COMMENT '购买商品单价',
`average_cost` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '平均成本价格',
`weight` float DEFAULT NULL COMMENT '商品重量',
`fee_money` decimal(8,2) NOT NULL DEFAULT '0.00' COMMENT '优惠分摊金额',
`w_id` int(10) unsigned NOT NULL COMMENT '仓库ID',
`modified_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '最后修改时间',
PRIMARY KEY (`order_detail_id`)
) ENGINE=InnoDB AUTO_INCREMENT=29698 DEFAULT CHARSET=utf8 COMMENT='订单详情表';

接下来,修改Mycat的schema.xml文件,将之前对order_master表的定义修改成如下所示。

<table name="order_master" primaryKey="order_id" dataNode = "orderdb01,orderdb02,orderdb03,orderdb04" rule="order_master" autoIncrement="true">
<childTable name="order_detail" primaryKey="order_detail_id" joinKey="order_id" parentKey="order_id" autoIncrement="true"/>
</table>

将order_detail表定义为order_master表的子表。修改完如上配置后,需要将schema.xml中原来配置的order_detail表删除,也就是说,需要在schema.xml文件中删除如下配置。

<table name="order_detail" primaryKey="order_detail_id" dataNode = "ordb"/>

最终schema.xml文件的内容如下所示。

<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://io.mycat/">

<schema name="shop" checkSQLschema="false" sqlMaxLimit="1000">
<table name="order_master" primaryKey="order_id" dataNode = "orderdb01,orderdb02,orderdb03,orderdb04" rule="order_master" autoIncrement="true">
<childTable name="order_detail" primaryKey="order_detail_id" joinKey="order_id" parentKey="order_id" autoIncrement="true"/>
</table>
<table name="order_cart" primaryKey="cart_id" dataNode = "ordb"/>
<table name="order_customer_addr" primaryKey="customer_addr_id" dataNode = "ordb"/>
<table name="region_info" primaryKey="region_id" dataNode = "ordb,prodb,custdb" type="global"/>
<table name="serial" primaryKey="id" dataNode = "ordb"/>
<table name="shipping_info" primaryKey="ship_id" dataNode = "ordb"/>
<table name="warehouse_info" primaryKey="w_id" dataNode = "ordb"/>
<table name="warehouse_proudct" primaryKey="wp_id" dataNode = "ordb"/>

<table name="product_brand_info" primaryKey="brand_id" dataNode = "prodb"/>
<table name="product_category" primaryKey="category_id" dataNode = "prodb"/>
<table name="product_comment" primaryKey="comment_id" dataNode = "prodb"/>
<table name="product_info" primaryKey="product_id" dataNode = "prodb"/>
<table name="product_pic_info" primaryKey="product_pic_id" dataNode = "prodb"/>
<table name="product_supplier_info" primaryKey="supplier_id" dataNode = "prodb"/>

<table name="customer_balance_log" primaryKey="balance_id" dataNode = "custdb"/>
<table name="customer_inf" primaryKey="customer_inf_id" dataNode = "custdb"/>
<table name="customer_level_inf" primaryKey="customer_level" dataNode = "custdb"/>
<table name="customer_login" primaryKey="customer_id" dataNode = "custdb"/>
<table name="customer_login_log" primaryKey="login_id" dataNode = "custdb"/>
<table name="customer_point_log" primaryKey="point_id" dataNode = "custdb"/>

</schema>

<dataNode name="mycat" dataHost="binghe151" database="mycat" />

<dataNode name="ordb" dataHost="binghe152" database="order_db" />
<dataNode name="prodb" dataHost="binghe153" database="product_db" />
<dataNode name="custdb" dataHost="binghe154" database="customer_db" />

<dataNode name="orderdb01" dataHost="binghe155" database="orderdb01" />
<dataNode name="orderdb02" dataHost="binghe156" database="orderdb02" />
<dataNode name="orderdb03" dataHost="binghe157" database="orderdb03" />
<dataNode name="orderdb04" dataHost="binghe158" database="orderdb04" />

<dataHost name="binghe151" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe51" url="192.168.175.151:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe152" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe52" url="192.168.175.152:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe153" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe53" url="192.168.175.153:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe154" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe54" url="192.168.175.154:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe155" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe55" url="192.168.175.155:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe156" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe56" url="192.168.175.156:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe157" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe57" url="192.168.175.157:3306" user="mycat" password="mycat"/>
</dataHost>

<dataHost name="binghe158" maxCon="1000" minCon="10" balance="1"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<writeHost host="binghe58" url="192.168.175.158:3306" user="mycat" password="mycat"/>
</dataHost>

</mycat:schema>

接下里,由于在order_detail表的配置中使用了autoIncrement="true"属性,所以还需要配置order_detail数据表的全局自增id。

在Mycat安装目录下的conf目录下的sequence_db_conf.properties文件中新增如下配置项。

order_detail=mycat

最终sequence_db_conf.properties文件中的内容如下所示。

GLOBAL=mycat
order_master=mycat
order_detail=mycat

接下来,还需要在binghe151服务器上的mycat数据库的mycat_sequence数据表中设置order_detail数据表的自增信息。登录binghe151服务器上的MySQL命令行,进入到mycat数据库中,如下所示。

[root@binghe151 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1946
Server version: 8.0.18 binghe edition

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use mycat;
Database changed

查询mycat_sequence数据表中的数据,如下所示。

mysql> select * from mycat_sequence;
+--------------+---------------+-----------+
| name | current_value | increment |
+--------------+---------------+-----------+
| GLOBAL | 1 | 1 |
| order_master | 1 | 1 |
+--------------+---------------+-----------+
2 rows in set (0.12 sec)

接下来,向mycat_sequence数据表中插入order_detail表的自增信息,如下所示。

mysql> insert into mycat_sequence values ('order_detail', 1, 1);
Query OK, 1 row affected (0.00 sec)

其中,插入的name字段的值需要和sequence_db_conf.properties文件中配置的属性Key相同。最终,mycat_sequence数据表中的数据如下所示。

mysql> select * from mycat_sequence;
+--------------+---------------+-----------+
| name | current_value | increment |
+--------------+---------------+-----------+
| GLOBAL | 1 | 1 |
| order_detail | 1 | 1 |
| order_master | 1 | 1 |
+--------------+---------------+-----------+
3 rows in set (0.00 sec)

接下来,重启Mycat使用配置生效。此时,再次执行如下SQL语句,就能够正确查询出数据了。

select * from order_master a join order_detail b on a.order_id = b.order_id;

接下来,我也给出server.xml文件的完整内容和rule.xml文件的完整内容。

  • server.xml文件的完整内容
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mycat:server SYSTEM "server.dtd">
<mycat:server xmlns:mycat="http://io.mycat/">
<system>
<property name="useHandshakeV10">1</property>
<property name="defaultSqlParser">druidparser</property>
<property name="serverPort">3307</property>
<property name="managerPort">3308</property>
<property name="nonePasswordLogin">0</property>
<property name="bindIp">0.0.0.0</property>
<property name="charset">utf8mb4</property>
<property name="frontWriteQueueSize">2048</property>
<property name="txIsolation">2</property>
<property name="processors">2</property>
<property name="idleTimeout">1800000</property>
<property name="sqlExecuteTimeout">300</property>
<property name="useSqlStat">0</property>
<property name="useGlobleTableCheck">0</property>
<property name="sequenceHandlerType">1</property>
<property name="defaultMaxLimit">1000</property>
<property name="maxPacketSize">104857600</property>
</system>

<user name="mycat" defaultAccount="true">
<property name="usingDecrypt">1</property>
<property name="password">cTwf23RrpBCEmalp/nx0BAKenNhvNs2NSr9nYiMzHADeEDEfwVWlI6hBDccJjNBJqJxnunHFp5ae63PPnMfGYA==</property>
<property name="schemas">shop</property>
</user>
</mycat:server>
  • rule.xml文件的完整内容
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mycat:rule SYSTEM "rule.dtd">
<mycat:rule xmlns:mycat="http://io.mycat/">
<tableRule name="order_master">
<rule>
<columns>customer_id</columns>
<algorithm>mod-long</algorithm>
</rule>
</tableRule>

<function name="mod-long" class="io.mycat.route.function.PartitionByMod">
<property name="count">4</property>
</function>

</mycat:rule>