activemq 集群 activemq集群负载均衡

转载

mob6454cc6faf88 2024-02-20 09:14:50

文章标签 activemq 集群网络 java 操作系统 spring 文章分类 机器学习人工智能

1. 官方文档

http://activemq.apache.org/networks-of-brokers.html

2. 搭建环境

搭建两套activemq集群，每个集群3个节点，两套集群一共6个节点：

集群A：

192.168.240.132:51511
192.168.240.133:51512
192.168.240.134:51513

集群B（伪集群，在一台机器上开三个activemq服务）：

192.168.240.131:53531
192.168.240.131:53532
192.168.240.131:53533

防火墙设置端口，设置/etc/hosts文件。

3. 配置负载均衡

使用static discovery方式配置负载均衡，官方文档例子：

<networkConnectors>
  <networkConnector uri="static:(tcp://host1:61616,tcp://host2:61616,tcp://..)"/>
</networkConnectors>

可以设置的一些属性:

property	default	description
initialReconnectDelay	1000	重连之前等待的时间(ms) (如果useExponentialBackOff为 false)
maxReconnectDelay	30000	重连之前等待的最大时间(ms)
useExponentialBackOff	true	每次重连失败时是否增大等待时间
backOffMultiplier	2	增大等待时间的系数

下面是集群A和集群B的active.xml配置文件内容，先是集群A：

<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <value>file:${activemq.conf}/credentials.properties</value>
        </property>
    </bean>

    <bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
          lazy-init="false" scope="singleton"
          init-method="start" destroy-method="stop">
    </bean>

    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="ems2" dataDirectory="${activemq.data}">

        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry topic=">" >
                  <pendingMessageLimitStrategy>
                    <constantPendingMessageLimitStrategy limit="1000"/>
                  </pendingMessageLimitStrategy>
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy>

        <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

        <networkConnectors> 
                <networkConnector userName="admin" password="admin" uri="static:(tcp://192.168.240.132:51511,tcp://192.168.240.133:51512,tcp://192.168.240.134:51513)" duplex="false"/> 
        </networkConnectors>

        <persistenceAdapter>
                <!-- kahaDB directory="${activemq.data}/kahadb"/ --> 
                <replicatedLevelDB 
                        directory="${activemq.data}/leveldb" 
                        replicas="3" 
                        bind="tcp://0.0.0.0:63631" 
                        zkAddress="192.168.240.132:2181,192.168.240.133:2182,192.168.240.134:2183" 
                        hostname="dubbo-tools" 
                        zkPath="/activemq2/leveldb-stores" 
                />
        </persistenceAdapter>

          <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage percentOfJvmHeap="70" />
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="100 gb"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="50 gb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>

        <transportConnectors>
            <!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
            <transportConnector name="openwire" uri="tcp://0.0.0.0:53531?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
        </transportConnectors>

        <!-- destroy the spring context on shutdown to stop jetty -->
        <shutdownHooks>
            <bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
        </shutdownHooks>
        <plugins>
                <simpleAuthenticationPlugin>
                        <users>
                                <authenticationUser username="admin" password="admin"
                                        groups="users,admins"/>
                                <authenticationUser username="user" password="password"
                                        groups="users"/>
                                <authenticationUser username="guest" password="password" groups="guests"/>
                        </users>
                </simpleAuthenticationPlugin>
        </plugins>
    </broker>

    <import resource="jetty.xml"/>

</beans>

接下来是集群B的activemq.xml：

<!--
    Licensed to the Apache Software Foundation (ASF) under one or more
    contributor license agreements.  See the NOTICE file distributed with
    this work for additional information regarding copyright ownership.
    The ASF licenses this file to You under the Apache License, Version 2.0
    (the "License"); you may not use this file except in compliance with
    the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
-->
<!-- START SNIPPET: example -->
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

    <!-- Allows us to use system properties as variables in this configuration file -->
    <bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <value>file:${activemq.conf}/credentials.properties</value>
        </property>
    </bean>

   <!-- Allows accessing the server log -->
    <bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
          lazy-init="false" scope="singleton"
          init-method="start" destroy-method="stop">
    </bean>

    <!--
        The <broker> element is used to configure the ActiveMQ broker.
    -->
    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="ems" dataDirectory="${activemq.data}">

        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry topic=">" >
                    <!-- The constantPendingMessageLimitStrategy is used to prevent
                         slow topic consumers to block producers and affect other consumers
                         by limiting the number of messages that are retained
                         For more information, see:

                         http://activemq.apache.org/slow-consumer-handling.html

                    -->
                  <pendingMessageLimitStrategy>
                    <constantPendingMessageLimitStrategy limit="1000"/>
                  </pendingMessageLimitStrategy>
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy>


        <!--
            The managementContext is used to configure how ActiveMQ is exposed in
            JMX. By default, ActiveMQ uses the MBean server that is started by
            the JVM. For more information, see:

            http://activemq.apache.org/jmx.html
        -->
        <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

        <networkConnectors>
                <networkConnector userName="admin" password="admin" uri="static:(tcp://192.168.240.131:53531,tcp://192.168.240.131:53532,tcp://192.168.240.131:53533)" duplex="false" />
        </networkConnectors>

        <!--
            Configure message persistence for the broker. The default persistence
            mechanism is the KahaDB store (identified by the kahaDB tag).
            For more information, see:

            http://activemq.apache.org/persistence.html
        -->
        <persistenceAdapter>
                <!-- kahaDB directory="${activemq.data}/kahadb"/ --> 
                <replicatedLevelDB 
                        directory="${activemq.data}/leveldb" 
                        replicas="3" 
                        bind="tcp://0.0.0.0:62623" 
                        zkAddress="192.168.240.132:2181,192.168.240.133:2182,192.168.240.134:2183" 
                        hostname="server03" 
                        zkPath="/activemq/leveldb-stores" 
                />
        </persistenceAdapter>


          <!--
            The systemUsage controls the maximum amount of space the broker will
            use before disabling caching and/or slowing down producers. For more information, see:
            http://activemq.apache.org/producer-flow-control.html
          -->
          <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage percentOfJvmHeap="70" />
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="100 gb"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="50 gb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>

        <!--
            The transport connectors expose ActiveMQ over a given protocol to
            clients and other brokers. For more information, see:

            http://activemq.apache.org/configuring-transports.html
        -->
        <transportConnectors>
            <!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
            <transportConnector name="openwire" uri="tcp://0.0.0.0:51513?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
            <transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
        </transportConnectors>

        <!-- destroy the spring context on shutdown to stop jetty -->
        <shutdownHooks>
            <bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
        </shutdownHooks>

        <plugins>
                <simpleAuthenticationPlugin>
                        <users>
                                <authenticationUser username="admin" password="admin"
                                        groups="users,admins"/>
                                <authenticationUser username="user" password="password"
                                        groups="users"/>
                                <authenticationUser username="guest" password="password" groups="guests"/>
                        </users>
                </simpleAuthenticationPlugin>
        </plugins>

    </broker>

    <!--
        Enable web consoles, REST and Ajax APIs and demos
        The web consoles requires by default login, you can disable this in the jetty.xml file

        Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
    -->
    <import resource="jetty.xml"/>

</beans>

要在persistenceAdapter标签前，添加networkConnectors！

启动6个节点后，开始进行测试。

4. 测试

编写4个小程序，生产者1、生产者2、消费者1和消费者2。

下面是生产者1的部分代码，生产者1将消息发送到集群A中：

package cn.net.bysoft.activemq.test;

import org.springframework.context.support.ClassPathXmlApplicationContext;

/*
 * failover:(
 * tcp://192.168.240.132:51511,
 * tcp://192.168.240.133:51512,
 * tcp://192.168.240.134:51513
 * )?randomize=false
 * */
public class Product01Test {

	private static ClassPathXmlApplicationContext context;

	public static void main(String[] args) throws InterruptedException {
		context = new ClassPathXmlApplicationContext("classpath:spring/spring-context.xml");
		context.start();

		Product01 product01 = (Product01) context.getBean("product01");

		for (int i = 0; i < Integer.MAX_VALUE; i++) {
			product01.sendMessage("P1 -> Message" + i);
			Thread.sleep(2000);
		}
	}

}

接下来是生产者02的部分代码，它将消息发送到集群B中：

package cn.net.bysoft.activemq.test;

import org.springframework.context.support.ClassPathXmlApplicationContext;

/*
 * failover:(
 * tcp://192.168.240.131:53531,
 * tcp://192.168.240.131:53532,
 * tcp://192.168.240.131:53533
 * )?randomize=false
 * */
public class Product02Test {

	private static ClassPathXmlApplicationContext context;

	public static void main(String[] args) throws InterruptedException {
		context = new ClassPathXmlApplicationContext("classpath:spring/spring-context.xml");
		context.start();

		Product02 product02 = (Product02) context.getBean("product02");
		
		for (int i = 0; i < Integer.MAX_VALUE; i++) {
			product02.sendMessage("P2 -> Message" + i);
			Thread.sleep(2000);
		}
	}

}

接下来是消费者01的代码，它从集群A中获取消息：

package cn.net.bysoft.activemq.test;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.context.support.ClassPathXmlApplicationContext;

/*
 * failover:(
 * tcp://192.168.240.132:51511,
 * tcp://192.168.240.133:51512,
 * tcp://192.168.240.134:51513
 * )?randomize=false
 * */
public class Consumer01Test {

	private static final Log log = LogFactory.getLog(Consumer01Test.class);
	private static ClassPathXmlApplicationContext context;

	public static void main(String[] args) {
		try {
			context = new ClassPathXmlApplicationContext("classpath:spring/spring-context.xml");
			context.start();
		} catch (Exception e) {
			log.error("==>MQ context start error:", e);
			System.exit(0);
		}
	}

}

接下来是消费者02的代码，它从集群B中获取消息：

package cn.net.bysoft.activemq.test;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.context.support.ClassPathXmlApplicationContext;

/*
 * failover:(
 * tcp://192.168.240.131:53531,
 * tcp://192.168.240.131:53532,
 * tcp://192.168.240.131:53533
 * )?randomize=false
 * */
public class Consumer02Test {

	private static final Log log = LogFactory.getLog(Consumer02Test.class);
	private static ClassPathXmlApplicationContext context;

	public static void main(String[] args) {
		try {
			context = new ClassPathXmlApplicationContext("classpath:spring/spring-context.xml");
			context.start();
		} catch (Exception e) {
			log.error("==>MQ context start error:", e);
			System.exit(0);
		}
	}

}

首先运行两个消费者的程序，然后观察控制台：

activemq 集群 activemq集群负载均衡_spring

activemq 集群 activemq集群负载均衡_spring_02

两个消费者分别连接两个集群，但是两个集群的控制台上都会发现这两个消费者。

接下来运行生产者01，它只向集群A发送消息，测试一下发送到集群A的消息，是否会被两个消费者共同消费：

activemq 集群 activemq集群负载均衡_spring_03

activemq 集群 activemq集群负载均衡_java_04

结果表明，生产者01向集群A发送的消息，分别被消费者01和消费者02使用。两个集群的网络桥接生效了，消息队列进行了共享。

最后，把消费者01停止掉，开启生产者02，测试一下，两者生产者对一个消费者的情况，看看这一个消费者是否会消费两个生产者发送的消息：

activemq 集群 activemq集群负载均衡_java_05

5. 待解决

使用该模式进行负载均衡时，监控日志，会频繁出现警告。

2个集群，6个节点中。两个 master 的连接不会出现警告，但是与 slave 连接时，会出现：

2017-02-10 00:38:38,849 | WARN  | Could not start network bridge between: vm://ems2?async=false&create=false and: tcp://192.168.240.132:51511 due to: 拒绝连接 (Connection refused) | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-95
2017-02-10 00:38:38,851 | WARN  | Could not start network bridge between: vm://ems2?async=false&create=false and: tcp://192.168.240.133:51512 due to: 拒绝连接 (Connection refused) | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-97

若有了解这个问题的朋友，请留言给我，感激不尽！

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。