Apache RocketMQ是一个开源的分布式消息队列，具备高可靠性和高性能的特点，被广泛应用于异步通信和数据处理场景。Spring Tips中与Apache RocketMQ相关的内容可能会涉及RocketMQ的集成、使用和优化等方面。
RocketMQ的核心组件包括Producer（生产者）、Consumer（消费者）、Broker（代理服务器）和Namesrv（名称服务器）。生产者负责将消息发送到Broker中的Topic，消费者负责从Broker中接收消息。RocketMQ支持消息发布订阅模式和点对点模式，适用于大规模分布式应用的消息通信。
RocketMQ的特点包括：
高可靠性：RocketMQ提供了消息的高可靠性传输，支持同步和异步复制机制，保证消息不会丢失。
高性能：RocketMQ具备出色的消息吞吐量和低延迟，适用于高并发场景。
分布式架构：RocketMQ采用分布式架构，支持水平扩展，可满足不同规模的应用需求。
丰富的特性：RocketMQ提供了消息过滤、延迟消息、顺序消息等丰富的特性，适用于不同业务场景。
在Spring框架中集成RocketMQ，可以使开发者更方便地使用RocketMQ进行消息通信。Spring Boot提供了对RocketMQ的自动配置支持，使得集成过程更加简单。通过Spring Boot的starter依赖，可以快速地添加RocketMQ的客户端依赖，并配置相关的属性。
在Spring Tips中，可能会介绍如何配置和使用RocketMQ的Producer和Consumer，以及如何处理消息的生命周期、异常处理等。此外，还可能会介绍一些优化RocketMQ性能的技巧，如调整消息存储策略、优化网络传输等。
总之，Apache RocketMQ是一个功能强大、性能优越的分布式消息队列，与Spring框架的集成可以进一步简化其使用过程。通过Spring Tips的学习，可以更好地理解和使用RocketMQ进行消息通信。
Engineering
Josh Long
February 25, 2020Hi, Spring fans! In this installment of Spring Tips, we’re going to look at Alibaba’s Apache RocketMQ. We’ve talked some about Alibaba in Spring Tips before. Check out the earlier Spring Tips installment in which we explore some of Spring Cloud Alibaba.
Running Apache RocketMQIn order to use Apache RocketMQ, you’ll need to follow the steps in the RocketMQ quickstart. This Spring Tips installment introduces Apache RocketMQ, originally a technology developed and used internally at Alibaba and proven in the forge of 11/11, the famous Chinese sales holiday, sort of like “Cyber Monday,” or “Black Friday,” in the US. Sort of like that, but waaaaaay bigger. In 2019, Alibaba (alone, with no other e-commerce engines involved), made almost $40 billion USD in 24 hours. This required that trillions of messages be sent through something that could scale to meet the demand. RocketMQ is the only thing they could trust.
You’ll need to use Java 8 when running Apache RocketMQ. (You can use any version of Java when writing Spring applications that connect to Apache RocketMQ, of course.) I use SDK Manager (“SDKman” - sdk) to switch to the appropriate version of Java.
sdk use java 8.0.242.hs-adpt
That’ll install a version that works if it’s not already installed. Once that’s done, you’ll then need to run the NameServer.
${ROCKETMQ_HOME}/bin/mqnamesrv
Then you’ll need to run the Broker itself.
${ROCKETMQ_HOME}/bin/mqbroker -n localhost:9876
If you want to use SQL-based filtering, you need to add a property to the broker’s configuration, $ROCKETMQ_HOME/conf/broker.conf, and then tell RocketMQ to use that configuration.
enablePropertyFilter = true
I use a script like this to launch everything.
export JAVA_HOME=$HOME/.sdkman/candidates/java/8.0.242.hs-adpt
${ROCKETMQ_HOME}/bin/mqnamesrv &
${ROCKETMQ_HOME}/bin/mqbroker -n localhost:9876 -c ${ROCKETMQ_HOME}/conf/broker.confUsing Apache RocketMQ from Java Code
Let’s look at a simple producer class that uses the Spring Boot autoconfiguration and the RocketMQTemplate.
In order to work with this, you’ll need to create a new project on the Spring Initializr. I generated a new project with the latest version of Java and then I made sure to include Lombok. We also need the Apache RocketMQ client and the appropriate Spring Boot autoconfiguration:
org.apache.rocketmq rocketmq-spring-boot-starter 2.0.4
The autoconfiguration will create a connection to the running Apache RocketMQ broker, informed by certain properties.
rocketmq.name-server=127.0.0.1:9876
rocketmq.producer.group=greetings-producer-groupThe first property, name-server, tells the application where the Apache RocketMQ nameserver lives. The nameserver, then, knows where the broker lives. You’ll need to also specify a group for both the producer and the consumer. Here, we use greetings-producer-group.
package com.example.producer;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.RequiredArgsConstructor;
import org.apache.rocketmq.spring.core.RocketMQTemplate;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.context.annotation.Bean;
import org.springframework.messaging.Message;
import org.springframework.messaging.core.MessagePostProcessor;
import org.springframework.messaging.support.MessageBuilder;import java.time.Instant;
@RequiredArgsConstructor
@SpringBootApplication
public class ProducerApplication {@Bean
ApplicationListener<ApplicationReadyEvent> ready(RocketMQTemplate template) {
return event -> {
var now = Instant.now();
var destination = "greetings-topic";
for (var name : "Tammie,Kimly,Josh,Rob,Mario,Mia".split(",")) {
var payload = new Greeting("Hello @ " + name + " @ " + now.toString());
var messagePostProcessor = new MessagePostProcessor() {
@Override
public Message<?> postProcessMessage(Message<?> message) {
var headerValue = Character.toString(name.toLowerCase().charAt(0));
return MessageBuilder
.fromMessage(message)
.setHeader("letter", headerValue)
.build();
}
};
template.convertAndSend(destination, payload, messagePostProcessor);
}
};
}
public static void main(String[] args) {
SpringApplication.run(ProducerApplication.class, args);
}}
@Data
@AllArgsConstructor
@NoArgsConstructor
class Greeting {
private String message;
}I don’t know if it can get much simpler than that! It’s a simple for-loop, processing each name, creating a new Greeting object, and then using the RocketMQTemplate to send the payload to an Apache RocketMQ topic, greetings-topic. Here, we’ve used the overload of the RocketMQTemplate object that accepts a MessagePostProcessor. The MessagePostProcessor is a callback in which we can transform the Spring Framework Message object that will be sent out. In this example, we contribute a header value, letter, that contains the first letter of the name. We’ll use this in the consumer.
Let’s look at the consumer. Generate a new Spring Boot application from the Spring Initializr and be sure to add the Apache RocketMQ autoconfiguration. You’ll need to specify the name server in application.properties for the client, too.
The autoconfiguration supports defining beans that implement RocketMQListener, where T is the type of the payload that the consumer will receive. The payload, in this case, is the Greeting.
package com.example.consumer;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.extern.log4j.Log4j2;
import org.apache.rocketmq.spring.annotation.RocketMQMessageListener;
import org.apache.rocketmq.spring.core.RocketMQListener;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.stereotype.Service;import static org.apache.rocketmq.spring.annotation.SelectorType.SQL92;
@SpringBootApplication
public class ConsumerApplication {public static void main(String[] args) {
SpringApplication.run(ConsumerApplication.class, args);
}}
@Data
@AllArgsConstructor
@NoArgsConstructor
class Greeting {
private String message;
}@Log4j2
@Service
@RocketMQMessageListener(
topic = “greetings-topic”,
consumerGroup = “simple-group”
)
class SimpleConsumer implements RocketMQListener {@Override
public void onMessage(Greeting greeting) {
log.info(greeting.toString());
}}
In this example, the SimpleConsumer simply logs all incoming messages from the greetings-topic topic in Apache RocketMQ. Here, the consumer will process every message on the topic. Let’s look at another nice feature - selectors - that let us selectively process incoming messages. Let’s replace the existing RocketMQ listener with two new ones. Each one will use a SQL92-compatible predicate to determine whether incoming messages should be processed. One listener processes only the messages that have a letter header
matching m, k, or t. The other matches only those whose letter header matches j.
@Log4j2
@Service
@RocketMQMessageListener(
topic = “greetings-topic”,
selectorExpression = " letter = ‘m’ or letter = ‘k’ or letter = ‘t’ ",
selectorType = SQL92,
consumerGroup = “sql-consumer-group-mkt”
)
class MktSqlSelectorConsumer implements RocketMQListener {@Override
public void onMessage(Greeting greeting) {
log.info("'m', 'k', 't': " + greeting.toString());
}}
@Log4j2
@Service
@RocketMQMessageListener(
topic = “greetings-topic”,
selectorExpression = " letter = ‘j’ ",
selectorType = SQL92,
consumerGroup = “sql-consumer-group-j”
)
class JSqlSelectorConsumer implements RocketMQListener {@Override
public void onMessage(Greeting greeting) {
log.info("'j': " + greeting.toString());
}}
Not bad, eh? There’s plenty of other things that Apache RocketMQ supports (besides processing trillions of messages in 24 hours!) It can store long tail messages on disk, without degrading performance. It supports serialization - the ordering of - of messages, transactions, batch processing, etc. It even supports scheduled messages - messages that are only delivered after a certain interval. Needless to say, I’m a big Apache RocketMQ fan.
comments powered by Disqus
Apache RocketMQ 是一个流行的开源分布式消息和流数据平台，为大数据领域的实时、可靠的数据传输、处理和集成提供了高性能的保障。RocketMQ 的设计目标是满足互联网领域的海量消息堆积、高并发、可靠和实时的消息传输需求。
以下是 Apache RocketMQ 的一些主要特点和优势：
高可靠性：RocketMQ 提供了消息的持久化存储，确保消息不会因系统崩溃而丢失。同时，RocketMQ 支持消息备份和同步，保证数据的高可用性。
高性能：RocketMQ 采用零拷贝、顺序写盘等技术优化 I/O 性能，支持高并发消息的生产和消费，满足大规模分布式系统的需求。
分布式架构：RocketMQ 的分布式架构可以水平扩展，支持多个 Broker 集群，以满足不同规模的应用需求。
多种消息模型：支持发布/订阅模型（Pub/Sub）和点对点模型（P2P），满足不同的业务场景需求。
消息顺序性：RocketMQ 可以保证全局顺序消息和部分顺序消息，满足对消息顺序有严格要求的业务场景。
丰富的消息过滤：支持在 Broker 端和 Consumer 端进行消息过滤，减少不必要的消息传输和处理。
消息重试和死信队列：Consumer 端消费消息失败后，RocketMQ 支持消息重试机制，并可以将无法消费的消息发送到死信队列，供后续处理。
监控和告警：RocketMQ 提供了丰富的监控指标和告警机制，方便用户监控系统的运行状态并及时处理潜在问题。
在 Spring 框架中集成 Apache RocketMQ，可以方便地实现消息的异步通信和数据处理。Spring Boot 提供了对 RocketMQ 的集成支持，通过简单的配置和注解即可实现消息的发送和接收。此外，Spring Cloud Stream 也支持 RocketMQ 作为消息传输层，方便在微服务架构中实现消息的通信和集成。
