redis找不到从机 redis集群查不到key

转载

mob64ca1409d8ea 2023-09-03 22:00:35

文章标签 redis找不到从机 redis 数据库缓存 spring 文章分类 Redis 数据库

redis集群监听key失效

文章目录

redis集群监听key失效
前言

问题分析

一、集群版redis监听

2.动态redis集群环境

前言

使用redis进行key过期丢失案例，集群环境下的redis就会出现问题。

问题分析

但是在集群模式下你会发现一个问题，就是客户端可能接收不到所有的key过期事件

根据github issue #53102里面的描述：

Note: keyspace notifications are node-specific, unlike regular pub/sub
which is broadcast from all nodes. So: you (or the library you are
using) would need to attach to all nodes to reliably get keyspace
notifications from cluster.

也就是说键空间通知是指定node的（自身），不像常规的pub/sub是广播到所有节点的，所以我们需要连接集群中所有的可用节点去获取键空间通知

as keyspace notifications are not broadcasted in the cluster you’ll
have to:
1.Open a connection to each of the cluster’s nodes
2.In each connection, subscribe to keyspace notifications from that node

换句话说，Redis Cluster集群中key采用的是分片存储，不同的key通过哈希计算放到不同的slot槽中，即可能是不同的node节点，而keyspace notification只在自己所在的node上发布，并没有发布到集群当中，我们redis-py-cluster客户端订阅监听的时候只监听随机的node（即每次建立连接的node是随机的），那么就有可能有些key过期没有被监听到，这就导致说我们收不到这个过期事件。

Where Pub/Sub messages are generally sent across the cluster, the
keyspace notifications are only sent locally. Broadcasting such events
cluster-wide could become very bandwidth intensive. However, to
simulate cluster-wide behaviour, clients can subscribe to all the
master nodes and merge the received events. In this approach the
clients should check the cluster configuration from time to time to
make sure to connect to other masters added in possible
reconfiguration.

即集群本身的pub/sub是节点之间交叉广播的，但是键空间通知只支持本地

继续翻，发现了antirez大神（Redis的作者）亲口的答复3：

redis找不到从机 redis集群查不到key_数据库

Hello. I’m not sure we are able to provide non-local events in Redis Cluster. This would require to broadcast the events cluster-wide, which is quite bandwidth intensive… Isn’t it better that the client just subscribes to all the master nodes instead, merging all the received events? However one problem with this is that from time to time it should check the cluster configuration to make sure to connect to other masters added during the live of the cluster.

也就是说因集群模式点对点之间网络带宽的压力，不考虑将键空间通知加入到集群广播中来，更建议是客户端直接连接节点获取键空间通知，但是有个问题就是需要客户端随时检查集群配置，以获取新加入的master节点

一、集群版redis监听

如果确定集群环境可以使用。

spring:
  application:
    name: redis-cluster
  redis:
    cluster:
      nodes: 127.0.0.1:6379,127.0.0.1:6380,127.0.0.1:6381,127.0.0.1:6382,127.0.0.1:6383,127.0.0.1:6384  
      max-redirects: 6  
    host1: 127.0.0.1
    host2: 127.0.0.1
    host3: 127.0.0.1
    port1: 6379
    port2: 6380
    port3: 6381
    password: 123456

package com.cxdq.ipower.data.ws.config;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import redis.clients.jedis.JedisPoolConfig;

/**
 * @Author  wyb
 * @Create  2019/8/6 14:46
 * @Desc    监听redis中Key过期事件
 **/
@Configuration
public class RedisListenerConfig {

	@Value("${spring.redis.host1}")
	private String host1;

	@Value("${spring.redis.host2}")
	private String host2;

	@Value("${spring.redis.host3}")
	private String host3;

	@Value("${spring.redis.host4}")
	private String host4;

	@Value("${spring.redis.host5}")
	private String host5;

	@Value("${spring.redis.host6}")
	private String host6;


	@Value("${spring.redis.port1}")
	private int port1;

	@Value("${spring.redis.port2}")
	private int port2;

	@Value("${spring.redis.port3}")
	private int port3;

	@Value("${spring.redis.port4}")
	private int port4;

	@Value("${spring.redis.port5}")
	private int port5;

	@Value("${spring.redis.port6}")
	private int port6;

	@Autowired
	private RedisTemplate redisTemplate ;
	@Bean
	JedisPoolConfig jedisPoolConfig(){
		JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
		jedisPoolConfig.setMaxIdle(100);
		jedisPoolConfig.setMaxWaitMillis(1000);
		return jedisPoolConfig;
	}




	// redis-cluster不支持key过期监听，建立多个连接，对每个redis节点进行监听
	@Bean
	RedisMessageListenerContainer redisContainer1() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host1);
		jedisConnectionFactory.setPort(port1);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}

	@Bean
	RedisMessageListenerContainer redisContainer2() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host2);
		jedisConnectionFactory.setPort(port2);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}

	@Bean
	RedisMessageListenerContainer redisContainer3() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host3);
		jedisConnectionFactory.setPort(port3);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}

	@Bean
	RedisMessageListenerContainer redisContainer4() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host4);
		jedisConnectionFactory.setPort(port4);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}

	@Bean
	RedisMessageListenerContainer redisContainer5() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host5);
		jedisConnectionFactory.setPort(port5);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}

	@Bean
	RedisMessageListenerContainer redisContainer6() {
		final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
		JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
		jedisConnectionFactory.setHostName(host6);
		jedisConnectionFactory.setPort(port6);
		jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
		jedisConnectionFactory.afterPropertiesSet();
		container.setConnectionFactory(jedisConnectionFactory);
		return container;
	}




	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener1() {
		return new RedisKeyExpirationListener(redisContainer1());
	}

	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener2() {
		return new RedisKeyExpirationListener(redisContainer2());
	}

	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener3() {
		return new RedisKeyExpirationListener(redisContainer3());
	}

	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener4() {
		return new RedisKeyExpirationListener(redisContainer4());
	}

	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener5() {
		return new RedisKeyExpirationListener(redisContainer5());
	}

	@Bean
	RedisKeyExpirationListener redisKeyExpirationListener6() {
		return new RedisKeyExpirationListener(redisContainer6());
	}

	@Bean
	public RedisTemplate<String, Object> stringSerializerRedisTemplate() {
		RedisSerializer<String> stringSerializer = new StringRedisSerializer();
		redisTemplate.setKeySerializer(stringSerializer);
		redisTemplate.setValueSerializer(stringSerializer);
		redisTemplate.setHashKeySerializer(stringSerializer);
		redisTemplate.setHashValueSerializer(stringSerializer);
		return redisTemplate;
	}

}

接受到redisKey过期

public class RedisKeyExpirationListener extends KeyExpirationEventMessageListener {


    @Autowired
    private RedisTemplate redisTemplate;

    public RedisKeyExpirationListener(RedisMessageListenerContainer listenerContainer) {
        super(listenerContainer);
    }
 
    @Override
    public void onMessage(Message message, byte[] pattern) {
        // 用户做自己的业务处理即可,message.toString()可以获取失效的key
        String mesg = message.toString();
        System.err.println(mesg);
    }
}

如果是分布式的话就需要给这个加上分布式🔒了

2.动态redis集群环境

动态读取配置文件里面节点，从而进行动态监听。自己使用的环境是三主三从

node:
  mapping:
    host1: 127.0.0.1:6379
    host2: 127.0.0.1:6380
    host3: 127.0.0.1:6381

package com.ipower.data.websocket.redis;

import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;


import java.util.Map;

/**
 * @author wyb
 * @date 2022-01-06 20:00
 */
@ConfigurationProperties(prefix = "node")
@Data
@Configuration
public class BeanProperties {

    private Map<String, String> mapping;
}

package com.ipower.data.websocket.redis;

import com.ipower.data.websocket.config.SpringContextUtils;
import org.springframework.beans.BeansException;
import org.springframework.beans.MutablePropertyValues;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.BeanDefinition;
import org.springframework.beans.factory.config.ConfigurableBeanFactory;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.beans.factory.support.BeanDefinitionRegistry;
import org.springframework.beans.factory.support.DefaultListableBeanFactory;
import org.springframework.beans.factory.support.RootBeanDefinition;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.context.EnvironmentAware;
import org.springframework.context.annotation.*;
import org.springframework.core.annotation.AnnotationUtils;
import org.springframework.core.env.Environment;
import org.springframework.core.type.AnnotationMetadata;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.serializer.RedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import redis.clients.jedis.JedisPoolConfig;

import javax.annotation.PostConstruct;
import java.util.Map;
import java.util.Objects;


@Configuration
@EnableConfigurationProperties(BeanProperties.class)
@Import(AppConfig.ImportConfig.class)
public class AppConfig  implements ImportBeanDefinitionRegistrar, EnvironmentAware  {


    @Autowired
    private RedisTemplate redisTemplate ;

    private BeanProperties beanProperties;


    @Bean
    static JedisPoolConfig jedisPoolConfig(){
        JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
        jedisPoolConfig.setMaxIdle(100);
        jedisPoolConfig.setMaxWaitMillis(1000);
        return jedisPoolConfig;
    }

    @Override
    public void setEnvironment(Environment environment) {
        // 通过Binder将environment中的值转成对象
        beanProperties = Binder.get(environment).bind(getPropertiesPrefix(BeanProperties.class), BeanProperties.class).get();
    }

    private String getPropertiesPrefix(Class<?> tClass) {
        return Objects.requireNonNull(AnnotationUtils.getAnnotation(tClass, ConfigurationProperties.class)).prefix();
    }



    @PostConstruct
    public void init(){
        for (Map.Entry<String, String> entry : beanProperties.getMapping().entrySet()) {
            redisKeyExpirationListener(entry.getKey());
        }
    }

    @Bean
    @Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE)
    RedisKeyExpirationListener redisKeyExpirationListener(String s) {
        return new RedisKeyExpirationListener((RedisMessageListenerContainer) SpringContextUtils.getContext().getBean(s));
    }

    public static class ImportConfig implements ImportBeanDefinitionRegistrar, EnvironmentAware {
        private BeanProperties beanProperties;


        @Override
        public void registerBeanDefinitions(AnnotationMetadata annotationMetadata, BeanDefinitionRegistry beanDefinitionRegistry) {
            for (Map.Entry<String, String> entry : beanProperties.getMapping().entrySet()) {
                // 注册bean
                RootBeanDefinition beanDefinition = new RootBeanDefinition();
                beanDefinition.setBeanClass(RedisMessageListenerContainer.class);
                JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory();
                String s = entry.getValue();
                jedisConnectionFactory.setHostName(s.substring(0,s.lastIndexOf(":")));
                jedisConnectionFactory.setPort(Integer.parseInt(s.substring(s.lastIndexOf(":")+1)));
                jedisConnectionFactory.setPassword("123456");
                jedisConnectionFactory.setPoolConfig(jedisPoolConfig());
                jedisConnectionFactory.afterPropertiesSet();
                MutablePropertyValues values = new MutablePropertyValues();
                values.addPropertyValue("connectionFactory", jedisConnectionFactory);
                beanDefinition.setPropertyValues(values);
                beanDefinitionRegistry.registerBeanDefinition(entry.getKey(), beanDefinition);
            }
        }

        @Override
        public void setEnvironment(Environment environment) {
            // 通过Binder将environment中的值转成对象
            beanProperties = Binder.get(environment).bind(getPropertiesPrefix(BeanProperties.class), BeanProperties.class).get();
        }

        private String getPropertiesPrefix(Class<?> tClass) {
            return Objects.requireNonNull(AnnotationUtils.getAnnotation(tClass, ConfigurationProperties.class)).prefix();
        }
    }



    @Bean
    public RedisTemplate<String, Object> stringSerializerRedisTemplate() {
        RedisSerializer<String> stringSerializer = new StringRedisSerializer();
        redisTemplate.setKeySerializer(stringSerializer);
        redisTemplate.setValueSerializer(stringSerializer);
        redisTemplate.setHashKeySerializer(stringSerializer);
        redisTemplate.setHashValueSerializer(stringSerializer);
        return redisTemplate;
    }

}

ImportConfig.registerBeanDefinitions方法是动态获取配置文件里面节点进行注册连接bean
redisKeyExpirationListener()方法是进行监听的节点bean需要多例

package com.ipower.data.websocket.redis;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.config.ConfigurableBeanFactory;
import org.springframework.context.annotation.Scope;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.KeyExpirationEventMessageListener;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;


/**
 * @Author
 * @Create
 * @Desc
 **/
public class RedisKeyExpirationListener extends KeyExpirationEventMessageListener {


    @Autowired
    private RedisTemplate redisTemplate;

    public RedisKeyExpirationListener(RedisMessageListenerContainer listenerContainer) {
        super(listenerContainer);
    }
 
    @Override
    public void onMessage(Message message, byte[] pattern) {
        // 用户做自己的业务处理即可,message.toString()可以获取失效的key
        String mesg = message.toString();
        System.err.println(mesg);
    }
}

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。