2021-06-01 安装kafka es尝试编码

原创

寂寞旅行 2024-10-21 13:56:13 博主文章分类：linux ©著作权

文章标签 kafka zookeeper java tomcat tdd 文章分类 Python 后端开发

©著作权归作者所有：来自51CTO博客作者寂寞旅行的原创作品，请联系作者获取转载授权，否则将追究法律责任

安装 kafka

上面步骤中安装了zookeeper 和 kafka ,其中 zookeeper是kafka自动集成的,因为 kafka需要依赖zookeeper.
安装好之后,编码集成编写配置写demo

引入pom

<dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

编写配置配置文件 .yml

server:
  port: 8188
spring:
  data:
    elasticsearch:
      cluster-name: elasticsearch
      cluster-nodes: 172.18.3.180:9300
  # kafka配置
  kafka:
    bootstrap-servers: 172.18.3.170:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
    consumer:
      group-id: test
      enable-auto-commit: true
      auto-commit-interval: 1000
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

编写demo 一个消息提供者,一个消息消费者
提供者 TestKafkaProducerController 消费者 TestConsumer

// 第一个类 消息提供者
@RestController
@RequestMapping("kafka")
public class TestKafkaProducerController {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @RequestMapping("send")
    public String send(String msg){
        kafkaTemplate.send("animal", msg);//topic 必须对应 animal
        return "success";
    }
}

// ===========================================
//第二个类 消息消费者
@Component
public class TestConsumer {

    @KafkaListener(topics = "animal") //topic 必须对应 animal
    public void listen (ConsumerRecord<?, ?> record) throws Exception {
        System.out.printf("topic = %s, offset = %d, value = %s \n", record.topic(), record.offset(), record.value());
    }
}

最后测试下kafka发送消息以及接收消息
页面访问

http://localhost:8188/kafka/send?msg="测试kafka发送消息"

效果如图

2021-06-01 安装kafka es尝试编码_tomcat

kafak消费者打印出消息

至此,kafka发送消息基本实现

===============================================================
下面说个半成品,es 就是安装好了,但是无法使用,暂时未找到原因,请大神指点
安装步骤参考

安装启动命令 docker run --name elasticsearch -d -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -e "discovery.type=single-node" -p 9200:9200 -p 9300:9300 elasticsearch:6.4.3

之后需要配置,进入容器后的elasticsearch.yml

network.host: 0.0.0.0
cluster.name: elasticsearch
http.cors.enabled: true
http.cors.allow-origin: "*"

配置好之后才可以在页面访问 elasticsearch 页面 ip+9200 和 elasticsearch-header 界面ip+9100
我安装,配置好之后,页面访问的样子

2021-06-01 安装kafka es尝试编码_tomcat_02

页面访问效果

仔细查看了各个版本的对应关系,以及springboot的支持版本,spring data es 版本2.x ,es版本6.4,配置es 的服务名称,与配置文件中的完全一直都是elasticsearch

server:
  port: 8188
spring:
  data:
    elasticsearch:
      cluster-name: elasticsearch
      cluster-nodes: 172.18.3.180:9300
      # cluster-nodes: 172.18.3.180:9200

测试类用的是 ElasticsearchTemplate

@RestController
public class TestController {

    @Autowired
    ElasticsearchTemplate elasticsearchTemplate;

    @RequestMapping("/test")
    public void test(){

        Item item = new Item();
        item.setId(0L);
        item.setTitle("67");
        item.setCategory("56");
        item.setBrand("576");
        item.setPrice(0.0D);
        item.setImages("98");

        elasticsearchTemplate.createIndex("124");

        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
//        if (params.get("uuid") != null && StringUtils.isNotBlank(params.get("uuid").toString())) {
//            boolQuery.must(QueryBuilders.matchPhraseQuery("article_id", params.get("uuid")));
//        }
//        if (params.get("subject_id") != null && StringUtils.isNotBlank(params.get("subject_id").toString())) {
//            boolQuery.must(QueryBuilders.matchPhraseQuery("subject_id", params.get("subject_id")));
//        }
//        if (params.get("title_like") != null && StringUtils.isNotBlank(params.get("title_like").toString())) {
//            boolQuery.must(QueryBuilders.matchPhraseQuery("title", params.get("title_like")));
//        }
//        if (params.get("content_like") != null && StringUtils.isNotBlank(params.get("content_like").toString())) {
//            boolQuery.must(QueryBuilders.matchPhraseQuery("content", params.get("content_like")));
//        }
//        Pageable pageable= new PageRequest(Integer.parseInt(String.valueOf(params.get("start"))), Integer.parseInt(String.valueOf(params.get("limit"))),new Sort(Sort.Direction.DESC, "update_time"));
        NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(boolQuery)
//                .withPageable(pageable)
                .build();
        List<Map> maps = elasticsearchTemplate.queryForList(searchQuery, Map.class);

        IndexQuery indexQuery = new IndexQueryBuilder().withId("12").withObject("980").build();

        elasticsearchTemplate.index(indexQuery);

    }
}

启动正常,执行测试方法的时候,报错,无可用的节点
页面访问路径

http://localhost:8188/test

报错

Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.

Tue Jun 01 10:24:00 CST 2021
There was an unexpected error (type=Internal Server Error, status=500).
None of the configured nodes are available: [{#transport#-1}{wRlyLXKuT9S8ox9yGLDu3g}{172.18.3.180}{172.18.3.180:9300}]
NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{wRlyLXKuT9S8ox9yGLDu3g}{172.18.3.180}{172.18.3.180:9300}]]
    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:349)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:247)
    at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:60)
    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:381)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:396)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1254)
    at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.create(AbstractClient.java:1381)
    at org.springframework.data.elasticsearch.core.ElasticsearchTemplate.createIndex(ElasticsearchTemplate.java:201)
    at com.learn.elastic.controller.TestController.test(TestController.java:37)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:189)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:102)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:800)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1038)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)