hermes 试用

hermes 是一个不错的基于kafaka 的event broker，基于push模型（webhook）
测试环境使用docker-compose 运行

环境准备

  • docker-compose
 
version: '3'
services:
  graphite:
    image: nickstenning/graphite
    ports:
      - "80:80"
      - "2003:2003"
  frontend:
    image: allegro/hermes-frontend
    ports:
      - "8080:8080"
    depends_on:
      - zk
      - kafka
      - graphite
  zk:
    image: wurstmeister/zookeeper:3.4.6
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka:1.1.0
    ports:
      - "9092:9092"
    depends_on:
      - zk
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: zk:2181
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'
      KAFKA_DELETE_TOPIC_ENABLE: 'true'
      KAFKA_BROKER_ID: 0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
  consumers:
    image: allegro/hermes-consumers
    depends_on:
      - zk
      - kafka
      - graphite
  benthos:
    image: jeffail/benthos
    volumes:
    - "./conf/webhook.yaml:/benthos.yaml"
    ports:
    - "4195:4195"
  management:
    image: allegro/hermes-management
    ports:
      - "8090:8090"
    depends_on:
      - zk
      - kafka
      - graphite
 
 
  • 运行
docker-compose up -d
 
  • webhook 服务
    webhook 服务使用benthos,配置如下
 
input:
  type: broker
  broker:
    inputs:
      - type: http_server
        http_server:
          path: /
        processors:
          - type: text
            text:
              operator: prepend
              value: "get message: "
output:
  type: stdout
 
 
  • 参考架构图
    hermes 试用_编程
  • 组件说明
    frontend 进行消息发送
    consumers 消息订阅
    manger 管理主题以及订阅
    message store 存储以及路由消息当前支持的是kafka
    medata store 共享状态，当前使用的是zk
    metrics store 可选的，存储metrics 信息，当前是graphite
    tracking store 存储追踪信息，当前是es

测试

  • 创建group && topic
    注意应该使用management 的rest api 地址，对应docker 环境为8090 端口,注意topic的格式
    应该为 groupname.topicname
 
group: 
curl -X POST -H "Content-Type: application/json" http://localhost:8090/groups -d '{"groupName": "dalong"}'
topic:
curl -X POST -H "Content-Type: application/json" http://localhost:8090/topics -d '{
    "name": "dalong.userlogin",
    "description": "This is userlogin topic",
    "contentType": "JSON",
    "retentionTime": {
        "duration": 1
    },
    "owner": {
        "source": "Plaintext",
        "id": "dalong"
    }
}'
 
  • 创建订阅
    注意应该使用management 的rest api 地址，对应docker 环境为8090 端口，注意topic的格式
    应该为 groupname.topicname
 
curl -X POST -H "Content-Type: application/json" http://localhost:8090/topics/dalong.userlogin/subscriptions -d '{
    "topicName": "dalong.userlogin",
    "name": "userlogin", 
    "description": "This is my subscription",
    "endpoint": "http://benthos:4195/", 
    "owner": {
        "source": "Plaintext",
        "id": "dalong"
    }
}'
 
 
http://localhost:8090/topics/dalong.userlogin/subscriptions/userlogin/state
 

应该为：ACTIVE
支持的状态有：SUSPENDED，ACTIVE

  • 发送
 
curl -X POST -H "Content-Type: application/json" http://localhost:8080/topics/dalong.userlogin -d '{"message": "Hello world!"}'
 
  • 查看发送消息状态
docker-compose logs -f frontend benthos
 

效果

Attaching to hermes-project_frontend_1, hermes-project_benthos_1
benthos_1 | {"@timestamp":"2018-12-16T03:28:15Z","@service":"benthos","level":"INFO","component":"benthos","message":"Launching a benthos instance, use CTRL+C to close."}
benthos_1 | {"@timestamp":"2018-12-16T03:28:15Z","@service":"benthos","level":"INFO","component":"benthos","message":"Listening for HTTP requests at: http://0.0.0.0:4195\n"}
benthos_1 | get message: {"message": "Hello world!"}
 

说明

hermes 使用起来还是比较方便的，但是官方的docker-compose 对于kafaka 的配置有点问题

参考资料

https://hermes-pubsub.readthedocs.io/en/latest/
https://github.com/rongfengliang/hermes-docker-compose