业务目标

借助logstash工具 将mysql中的数据上传到elasticsearch中
并借助kibana工具可是化展示

a.下载解压

wget https://artifacts.elastic.co/downloads/logstash/logstash-7.1.1.tar.gz
tar -zxvf logstash-7.1.1.tar.gz  /usr/local/logstash

b.安装插件
cd /usr/local/logstash/bin

./logstash-plugin install logstash-input-jdbc
./logstash-plugin install logstash-output-elasticsearch

c.获取mysql驱动包
放在bin目录下，与配置文件同级

d.创建启动配置文件stdmysql.conf
借助 config/pipelines.yml 可以同时启动加载多个配置文件

vim stdmysql.conf
input {
  jdbc {
    jdbc_driver_library => "mysql-connector-java-5.1.7-bin.jar"
    jdbc_driver_class => "com.mysql.jdbc.Driver"
    jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/luoforTest"
    jdbc_user => "root"
    jdbc_password => "092248"
    schedule => "* * * * *"
    statement => "SELECT * FROM UserDto WHERE update_time >= :sql_last_value"
    use_column_value => true
    tracking_column_type => "timestamp"
    tracking_column => "update_time"
    last_run_metadata_path => "syncpoint_table"
  }
}


output {
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
    index => "userdto"  #不支持大写
    document_id => "%{id}"
  }
}

e.启动 logstash(通过kibana可视化展示)

./logstash -f stdmysql.conf &

mysql可视化_mysql

f.多配置同时启动
vim config/pipelines.yml

- pipeline.id: elk
  path.config: "../bin/stdin.conf"
- pipeline.id: mysql
  path.config: "../bin/stdmysql.conf"

直接启动无须指定配置

./logstash &

mysql可视化_logstash_02