文章目录
- 需求背景
- 安装各个组件
- 安装ruby
- 获取fluentd源码
- 修改gem源
- 安装Bundle
- 构建fluentd
- 运行fluentd
- 安装过程遇到的问题
- 1.安装bundler没有zlib包
- 2. 没有openssl
需求背景
公司需要搭建一个日志收集服务器,用于将公司的项目日志汇总到一台服务器上面,方便查看和减轻各项目服务器压力。但是由于目前资源不够充足,所以放弃使用ELK、EFK。最后在调研尝试过Linux自带的Syslog和fluentd之后,Linux自带的Syslog虽然更简单,但是汇总的数据中会有乱码的情况(怀疑是Appender问题),并且将日志信息组合成Json格式比较麻烦,因此选择使用fluentd来做日志收集。
安装各个组件
选用fluentd来做日志收集选择使用源码来进行安装,并且需要安装一些依赖的插件,如Ruby等,下面依次介绍需要安装的组件。
安装ruby
下载ruby压缩包 ruby-2.6.5.tar.gz,版本:2.6.5,并解压
tar -zxvf ruby-2.6.5.tar.gz
cd ruby-2.6.5
安装ruby,安装需要gcc
yum install -y gcc
./configure prefix=/export/source/ruby
make && make install
安装完成后修改系统配置文件 /etc/profile,在最后添加ruby目录并加到path中
export RUBY_HOME=/export/source/ruby
export PATH=$PATH:$JAVA_HOME/bin:$RUBY_HOME/bin
验证是否安装成功
[root@localhost ruby-2.6.5]# ruby -v
ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
显示ruby版本号,安装成功
获取fluentd源码
从github上拉取fluentd源码,github地址:fluentd
git clone https://github.com/fluent/fluentd.git
cd fluentd
修改gem源
先查看当前默认源
gem source
*** CURRENT SOURCES ***
https://rubygems.org/
移除默认源
gem sources -r https://rubygems.org/
添加源,目前可用的有ruby-china的源,添加ruby-china源
gem sources -a https://gems.ruby-china.com
https://gems.ruby-china.com added to sources
# 查看当前源
gem source
*** CURRENT SOURCES ***
https://gems.ruby-china.com
更新缓存
gem sources -u
安装Bundle
gem install bundler
等待提示安装成功
构建fluentd
进入fluentd文件夹后,构建项目
[root@localhost fluentd]# bundle install
Fetching gem...
...
Bundle complete! 11 Gemfile dependencies, 37 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
[root@localhost fluentd]# bundle exec rake build
fluentd 1.7.3 built to pkg/fluentd-1.7.3.gem.
[root@localhost fluentd]# gem install pkg/fluentd-1.7.3.gem
Successfully installed fluentd-1.7.3
Parsing documentation for fluentd-1.7.3
Installing ri documentation for fluentd-1.7.3
Done installing documentation for fluentd after 3 seconds
1 gem installed
安装完成
运行fluentd
[root@localhost fluentd]# fluentd --setup ./fluent
[root@localhost fluentd]# fluentd -c ./fluent/fluent.conf -vv &
最后在控制台显示fluentd读取的配置文件信息并且打印相关日志。至此日志收集服务器所需要的fluentd已经搭建完成,接下来就是配置fluentd以完成日记收集功能。
安装过程遇到的问题
1.安装bundler没有zlib包
ERROR: Loading command: install (LoadError)
cannot load such file -- zlib
ERROR: While executing gem ... (NoMethodError)
undefined method `invoke_with_build_args' for nil:NilClass
解决方案
安装zlib-devel
yum -y install zlib-devel
进入ruby源码文件夹,安装ruby自身提供的zlib包ruby-2.5.1/ext/zlib
cd ruby-2.6.5/ext/zlib
ruby ./extconf.rb
make
make install
在make时会报错
make: *** No rule to make target `/include/ruby.h', needed by `zlib.o'. Stop.
修改目录中Makefile文件
zlib.o: $(top_srcdir)/include/ruby.h
改成
zlib.o: …/…/include/ruby.h
到这里就可以make成功了。
2. 没有openssl
错误描述:cannot load such file – openssl
/export/source/ruby/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- openssl (LoadError)
29: from /export/source/ruby/bin/gem:21:in `<main>'
28: from /export/source/ruby/lib/ruby/2.6.0/rubygems/gem_runner.rb:59:in `run'
27: from /export/source/ruby/lib/ruby/2.6.0/rubygems/command_manager.rb:148:in `run'
26: from /export/source/ruby/lib/ruby/2.6.0/rubygems/command_manager.rb:178:in `process_args'
...
/export/source/ruby/lib/ruby/2.6.0/rubygems/request.rb:84:in `rescue in configure_connection_for_https': Unable to require openssl, install OpenSSL and rebuild Ruby (preferred) or use non-HTTPS sources (Gem::Exception)
22: from /export/source/ruby/bin/gem:21:in `<main>'
21: from /export/source/ruby/lib/ruby/2.6.0/rubygems/gem_runner.rb:59:in `run'
20: from /export/source/ruby/lib/ruby/2.6.0/rubygems/command_manager.rb:148:in `run'
19: from /export/source/ruby/lib/ruby/2.6.0/rubygems/command_manager.rb:178:in `process_args'
...
解决方案
yum install openssl-devel -y
在ruby自身提供的额外文件夹中ruby-2.6.5/ext/openssl,执行ruby ./extconf.rb,再make
ruby ./extconf.rb
make
make install
make时出错
make: *** No rule to make target `/include/ruby.h', needed by `ossl.o'. Stop.
修改Makefile文件,在文件顶部添加top_srcdir = …/…