1、安装elk：elasticsearch+kibana+logstash
方法一：
使用elk官方的yum源，直接yum在线安装，缺点是网速很慢要等很久。
新增yum源elasticsearch
cat >>/etc/yum.repos.d/elasticsearch.repo <<EOF
[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=0
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
新增yum源kibana
cat >>/etc/yum.repos.d/kibana.repo <<EOF
[kibana-7.x]
name=Kibana repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=0
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
新增yum源logstash
cat >>/etc/yum.repos.d/logstash.repo <<EOF
[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=0
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
yum install elasticsearch kibana logstash
方法二：
直接下载elk的3个rpm包进行本地安装。
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-x86_64.rpm
https://artifacts.elastic.co/downloads/kibana/kibana-7.0.0-x86_64.rpm
https://artifacts.elastic.co/downloads/logstash/logstash-7.0.0.rpm
rpm -ivh elasticsearch-7.0.0-x86_64.rpm
rpm -ivh kibana-7.0.0-x86_64.rpm
rpm -ivh logstash-7.0.0.rpm
方法三：源码包安装
省略。。。
2、启动elasticsearch
配置文件说明：
elasticsearch.yml for configuring Elasticsearch
jvm.options for configuring Elasticsearch JVM settings
log4j2.properties for configuring Elasticsearch logging
systemctl start elasticsearch 出现以下监听端口：
tcp6 0 0 127.0.0.1:9200 :::* LISTEN 30057/java
tcp6 0 0 ::1:9200 :::* LISTEN 30057/java
tcp6 0 0 127.0.0.1:9300 :::* LISTEN 30057/java
tcp6 0 0 ::1:9300 :::* LISTEN 30057/java
3、配置kibana
先修改kibana的监听范围，否则只能本机访问
vim /etc/kibana/kibana.yml，增加如下内容：
server.host: ‘0.0.0.0’
systemctl start kibana
访问http://192.168.1.126:5601
4、配置logstash
新建一个logstash配置用于支持syslog收集
vim /etc/logstash/conf.d/syslog.conf
使用1514端口进行监听，类型为syslog
===========================================================
input {
tcp {
port => 1514
type => syslog
}
udp {
port => 1514
type => syslog
}
}
output {
elasticsearch { hosts => [“localhost:9200”] }
stdout { }
}
===========================================================
systemctl restart logstash
5、配置syslog客户端
找一台客户端，编辑rsyslog配置，在最后加上一句并重启
（logstash服务器ip是192.168.1.126）
cat >>/etc/rsyslog.conf <<EOF
*.* @@192.168.1.126:1514
EOF
systemctl restart rsyslog
此时在服务端的messages日志可以看到客户端发来的json报文：
Apr 26 00:59:59 dakvm logstash: {
Apr 26 00:59:59 dakvm logstash: “type” => “syslog”,
Apr 26 00:59:59 dakvm logstash: “@version” => “1”,
Apr 26 00:59:59 dakvm logstash: “port” => 44298,
Apr 26 00:59:59 dakvm logstash: “@timestamp” => 2019-04-25T16:59:59.028Z,
Apr 26 00:59:59 dakvm logstash: “host” => “192.168.1.125”,
Apr 26 00:59:59 dakvm logstash: “message” => “<77>Apr 26 01:01:01 dacentos run-parts(/etc/cron.hourly)[4295 finished 0anacron”
Apr 26 00:59:59 dakvm logstash: }
6、遇到的问题
=======================================================
绑定到514端口日志出现报错，报错内容见最下方：
原因是因为1-1024端口只能root用户使用来进行监听，而logstash启动时是用的logstash这个用户，故没有权限使用端口514。
因为使用yum来安装，默认会新建logstash此用户并由其启动进程。应该可以对启动用户进行修改为root从而避免该问题。
解决方法类似转发，通过iptables将普通端口收到的数据转发给514端口。
国外论坛给出的方法：
Ports in range 1 to 1024 are privileged and only root user can listen on it.
Options:
run logstash as root (not a good idea)
use setcap to grant java permission to use privileged ports
use iptables or a proxy to forward port 514 to an unprivileged port.
— https://github.com/elastic/logstash/issues/1587#issuecomment-50939823
所以，还是直接改端口吧。。。将端口改为1514
Apr 26 00:32:25 dakvm logstash: [2019-04-26T00:32:25,721][INFO ][logstash.inputs.tcp ] Starting tcp input listener {:address=>”0.0.0.0:514″, :ssl_enable=>”false”}
Apr 26 00:32:25 dakvm logstash: [2019-04-26T00:32:25,723][ERROR][logstash.javapipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Apr 26 00:32:25 dakvm logstash: Pipeline_id:main
Apr 26 00:32:25 dakvm logstash: Plugin: <LogStash::Inputs::Tcp type=>”syslog”, port=>514, id=>”b9ddfe239cb4ce591abd9ae12dbe28e96dafc38941029217a522fe7386568936″, enable_metric=>true, codec=><LogStash::Codecs::Line id=>”line_053fee24-ab6f-4201-86f3-38f07fb19469″, enable_metric=>true, charset=>”UTF-8″, delimiter=>”\n”>, host=>”0.0.0.0″, mode=>”server”, proxy_protocol=>false, ssl_enable=>false, ssl_verify=>true, ssl_key_passphrase=><password>, tcp_keep_alive=>false, dns_reverse_lookup_enabled=>true>
Apr 26 00:32:25 dakvm logstash: Error: Permission denied
Apr 26 00:32:25 dakvm logstash: Exception: Java::JavaNet::SocketException
Apr 26 00:32:25 dakvm logstash: Stack: sun.nio.ch.Net.bind0(Native Method)
Apr 26 00:32:25 dakvm logstash: sun.nio.ch.Net.bind(sun/nio/ch/Net.java:433)
Apr 26 00:32:25 dakvm logstash: sun.nio.ch.Net.bind(sun/nio/ch/Net.java:425)
Apr 26 00:32:25 dakvm logstash: sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:223)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:128)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:558)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1283)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:501)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:486)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:989)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:254)
Apr 26 00:32:25 dakvm logstash: io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:364)
Apr 26 00:32:25 dakvm logstash: io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:163)
Apr 26 00:32:25 dakvm logstash: io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:403)
Apr 26 00:32:25 dakvm logstash: io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:463)
Apr 26 00:32:25 dakvm logstash: io.netty.util.concurrent.SingleThreadEventExecutor$5.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:858)
Apr 26 00:32:25 dakvm logstash: io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
Apr 26 00:32:25 dakvm logstash: java.lang.Thread.run(java/lang/Thread.java:748)
Apr 26 00:32:26 dakvm logstash: [2019-04-26T00:32:26,396][INFO ][logstash.inputs.udp ] Starting UDP listener {:address=>”0.0.0.0:514″}
Apr 26 00:32:26 dakvm logstash: [2019-04-26T00:32:26,397][ERROR][logstash.inputs.udp ] UDP listener died {:exception=>#<Errno::EACCES: Permission denied – bind(2) for “0.0.0.0” port 514>, :backtrace=>[“org/jruby/ext/socket/RubyUDPSocket.java:215:in `bind'”, “/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:116:in `udp_listener'”, “/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:68:in `run'”, “/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:297:in `inputworker'”, “/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:290:in `block in start_input'”]}
Apr 26 00:32:26 dakvm logstash: [2019-04-26T00:32:26,731][INFO ][logstash.inputs.tcp ] Starting tcp input listener {:address=>”0.0.0.0:514″, :ssl_enable=>”false”}
Apr 26 00:32:26 dakvm logstash: [2019-04-26T00:32:26,734][ERROR][logstash.javapipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Apr 26 00:32:26 dakvm logstash: Pipeline_id:main
Apr 26 00:32:26 dakvm logstash: Plugin: <LogStash::Inputs::Tcp type=>”syslog”, port=>514, id=>”b9ddfe239cb4ce591abd9ae12dbe28e96dafc38941029217a522fe7386568936″, enable_metric=>true, codec=><LogStash::Codecs::Line id=>”line_053fee24-ab6f-4201-86f3-38f07fb19469″, enable_metric=>true, charset=>”UTF-8″, delimiter=>”\n”>, host=>”0.0.0.0″, mode=>”server”, proxy_protocol=>false, ssl_enable=>false, ssl_verify=>true, ssl_key_passphrase=><password>, tcp_keep_alive=>false, dns_reverse_lookup_enabled=>true>
Apr 26 00:32:26 dakvm logstash: Error: Permission denied
Apr 26 00:32:26 dakvm logstash: Exception: Java::JavaNet::SocketException
Apr 26 00:32:26 dakvm logstash: Stack: sun.nio.ch.Net.bind0(Native Method)
Apr 26 00:32:26 dakvm logstash: sun.nio.ch.Net.bind(sun/nio/ch/Net.java:433)
Apr 26 00:32:26 dakvm logstash: sun.nio.ch.Net.bind(sun/nio/ch/Net.java:425)
Apr 26 00:32:26 dakvm logstash: sun.nio.ch.ServerSocketChannelImpl.bind(sun/nio/ch/ServerSocketChannelImpl.java:223)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.socket.nio.NioServerSocketChannel.doBind(io/netty/channel/socket/nio/NioServerSocketChannel.java:128)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.AbstractChannel$AbstractUnsafe.bind(io/netty/channel/AbstractChannel.java:558)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.DefaultChannelPipeline$HeadContext.bind(io/netty/channel/DefaultChannelPipeline.java:1283)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.AbstractChannelHandlerContext.invokeBind(io/netty/channel/AbstractChannelHandlerContext.java:501)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.AbstractChannelHandlerContext.bind(io/netty/channel/AbstractChannelHandlerContext.java:486)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.DefaultChannelPipeline.bind(io/netty/channel/DefaultChannelPipeline.java:989)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.AbstractChannel.bind(io/netty/channel/AbstractChannel.java:254)
Apr 26 00:32:26 dakvm logstash: io.netty.bootstrap.AbstractBootstrap$2.run(io/netty/bootstrap/AbstractBootstrap.java:364)
Apr 26 00:32:26 dakvm logstash: io.netty.util.concurrent.AbstractEventExecutor.safeExecute(io/netty/util/concurrent/AbstractEventExecutor.java:163)
Apr 26 00:32:26 dakvm logstash: io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(io/netty/util/concurrent/SingleThreadEventExecutor.java:403)
Apr 26 00:32:26 dakvm logstash: io.netty.channel.nio.NioEventLoop.run(io/netty/channel/nio/NioEventLoop.java:463)
Apr 26 00:32:26 dakvm logstash: io.netty.util.concurrent.SingleThreadEventExecutor$5.run(io/netty/util/concurrent/SingleThreadEventExecutor.java:858)
Apr 26 00:32:26 dakvm logstash: io.netty.util.concurrent.FastThreadLocalRunnable.run(io/netty/util/concurrent/FastThreadLocalRunnable.java:30)
Apr 26 00:32:26 dakvm logstash: java.lang.Thread.run(java/lang/Thread.java:748)