之前我们在Windows平台上安装过Elasticsearch+X-Pack+Kibana工具（具体参考：Windows 安装Elasticsearch&Kibana&X-Pack），这里我们在Linux系统中做一个日志分析平台。
一.安装Elasticsearch
#wget -c https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.4.0.tar.gz #tar -zxvf elasticsearch-5.4.0.tar.gz #mkdir /usr/elk #mv elasticsearch-5.4.0 /usr/elk/elasticsearch #cd /usr/elk/elasticsearch/bin
然后我们可以使用如下命令启动
./elasticsearch
启动过程中可能遇到如下问题
①.jvm内存不足
解决方法，修改如下配置文件，调整-xms2g -Xmx2g为-xms1g -Xmx1g
vim /usr/elk/elasticsearch/config/jvm.options
②.要求openjdk版本至少1.8，oracle jdk 1.7
这个时候需要升级openjdk或者使用oracle jdk替换
#查询本地安装的数据库,并且将其删除
#rpm -qa | grep jdk java-1.6.0-openjdk-1.6.0.0-1.45.1.11.1.el6.x86_64 java-1.7.0-openjdk-1.7.0.0-1.32.1.11.1.el6.x86_64 #yum remove java-1.6.0-openjdk-1.6.0.0-1.45.1.11.1.el6.x86_64 #yum remove java-1.7.0-openjdk-1.7.0.0-1.32.1.11.1.el6.x86_64
#当然，如果你觉得上述操作麻烦，建议直接使用如下方式
#yum remove java-1.6.0-openjdk* #yum remove java-1.7.0-openjdk*
#然后通过如下方式检索java-1.8.0-openjdk
#yum search java-1.8.0-openjdk
============== N/S Matched: java-1.8.0-openjdk ============== java-1.8.0-openjdk.x86_64 : OpenJDK Runtime Environment java-1.8.0-openjdk-debug.x86_64 : OpenJDK Runtime Environment with full debug on java-1.8.0-openjdk-demo.x86_64 : OpenJDK Demos java-1.8.0-openjdk-demo-debug.x86_64 : OpenJDK Demos with full debug on java-1.8.0-openjdk-devel.x86_64 : OpenJDK Development Environment java-1.8.0-openjdk-devel-debug.x86_64 : OpenJDK Development Environment with full debug on java-1.8.0-openjdk-headless.x86_64 : OpenJDK Runtime Environment java-1.8.0-openjdk-headless-debug.x86_64 : OpenJDK Runtime Environment with full debug on java-1.8.0-openjdk-javadoc.noarch : OpenJDK API Documentation java-1.8.0-openjdk-javadoc-debug.noarch : OpenJDK API Documentation for packages with debug on java-1.8.0-openjdk-src.x86_64 : OpenJDK Source Bundle java-1.8.0-openjdk-src-debug.x86_64 : OpenJDK Source Bundle for packages with debug on
一般都能检索出来，如果检索不出来，建议去下载安装oracle jdk
#如果检索到java-1.8.0-openjdk，直接安装即可
#yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel java-1.8.0-openjdk-headless
③.root用户不允许运行
修改运行文件
#groupadd elkstack #创建组 #useradd elkstack -g elkstack -d /usr/elk -s /bin/bash #创建用户 #passwd elkstack #给用户创建密码 #chown -R elkstack:elkstack /usr/elk #目录的拥有者 ######################################################### #cd /etc/skel/ #进入用户登录状态管理目录，如果不执行此操作，则登录界面在sh中 #ls -a . .. .bash_logout .bash_profile .bashrc .mozilla #复制文件到新用户的创建目录 #cp .bash_logout /home/MysqL/ #cp .bash_profile /home/MysqL/ #cp .bashrc /home/MysqL #cd / ######################################################### #su -l elkstack #切换用户 #./elasticsearch/bin/elasticsearch #启动elasticsearch
④.当前用户创建文件的大小，可使用的内存有限制
max file descriptors [4096] for elasticsearch process is too low,increase to at least [65536] max number of threads [1024] for user [elkstack] is too low,increase to at least [2048] max virtual memory areas vm.max_map_count [65530] is too low,increase to at least [262144] system call filters Failed to install; check the logs and fix your configuration or disable system call filters at your own risk
Q:max file descriptors [4096] for elasticsearch process is too low,increase to at least [65536] #vi /etc/security/limits.conf 添加如下内容: * soft nofile 65536 * hard nofile 131072 * soft nproc 2048 * hard nproc 4096
Q:max number of threads [1024] for user [elkstack] is too low,increase to at least [2048] #vi /etc/security/limits.d/90-nproc.conf 修改如下内容： * soft nproc 1024 #修改为 * soft nproc 2048
Q:max virtual memory areas vm.max_map_count [65530] is too low,increase to at least [262144] #vi /etc/sysctl.conf 添加下面配置： vm.max_map_count=655360
system call filters Failed to install; check the logs and fix your configuration or disable system call filters at your own risk #vim config/elasticsearch.yml 添加 bootstrap.system_call_filter: false
配置完成之后，执行命令
sysctl -p
以上是安装过程中遇到的比较多的问题
使用curl 检查是否成功启动
#curl -i http://127.0.0.1:9200
HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 327
{
"name" : "rGlFyHB","cluster_name" : "elasticsearch","cluster_uuid" : "7sEFicrvQW-RPbJTjekbHg","version" : {
"number" : "5.4.0","build_hash" : "780f8c4","build_date" : "2017-04-28T17:43:27.229Z","build_snapshot" : false,"lucene_version" : "6.5.0"
},"tagline" : "You KNow,for Search"
}
二.安装kibana
#wget -c https://artifacts.elastic.co/downloads/kibana/kibana-5.4.0-linux-x86_64.tar.gz #tar -zxvf kibana-5.4.0-linux-x86_64.tar.gz #mv kibana-5.4.0-linux-x86_64 /usr/elk/kibana #cd /usr/elk/ #chown -R elkstack:elkstack kibana #./kibana/bin/kibana
使用curl检测是否成功启动（注意：必须先启动elasticsearch）
#curl -i http://localhost:5601
HTTP/1.1 200 OK
kbn-name: kibana
kbn-version: 5.4.0
cache-control: no-cache
content-type: text/html; charset=utf-8
content-length: 217
accept-ranges: bytes
Date: Mon,22 May 2017 06:45:26 GMT
Connection: keep-alive
<script>var hashRoute = '/app/kibana';
var defaultRoute = '/app/kibana';
var hash = window.location.hash;
if (hash.length) {
window.location = hashRoute + hash;
} else {
window.location = defaultRoute;
}
此外，kibana中需要配置elasticsearch的信息，如果elasticsearch的访问信息更新了，同样也需要更新kibana中的配置信息
#vim kibana/config/kibana.yml
#elasticsearch默认配置信息如下 elasticsearch.url: "http://localhost:9200"
三.安装logstash
#wget -c https://artifacts.elastic.co/downloads/logstash/logstash-5.4.0.tar.gz #tar -zxvf logstash-5.4.0.tar.gz #mv logstash-5.4.0 /usr/elk/logstash #cd /usr/elk/ #chown -R elkstack:elkstack logstash #./logstash/bin/logstash
测试是否安装成功
注意：最好以root用户运行，或者在sudoers中添加用户的sudo命令权限，否则可能产生好多问题
#./logstash/bin/logstash -e 'input{stdin{}} output{stdout{}}'
#启动之后，执行如下操作，测试是否有回显，如果有回显，则表示正确
#Hello World
四.配置&插件安装
①.远程访问
kibana.yml:
server.port: 5601 server.host: "192.168.1.210" elasticsearch.url: "http://192.168.1.210:9200"
elasticsearch.yml:
network.host: "192.168.1.210" http.port: 9200 #加入新集群时使用的ip地址，默认是回环地址 #discovery.zen.ping.unicast.hosts: ["192.168.1.210"] #集群中最少的master数量 #discovery.zen.minimum_master_nodes: 3 #bootstrap.system_call_filter: false
logstash.yml
http.host: "172.20.11.62"
②.安装X-Pack
注意：安装前必须停止elasticsearch与kibana服务
#cd /usr/elk/elasticsearch/bin #./elasticsearch-plugin install x-pack #cd /usr/elk/kibana/bin #./kibana-plugin install x-pack
安装X-Pack完成之后，穷elasticsearch与kibana ，会进行用户登录校验，默认用户名和密码如下
username : elastic passowrd : changeme
但是，对于logstash来说，需要在配置文件中配置用户名才行，否则无法链接到elasticsearch
input {
file {
type =>"syslog"
path => ["/var/log/messages","/var/log/secure" ]
}
syslog {
type =>"syslog"
port =>"5544"
}
}
output {
stdout { codec=> rubydebug }
elasticsearch {
hosts => ["192.168.1.210:9200"]
user => elastic
password => changeme
index => "syslogstash-%{+YYYY.MM.dd}"
template_overwrite => true
}
}
③.Kibana创建索引
Kibana创建索引的前提是logstash的pipline配置文件中存索引，并且logstash已经向elasticsearch注册了索引
index => "syslogstash-%{+YYYY.MM.dd}"
input {
file {
type =>"syslog"
path => ["/var/log/messages","/var/log/secure" ]
}
syslog {
type =>"syslog"
port =>"5544"
}
}
output {
stdout { codec=> rubydebug }
elasticsearch {
hosts => ["192.168.1.210:9200"]
user => elastic
password => changeme
index => "logstash-%{+YYYY.MM.dd}"
template_overwrite => true
}
}
检测配置文件是否正确
#logstash/bin/logstash -f test_logstash.conf -t
启动logstash
#logstash/bin/logstash -f test_logstash.conf
触发Input Event，让logstash主动注册index到elasticsearch
#logger -p info "hello,remote rsyslog"
然后登录Kibana，点击Management->Index Patterns打开索引注册页面，点击左侧菜单栏中的【+】，新增索引。
如果你看不到Create按钮，那么很可能意味着索引没有注册成功，注册可能需要一个Input Event输入触发才行。
如果索引注册成功，那么点击Kibana菜单discover，选择syslogstash-*索引，便能看到相应的事件。
参考：
ELK集群部署及收集nginx日志
Elasticsearch5.0 版本安装错误
记录Linux下安装elasticSearch时遇到的一些错误
centos7虚拟机安装elasticsearch5.0.x-安装篇
ElasticSearch 5.0.0 安装部署常见错误或问题
已解决：登录Linux的 -bash-4.2$ 问题