ELK日志采集

• Elasticsearch：近乎全文搜索引擎
• Logstash：读取原始日志进行过滤和分析
• Kibana：web图形页面

192.168.100.211

下载elk

[root@localhost ~]# docker pull sebp/elk:760

修改参数

主机内存 最少4g
[root@localhost ~]# echo 'vm.max_map_count = 262144' >> /etc/sysctl.conf 
[root@localhost ~]# sysctl -p
vm.max_map_count = 262144

开启容器

[root@localhost ~]# docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -itd --restart always -e ES_HEAP_SIZE="2g" -e LS_HEAP_SIZE="2g" --name elk sebp/elk:760
16b5fcbb44b23da6e1698ef041f0abd91e4f5a0f79c7fdd7bef8c9335419548a

# 查看是否启动
[root@localhost ~]# netstat -anput |grep docker
tcp6       0      0 :::5601                 :::*                    LISTEN      17140/docker-proxy  
tcp6       0      0 :::9200                 :::*                    LISTEN      17129/docker-proxy  
tcp6       0      0 :::5044                 :::*                    LISTEN      17151/docker-proxy  

访问页面
192.168.100.211:5601

• 根据页面提示来在命令行内输入命令
  
• 下面的操作是根据上面这张图来进行的

---

安装filebeat

[root@localhost ~]# curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.6.0-x86_64.rpm
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 23.5M  100 23.5M    0     0  4967k      0  0:00:04  0:00:04 --:--:-- 5677k
[root@localhost ~]# sudo rpm -vi filebeat-7.6.0-x86_64.rpm
警告：filebeat-7.6.0-x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID d88e42b4: NOKEY
软件包准备中...
filebeat-7.6.0-1.x86_64

修改配置文件

• 这里页面写了 要指定 elasticsearch 和 kibana 的ip
• 另外还需要加入 docker 的日志文件才可以监控到数据

  [root@localhost ~]# vim /etc/filebeat/filebeat.yml
   # 开启日志监控 24行
    enabled: true
   # 指定 docker的日志文件 29行
     paths:
     - /var/log/*.log
     - /var/lib/docker/containers/*/*.log # 这里是docker的日志文件
  
   # 指定kibana的ip和端口 125行
     host: "192.168.100.211:5601"
   # 指定elasticsearch 的ip和端口  152行
     hosts: ["192.168.100.211:9200"]

  照着页面提示启动filebeat

    # 启用 elasticsearch
  [root@localhost ~]# sudo filebeat modules enable elasticsearch
  Enabled elasticsearch
  # 初始化
  [root@localhost ~]# sudo filebeat setup
  Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.
  Index setup finished.
  Loading dashboards (Kibana must be running and reachable)
  Loaded dashboards
  Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.
  See more: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-ml.html
  Loaded machine learning job configurations
  Loaded Ingest pipelines
  # 启动
  [root@localhost ~]# sudo service filebeat start
  Starting filebeat (via systemctl):                         [  确定  ]
  

  点击最后的蓝色按钮就可以了

• 测试数据日志是否被监控

   [root@localhost ~]# docker run busybox sh -c 'while true; do echo "hello_lmk"; sleep 10s; done;'
   hello_lmk
  # 该容器每10秒会输出一个 hello_lmk
  #在elk中查看是否被采集到了