江苏龙网

  1. 主页 > 生活百科 > >

Filebeat的入门安装使用

Filebeat

日志采集的工具有很多种，如logagent, flume, logstash,betas等等 。首先要知道为什么要使用filebeat呢？因为logstash是jvm跑的，资源消耗比较大，启动一个logstash就需要消耗500M左右的内存，而filebeat只需要10来M内存资源 。常用的ELK日志采集方案中，大部分的做法就是将所有节点的日志内容通过filebeat送到kafka消息队列，然后使用logstash集群读取消息队列内容，根据配置文件进行过滤 。然后将过滤之后的文件输送到elasticsearch中，通过kibana去展示 。
1.下载安装包https://www.elastic.co/cn/downloads/beats/filebeat
2.安装解压：tar -zxvf filebeat-5.5.2-linux-x86_64.tar.gz
创建软链接：ln -s filebeat-5.5.2-linux-x86_64 filebeat
3.配置文件(同时输出到es和logstash以及kafka）vi filebeat.yml
filebeat.prospectors:- input_type: logpaths:- /home/pgxl/elk/a.txt# ----------------output.elasticsearch-------------------------output.elasticsearch:hosts: ["localhost:9200"]protocol: "http"index: "stat_filebeat" # template.name: "stat_ilebeat" # template.path: "filebeat.template.json" # template.overwrite: false#----------------output.logstash-----------------------------output.logstash:hosts: ["localhost:5044"]#---------------output.kafka----------------------------------output.kafka:enabled: truehosts: ["192.168.10.1:9092","192.168.10.2:9092","192.168.10.3:9092"]topic: "liuzc_test"partition.round_robin:reachable_only: trueworker: 2required_acks: 1compression: gzipmax_message_bytes: 10000000

Filebeat的入门安装使用

文章插图
 
4.启动filebeat./filebeat -e -c filebeat.yml
5.logstash监听filebeat的配置文件(只是输出监听到的数据到控制台，不写入别的组件)input {beats {port => 5044}}output {stdout{codec=>"rubydebug"}}
Filebeat的入门安装使用

文章插图
 
 6.kibana查看结果
Filebeat的入门安装使用

文章插图
 

Filebeat的入门安装使用

文章插图
 
 
7.遇到的坑用上述配置文件启动logstash的时候居然报错
The error reported is:
Couldn't find any input plugin named 'beats'. Are you sure this is correct? Trying to load the beats input plugin resulted in this error: no such file to load -- logstash/inputs/beats
Filebeat的入门安装使用

文章插图
 

Filebeat的入门安装使用

文章插图
 
?
大概意思就说说缺少logstash-input-beats这个组件，然后就查看一下安装的组件有哪些：
[pgxl@lx33 logstash]$ bin/plugin listlogstash-codec-collectdlogstash-codec-dotslogstash-codec-ednlogstash-codec-edn_lineslogstash-codec-es_bulklogstash-codec-fluentlogstash-codec-graphitelogstash-codec-jsonlogstash-codec-json_lineslogstash-codec-linelogstash-codec-msgpacklogstash-codec-multilinelogstash-codec-netflowlogstash-codec-oldlogstashjsonlogstash-codec-plainlogstash-codec-rubydebuglogstash-filter-anonymizelogstash-filter-checksumlogstash-filter-clonelogstash-filter-csvlogstash-filter-datelogstash-filter-DNSlogstash-filter-droplogstash-filter-fingerprintlogstash-filter-geoiplogstash-filter-greenlinelogstash-filter-groklogstash-filter-jsonlogstash-filter-kvlogstash-filter-metricslogstash-filter-multilinelogstash-filter-mutatelogstash-filter-rubylogstash-filter-sleeplogstash-filter-splitlogstash-filter-syslog_prilogstash-filter-throttlelogstash-filter-urldecodelogstash-filter-useragentlogstash-filter-uuidlogstash-filter-xmllogstash-input-couchdb_changeslogstash-input-elasticsearchlogstash-input-eventloglogstash-input-execlogstash-input-filelogstash-input-ganglialogstash-input-gelflogstash-input-generatorlogstash-input-graphitelogstash-input-heartbeatlogstash-input-httplogstash-input-imaplogstash-input-irclogstash-input-kafkalogstash-input-log4jlogstash-input-lumberjacklogstash-input-pipelogstash-input-rabbitmqlogstash-input-redislogstash-input-s3logstash-input-snmptraplogstash-input-sqslogstash-input-stdinlogstash-input-sysloglogstash-input-tcplogstash-input-twitterlogstash-input-udplogstash-input-unixlogstash-input-xmpplogstash-input-zeromqlogstash-output-cloudwatchlogstash-output-csvlogstash-output-elasticsearchlogstash-output-elasticsearch_httplogstash-output-emaillogstash-output-execlogstash-output-filelogstash-output-ganglialogstash-output-gelflogstash-output-graphitelogstash-output-hipchatlogstash-output-httplogstash-output-irclogstash-output-juggernautlogstash-output-kafkalogstash-output-lumberjacklogstash-output-nagIOSlogstash-output-nagios_nscalogstash-output-nulllogstash-output-opentsdblogstash-output-pagerdutylogstash-output-pipelogstash-output-rabbitmqlogstash-output-redislogstash-output-s3logstash-output-snslogstash-output-sqslogstash-output-statsdlogstash-output-stdoutlogstash-output-tcplogstash-output-udplogstash-output-xmpplogstash-output-zeromqlogstash-patterns-core


推荐阅读


上一篇：Google开源Java图片处理类库Thumbnail

下一篇：方便携带的蓝牙耳机有哪些，方便小巧佩戴舒适的蓝牙耳机推荐