Posted on

from filebeat to logstash

In logstash.yml you can parse the data coming from the earlier filebeat example with a ‘filter’.

A ‘kv’ filter is for splitting data in key-value pairs and the default is to expect a comma as a separator.

The values are converted to strings unless otherwise specified.  The ‘mutuate’ section allows them to be converted to integers. Finally, the timestamp of the event needs to be when it happened and not when logstash received the data. Therefore, the ‘date’ filter is used to convert the value in logdate and use that as the timestamp .

filter {
if [type] == "mylog" {

kv {
field_split => ","

}
mutate {
convert => { "date" => "integer" }
convert => { "month" => "integer" }
convert => { "year" => "integer" }
convert => { "hour" => "integer" }
convert => { "minute" => "integer" }
convert => { "second" => "integer" }
}
date {
locale => "en"
match => [ "logdate" , "yyyyMMddHHmm" ]
target => "@timestamp"
}
}

}