Logstash – parsing multiline log entries

In my previous post I’ve shown how to configure Logstash so that, it would be able to parse the logs in custom format. Configuration presented in that post had one significant drawback – it wasn’t able to parse multiline log entries. This is a rather common scenario, especially when you log exceptions with a stack trace. Log entry, in that case may look as follows

Parsing these kind of messages using current configuration will result in grok parsing error
and we won’t be able to search against predefined fields. Fortunately Logstash allows you to configure something called input codecs which basically allows you to transform input data into some other form. One of those codes is multiline codec, which is responsible for “merging” multiline logs into one entry.
Here is example of codec configuration

The code above says that any line not starting with a “TimeStamp=timestamp value” should be merged with the previous line.
Multiline coded can be added to a variety of inputs. Here is how you can apply it to file input

Thanks to that change, Logstash is now able to correctly parse exceptions from our logs.

Source code for this post can be found here

Logstash – parsing multiline log entries