Logstash Filter OTD : NGiNX Access Log

In today’s Logstash Filter Of The Day, we look at encoding NGiNX access logs into JSON. To save the default NGiNX log format into Elasticsearch requires transcoding it to JSON. Read this post to learn how to format your logs

NGiNX JSON Access Log

As of Nginx 1.11.8 you can use the parameter escape=json in your log_format definition to handle JSON encoding and escaping. Here’s a sample log format you can use. Configure server to use json-log definition. Example log line. Passed through jq.

Elasticsearch Document Deduplication With Logstash

Removing duplicate documents from Elasticsearch saves disk space and will speed-up searches. This saves you time and makes you more productive. I setup and manage ELK (Elasticsearch, Logstash and Kibana) clusters that process hundreds of millions log lines per day.