In today’s Logstash Filter Of The Day, we look at encoding NGiNX access logs into JSON. To save the default NGiNX log format into Elasticsearch requires transcoding it to JSON. Read this post to learn how to format your logs
As of Nginx 1.11.8 you can use the parameter escape=json in your log_format definition to handle JSON encoding and escaping. Here’s a sample log format you can use. Configure server to use json-log definition. Example log line. Passed through jq.
Elasticsearch provides a few built-in analyzers. Here’s a breakdown of each and where best to use it. No Analyzer Disabling the Analyzer will store your data with no tokens and can only be searched by exact matches of the full
The _source field contains the original JSON document body of an Elaticsearch document. It is not indexed nor searchable, but it can be returned when executing fetch requests. Keeping the _source field saved with the document comes with substantial storage
Index templates allow you to specify index settings and mappings that will automatically be applied when an index is first created. It uses a simple pattern match to control when a template is applied. An index can have multiple patterns
Removing duplicate documents from Elasticsearch saves disk space and will speed-up searches. This saves you time and makes you more productive. I setup and manage ELK (Elasticsearch, Logstash and Kibana) clusters that process hundreds of millions log lines per day.
Example assumes the node IP Address is 172.21.1.21 and that you have jq installed. If you do not have jq remove | jq . from the commands. Expand the number of shards that can be reallocated Choose settings appropriate for
Overview Nagios Core Server combined with the Nagios Remote Plugin Executor (NRPE) Server allows Nagios to execute remote plugins on client servers. This post discusses automated ways to override the default plugin tests and parameters. Assumptions This post assumes you
Consul and Consul-Template are two powerful tools for updating server files and executing a command after the file has been updated. This makes it a great and lightweight tool for managing configuration files for applications that need to reload or restart after
sel.datum(aryData) vs. sel.data(aryData) .datum() gives aryData to all elements in the selection. .data() gives one unique array value in aryData to each element in the selection (i.e. .update()). It also has .enter() and .exit() as methods. Example Here’s the data we’ll