Kawika's SRE/Devops Blog

Deliver – Measure – Grow

Menu

  • Home
  • About
  • Contact

Elasticsearch

You are here: Kawika's SRE/Devops Blog > Elasticsearch

Which Elasticsearch Analyzer for the job?

Elasticsearch provides a few built-in analyzers. Here’s a breakdown of each and where best to use it. No Analyzer Disabling the Analyzer will store your data with no tokens and can only be searched by exact matches of the full

Kawika Ohumukini March 26, 2020April 7, 2020 Elasticsearch Read more

Elasticsearch _source field

The _source field contains the original JSON document body of an Elaticsearch document. It is not indexed nor searchable, but it can be returned when executing fetch requests. Keeping the _source field saved with the document comes with substantial storage

Kawika Ohumukini March 25, 2020April 7, 2020 Elasticsearch Read more

Elasticsearch Index Templates

Index templates allow you to specify index settings and mappings that will automatically be applied when an index is first created. It uses a simple pattern match to control when a template is applied. An index can have multiple patterns

Kawika Ohumukini March 24, 2020April 7, 2020 Elasticsearch Read more

Elasticsearch Document Deduplication With Logstash

Removing duplicate documents from Elasticsearch saves disk space and will speed-up searches. This saves you time and makes you more productive. I setup and manage ELK (Elasticsearch, Logstash and Kibana) clusters that process hundreds of millions log lines per day.

Kawika Ohumukini March 24, 2020April 7, 2020 Elasticsearch, Logstash Read more

Remove Elasticsearch Node

Example assumes the node IP Address is 172.21.1.21 and that you have jq installed. If you do not have jq remove | jq . from the commands. Expand the number of shards that can be reallocated Choose settings appropriate for

Kawika Ohumukini August 31, 2018April 7, 2020 Elasticsearch Read more

Recent Posts

  • Logstash Filter OTD : NGiNX Access Log
  • NGiNX JSON Access Log
  • Which Elasticsearch Analyzer for the job?
  • Elasticsearch _source field
  • Elasticsearch Index Templates

Categories

  • DevOps
  • Elasticsearch
  • Logstash
  • Miscellaneous
  • One-Liners

Recent Posts

  • Logstash Filter OTD : NGiNX Access Log

    Logstash Filter OTD : NGiNX Access Log

  • NGiNX JSON Access Log

    NGiNX JSON Access Log

  • Which Elasticsearch Analyzer for the job?

    Which Elasticsearch Analyzer for the job?

Pages

  • Home
  • Contact Me
  • Miscellaneous

Contact

  • Kawika Ohumukini
  • San Diego | CA | USA
  • Email: kawikao@kickin6.com

Copyright © 2021 Kawika's SRE/Devops Blog

  • Home
  • Contact Me
  • Miscellaneous
Logstash Default Template
{
  "order": 0,
  "version": 60001,
  "index_patterns": [
    "logstash-*"
  ],
  "settings": {
    "index": {
      "number_of_shards": "1",
      "refresh_interval": "5s"
    }
  },
  "mappings": {
    "dynamic_templates": [
      {
        "message_field": {
          "path_match": "message",
          "mapping": {
            "norms": false,
            "type": "text"
          },
          "match_mapping_type": "string"
        }
      },
      {
        "string_fields": {
          "mapping": {
            "norms": false,
            "type": "text",
            "fields": {
              "keyword": {
                "ignore_above": 256,
                "type": "keyword"
              }
            }
          },
          "match_mapping_type": "string",
          "match": "*"
        }
      }
    ],
    "properties": {
      "@timestamp": {
        "type": "date"
      },
      "geoip": {
        "dynamic": true,
        "properties": {
          "ip": {
            "type": "ip"
          },
          "latitude": {
            "type": "half_float"
          },
          "location": {
            "type": "geo_point"
          },
          "longitude": {
            "type": "half_float"
          }
        }
      },
      "@version": {
        "type": "keyword"
      }
    }
  },
  "aliases": {}
}