Elasticsearch 5.4: What's New, Release History & Support Lifecycle

What Is New in Elasticsearch 5.4

Elasticsearch 5.4 delivers significant enhancements across search, security, and data management. This release focuses on making complex operations more intuitive and improving the overall resilience of the cluster.

How does Painless scripting change the game?

Painless is a new secure scripting language designed specifically for Elasticsearch. It's the new default for inline and stored scripts, offering significant performance and security advantages over Groovy or other previous options.

It's sandboxed by default, which prevents potentially dangerous operations right out of the box. In practice, this means you get the flexibility of scripting without the operational headaches of managing fine-grained security permissions.

Example Usage

GET /_search
{
  "script_fields": {
    "total_discount": {
      "script": {
        "lang": "painless",
        "source": "doc['price'].value * params.discount_rate",
        "params": {
          "discount_rate": 0.1
        }
      }
    }
  }
}

What's new with reindexing data?

The reindex API now supports pulling data directly from a remote Elasticsearch cluster. This is a huge quality-of-life improvement for migrations or consolidating data from different deployments.

Before this, you had to use external tools or write custom scripts to transfer data between clusters. Now, it's a single API call that handles the entire process, including setting up a remote connection and the reindex operation itself.

Reindex from Remote Syntax

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://other-cluster:9200"
    },
    "index": "old_index"
  },
  "dest": {
    "index": "new_index"
  }
}

Are there better ways to control index readiness?

Yes, the new Wait for Refresh API allows you to block until a refresh has occurred, making the operations visible to search. This provides more explicit control over data visibility compared to the `refresh` parameter.

This matters for testing and scenarios where you need to ensure that a document is immediately searchable after an indexing operation without forcing a refresh on every single write, which is expensive.

API Call

POST /my_index/_wait_for_refresh

How is data processing improved in Ingest Nodes?

Ingest nodes get two powerful new processors: the Date Index Name processor and the Dot Expander processor. These simplify common data transformation tasks during the ingestion pipeline.

The Date Index Name processor automatically routes documents to time-based indices (like `logstash-2017.05.15`) based on a date field in the document itself. This is fantastic for log data and time-series use cases, moving logic out of your application and into the cluster.

Dot Expander Processor

{
  "dot_expander": {
    "field": "foo.bar"
  }
}