Elasticsearch 8.4: What's New, Release History & Support Lifecycle

What Is New in Elasticsearch 8.4

Elasticsearch 8.4 delivers key enhancements across search, security, and operations. This version focuses on making vector search more practical and improving the developer experience with better APIs and diagnostics.

How is vector search getting faster and more efficient?

The big news is native k-NN indexing support. Before 8.4, approximate k-nearest neighbor (kNN) search was efficient but building the index happened on the fly, which was slow for large datasets.

Now you can create a knn_vector field with an index option set to true. This builds the HNSW graph during indexing, making subsequent searches significantly faster. This matters because it shifts the computational heavy-lifting from query time to index time, which is a much better trade-off for production workloads.

Example Index Mapping

PUT my-knn-index
{
  "mappings": {
    "properties": {
      "my_vector": {
        "type": "knn_vector",
        "dimension": 3,
        "index": true,
        "similarity": "cosine"
      }
    }
  }
}

What new query language is in tech preview?

Elasticsearch introduces ESQL, a new piped query language, in tech preview. ESQL aims to simplify data exploration and aggregation by using a more intuitive syntax reminiscent of SQL but with a pipeline structure.

Instead of nesting complex JSON aggregations, you can write a linear query. For instance, to find the average salary by department, you'd write: FROM employees | STATS avg(salary) BY department. This is a big win for developers who find the traditional Elasticsearch aggregation DSL difficult to write and debug.

How does downsampling help with time series data?

The new downsampling API lets you reduce the storage footprint of time series data while preserving its overall trends. It works by rolling up old, high-resolution data into a lower resolution, storing only the aggregated metrics.

You configure a downsampling policy with an interval (e.g., 1h) and the aggregation to use (e.g., max, avg). In practice, this is a more automated and integrated solution than trying to manage this yourself with Rollups or ILM, though it's currently limited to metrics and count aggregations.

What security improvements should I know about?

Elasticsearch 8.4 is now FIPS 140-2 compliant when running on a FIPS-enabled JVM. This is crucial for deployments in government or financial sectors that require this certification.

For everyday use, Service Account tokens are now generally available. These are long-lived tokens specifically designed for machine-to-machine authentication, like for CLI scripts or application integrations, making them a safer alternative to user-based tokens for automation.

Are there any new tools for working with geospatial data?

Yes, the new Vector Tile API is a game-changer for building map visualizations. It returns data in the standard Mapbox Vector Tile format, which is highly efficient for rendering complex maps at high zoom levels.

This means your Kibana maps or custom map clients can now request only the vector data they need for a specific viewport and zoom level, drastically improving performance and reducing bandwidth compared to returning raw GeoJSON geometries.