What Is New in Elasticsearch 7.13
| Category | Key Updates |
|---|---|
| New Features | Vector search with knn query, runtime_mappings in search, EQL sequence
joins, downsample ILM action |
| Enhancements | Faster index recovery, improved frozen tier storage, better CCS performance, enhanced GeoIP database handling |
| Deprecations | Deprecated the _type field in EQL queries |
How does vector search work in 7.13?
The big news is the introduction of a dedicated knn query for approximate k-nearest neighbors
search. This lets you perform vector similarity searches natively within your Elasticsearch queries. You index
your dense vector data using the dense_vector field type and then use the new knn
clause to find similar documents.
In practice, this is a game-changer for building semantic search, recommendation systems, and anomaly detection directly on your Elastic stack without needing external services. The query integrates seamlessly with the standard Elasticsearch query DSL, allowing you to combine lexical search with vector-based semantic search in a single request.
What are runtime mappings in search?
You can now define runtime_mappings directly within a search request. This creates transient fields
that exist only for the duration of that specific query, calculated on the fly from your existing document data.
This is incredibly useful for on-the-fly data transformation without reindexing. Need to tweak a field for a one-off analysis or A/B test? Just define it in your search request. It keeps your index mapping clean and gives you massive flexibility for data exploration and aggregation.
GET my_index/_search
{
"runtime_mappings": {
"temp_field": {
"type": "long",
"script": "emit(doc['price'].value * params.multiplier)",
"params": {
"multiplier": 1.1
}
}
},
"query": { ... }
}
How is EQL getting better?
Event Query Language (EQL) now supports sequence joins, which allows you to correlate events across multiple series of data based on keys. You can write queries to find sequences of events that happen across different entities with a common identifier.
This makes EQL much more powerful for security and observability use cases. Think detecting a pattern where a user logs in from one IP address and then, within minutes, logs in from a completely different geographic location--sequence joins can now model that complex relationship easily.
What storage improvements were made?
The frozen tier sees significant storage efficiency gains. When a shard is allocated to the frozen tier, Elasticsearch now uses the recoverable bytes from the local node cache more effectively, drastically reducing the amount of heap memory required per shard.
For cross-cluster search (CCS), the process now skips unnecessary phases when possible, making queries against remote clusters faster and more efficient. Index recovery is also faster due to optimizations in how segments are fetched from the repository.
How is ILM more powerful now?
A new downsample action is available in Index Lifecycle Management (ILM). This automates the process
of rolling up time-series data from a high-resolution index into a new, lower-resolution index to save on
storage costs for older data.
You configure the fixed interval for the downsampled data (e.g., 1h, 1d), and ILM handles the rest. This matters because it provides a native, automated path for cost-effective data retention for metrics and logging use cases, complementing the existing rollup and shrink actions.
FAQ
Can I use the new `knn` query with regular queries?
Yes, absolutely. The knn
clause can be combined with other query clauses in a bool query. This hybrid approach lets you
filter documents based on traditional criteria (e.g., date ranges, keywords) and then perform the vector
similarity search on the filtered subset.
Do runtime mappings in search affect my index mapping?
No, that's the best part. Runtime
mappings defined in a search request are completely transient. They are not persisted to the index mapping and
only exist for the life of that single API call, so there's no risk of polluting your cluster state.
What happens to the old Rollup feature with the new Downsample action?
They serve different
purposes. The downsample action creates a summarized version of a single index, preserving the document
structure. The older rollup feature aggregates data from multiple indices into a single, entirely new rollup
index with a different schema. Downsample is simpler and built for ILM automation.
Is the deprecated `_type` field in EQL going away immediately?
No, it's just a deprecation
warning in 7.13. You can still use it, but you should start migrating your EQL queries to use a different field
for filtering, like an explicit event category field, as support for _type will be removed in a
future version.
How do the frozen tier improvements affect my node sizing?
The optimizations mean you can
host many more frozen indices on the same hardware. The reduced heap usage per shard allows for a higher density
of shards on a node, which translates to lower storage costs for your archival data.