What Is New in Elasticsearch 7.7
Elasticsearch 7.7 delivers significant enhancements in observability, security, and data management. This release focuses on making it easier to manage complex deployments and extract insights from your data.
| Category | Key Updates |
|---|---|
| New Features | New EQL language, Data streams, SAML authentication for Kibana, Logs UI |
| Enhancements | Faster index recovery, Frozen tier storage improvements, Better CCS performance |
| Security | FIPS 140-2 readiness, New SAML realm settings, Service account tokens |
| Deprecations | Deprecation of the `_field_stats` API |
How does the new EQL language change search?
The Event Query Language (EQL) introduces a powerful way to search for event sequences. It's designed for tracing complex behaviors across time, like tracking a user's journey through a system or identifying multi-step attack patterns in security data.
Instead of just finding matching documents, EQL lets you find sequences of related events. This is a game-changer for security and observability use cases where context and order matter. You can now write queries that find events where A happened, then B, then C.
In practice, this means you can hunt for sophisticated threats or analyze complex application workflows with a single, expressive query. It moves beyond traditional keyword search into the realm of behavioral analysis.
What are data streams and why use them?
Data streams provide a unified interface for managing time-series data across multiple backing indices. They simplify how you handle append-only data like logs, metrics, and events by abstracting away the underlying index management.
You write to a stream, and Elasticsearch automatically handles routing documents to the correct backing index based on the timestamp. This makes rollovers and lifecycle management much cleaner for time-series use cases. It's a more logical abstraction for logs and metrics than managing individual indices.
This matters because it reduces the operational overhead of managing time-series data at scale. You interact with a single named resource while Elasticsearch handles the distribution across the index lifecycle.
How is security improved in this release?
Elasticsearch 7.7 adds several key security enhancements including FIPS 140-2 compliance, expanded SAML capabilities, and service account tokens. The FIPS support ensures the stack can operate in environments requiring certified cryptographic modules.
New SAML realm settings provide greater flexibility for single sign-on configurations, particularly for Kibana. The introduction of service account tokens offers a secure way for Kibana and other applications to authenticate with Elasticsearch without using user credentials.
These improvements make it easier to deploy Elasticsearch in strict security environments while maintaining a smooth user authentication experience. The service account tokens are particularly useful for automating system-to-system communication.
What performance optimizations were made?
This release includes significant performance improvements for index recovery and cross-cluster search. Index recovery is now faster due to optimizations in how segments are transferred during the process, reducing downtime when restoring or relocating shards.
Cross-cluster search (CCS) performance has been enhanced through better network handling and reduced overhead. For frozen tier storage, the efficiency of searching data in the frozen tier has been improved, making archival data more accessible without warming it up.
These optimizations matter most for large-scale deployments where recovery times and cross-cluster latency directly impact user experience and operational efficiency. The frozen tier improvements make cold data searches more practical for analytical queries.
FAQ
How do I start using EQL in my queries?
You can use EQL through the new `eql` search API. The syntax is different from standard Elasticsearch queries, focusing on event sequences with operators like `sequence` and `join` for correlating events over time.
Can I convert my existing time-series indices to data streams?
No, data streams are designed for new time-series data. You cannot convert existing indices into data streams. You need to create a new data stream and index future documents into it while keeping existing indices as-is.
What exactly does FIPS 140-2 readiness mean?
It means Elasticsearch can use FIPS-validated cryptographic modules for TLS, hashing, and encryption when running on a FIPS-enabled JVM. This is required for US federal government deployments and other regulated sectors.
Are service account tokens replacing user authentication?
No, they complement it. Service account tokens are designed for machine-to-machine communication, like Kibana connecting to Elasticsearch, while user authentication handles human access through Kibana or other clients.
Why was the _field_stats API deprecated?
The `_field_stats` API was deprecated in favor of using aggregations which provide more flexibility and better performance. The field statistics functionality can be achieved through existing aggregation endpoints with better results.