What Is New in Elasticsearch 2.1
Elasticsearch 2.1 brings significant enhancements to performance, query functionality, and cluster management. This release focuses on making aggregations faster, queries more powerful, and the overall system more robust for production use.
| Category | Key Updates |
|---|---|
| New Features | Pipeline aggregations, Significant Terms aggregation, Doc Values by default |
| Improvements | Query and filter merge, Faster aggregations, Enhanced recovery throttling |
| Bug Fixes | Multiple issues resolved in indexing, search, and node discovery |
| Deprecations | Preparation for removal of deprecated features in future versions |
How did query and filter execution change?
Queries and filters have been merged into a single concept. The separate filter execution path was removed, meaning all clauses in the bool query are now executed in query context.
This simplifies the API but means you need to be more explicit about caching. You now use the bool query's filter clause for filtering, which is cacheable, instead of the old top-level filter objects.
In practice, this change makes query execution more consistent. You'll want to audit your existing queries to ensure they use the new bool structure for optimal performance.
What are the new aggregation capabilities?
Pipeline aggregations are the headline feature, allowing you to perform operations on the output of other aggregations. This enables complex data analysis directly within your Elasticsearch queries.
The Significant Terms aggregation identifies unusually frequent terms in a subset of your data compared to the whole dataset. It's useful for finding anomalies or correlations that stand out from background noise.
These new aggregations reduce the need for post-processing data outside of Elasticsearch. You can now build more sophisticated analytics pipelines without extra application logic.
How is cluster recovery improved?
Recovery throttling now dynamically adjusts based on ongoing indexing activity. The system better manages resource contention between recovery processes and active indexing operations.
This means large segment files won't monopolize resources during recovery. The cluster remains more responsive to indexing and search requests while rebuilding shards.
For operations teams, this translates to more predictable performance during node failures or cluster expansions. You'll see less impact on latency when bringing new nodes online.
Why are Doc Values now default?
Doc Values are enabled by default for most field types because they provide better performance for aggregations, sorting, and scripting. They store data in a column-oriented format that's more efficient for these operations.
The change reduces heap usage since fielddata is no longer the default for non-text fields. This helps prevent out-of-memory errors during large aggregation operations.
You might need to check your field mappings if you explicitly disabled Doc Values. The performance benefits are usually worth the slight increase in disk usage.
FAQ
Do I need to rewrite all my queries that use filters?
Yes, you should migrate them to use the filter clause within a bool query. The old top-level filter syntax is no longer valid in 2.1.
What happens to my existing fielddata settings?
Fields that had doc_values: false will continue to use fielddata, but new string fields will use Doc Values by default. Review your mappings to avoid unexpected heap usage.
Are there any breaking changes in the Java API?
Yes, the filter and query merge affects the Java API. Methods related to separate filters have been deprecated in favor of the new query structure.
How do pipeline aggregations improve my analytics?
They let you compute derivatives, cumulative sums, and moving averages on aggregation results without external processing, enabling more complex time series analysis.
Will my cluster perform better during node recovery?
Yes, the improved recovery throttling should minimize impact on active operations, though recovery might take slightly longer in some cases due to more conservative resource allocation.