What Is New in Elasticsearch 2.0
Elasticsearch 2.0 delivers significant enhancements focused on stability, performance, and a more intuitive API. This release tightens query validation, improves aggregation capabilities, and introduces new snapshot/restore features. It's a foundational update that makes the cluster more robust for production workloads.
| Category | Key Changes |
|---|---|
| New Features | Pipeline Aggregations, Doc Values by default, Query/Filters merge |
| Improvements | Enhanced compression, Performance optimizations, Better circuit breakers |
| Breaking Changes | Tighter query validation, Strict duplicate field handling, API simplifications |
| Deprecations | Facets, Index-time boosting, Several plugin APIs |
How did querying change in 2.0?
The old separation between queries and filters has been unified. The filtered query and filter DSL are now deprecated in favor of a bool query with must (for queries) and filter (for filters) clauses. This simplifies your DSL while maintaining the performance benefits of filters for non-scoring clauses.
In practice, this means you need to rewrite your existing filtered queries. The execution is smarter too, as the optimizer can now automatically cache frequently used filters and rearrange bool clauses for better performance.
Before and After Example
// Old way (deprecated)
{
"query": {
"filtered": {
"query": { "match": { "title": "elasticsearch" } },
"filter": { "term": { "status": "published" } }
}
}
}
// New way
{
"query": {
"bool": {
"must": { "match": { "title": "elasticsearch" } },
"filter": { "term": { "status": "published" } }
}
}
}
What are the new aggregation features?
Pipeline aggregations are the headline addition, allowing you to perform operations on the output of other aggregations. This unlocks complex analytics like moving averages, derivatives, and cumulative sums directly in your aggregation pipeline.
Bucket aggregations also got smarter with the new terms aggregation ordering options. You can now order buckets by a specific metric aggregation, like the top 10 tags by their average click-through rate, which is far more useful than just count.
{
"aggs": {
"sales_per_month": {
"date_histogram": { "field": "date", "interval": "month" },
"aggs": {
"sales": { "sum": { "field": "price" } },
"cumulative_sales": {
"cumulative_sum": { "buckets_path": "sales" } // Pipeline agg
}
}
}
}
}
Why is the cluster more stable now?
Circuit breakers were significantly improved to prevent nodes from running out of memory and crashing. New breakers track memory usage for field data, request sizes, and in-flight requests more accurately, giving the cluster a much better chance of gracefully handling heavy load instead of failing catastrophically.
Doc Values are now enabled by default for all fields except analyzed strings. This shifts field value loading from heap-intensive fielddata to off-heap, disk-based operations, drastically reducing garbage collection pressure and memory overhead. This is a huge win for cluster stability with large datasets.
What breaking changes require immediate attention?
Query parsing is now strict. Malformed queries that were previously silently ignored will now throw an exception. You must audit and fix any queries in your application that rely on lenient parsing.
Mapping definitions also became strict. You can no longer have multiple fields with the same name in a JSON object. This was a common source of mapping conflicts and data ambiguity that is now enforced upfront.
- Facets are completely removed; you must migrate all usage to aggregations.
- Index-time boosting is removed; use query-time boosting instead.
- Multiple mapping and API endpoints have been removed or simplified (e.g.,
/_optimizeis now/_forcemerge).
FAQ
My filtered queries broke. What's the quick fix?
Replace the deprecated filtered query with a bool query. Move your main query into the must clause and your filter into the filter clause. This maintains performance while using the modern syntax.
Why is my application getting parsing exceptions after upgrading?
Elasticsearch 2.0 introduced strict query parsing. Queries with invalid JSON structures, unsupported parameters, or syntax errors that were previously ignored will now throw an exception. Check your application logs for the exact error and fix the malformed query.
What should I use instead of facets?
You must migrate all facet functionality to aggregations. The terms aggregation is the direct replacement for most term facets. Check the migration guide for specific examples on converting statistical, date histogram, or range facets to their aggregation counterparts.
Are doc values always good? When should I turn them off?
Doc values are great for most use cases (sorting, aggregations, scripting) as they reduce heap usage. The main exception is for analyzed text fields, which cannot use doc values. For these, you still need fielddata if you plan to use them in aggregations or sorting.
How do I handle the duplicate field mapping restriction?
You can no longer have JSON objects containing two fields with the same name. You must clean your data source to prevent this before indexing. If you encounter a mapping update failure, check your input data for duplicate keys and remove them.