Apache Airflow 3: What's New, Release History & Support Lifecycle

What Is New in Apache Airflow 3

Apache Airflow 3.0 is the most significant release in the project's history. It introduces a service-oriented architecture, a fully rewritten React UI, native DAG versioning, first-class Asset support, event-driven scheduling, and a new Task Execution API that lets tasks run anywhere -- in any language -- with full isolation from the metadata database.

What does the new Task Execution API mean for how Airflow runs tasks?

The Task Execution API (AIP-72) fundamentally changes Airflow's execution model: tasks no longer need direct access to the metadata database or the Airflow process tree -- they communicate with the scheduler via a dedicated API server instead.

In practice, this means worker processes are now fully decoupled from the core Airflow stack. The new Task SDK provides a language-agnostic contract for task execution, which opens the door to tasks written in languages other than Python. This also improves security posture considerably -- a compromised worker can no longer directly query or modify the Airflow DB.

Watch out for the hard removal of direct DB access from task code. If your tasks use internal Airflow models (e.g. DagRun, TaskInstance queries from within a task), those calls will fail on Airflow 3. The recommended migration path is to replace them with REST API calls.

# Airflow 2 -- direct DB access (no longer supported in Airflow 3)
from airflow.models import DagRun
runs = DagRun.find(dag_id="my_dag")  # BREAKS in Airflow 3

# Airflow 3 -- use the REST API instead
import requests
resp = requests.get(
    "http://airflow-api/api/v1/dags/my_dag/dagRuns",
    headers={"Authorization": "Bearer <token>"}
)

How does DAG versioning work in Airflow 3 and why does it matter?

Airflow 3 introduces native DAG versioning (AIP-65, AIP-66): every structural change to a DAG is now tracked and stored, and users can inspect the full version history directly from the new UI.

This matters if you have ever wondered why a historical DAG run shows a task that no longer exists, or struggled to reproduce failures from weeks ago when the DAG code had since changed. With versioning, the serialized DAG structure is associated with each run, giving you a reliable audit trail.

Most teams will also notice that the brand-new React UI (AIP-38, AIP-84), built on FastAPI, delivers noticeably improved Grid, Graph, and Asset views. The legacy Jinja/Flask-AppBuilder UI is gone -- this is the UI going forward.

How has data-aware scheduling changed with Assets in Airflow 3?

What was called "Datasets" in Airflow 2.x has been elevated to a first-class concept called Assets in Airflow 3, with a new @asset decorator syntax and dedicated Asset-based execution paths (AIP-74, AIP-75).

External event scheduling (AIP-82) extends this further -- DAGs can now be triggered by events arriving via a pluggable message bus. The initial release ships with AWS SQS support. This is a significant step toward true event-driven orchestration without polling hacks.

from airflow.decorators import asset

@asset(uri="s3://my-bucket/processed/output.parquet")
def process_data():
    # task body -- Airflow tracks this as a producible Asset
    ...

The scheduler-managed backfill overhaul (AIP-78) is also worth noting: backfills are now a first-class scheduler concern with UI visibility and better diagnostics. In Airflow 2, backfills were a CLI-only affair with limited observability.

What are the most important breaking changes when upgrading to Airflow 3?

The scheduling API received a sweeping cleanup: schedule_interval and timetable DAG constructor arguments are removed entirely -- use schedule exclusively. The new defaults are schedule=None and catchup=False, which is a sensible production default but may silently change behaviour for DAGs that previously relied on implicit catchup.

Several task context variables have been removed, including conf, execution_date, and dag_run.external_trigger. The fail_stop DAG parameter is now fail_fast. The .airflowignore file now uses glob syntax by default rather than regex.

The removed features table captures the most impactful removals:

The Airflow team provides two automated tools to ease the migration:

# Flag and auto-fix DAG-level breaking changes
ruff check --preview --select AIR30 --fix

# Identify removed or renamed config options
airflow config lint

The minimum supported version to upgrade from is Airflow 2.7. Anthropic strongly recommends reaching Airflow 2.10.x first to surface all deprecation warnings before jumping to 3.0.

Does Airflow 3 add anything specifically for ML and AI workloads?

Yes -- Airflow 3 introduces support for non-data-interval DAGs (AIP-83), which removes the uniqueness constraint on execution dates. This is directly targeted at ML use cases such as inference pipelines, hyperparameter tuning runs, and any workload that needs to trigger multiple identical runs without manufacturing artificial data intervals.

The Edge Executor (AIP-69) is an experimental addition that enables edge compute patterns -- running tasks on lightweight remote agents without standing up a full Airflow worker fleet. This is early-stage but worth watching for teams deploying to constrained or distributed environments.

In practice, the combination of the Task SDK's language-agnostic execution model plus non-data-interval scheduling means Airflow 3 is considerably more practical as an ML orchestration platform than its predecessors.