What Is New in Apache Airflow 1.10
Apache Airflow 1.10 is a major milestone release that spans 15 patch versions (1.10.0 through 1.10.15), delivering a new RBAC-based web UI, sweeping CLI restructuring, extensive GCP ecosystem improvements, Kubernetes executor hardening, and critical behavioral fixes across the scheduler, variables, sensors, and trigger rules. This guide consolidates the most operationally significant changes across the entire 1.10.x line so your upgrade decision is well-informed.
| Category | Highlights |
|---|---|
| New Features |
Flask-AppBuilder RBAC web UI; DAG-level access control; Dask executor; KubernetesPodOperator
pod_template_file support; infinite pool size (-1); new none_failed_or_skipped
trigger rule; session_lifetime_minutes config; DAG discovery safe mode;
ts_nodash_with_tz macro
|
| Improvements |
CLI commands reorganized into subcommand groups (dags, tasks, db, users, celery);
GCP operators now accept optional PROJECT_ID; pool queries optimized to prevent n^2
MySQL performance; parsing_processes config rename for clarity;
pod_mutation_hook upgraded to accept full V1Pod objects;
success/failure callbacks now fire on manual state changes from UI
|
| Bug Fixes |
none_failed trigger rule behavior now matches documentation; empty-string
Airflow Variables now correctly return '' instead of None;
TimeSensor is now timezone-aware; tasks cleared after SkipMixin now remain skipped;
ts_nodash no longer embeds timezone offset
|
| Breaking Changes |
Experimental REST API denies all requests by default; XCom values cannot be added or edited
from the web UI; Kubernetes run_as_user defaults to 50000; LDAP auth now
requires TLS; BaseOperator::render_template signature changed;
get_task_instances signature changed in BaseOperator and DAG classes;
User.superuser is now a database column defaulting to False;
UTC is the new default timezone
|
| Deprecations |
Importing operators/hooks/sensors via the plugin mechanism; old flat CLI commands
(still work but hidden from help); bql parameter on BigQueryOperator;
XCom pickling (JSON is now preferred); session_lifetime_days and
force_log_out_after config keys; DAG processing metrics renamed for 2.0 alignment;
Kerberos support for HDFS hook removed
|
How does the new RBAC web UI change authentication and access control in Airflow 1.10?
Airflow 1.10 replaces the Flask-Admin UI with a new interface built on Flask-AppBuilder (FAB), bringing proper Role-Based Access Control, DAG-level permissions, and support for multiple authentication backends out of the box.
To enable the new UI, set rbac = True in the [webserver] section of
your airflow.cfg, then run airflow db init to create the new
ab_* tables. After that, create an admin user:
airflow users create \
--username admin \
--firstname Admin \
--lastname User \
--role Admin \
--email [email protected]
Watch out for these breaking changes when migrating to the RBAC UI:
- The Airflow home page moves from
/adminto/home. - All model view URLs change -- e.g.
/admin/connectionbecomes/connection/list. - Users stored in the old users table are not migrated automatically. You must recreate accounts.
- Data Profiling features (Ad Hoc Query, Charts, Known Events) are removed for security reasons.
User.superuseris now a database column that defaults toFalse. Any users who previously relied on superuser access must be explicitly granted it.
DAG-level ACL is also introduced in 1.10.2. Each DAG gets two permissions --
can_dag_edit and can_dag_read. Admins assign these per-role
and can use airflow sync-perm to keep permissions in sync after DAG changes.
In practice, teams running multi-tenant Airflow deployments should plan a dedicated migration
window to avoid role gaps during the switchover.
What CLI commands changed in Airflow 1.10 and how do I update my automation scripts?
Starting with Airflow 1.10.14, all CLI commands are reorganized into subcommand groups to align with the Airflow 2.0 interface -- this is the most widespread operational change that will break automation scripts and CI/CD pipelines if not updated.
The old flat commands still function for backward compatibility but are hidden from
--help output. Migrate your scripts before Airflow 2.0 removes them entirely.
Here is a mapping of the most commonly used commands:
| Old Command | New Command |
|---|---|
airflow trigger_dag | airflow dags trigger |
airflow list_dags | airflow dags list |
airflow pause | airflow dags pause |
airflow backfill | airflow dags backfill |
airflow test | airflow tasks test |
airflow run | airflow tasks run |
airflow clear | airflow tasks clear |
airflow initdb | airflow db init |
airflow upgradedb | airflow db upgrade |
airflow create_user | airflow users create |
airflow worker | airflow celery worker |
airflow flower | airflow celery flower |
airflow config | airflow config list |
Most teams should audit shell scripts, Dockerfiles, Kubernetes CronJob manifests, and any
wrapper scripts that invoke Airflow CLI directly. A simple grep -r "airflow trigger_dag"
across your infrastructure repositories will surface the most common hits quickly.
What Kubernetes executor changes in Airflow 1.10 require action before upgrading?
Airflow 1.10 introduces several Kubernetes executor hardening changes that affect pod security, mutation hooks, and operator configuration -- each worth a targeted review before rolling out to a production cluster.
Default UID changed to 50000 (1.10.11)
The run_as_user setting in the [kubernetes] section previously
defaulted to an empty string, which the code interpreted as 0 (root). It now
defaults to 50000. This is a security improvement but can break existing workloads
if your container images do not have the appropriate file permissions for UID 50000. Audit your
worker images before upgrading.
# airflow.cfg -- explicit setting post-1.10.11
[kubernetes]
run_as_user = 50000
pod_mutation_hook now accepts V1Pod (1.10.12)
If you have a custom pod_mutation_hook using the legacy
airflow.contrib.kubernetes.Pod class, migrate to the
kubernetes.client.models.V1Pod object. The old class is deprecated and
gives full Kubernetes API access as a bonus. In practice, any mutation hook touching
pod affinity, tolerations, or resource requests should benefit from this richer interface.
pod_template_file support in KubernetesPodOperator (1.10.12)
Teams that manage complex pod specs inline in DAG code can now offload that configuration to a YAML template file:
from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
task = KubernetesPodOperator(
task_id="my_task",
pod_template_file="/path/to/pod_template.yaml",
...
)
This matters if your team enforces pod specs through GitOps workflows or shares templates across multiple operators -- the pod_template_file parameter reduces DAG verbosity significantly.
XCom editing disabled from the UI (1.10.11)
XCom values can no longer be created or modified through the web interface because pickled payloads posed a remote code execution risk. Any tooling or runbook steps that relied on manual XCom injection via the UI must switch to programmatic methods.
Which trigger rule and scheduler behavior changes in Airflow 1.10 could silently break existing DAGs?
Several behavioral fixes in Airflow 1.10 correct longstanding inconsistencies, but because they alter runtime behavior rather than configuration, they can silently change how existing DAGs execute after an upgrade.
none_failed trigger rule fixed -- and a new rule added (1.10.10)
The none_failed trigger rule was previously skipping downstream tasks when all
parents were skipped, contrary to its documented behavior. The fix aligns the rule with its
documentation: tasks downstream of none_failed now run when all parents have
succeeded or been skipped. If your DAG depended on the old (buggy) behavior, migrate those
tasks to the new none_failed_or_skipped rule to preserve the original flow.
# Before: used none_failed expecting skip-if-all-parents-skipped behavior
my_task = PythonOperator(
task_id="my_task",
trigger_rule="none_failed",
...
)
# After: use none_failed_or_skipped to keep old behavior
my_task = PythonOperator(
task_id="my_task",
trigger_rule="none_failed_or_skipped",
...
)
SkipMixin-cleared tasks now stay skipped (1.10.12)
Previously, clearing a task that was skipped by a BranchPythonOperator, BaseBranchOperator,
or ShortCircuitOperator would cause it to execute on the next scheduler cycle. After 1.10.12
it will be skipped again via the new NotPreviouslySkippedDep dependency.
This is the correct behavior, but watch out for any manual reruns where the intent was to
force a skipped branch to execute -- you will need to use a different approach such as
modifying the branching logic itself.
Empty Airflow Variables now return empty strings (1.10.10)
Setting a Variable to '' and calling Variable.get('key') previously
returned None. It now correctly returns ''. Any DAG code that
checks if value is None to detect "unset" variables may now miss the empty-string
case. Audit variable handling logic after upgrading.
TimeSensor is now timezone-aware (1.10.13)
Before 1.10.13, TimeSensor always compared target_time against UTC regardless
of the DAG's configured timezone. It now uses the DAG's timezone, falling back to
default_timezone from the global config. This is a silent behavioral change --
sensors on DAGs with non-UTC timezones will fire at a different wall-clock time after upgrade.
Success callbacks now fire on manual UI state changes (1.10.10)
When a user marks a task as success through the Airflow UI, on_success_callback
is now invoked. Similarly, on_failure_callback fires when a task is marked failed
manually (1.10.8). This is the expected behavior, but if callbacks have side effects like
sending alerts or updating external systems, expect them to trigger from UI interactions too.
What GCP operator and hook changes in Airflow 1.10 affect existing Google Cloud workflows?
Airflow 1.10 ships extensive improvements to GCP operators and hooks, making PROJECT_ID optional, replacing deprecated client libraries, and consolidating connection IDs -- but several of these come with migration requirements.
PROJECT_ID is now optional on many GCP operators (1.10.3)
GCE, GCF, and Cloud SQL operators no longer require a hardcoded PROJECT_ID.
If omitted, the project ID from the GCP connection is used. This simplifies DAG code and
makes DAGs portable across environments, but the GCP connection itself must have a project ID
configured or an AirflowException will be raised at runtime.
GoogleCloudStorageHook migrated to client library (1.10.4)
The GCS hook now uses the google-cloud-storage client library instead of the
discovery-based googleapiclient.discovery API. Key impacts:
- The
multipartandnum_retriesparameters onGoogleCloudStorageHook.uploadare deprecated -- multipart upload is now automatic for objects over 8 MB. - The
generationparameter ondeleteandinsert_object_aclis deprecated. - If you call
GoogleCloudStorageHook().get_conn().get_bucket(...)directly, the signature changed ingoogle-cloud-storage >= 1.16.
google_cloud_storage_default connection replaced (1.10.0)
The google_cloud_storage_default connection ID has been retired in favor of
google_cloud_default across all GCP operators. Update any hardcoded connection
references in your DAGs and connection configuration.
MySqlToGoogleCloudStorageOperator now exports timestamps as UTC (1.10.4)
TIMESTAMP columns are now exported in UTC by default, which is correct for BigQuery.
If you need to preserve server-local timezone behavior, set ensure_utc=False
explicitly. This change is particularly relevant for pipelines loading time-series data
into BigQuery where timezone correctness affects query results.
Frequently Asked Questions about Apache Airflow 1.10
Do I need to update my automation scripts when upgrading to Airflow 1.10.14 or later?
Yes, if your scripts use old flat CLI commands like airflow trigger_dag or airflow list_dags you should migrate them to the new subcommand structure (airflow dags trigger, airflow dags list, etc.) before moving to Airflow 2.0, where the old commands will be removed entirely. The old commands still work in 1.10.x but are hidden from help output.
What happens to the existing users table when I enable the new RBAC UI in Airflow 1.10?
Users stored in the old Flask-Admin users table are not migrated automatically to the new Flask-AppBuilder tables. You must recreate user accounts using airflow users create after enabling rbac = True and running airflow db init to generate the new ab_ prefixed tables.
Why is my BranchPythonOperator downstream task now executing after I clear it in Airflow 1.10.12?
This is a known behavioral change -- prior to 1.10.12, clearing a task that was skipped by SkipMixin-based operators (BranchPythonOperator, ShortCircuitOperator) would cause it to run. Starting in 1.10.12, the new NotPreviouslySkippedDep dependency ensures cleared tasks that were previously skipped remain skipped, matching the intended workflow logic.
How do I preserve the old none_failed behavior after upgrading to Airflow 1.10.10?
Change the trigger_rule on affected tasks from none_failed to the new none_failed_or_skipped. The old behavior -- where a task would be skipped if all its parents were also skipped -- is now captured by none_failed_or_skipped, while none_failed was corrected to match its documented meaning of running when all parents have succeeded or been skipped.
Does Airflow 1.10 require any changes to how I configure session expiry for the web UI?
Yes, the session_lifetime_days and force_log_out_after configuration options are removed in 1.10.13 and replaced by a single session_lifetime_minutes option. For example, to set a 30-day session lifetime, add session_lifetime_minutes = 43200 to the webserver section of airflow.cfg and remove the old keys.
Is it safe to rely on the Experimental REST API without authentication in Airflow 1.10.11?
No, starting in 1.10.11 the Experimental API denies all requests by default on new installations to prevent unintentional open access. If you have an authentication layer in front of Airflow or are aware of the risks, you can restore the old behavior by setting auth_backend = airflow.api.auth.backend.default in the api section of airflow.cfg, but note that this change does not affect existing installations automatically.