Airflow State Store
by astronomercbe1141f547bNo license451 starsListed Oct 8, 2026Updated Oct 8, 2026Repository updated today
Persists task and asset state across retries and DAG runs using Airflow 3.3's AIP-103 key/value stores (`task_state_store`, `asset_state_store`) and the crash-safe `ResumableJobMixin`. Use when the user asks about task state store, checkpointing in tasks, persisting state across retries, job IDs surviving worker crashes, watermarks, asset metadata, resumable tasks, crash-safe operators, or "what's new in Airflow 3.3". Also use proactively when reading a DAG that uses Variables or XCom for intra-task coordination state (flag the anti-pattern), or when reviewing any DAG that submits a job to an external system (Databricks, Snowflake, BigQuery, Redshift, Spark, dbt Cloud, EMR, AWS Batch, etc.) and waits for it to finish, whether as one submit-and-wait operator or split into submit + sensor/polling tasks; covers `wait_for_termination`, `deferrable`, `durable`, and hand-rolled polling sensors. State-persistence pieces require Airflow 3.3+; submit+poll architecture guidance applies on any version.
- cbe1141f547bCurrentcommit cbe1141Published Oct 8, 2026
Source and attribution
Source:astronomer/agentsinskills/airflow-state-storeat commitcbe1141
License: No license
Content belongs to its original authors. SourceWeft indexes it from a public repository.
More from astronomer/agents
Warehouse Init
astronomer
Discovers data warehouse schema metadata and generates a shareable .astro/warehouse.md reference file for fast concept-to-table lookups.
Tracing Upstream Lineage
astronomer
Trace upstream data lineage. Use when the user asks where data comes from, what feeds a table, upstream dependencies, data sources, or needs to understand data origins.
Tracing Downstream Lineage
astronomer
Traces downstream data lineage to assess what breaks when a table or DAG changes.
Testing Dags
astronomer
Guides iterative testing, debugging and fixing of Airflow DAGs using af CLI commands.
Setting Up Astro Project
astronomer
Initializes and configures Astro/Airflow projects with the Astro CLI, covering dependencies, connections, and variables.
Profiling Tables
astronomer
Profiles a database table with SQL statistics, cardinality, samples and a data quality assessment.
More in DevOps & Cloud
Amazon Location Service
awslabs
Guides integration of Amazon Location Service APIs for maps, geocoding, routing, places search, geofencing and tracking.
Managing Astro Local Env
astronomer
Manage a local Airflow environment with the Astro CLI: start, stop, restart, logs, API queries and troubleshooting.
Managing Astro Deployments
astronomer
Guides use of the Astro CLI to authenticate, manage workspaces, and create, update, delete, and deploy Astronomer production deployments.
Deploying Airflow
astronomer
Guides deploying Airflow DAGs and projects to production via Astro, Docker Compose, or Kubernetes Helm.
Airflow
astronomer
Operates and troubleshoots Apache Airflow instances through the af CLI, covering DAGs, runs, tasks, logs, config and health.
Monitor Reliability
amplitude
Produces a proactive product reliability health check from Amplitude error, network, and error-click data.