Extract data from your sources, transform it, and load it into Snowflake.

Most teams now run ELT instead which means you load raw data first, transform it inside Snowflake. In practice, real pipelines are usually a hybrid.

Apache Airflow, run on Astronomer.

It's the most widely adopted orchestrator for Snowflake pipelines because it gives you:

• Native Snowflake operators
• Dependency management between tasks
• Built-in data quality checks
• End-to-end observability

Where the transform happens.

ELT is more common today because Snowflake's compute handles transformations at scale.

• ETL — transform before loading into Snowflake
• ELT — load raw, then transform inside Snowflake (usually in SQL)

A reliable pattern looks like this:

• Stage files in S3
• Point an external Snowflake Stage at that location
• Use Airflow's CopyFromExternalStageToSnowflakeOperator to load
• Run dedup and validation before promoting to production tables

Snowflake handles ingestion, but not orchestration.

For dependencies, quality gates, and alerting, most teams add Airflow on top. Snowflake handles storage and compute; Airflow handles coordination.

• Snowpipe handles continuous loading
• Stages handle file staging

With Airflow Dags — each step a task with explicit dependencies.

Airflow 3's asset-aware scheduling triggers downstream Dags automatically when upstream data is ready. More reliable than time-based scheduling, which breaks the moment a step runs long.

A short list that prevents most outages:

• Plan the data flow before writing any Dag code
• Keep SQL in separate files from orchestration logic
• Use driver-level parameter substitution to prevent SQL injection
• Run Airflow's quality check operators before data hits production
• Authenticate Snowflake with key pair or OAuth — not passwords
• Use asset-aware scheduling so dependencies stay explicit