Guide

Managing your Connections in Apache Airflow


This document covers how to set up various connections in Airflow. Connections defined by this process are used by Hooks in order to encapsulate authentication code and common functions that are used by Operators.

Connections can be maintained in the Airflow Interface (Menu --> Admin --> Connections).

Example Connection Configurations

Microsoft SQL Server

  • Host: localhost
  • Schema: n/a
  • Login: your username
  • Password: blank
  • Port: 1433
  • Extras: n/a

MongoDb

  • Host:
  • Schema: Authentication Database
  • Login:
  • Password:
  • Port: 27017
  • Extras: JSON Object of connection options

MySQL

  • Host: localhost
  • Schema: your database name
  • Login: your username
  • Password: blank
  • Port: 3306
  • Extras: n/a

S3

  • Host: n/a
  • Schema: n/a
  • Login: n/a
  • Password: n/a
  • Port: n/a
  • Extras: {"aws_access_key_id":" ","aws_secret_access_key":" "}

Postgres

  • Host: localhost
  • Schema: your database name
  • Login: your username
  • Password: blank
  • Port: 5432
  • Extras: n/a

A note about the Schema field

The Schema field in Airflow can potentially be a source of confusion as many databases have different meanings for the term. In Airflow a schema refers to the database name to which a connection is being made. For example, for a Postgres connection the name of the database should be entered into the Schema field and the Postgres idea of schemas should be ignored (or put into the Extras field) when defining a connection.


Subscribe to RSS
Ready to build your data workflows with Airflow?

Astronomer is the data engineering platform built by developers for developers. Send data anywhere with automated Apache Airflow workflows, built in minutes...