Blog archive

How to Configure Apache Airflow SSO with Keycloak

Configure Apache Airflow single sign-on with Keycloak to centralize user authentication and control access through an identity provider.

Published · Republished on Medium

KeycloakKubernetesAirflowSecurity

Screenshot from How to Implement SSO for Apache Airflow Using Keycloak for Secure Authentication

Airflow authentication changed substantially in Airflow 3. A configuration copied from an Airflow 2 webserver can use the wrong component, callback path, or API-authentication model when applied to Airflow 3.

This guide focuses on Airflow 3 with the official Keycloak auth-manager provider. At the time of this revision, the provider documentation identifies the Keycloak auth manager as alpha/experimental. That makes it suitable for controlled evaluation, but production adoption needs compatibility tests and a rollback plan. Teams that require the mature Flask AppBuilder authorization model should evaluate the maintained FAB auth-manager provider instead.

Choose one auth-manager path

Airflow can use only one auth manager at a time:

  • Keycloak auth manager delegates authentication and authorization to Keycloak. The provider requires Airflow 3 or newer and is currently experimental.
  • FAB auth manager supports OAuth2 SSO and remains maintained, but uses different configuration and role-mapping behavior.

Do not combine their settings in one deployment. The old Airflow 2 callback path /oauth-authorized/keycloak also differs from the Airflow 3 FAB example path /auth/oauth-authorized/keycloak.

The remaining steps use the Keycloak auth manager.

Version and deployment prerequisites

Before changing authentication:

  1. Record the current Airflow, Helm chart, and provider versions.
  2. Confirm the Airflow API server is reachable through its final HTTPS URL.
  3. Keep a tested administrative recovery path.
  4. Export or document existing users, roles, and permissions.
  5. Test the change in a non-production environment.

Build a custom Airflow image with a pinned provider version rather than installing Python packages every time a Pod starts:

text

apache-airflow>=3.0.0
apache-airflow-providers-keycloak==0.8.2

The provider version above matches the official documentation reviewed for this revision. Before deploying, check for a newer security or compatibility release, pin the selected version in the image, and run the same login and authorization tests against it.

Configure the Keycloak client

Create or select a dedicated realm, then create a confidential OpenID Connect client for Airflow.

Use the public Airflow URL consistently. For https://airflow.example.com, the provider documentation defines these client access settings:

SettingValue
Root URLhttps://airflow.example.com
Home URLhttps://airflow.example.com
Valid redirect URIshttps://airflow.example.com/*
Valid post-logout redirect URIshttps://airflow.example.com/*
Web originshttps://airflow.example.com

Enable client authentication and the standard authorization-code flow. Keep the implicit flow disabled. Do not use * for Web Origins. If a narrower set of redirect paths is supported by the provider version you test, prefer it over a wildcard.

Keycloak is not only the login screen for this auth manager: Airflow delegates authorization decisions to it. Define users, groups, roles, permissions, and multi-team resources deliberately. A successful login does not prove that DAG-level access is correct.

Configure Airflow 3

Enable the auth manager and set the public API-server URL:

ini

[core]
auth_manager = airflow.providers.keycloak.auth_manager.keycloak_auth_manager.KeycloakAuthManager

[api]
base_url = https://airflow.example.com

[keycloak_auth_manager]
client_id = airflow
realm = platform
server_url = https://keycloak.example.com

The client secret should not be committed to airflow.cfg or Helm values. Airflow supports secret-backed configuration variants. For example, expose the client secret through the deployment's secret backend or a mounted command and set the corresponding _SECRET or _CMD configuration variable supported by your environment.

The equivalent non-secret environment variables are:

text

AIRFLOW__CORE__AUTH_MANAGER=airflow.providers.keycloak.auth_manager.keycloak_auth_manager.KeycloakAuthManager
AIRFLOW__API__BASE_URL=https://airflow.example.com
AIRFLOW__KEYCLOAK_AUTH_MANAGER__CLIENT_ID=airflow
AIRFLOW__KEYCLOAK_AUTH_MANAGER__REALM=platform
AIRFLOW__KEYCLOAK_AUTH_MANAGER__SERVER_URL=https://keycloak.example.com

The base_url is required for correct absolute redirect and logout URLs. The server_url is the URL the Airflow API server uses to contact Keycloak; it may be an internal address only if TLS trust, issuer behavior, and network policy remain correct for that topology.

Kubernetes secret boundary

Store the Keycloak client secret in your existing secret-management path. A plain Kubernetes Secret is only a transport mechanism unless encryption at rest, RBAC, rotation, and audit controls are also configured.

The Pod should receive the secret without embedding it in the public chart values:

yaml

env:
  - name: AIRFLOW__KEYCLOAK_AUTH_MANAGER__CLIENT_SECRET
    valueFrom:
      secretKeyRef:
        name: airflow-keycloak
        key: client-secret

Prefer the Airflow _SECRET or _CMD variant when the deployment's secret backend supports it. Ensure every API-server replica uses the same authentication configuration and session-signing material.

Network and TLS requirements

Allow only the required flows:

text

User browser -> Airflow HTTPS endpoint
User browser -> Keycloak HTTPS endpoint
Airflow API server -> Keycloak HTTPS endpoint

Validate the certificate chain and hostname from inside the Airflow API-server Pod. Do not make verify=False or insecure TLS a permanent workaround. If Keycloak is behind a reverse proxy, configure Keycloak hostname and proxy headers correctly so its issuer and generated URLs match the public identity-provider URL.

Rollout sequence

  1. Deploy the pinned provider image without switching the auth manager.
  2. Verify the image starts and the provider imports successfully.
  3. Configure the Keycloak realm, client, roles, groups, and permissions.
  4. Apply the Airflow auth-manager configuration in a test environment.
  5. Test login, logout, expired sessions, denied access, and role changes.
  6. Roll out to production during a controlled change window.
  7. Retain the previous image and configuration for rollback.

Changing auth managers affects every user and permission mapping. Communicate the cutover and verify that at least two authorized administrators can log in before closing the change.

Validation checklist

Test outcomes, not just redirects:

  • an authorized user can log in and see only permitted DAGs
  • an unassigned user is denied by default
  • an operator cannot perform administrator actions
  • removal from a Keycloak role takes effect as expected
  • logout ends both the Airflow and identity-provider session as designed
  • an invalid redirect URI is rejected
  • Airflow handles a temporary Keycloak outage predictably
  • API authentication follows the Airflow 3 token model for the selected auth manager
  • audit logs identify login, denial, and administrative changes

For multi-team environments, add a test for a DAG whose authorization resource has not yet been created in Keycloak. The provider documents deny behavior for that case; automation should keep Keycloak resources synchronized with team and DAG onboarding.

Troubleshooting

Redirect URI mismatch

Compare the exact URI in the Keycloak event with the configured public Airflow URL. Check scheme, hostname, path, and trailing slash. Do not fix a mismatch by allowing every origin.

Login succeeds but access is denied

Authentication and authorization are separate. Inspect the user's Keycloak roles/groups and the Airflow resources or permissions assigned to them.

Airflow cannot reach Keycloak

From the API-server Pod, verify DNS, network policy, proxy routing, certificate trust, and the Keycloak discovery endpoint. A browser reaching Keycloak does not prove the Pod can reach it.

Switching from Airflow 2

Do not reuse an Airflow 2 webserver_config.py blindly. Inventory the previous FAB roles and users, choose the Airflow 3 auth manager explicitly, and test the new callback and public-API authentication behavior before migration.

References