Astronomer Software release notes
This document contains release notes for each version of Astronomer Software.
Version 0.34 is the latest long-term support (LTS) version of Astronomer Software. To upgrade to version 0.34, see Upgrade Astronomer. For more information about Software release channels, see Release and lifecycle policies. To read release notes specifically for the Astro CLI, see Astro CLI release notes.
Because Astronomer has separate maintenance life cycles for each minor version of Astronomer Software, the same change can be introduced multiple times across minor versions, resulting in multiple identical release notes. When a new minor version releases, such as version 0.33.0, all changes from previously released versions are included in the new minor version.
If you're upgrading to receive a specific change, ensure the release note for the change appears either:
- Within your target minor version.
- In a patch version that was released before the first release of your target minor version. For example, a change in 0.32.5, which released 12/8/2023, is not guaranteed to appear in the 0.33 series, which released 9/8/2023, unless there is a release note for it in an 0.33 patch. However, all changes in 0.32.1, which released June 12, 2023, are guaranteed to be in the 0.33 series, because 0.32.1 was released before 0.33.0.
0.35.3
Release date: October 4, 2024
Additional improvements
-
Improved the Deployment Rollbacks user experience by alerting you when you enable or disable rollbacks.
-
Added a new tab, AirflowDowngrade, to the Logging UI to show Deployment rollback status logs.
-
Added a new tab, DagServer, to the Logging UI to show DAG server events, without requiring you to log in to the kubernetes cluster.
-
You can now configure K8s tolerations to the containerd DaemonSet. The following example shows a configuration to enable this functionality. Your specific configuration values might be different:
global:
privateCaCertsAddToHost:
addToContainerd: true
containerdTolerations:
- effect: NoExecute
key: special-purpose
operator: Equal
value: workers-spot-vms
enabled: true
Behavior changes
- Removed PSP features.
Bug fixes
- Fixed a bug with fluentd liveness probes where
periodSeconds
was erroneously calledperiodSeconds5
. You can now setperiodSeconds
withperiodSeconds
, as expected. - Resolved an issue which created two entries in the deploy revision history when you executed the
astro deploy
command for Deployments with DAG-only deploys enabled. Now, this type of deploy scenario creates a single entry in Deploy History page. Requires Astro CLI version 1.29 or higher. - Fixed a bug where switching between Image and DAG-only deploys would cause the scheduler to disappear. Now you can switch between deploy types without affecting the scheduler.
- Fixed a bug where users with insufficient permissions could edit service accounts. See See System permissions on Software for more information about user permissions.
- Fixed a bug where HA scheduler pods were all taken down simultaneously when doing a deploy.
- Resolved an issue that sometimes caused out of memory errors during db migration.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-45288
- CVE-2024-2398
- CVE-2024-6345
- CVE-2024-24790
- CVE-2024-26147
- CVE-2024-26308
- CVE-2024-28757
- CVE-2024-34156
- CVE-2024-41110
0.35.2
Release date: August 1, 2024
End of support for Kubernetes versions 1.25 and 1.26
Kubernetes versions 1.25 and 1.26 are not supported in Software version 0.35.2 and future versions. See Kubernetes version support table and policy for a full list of supported Kubernetes versions and the Software versions they correspond to.
Additional improvements
-
The Astronomer Software UI now lists yanked versions of the Astro Runtime. These versions contain potential bugs and are to be used as per user discretion. See Restricted Runtime versions for more information about yanked versions.
-
Added
priorityClass
support forfluentd
andprometheus-node-exporter
daemonsets. Include the following configuration to use this functionality.fluentd:
priorityClassName: <valid-class-name>
prometheus-node-exporter:
priorityClassName: <valid-class-name> -
Added support for
extraEnv
tofluentd
so that you can pass custom variables. You can use the following configuration for this functionality:fluentd:
extraEnv:
RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR: 1 -
Added persistence section to the DAG server config, which can be passed to the Airflow Helm chart.
global:
dagOnlyDeployment:
enabled: true
persistence:
persistentVolumeClaimRetentionPolicy:
whenDeleted: Delete
whenScaled: Retain
Behavior changes
-
When migrating to an Astro Runtime from Astronomer Certified (AC), only AC Versions that are 2.2.5 and greater have equivalent Runtime versions. Previously, you could see equivalent versions of the Astro Runtime you could choose to migrate to in the UI. Now, to see the equivalent version of Astro Runtime for a Deployment running AC in the Deployment Settings page, you need to set the following configuration:
astronomer:
houston:
config:
deployments:
enableListAllRuntimeVersions: true
Bug fixes
- Fixed a bug where the DAG server on Openshift platform failed to replace default security context Helm values.
- Fixed a bug that causes DAG-only deploys to fail when rollbacks were disabled.
- Fixed a bug in DAG-only deploys that were causing unhandled exceptions.
- Fixed a bug that caused the DAG folder PVC to be depleted when a user switches from DAG-only deploys to any other code deploy type.
- Fixed a bug in the deploy revision history page in the Astronomer Software UI that caused the date range query to fail if you set it to less than 90 days for the cleanup policy.
0.35.1
Release date: July 15, 2024
Security fix for DAG-only deploys
Fixed an issue where if you had a DAG-only deploy, it was possible for users to access its dag-server from any other kubernetes namespace. See Upgrade considerations for 0.35 for more details about the security fix.
Security fixes
Resolved the following vulnerabilities:
0.35.0
Release date: July 1, 2024
Use deploy rollbacks to run previous versions of your code
Deploy rollbacks are a new way to ensure that your Deployments can keep running after a broken code deploy or upgrade. When you trigger a rollback to a past deploy, your previous image and/or DAG code are redeployed and start running.
To configure deploy rollbacks, see Deploy rollbacks.
Support for MySQL
You can now configure Astronomer Software to use a MySQL database as the backend for both Houston and your Airflow Deployments. To connect your Astronomer Software installation to a MySQL database, you update the astronomer-bootstrap
secret in your cluster to point to your database:
kubectl create secret generic astronomer-bootstrap \
--from-literal connection="jdbc:mysql://host1:33060/mydb" \
--namespace astronomer
Create Teams without using an IdP
Astronomer Teams can now be created without associating the team to an existing identity provider (IdP) group. Use local Teams to efficiently manage permissions for groups that might not exist in your identity provider.
Addiitonal improvements
- DAG-only deploys are now fully compatible with OpenShift clusters.
- You can now set a
priorityClass
to pods created bytemplates/trust-private-ca-on-all-nodes/containerd-daemonset.yaml
. This allows you to set a high priority class for certain pods to force an evication for important processes.
Bug fixes
- The Astro CLI now shows an error if you attempt to deploy code to a Software installation where
astronomer.houston.config.deployments.registry.protectedCustomRegistry.updateRegistry.host
is not set. - Fixed an issue where Jetstream resources were being created even when
global.nats.jetStream.enabled = false
in the Software platform configuration. - Fixed an issue where the default
astroUnitsEnabled
value invalues.yaml
was not respected when creating Deployments via the Houston API, resulting in Deployment creation failing when the resource strategy is not explicitly specified. - Fixed an issue where Houston could produce errors if the password for a Postgres database included special characters.
- Fixed an issue where pagination on specific Software UI screens did not behave as expected.
- Fixed an issue where the
lastUsedAt
data for a service account was not updated when the service account deployed an Airflow image. - Fixed a bug causing disruption to retrieval of logs for users with External Elasticsearch.
- Fixed a bug where a blank page is displayed when navigating to the user detail of a user who has been invited to a Workspace, but have not yet accepted an invitation.
- Fixed an issue where a pod limit was enforced for a custom resource based Deployment with the K8s executor.
- Fixed a bug where the pgbouncer calculator failed to provision Deployment resources.
- Resolved an issue where switching from DAG deploy to image deploy caused an error state.
- Fixed an issue where the Astro Runtime 11.2.0 wouldn't load the webserver when creating a Deployment.
- Fixed a bug where containerd images failed to load for some cloud providers.
- Fixed an issue where a patch error occured during an Astro Runtime upgrade when Pod Disruption Budget (PDB) was enabled.
- Resolved an issue where custom based resource Deployments were converted to AU-based after a platform upgrade.
- Fixed a bug where the the dag-deploy server pod failed in OpenShift due to an fsGroup error.
- Fixed a bug that sometimes prevented
dagDeployment
server and client resources from being configurable.
Security fixes
Resolved the following vulnerabilities:
- GHSA-m425-mq94-257g
- GHSA-36jr-mh4h-2g58
- GHSA-xpw8-rcwv-8f8p
- GHSA-9763-4f94-gfch
- CVE-2021-33194
- CVE-2021-38561
- CVE-2022-21698
- CVE-2023-1370
- CVE-2023-36665
- CVE-2023-39325
- CVE-2023-45283
- CVE-2023-45288
- CVE-2024-21626
- CVE-2024-25710
- CVE-2024-26147
- CVE-2024-26308
- CVE-2024-28757
- CVE-2024-36361
0.34.3
Release date: July 17, 2024
Security fix for DAG-only deploys
Fixed an issue where if you had a DAG-only deploy, it was possible for users to access its dag-server from any other kubernetes namespace. See Upgrade considerations for 0.34 for more details about the security fix.
Security fixes
Resolved the following vulnerabilities:
0.34.2
Release date: May 31, 2024
Additional improvements
-
DAG-only deploys are now fully compatible with OpenShift clusters.
-
You can now configure resource usage for the DAG-only deploy server in your
values.yaml
file like in the following example:global:
dagOnlyDeployment:
enabled: true
resources:
limits:
cpu: 100m
memory: 400Mi
requests:
cpu: 100m
memory: 400Mi -
You can now configure a priorityClass for all Pods created by the Containerd daemonset. This ensures that Pods generated by the daemonset can be reliably spun up on new nodes whenever they're removed from an existing node. To configure this, add the following lines to your
values.yaml
file and apply the configuration to your platform:global:
privateCaCertsAddToHost:
enabled: true
addToContainerd: true
priorityClassName: <priority-class-name>
Bug fixes
- Fixed an issue where larger DAG deploys had significant performance issues.
- Fixed an issue where Deployments using custom resources would be reverted to using AUs when you upgraded the Astronomer Software platform using Helm.
- Fixed an issue where the default
astroUnitsEnabled
value invalues.yaml
was not respected when creating Deployments via the Houston API, resulting in Deployment creation failing when the resource strategy is not explicitly specified. - Fixed an issue where using a service account to deploy DAGs without the Astro CLI would result in the service account's
lastUsedAt
value not updating as expected. - Fixed an issue where creating Deployments would fail when using the the
airflowStratV2
PGBouncer calculation strategy. - Fixed an issue where upgrading within the 0.34 series could disrupt external Elasticsearch configurations.
- Fixed an issue where tasks on a Deployment could start to fail if the Deployment used a custom resource strategy, used the Kubernetes executor, and attempted to run many tasks at once.
Security fixes
Resolved the following vulnerabilities:
- GHSA-m425-mq94-257g
- CVE-2023-39325
- CVE-2024-21626
- CVE-2024-25062
- GHSA-36jr-mh4h-2g58
- GHSA-xpw8-rcwv-8f8p
- CVE-2022-21698
- CVE-2021-33194
- CVE-2021-38561
- CVE-2024-28757
- CVE-2023-1370
- CVE-2024-25710
- CVE-2023-45283
- CVE-2024-26308
- GHSA-9763-4f94-gfch
0.34.1
Release date: April 8, 2024
Documentation refactor
- To make the documentation more consistent with Helm standards, all instances of
config.yaml
have been replaced withvalues.yaml
.
Additional improvements
-
NGinx now has a dedicated service account with only the required permissions for the service
-
You can now specify an image pull secret for the
privateCaCertsAddToHost.certCopier
image so that it can pull the image from a private registry. -
Added support for Kubernetes 1.29.
-
You can now define
additionalScrapeJobs
in the Prometheus Helm chart so that Prometheus scrapes additional jobs on the Astronomer Software cluster. For example, you could add the following configuration tovalues.yaml
to scrape a static job:astronomer:
prometheus:
additionalScrapeJobs:
- job_name: example-static-job
static_configs:
- targets:
- localhost:9090
Bug fixes
- Fixed an issue where the value for
houston.config.deployments.resourceProvisioningStrategy.astroUnitsEnabled
was not respected when creating Deployments using the Houston API. - Fixed an issue where skipping multiple pages of a list in the Software UI could skip an extra page.
- Removed support for Kubernetes 1.24.
- Fixed an issue where task usage metrics didn't work when using a private certificate authority.
Security fixes
Resolved the following vulnerabilities:
- GHSA-m425-mq94-257g
- CVE-2023-7104
- GHSA-xpw8-rcwv-8f8p
- CVE-2023-1370
- CVE-2024-25062
- GHSA-36jr-mh4h-2g58
- GHSA-9763-4f94-gfch
- CVE-2024-21626
- CVE-2022-2625
- CVE-2024-0985
- CVE-2022-21698
- CVE-2021-33194
- CVE-2023-39325
- CVE-2021-38561
- CVE-2023-52425
0.34.0
Release date: February 12, 2024
Updated permissions for upgrading Deployments to unsupported Astro Runtime versions
This update includes a breaking change to the Astronomer Software Helm chart. The setting enableSystemAdminCanUseAllRuntimes
no longer exists and has been replaced with:
astronomer:
houston:
config:
deployments:
enableListAllRuntimeVersions: true
This new setting changes whether all Admin-level users can view and upgrade to deprecated versions of Astro Runtime from the Software UI.
By default, any user with an Admin-level role (Deployment Admin, Workspace Admin, System Admin) can now upgrade a Deployment to an unsupported version of Astro Runtime using the Astro CLI and the Houston API.
You can additionally set the following value in your values.yaml
file to enable these users to view and upgrade to unsupported Runtime versions through the Software UI:
astronomer:
houston:
config:
deployments:
enableListAllRuntimeVersions: true
Change resource provisioning strategy per Deployment
You can now set a Resource Strategy for each Deployment to fine-tune how Astronomer Software reserves resources for the Deployment within the cluster. The new resource strategies don't use AUs, meaning you can specify the exact CPU/ Memory requests and limits based on the needs of the Deployment. For example, you can configure a Deployment to have significantly more memory than CPU in extra capacity so that memory-intensive tasks are always guaranteed to run. See Customize resource usage for more information.
Deploy only DAGs with astro deploy -—dags
You can now deploy only the DAGs folder of an Astro project to a Deployment. If you only need to deploy DAG code changes, DAG-only deploys are faster and safer than a full image deploy. This also allows you to configure CI/CD pipelines that allow certain team members to only push DAGs, while allowing other team members to push Astro project configuration updates. See Deploy DAGs for more information.
Additional improvements
-
You can now configure a global label that is applied to all Astronomer Software Pods.
-
You can now filter on
release_name
when you make adeployments()
query to the Houston API. -
You can now use containerd-based Astro Runtime images on an Astronomer Software cluster with a self-managed private CA certificate. To configure a self-managed private CA certificate, add the following configuration to your
values.yaml
file and apply the configuration to your cluster:astronomer:
privateCaCertsAddToHost:
enabled: true
hostDirectory: /path/to/docker/certs.d
addToContainerd: false
containerdCertConfigPath: /path/to/containerd/certs.d
containerdConfigToml: ~
containerdnodeAffinitys: [] -
You can now make a
createDeployment
orupsertDeployment
query by specifying a Workspace name or label instead of a Workspace ID. -
You can now disable the
astro-cli
Pod to free up resources on your cluster. This Pod is typically only used in airgapped clusters that can't accesshttps://install.astronomer.io
. To disable the Pod, add the following configuration to yourvalues.yaml
file and apply the change to your cluster:astronomer:
install:
cli-enabled: true -
Astronomer Software now redeploys your Deployment when you switch your executor type.
-
You no longer have to manually define your private registry in the configuration for Vector logging sidecars..
Bug fixes
- Fixed an issue where Astronomer users would occasionally not be associated with their related Azure AD/ Microsoft Entra ID accounts when added to Astronomer using SCIM.
- The Houston API now validates
updateDeployment
queries to ensure that Deployment resource limits and requests are set correctly. - Fixed an issue where the Core Container Status section of the Metrics tab would occasionally show unhealthy containers with a healthy status.
- Fixed an issue where Deployments would occasionally not recreate the correct resources when switching from the Kubernetes executor to the Celery executor.
- Fixed an issue where deploys could fail when using a self-signed certificate signed by a private certificate authority.
- Fixed an issue where Deployments would not have default configuration values as expected when a configuration was missing.
- Fixed an issue where you couldn't search for a user in the Software UI by their user ID.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-46233
- GHSA-36jr-mh4h-2g58
- CVE-2023-1370
- GHSA-xpw8-rcwv-8f8p
- GHSA-fr2g-9hjm-wr23
- CVE-2023-25653
- CVE-2023-36665
- CVE-2023-2976
- CVE-2023-1370
- CVE-2023-40690
- CVE-2023-2253
- CVE-2023-37788
- CVE-2022-21698
- CVE-2021-33914
- CVE-2021-38561
0.33.3
Release date: January 19, 2024
Additional improvements
- Added support for Kubernetes 1.29.
Bug fixes
- Fixed an issue where the Workspace list page in the Software UI would show an error if any Workspace had an empty description.
- Fixed an issue in 0.33.2 where you couldn't migrate a Deployment from Astronomer Certified to Astro Runtime.
- Fixed an issue where a Deployment would incorrectly appear as unhealthy if you scaled down its number of triggerers to zero.
Security fixes
Resolved the following vulnerabilities:
0.33.2
Release date: November 20, 2023
Change to behavior for interacting with deprecated Runtime versions
In 0.33.2, non-System Admin users can now update Deployments using deprecated versions of Astro Runtime. By default, non-System Admin users can now interact with Deployments using deprecated Runtime versions, but only System Admins can create new Deployments with deprecated Runtime versions. As a result of this change, the enableSystemAdminCanUseNonSupportedRuntime
key has been replaced with enableSystemAdminCanUseAllRuntimes
. To set the new flag, add the following configuration to your values.yaml
file:
astronomer:
houston:
config:
deployments:
enableSystemAdminCanUseAllRuntimes: true
Additional improvements
-
You can now configure a global index name prefix to use for both Fluentd and sidecar-based logging, which allows Astronomer Software to retain logs when you migrate between logging solutions. To enable this feature, add the following lines to your
values.yaml
file:global:
logging:
indexNamePrefix: <your-index-prefix> -
Deprecated Kubernetes 1.23
Bug fixes
- Fixed an issue where deploys were failing for Deployments that used older versions of the Airflow Helm chart.
- Fixed an issue where some dashboards in Grafana were not visible if you didn't manually log in to Grafana as an admin.
- Fixed an issue where the Software UI showed Deployments as healthy even if the triggerer Pod was down.
- Fixed an issue where you couldn't search for a user in the Software UI by their user ID.
- Fixed an issue where the
invites
query returned an error instead of an empty array when no users invites were found.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-43787
- GHSA-m425-mq94-257g
- CVE-2023-32002
- CVE-2023-32006
- CVE-2023-32559
- CVE-2023-36665
- CVE-2023-24540
- CVE-2023-24538
- CVE-2023-45133
- CVE-2022-48174
- CVE-2023-39320
- CVE-2023-29402
- CVE-2023-29404
- CVE-2023-29405
0.33.1
Release date: October 13, 2023
Additional improvements
- You can now set
astronomer.auth.microsoft.useExternalProxy: false
in yourvalues.yaml
file to bypass proxy support for Azure logins. - You can now list System-level Service Accounts using the Houston API.
- You can now configure a service account specifically for your image registry using by setting
astronomer.registry.serviceaccount
in yourvalues.yaml
file. - The Kibana logging dashboard now includes a default index.
Bug fixes
- Fixed an issue where if you queried a Deployment name that belonged to two different Deployments in two different Workspaces, the Houston API might retrieve the unintended Deployment.
- Fixed an issue where you could create users with the Houston API when SCIM was enabled and
userManagement.enable
wasfalse
. - Fixed an issue where some dashboards on Grafana didn't load properly.
- Fixed an issue where the Houston API
CreateDeployment
request would return a successful response even though no Deployment was created due to no specified Airflow version. - Fixed an issue where a System Admin user that did not belong to a Team could delete the Team from the Software UI.
- Fixed an issue where syncing an IdP group from Okta failed when SCIM was enabled and a user account was removed only from Astronomer Software.
- Fixed an issue where adding a user through SCIM provisioning that already exists on an Astronomer Software Azure cluster would throw an error instead of updating the existing user.
- Fixed an issue where the user login process would be unresponsive if the Houston API failed to retrieve IdP group information from Azure.
- Fixed an issue where the Software UI would show Deployments as healthy even when triggerer Pods were failing.
- Fixed an issue where a user who was deleted from Astronomer Software would not appear when they were reinvited through a Team.
- Fixed an issue where the System Admin page in the Software UI had entries that didn't fit the page.
- Astronomer Software now throws an error if you attempt to install it with an unsupported version of Kubernetes.
- Fixed an issue where using the Houston API to query for a Deployment that didn't exist returned a non-descriptive error.
- Fixed an issue where
ap-blackbox-exporter
did not respect global network policies.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-38325
- CVE-2023-4863
- CVE-2023-37788
- CVE-2023-37920
- CVE-2023-36665
- CVE-2023-26115
- CVE-2023-29526
- CVE-2022-48174
- CVE-2022-2253
- CVE-2017-11468
- CVE-2023-28840
- CVE-2023-2976
- CVE-2023-1370
- CVE-2021-40690
- CVE-2023-40577
- CVE-2022-41721
- CVE-2023-25653
- CVE-2022-21698
- CVE-2021-33194
- CVE-2021-38561
- CVE-2023-25653
- CVE-2023-25653
- CVE-2023-25653
- CVE-2023-25653
- CVE-2023-25653
0.33.0
Release date: September 8, 2023
Automatic PGBouncer connection scaling
Astronomer Software can now automatically scale the size of PGBouncer connection pools based on your Airflow component counts and Airflow configuration, instead of solely based on total AU. This improves performance, scalability, and utilization of database connections across all Deployments.
This feature is off by default. You can enable it by setting deployments.pgBouncerResourceCalculationStrategy: airflowStratV2
in your values.yaml
file. To revert back to previous behavior, set this key to auStratV1
instead.
Additional improvements
- You can now disable Airflow and platform alerts on the Prometheus alerts dashboard by setting
prometheus.defaultAlerts.airflow.enabled
andprometheus.defaultAlerts.airflow.enabled
tofalse
in your Prometheus Helm chart. If you disable these alerts, you can still add back specific alerts or configure custom alerts usingprometheus.defaultAlerts.additionalAlerts
. See Create custom alerts. - Added support for Kubernetes 1.27.
- The Workspace Deployments page is now paginated in the Astronomer UI.
- The Extra Capacity field in the Astronomer UI now shows up to 6 digits of AU.
- You no longer have to set
elasticsearch.curator.age.timestring
when you configure a custom indexing pattern for Vector logging sidecars. The only required value is nowastronomer.houston.config.deployments.helm.loggingSidecar.indexPattern
. - When you create or update a Deployment and select a Runtime version, the Astronomer UI now shows only the latest supported Astro Runtime patch for each supported Astro Runtime major version.
- You can now set
deployments.canUpsertDeploymentFromUI: false
to prevent all users besides System Admins from updating Deployments and environment variables through the Astronomer UI. - You can now overprovision the
triggerer-log-groomer
component.
Bug fixes
- Fixed an issue where a Deployment using Runtime 8 or earlier with the Celery executor would show as healthy in the Software UI even when workers were unavailable.
- Fixed an issue where Grafana could not start up on an OpenShift cluster.
- Fixed an issue where configurations in
astronomer.houston.config.deployments.components
applied only to Deployments that were created after the configuration was set. - Fixed an issue where a Workspace-level service account would improperly inherit lesser permissions for Deployments it was added to.
- The Astronomer UI now shows an error if you click the Delete button for Teams and you don't have the
system.teams.remove
permission. - Fixed an issue where you couldn't upgrade a Deployment's Airflow version if the Deployment used git-sync deploys and had default resources.
- Fixed an issue where you could get a 500 internal server error from the Airflow UI when switching between pages for a DAG.
- Fixed an issue where you couldn't set
properties.email
using theupsertDeployment
mutation. - Fixed an issue where the Astronomer UI would not show the right error screen when a user without the appropriate permissions viewed service accounts.
Security fixes
Resolved the following vulnerabilities:
0.32.6
Release date: February 16, 2024
Additional improvements
- Added support for Kubernetes 1.29
- You no longer have to manually define your private registry in the configuration for Vector logging sidecars.. Your registry details are now automatically pulled from your core private registry configuration.
- Fixed an issue where container status and usage did not appear in the Metrics tab for Deployments with pre-created namespaces.
Bug fixes
- Fixed an issue where Deployments would not have default configuration values as expected when a configuration was missing.
- Fixed an issue where the Core Container Status section of the Metrics tab would occasionally show unhealthy containers with a healthy status.
- Fixed an issue where a Deployment would incorrectly appear as unhealthy if you scaled down its number of triggerers to zero.
- Fixed an issue where you couldn't specify an image pull secret for the
certCopier
image, meaning that you couldn't pull it from a private registry.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-39325
- CVE-2023-52425
- GHSA-m425-mq94-257g
- CVE-2023-7104
- CVE-2023-34054
- CVE-2023-34062
- GHSA-xpw8-rcwv-8f8p
- CVE-2023-1370
- CVE-2024-0985
- CVE-2024-21626
- CVE-2023-47090
- CVE-2023-46129
- CVE-2023-46233
- GHSA-36jr-mh4h-2g58
- GHSA-9763-4f94-gfch
- CVE-2023-49569
- CVE-2023-49568
- CVE-2023-47108
- CVE-2023-45142
- CVE-2022-2625
- CVE-2023-5869
- CVE-2023-2253
- CVE-2022-21698
- CVE-2021-33194
- CVE-2021-38561
0.32.5
Release date: December 8, 2023
Additional improvements
-
You can now configure a global index name prefix to use for both Fluentd and sidecar-based logging, which allows Astronomer Software to retain logs when you migrate between logging solutions. To enable this feature, add the following lines to your
values.yaml
file:global:
logging:
indexNamePrefix: <your-index-prefix> -
Deprecated Kubernetes 1.23.
-
Astronomer Software now throws an error if you attempt to install it with an unsupported version of Kubernetes.
-
You can now set the following configuration to bypass proxy support when Astronomer Software redirects to Azure login. This might be required in some circumstances when importing identity provider groups as Teams:
astronomer:
houston:
config:
auth:
microsoft:
useExternalProxy: false
Bug fixes
- Fixed an issue where deleting a user from Astronomer Software without deleting the user from an Idp Team could result in the user not being active if they're re-added.
- Fixed an issue that allowed user creation with the Houston API when SCIM was enabled and
userManagement.enable
wasfalse
. - Fixed an issue where the Software UI showed Deployments as healthy even if the triggerer Pod was down.
- Fixed an issue where using the Houston API to query for a Deployment that didn't exist returned a non-descriptive error.
- Fixed an issue where you couldn't update the value for an existing secret environment variable.
- Fixed an issue where you couldn't search for a user in the Software UI by their user ID.
- Fixed an issue where you couldn't access Grafana or Kibana on GKE V2 clusters.
- Fixed an issue where the
invites
query returned an error instead of an empty array when no users invites were found.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-39325
- CVE-2023-5363
- CVE-2023-43787
- CVE-2023-4911
- CVE-2023-5678
- CVE-2023-37788
- CVE-2023-32002
- CVE-2023-32006
- CVE-2023-32559
- CVE-2022-37434
- CVE-2022-48174
- CVE-2023-39320
- CVE-2023-29402
- CVE-2023-29404
- CVE-2023-29405
- CVE-2023-37788
- CVE-2023-40577
- CVE-2023-2253
- CVE-2022-41721
0.32.4
Release date: October 19, 2023
Additional improvements
You can now create a default Kibana index when installing Software.
Bug fixes
- Fixed an issue where
ap-blackbox-exporter
did not respect global network policies.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-4863
- CVE-2023-4911
- CVE-2023-11468
- CVE-2023-28840
- CVE-2023-29491
- CVE-2023-37788
- CVE-2023-37920
- CVE-2023-38039
- CVE-2023-38325
- CVE-2023-38545
- CVE-2023-39417
- CVE-2023-41721
- CVE-2023-44487
- CVE-2023-45133
- CVE-2023-48174
- CVE-2021-33194
- CVE-2021-38561
- CVE-2023-40577
0.32.3
Release date: August 31, 2023
Additional improvements
- You can now disable Airflow and platform alerts on the Prometheus alerts dashboard by setting
Values.defaultAlerts.airflow.enabled
andprometheus.defaultAlerts.platform.enabled
tofalse
in your Prometheus Helm chart. If you disable these alerts, you can still add back specific alerts or configure custom alerts usingprometheus.defaultAlerts.additionalAlerts
. See Create custom alerts. - You no longer have to set
elasticsearch.curator.age.timestring
when you configure a custom indexing pattern for Vector logging sidecars. The only required value is nowglobal.loggingSidecar.indexPattern
. - You can now configure a service account specifically for your image registry using by setting
astronomer.registry.serviceaccount
in yourvalues.yaml
file. - You can now overprovision the
triggerer-log-groomer
component. - You can now set
astronomer.houston.enableHoustonInternalAuthorization
in yourvalues.yaml
file to redirect all authorization requests from the ingress controller to the Houston API internal service endpoint. This can increase performance and decrease network latency. - Upgraded ElasticSearch to 8.x.
- Added support for Kubernetes 1.27.
Bug fixes
- Fixed an issue where Helm changes to statsd Pod resources would apply only to new Deployments.
- Fixed an issue where Grafana could not start up on an OpenShift cluster.
- Fixed an issue where a Deployment using Runtime 8 or earlier with the Celery executor would show as healthy in the Software UI even when workers were unavailable.
- Fixed an issue where a System Admin user that did not belong to a Team could delete the Team from the Software UI.
- Fixed an issue where syncing an IdP group from Okta failed when SCIM was enabled and a user account was removed only from Astronomer Software.
- Fixed an issue where you couldn't upgrade a Deployment's Airflow version if the Deployment used git-sync deploys and had default resources.
- Fixed an issue where the user login process would be unresponsive if the Houston API failed to retrieve IdP group information from Azure.
Security fixes
Resolved the following vulnerabilities:
0.32.2
Release date: June 23, 2023
Bug fixes
- Reverted a change which prevented Workspace-level service accounts from accessing a Deployment when the service account didn't have specific permissions for that Deployment.
- Fixed an issue where some screens of the Airflow UI would produce an HTTP 500 internal server error.
0.32.1
Release date: June 12, 2023
Additional improvements
-
Overprovisioning now also applies to the following components:
- PGBouncer
- Statsd
- Flower
-
You can now configure
astronomer.houston.config.deployments.overProvisioningComponents
to limit the scope of overprovisioning only to specific Airflow components. -
Teams without any users are now automatically deleted when SCIM is disabled.
-
You can now authenticate to an external storage service for archiving task metadata using Workload Identity.
-
You can now set
prometheus.config.scrape_configs.kubernetes_apiservers.tls_config.insecure_skip_verify
in the Prometheus Helm chart. -
You can now set
astronomer.houston.config.deployments.helm.prometheus.certgenerator.extraAnnotations
in yourvalues.yaml
file. -
You can now configure credentials for a registry backend as Kubernetes secrets in your
values.yaml
file. See Configure a registry backend.
Bug fixes
- Fixed an issue where
git-sync-relay
containers wouldn't restart as expected after being terminated. - Fixed an issue where a service account with the Workspace Editor role could update a Deployment when it didn't have any Deployment-level permissions for the Deployment.
- Fixed an issue where data for Disk Usage and Platform Overview did not appear in Grafana.
- System Admins can no longer change a user's system role if the user is imported to Astronomer through an IdP group and
manageSystemPermissionsViaIdpGroups
is set totrue
. - Fixed an issue where you could not create a new Deployment from the Astro UI if you updated its scheduler count using the text-based input field.
- Fixed an issue where container status and usage did not appear in the Metrics tab for Deployments with pre-created namespaces.
- Fixed an issue where resource requests configured from the Software UI could get out of sync with the Houston database.
- Fixed an issue where where updating a Deployment's resource configuration did not persist in the Houston database when that Deployment had overprovisioning enabled.
- Reduced the number of redundant calls that Astronomer Software makes to your identity provider (IdP) when a user logs in.
- Fixed a security vulnerability in logging.
Security fixes
Resolved the following vulnerabilities:
0.32.0
Release date: April 28, 2023
Clean Deployment task metadata
You can now clean task data from your Deployments by exporting it to an external storage service. This workflow reduces the amount of data Airflow stores in your Deployment metadata database by archiving data that you don't need to access on a regular basis. To configure this job, see Clean Deployment task metadata.
Programmatically create and update Deployments with the Houston API
You can now programmatically create or update Deployments using the Houston API upsertDeployment
mutation. Unlike createDeployment
, the upsertDeployment
mutation includes keys for configuring Deployment resources such as environment variables. See Create or update a Deployment with configurations.
Reduce resource requests for Airflow components in development environments
You can reduce the amount of CPU and memory that an Airflow component requests in development environments, allowing you to more efficiently provision resources based on the requirements for your development Deployments. See Underprovision Airflow resources for configuration steps.
New cron job to clean Deployment task data
You can now clean task data from your Deployments by exporting it to an external storage service. This workflow reduces the amount of storage Astronomer Software uses by archiving data that you don't need to access on a regular basis. See Configure a Deployment for configuration steps.
Assign System-level permissions to Teams
You can assign the System Admin, System Editor, and System Viewer permissions to teams by setting the following values in your values.yaml
file:
# Auth configuration.
auth:
openidConnect:
idpGroupsImportEnabled: true
# Optional configuration. Set to assign System-level permissions using Teams.
manageSystemPermissionsViaIdpGroups:
enabled: true
systemAdmin: ["<your-system-admin-groups>"] // Only these groups will be treated as SysAdmin Groups
systemEditor: ["<your-system-editor-groups>"]
systemViewer: ["<your-system-viewer-groups>"]
When coupled with disabling individual user management, this feature allows you to control all user permissions on Astronomer Software exclusively through your identity provider. For more information, see Import IdP groups.
PostgreSQL 15
Astronomer Software version 0.32 upgrades PostgreSQL from 11.18.0-1 to 15. If you use in-cluster PostgreSQL for your workflows, upgrading to Software 0.32 without pinning your PostgreSQL version can impact your workflows. See the Upgrade to Postgres 15 for upgrade considerations and steps.
Additional improvements
- Added support for using git-sync with a private image registry.
- The root user feature introduced in Astronomer Software version 0.31 has been deprecated. System Admins now have the highest level of permissions on the platform.
- Workspaces are now required to have unique names. If you have existing Workspaces with identical names, upon upgrade the duplicate names will be appended with an underscore and a number.
- If you configured git-sync deploys for a Deployment, you can now view error logs emitted from the git-sync Kubernetes Pod in the Software UI.
- You can now configure a custom indexing pattern for Vector logging sidecars by setting both
elasticsearch.curator.age.timestring
andastronomer.houston.config.deployments.helm.loggingSidecar.indexPattern
in yourvalues.yaml
file. - You can now configure custom environment variables for ElasticSearch-based custom logging using the
astronomer.customLogging.extraEnv
value in yourvalues.yaml
file. - The
astronomer.houston.config.deployments.sysAdminScalabilityImprovementsEnabled
key has been replaced withastronomer.houston.config.deployments.performanceOptimizationModeEnabled
for improved performance across additional Software UI views.
Bug fixes
- Fixed an issue where ElasticSearch Curator version 7 and later did not work as expected.
- Fixed an issue where sidecar containers would sometimes not terminate properly after their primary container was terminated.
- Fixed an issue in the Software UI where you could not view Deployment details for a Deployment that included "team" in its name.
- Fixed an issue where a service account with Workspace Editor permissions could update Deployments.
- Fixed an issue where Prometheus was using more memory than expected due to a misconfiguration of statsd. Fixed an issue in the Software UI where a text search returned duplicate entries for a single Deployment name.
- Fixed an issue where authentication tokens were visible in Nginx logs produced by the Software UI.
Security fixes
Resolved the following vulnerabilities:
- CVE-2022-46146
- CVE-2022-27664
- CVE-2021-32149
- CVE-2021-2625
- CVE-2023-0286
- CVE-2023-25881
- CVE-2023-27536
- CVE-2023-27533
- CVE-2023-27534
- CVE-2023-27535
- CVE-2023-0464
- CVE-2023-27561
- CVE-2022-27664
- CVE-2022-41721
- CVE-2022-41723
- CVE-2022-32149
- CVE-2020-25649
- CVE-2020-36518
- CVE-2022-42003
- CVE-2022-42004
- CVE-2022-3171
- CVE-2022-3509
- CVE-2022-3510
- CVE-2022-25857
- CVE-2022-42898
- CVE-2022-3970
0.31.3
Release date: February 24, 2023
Additional improvements
- You can now configure
extraVolumes
andextraVolumeMounts
in the Alertmanager Helm chart, which can be useful for storing secret credentials for services that read your alerts. - You can now use
astronomer.houston.ingress.annotation
in the Astronomer Helm chart to configure custom ingress annotations for Houston. - You can now upgrade the Airflow Helm chart for individual Deployments by running
yarn upgrade-deployments <deployment-id>
from within the Houston Pod.
Bug fixes
- Fixed an issue where you could not set
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER
in a Deployment if you were using an Astronomer Certified image. - Astronomer Software now filters orphaned Deployments and Workspaces owned by users who were removed from an identity provider (IdP) group with SCIM enabled.
- Fixed a security vulnerability where you could query Elasticsearch logs for a Deployment from a different Deployment.
- Fixed an issue where authentication tokens were visible in Nginx logs produced by the Software UI.
- Fixed an issue where deploying an image with the
docker/build-push-action
GitHub action could produce errors in Houston that affected the entire Astronomer Software installation.
Security fixes
Resolved the following vulnerabilities:
0.31.2
Release date: February 2, 2023
Additional improvements
-
You can now configure custom annotations for Houston ingress by setting
astronomer.houston.ingress.annotation
in yourvalues.yaml
file. -
The System Admin Deployments list in the Software UI is now paginated.
-
You can now use the following values in your
values.yaml
file to configure resource allocation for the git-sync relay service:astronomer.gitSyncRelay.gitSyncResources
astronomer.gitSyncRelay.gitDaemonResources
astronomer.gitSyncRelay.securityContext
-
You can now set
timeoutSeconds
forreadinessProbe
andlivenessProbe
in the Prometheus Helm chart. -
Fixed an issue where Deployments with many DAGs could not be successfully upgraded due to a short timeout.
-
Houston API now logs an installation's deployed image versions whenever a GraphQL mutation is completed.
Bug fixes
- To limit Out of Memory (OOM) errors when migrating large DAGs, Deployment database migrations now use the same resources as the Deployment's scheduler.
- Fixed an issue in the Software UI where refreshing pages listing Workspace or Deployment service accounts returned an error.
- Fixed an issue where PgBouncer didn't work if you pulled its image from a private registry.
- When you view a user through a Teams list as a System Admin and return to the list, you now return to the Teams list instead of the System Admin users list.
Security fixes
Resolved the following vulnerabilities:
- CVE-2022-23529
- CVE-2021-44906
- CVE-2022-23540
- CVE-2022-23541
- CVE-2022-3996
- CVE-2022-43551
- CVE-2021-46848
- CVE-2022-21698
- CVE-2021-44716
- CVE-2022-27664
- CVE-2021-43565
- CVE-2021-38561
0.31.1
Release date: December 23, 2022
Additional improvements
- You can now configure
extraFlags
for the Prometheus startup command in the Prometheus Helm chart.
Bug fixes
- Fixed an issue where logging sidecars would occasionally fail to terminate.
Security fixes
Resolved the following vulnerabilities:
- CVE-2021-46848
- CVE-2021-44716
- CVE-2022-27191
- CVE-2022-27664
- CVE-2022-32149
- CVE-2022-37454
- CVE-2022-41717
- CVE-2022-42919
- CVE-2022-45061
- CVE-2022-46146
0.31.0
Release date: December 7, 2022
View and export task usage metrics
You can now view task usage metrics from the Software UI.
Task usage metrics provide an overview of your Airflow task runs and can help you quickly identify Deployments where more tasks are running or failing than expected.
To configure the feature, see Set up task usage metrics.
New root user role
Astronomer Software's role-based access control (RBAC) system now supports a single root user for each installation. The root user has a non-configurable username and autogenerated password stored as a Kubernetes secret in your installation.
See Manage the root user.
Manage Astronomer users through a SCIM integration
Astronomer Software now supports managing users through System for Cross-domain Identity Management (SCIM), which allows you to automatically provision and deprovision users based on templates for access and permissions. See Manage users with SCIM.
Invite users only through Teams
Using the new root user feature, you can now configure Astronomer Software so that users are managed exclusively through Teams. This helps you better integrate with your identity provider (IdP) by ensuring that all users on your platform are authenticated and managed through the IdP. See Disable individual user management.
New default resource limits and requests
Astronomer Software 0.31 includes new default resource limits and requests on the following resources:
- Alertmanager
- Elasticsearch
- NATS
- PostrgeSQL
- STAN
- Nginx
- Grafana
- Blackbox exporter
You might experience OOMKill errors or unexpected behavior after upgrading if you use resources beyond the new default limits. To minimize disruption, view resource usage for these components in Grafana prior to upgrade and compare this usage to the default resource limits in the Astronomer Helm chart.
If your current usage is expected and higher than the default resource limits, update the limits in your values.yaml
file before upgrading to Astronomer Software 0.31.
Additional improvements
- You can now set a custom security context for
es-client
by settingelasticsearch.client.securityContext.capabilities.add={}
in the ElasticSearch Helm chart. - The Deployment users page is now paginated in the Software UI.
- You can now set
astronomer.registry.logLevel
to filter which types of logs appear in your Docker registry. - The default Git-sync interval is now 1 instead of 0.
- You can now configure a Deployment to have 0 triggerer components.
- You can now set
astronomer.houston.config.useAutoCompleteForSensativeFields=false
to disable autocomplete on sensitive fields in the Software UI. - You can now set
astronomer.houston.config.shouldLogUsername=true
to include user email addresses in audit logs for logins through the Houston API. - Git sync-based Deployments now have a dedicated git-sync relay pod, service, and network policy.
Bug fixes
- The Software UI now stores user tokens with
httpOnly
andsecure
flags. - Fixed an issue where the Software UI would occasionally show an incorrect Extra AU number for Deployments.
- Fixed an issue where you could not access Astronomer Software's Docker registry if you had access to more than 100 Deployments.
- Fixed an issue where the Software UI did not show the correct last used dates for service accounts.
- Fixed an issue where NATS would send false Deployment alert emails.
- Fixed an issue where the configuration in
astronomer.houston.updateRuntimeCheck.url
was ignored if not all supported Deployment image versions were present in the destination URL.
Security fixes
Resolved the following vulnerabilities:
0.30.8
Release date: September 15, 2023
Additional improvements
- You can now configure credentials for a registry backend as Kubernetes secrets in your
values.yaml
file. See Configure a registry backend. - You can now disable Airflow and platform alerts on the Prometheus alerts dashboard by setting
prometheus.defaultAlerts.airflow.enabled
andprometheus.defaultAlerts.airflow.enabled
tofalse
in your Prometheus Helm chart. If you disable these alerts, you can still add back specific alerts or configure custom alerts usingprometheus.defaultAlerts.additionalAlerts
. See Create custom alerts. - You no longer have to set
elasticsearch.curator.age.timestring
when you configure a custom indexing pattern for Vector logging sidecars. The only required value is nowastronomer.houston.config.deployments.helm.loggingSidecar.indexPattern
. - You can now configure a service account specifically for your image registry using by setting
astronomer.registry.serviceaccount
in yourvalues.yaml
file. - The Kibana logging dashboard now includes a default index.
- Added support for Kubernetes 1.27.
Bug fixes
- Fixed an issue where if you queried a Deployment name that belonged to two different Deployments in two different Workspaces, the Houston API might retrieve the unintended Deployment.
- Fixed an issue where Helm changes to statsd Pod resources would apply only to new Deployments.
- Fixed an issue where data for Disk Usage and Platform Overview did not appear in Grafana.
- Fixed an issue where you could get a 500 internal server error from the Airflow UI when switching between pages for a DAG.
- Astronomer Software now throws an error if you attempt to install it with an unsupported version of Kubernetes.
- Removed support for Kubernetes 1.22.
- Fixed an issue where using the Houston API to query for a Deployment that didn't exist returned a non-descriptive error.
- Fixed an issue where you couldn't create registry service accounts on Openshift clusters.
Security fixes
Resolved the following vulnerabilities:
- CVE-2022-48174
- CVE-2023-38325
- CVE-2023-36665
- CVE-2022-48174
- CVE-2022-41723
- CVE-2022-29458
- CVE-2023-0464
- CVE-2023-2650
- CVE-2022-29458
- CVE-2023-27561
- CVE-2017-11468
- CVE-2023-2253
- CVE-2023-28840
- CVE-2022-21698
- CVE-2023-28319
- CVE-2023-29491
- CVE-2023-35945
- CVE-2023-37920
- CVE-2022-41721
- CVE-2023-39417
- CVE-2023-37788
- CVE-2023-40577
- CVE-2021-33194
- CVE-2021-38561
0.30.7
Release date: May 26, 2023
Additional improvements
- You can now configure custom environment variables for ElasticSearch-based custom logging using the
astronomer.customLogging.extraEnv
value in yourvalues.yaml
file. - You can now configure
prometheus.config.scrape_configs.kubernetes_apiservers.tls_config.insecure_skip_verify
in the Prometheus Helm chart. - You can now set
astronomer.houston.config.deployments.helm.prometheus.certgenerator.extraAnnotations
in yourvalues.yaml
file. - You can now configure a custom indexing pattern for Vector logging sidecars by setting both
elasticsearch.curator.age.timestring
andastronomer.houston.config.deployments.helm.loggingSidecar.indexPattern
in yourvalues.yaml
file. - The Software UI now shows a warning message for Deployments currently running an Astronomer Certified image. Only System Admins can create Deployments with deprecated Astronomer Certified images by setting
deployments.enableSystemAdminCanCreateDeprecatedAirflows
totrue
. - Grafana now includes an Astronomer Houston Dashboard that you can use to view Houston metrics.
- Improved signalling between primary Kubernetes containers and the logging sidecar so that you no longer have to set
global.loggingSidecar.terminationEndpoint
in yourvalues.yaml
file.
Bug fixes
- Fixed an issue where container status and usage did not appear in the Metrics tab for Deployments with pre-created namespaces.
- Fixed a security vulnerability in logging.
- Fixed an issue where sidecar containers would sometimes not terminate properly after their primary container was terminated.
- Fixed an issue where Prometheus was using more memory than expected due to a misconfiguration of statsd.
- Fixed an issue where a service account with the Workspace Editor role could update a Deployment when it didn't have any Deployment-level permissions for the Deployment.
- Fixed an issue in the Software UI where you could not view Deployment details for a Deployment that included "team" in its name.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-28840
- CVE-2023-27536
- CVE-2023-27533
- CVE-2023-27534
- CVE-2023-27535
- CVE-2023-0464
- CVE-2023-27561
- CVE-2022-27664
- CVE-2022-41721
- CVE-2022-41723
- CVE-2022-32149
- CVE-2020-25649
- CVE-2020-36518
- CVE-2022-42003
- CVE-2022-42004
- CVE-2022-3171
- CVE-2022-3509
- CVE-2022-3510
- CVE-2022-25857
- CVE-2022-42898
0.30.6
Release date: March 2, 2023
Additional improvements
- Support for Kubernetes 1.25 and 1.26.
- You can now configure
extraVolumes
andextraVolumeMounts
in the Alertmanager Helm chart, which can be useful for storing secret credentials for services that read your alerts.
Bug fixes
- Fixed a security vulnerability where you could query Elasticsearch logs for a Deployment from a different Deployment.
- Fixed an issue where deploying an image with the
docker/build-push-action
GitHub action could produce errors in Houston that affected the entire Astronomer Software installation. - Fixed an issue where authentication tokens were visible in Nginx logs produced by the Software UI.
Security fixes
Resolved the following vulnerabilities:
- CVE-2023-24807
- CVE-2023-0286
- CVE-2023-25881
- CVE-2022-27664
- CVE-2022-41721
- CVE-2022-32149
- CVE-2022-23529
- CVE-2021-44906
- CVE-2022-23540
- CVE-2022-23541
0.30.5
Release date: January 11, 2023
Additional improvements
- You can now set
timeoutSeconds
for bothreadinessProbe
andlivenessProbe
in the Prometheus Helm chart. - You can now roll back from Software version 0.30 to 0.28.
Bug fixes
- Fixed an issue where PgBouncer didn't work if you pulled its image from a private registry.
- Fixed an issue where the Software UI would occasionally show an incorrect Extra AU number for Deployments.
- Fixed an issue where users who had access to more than 100 Deployments could not access the Astronomer Software Docker registry.
- Fixed an issue where Deployments with many DAGs could not be successfully upgraded due to a short timeout.
- Fixed an issue where users couldn't log in through Azure Active Directory (AD) if they belonged to more than 100 teams.
- Fixed an issue where service accounts with System Admin permissions could not create Deployments for deprecated Airflow versions.
- Fixed an issue where you could not set
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER
in a Deployment if you were using an Astronomer Certified image. - Fixed an issue in the Software UI where refreshing pages listing Workspace or Deployment service accounts resulted in an error.
- Fixed an issue where logging sidecars would occasionally fail to terminate.
- Fixed an issue where NATS would send false Deployment alert emails.
Security fixes
Resolved the following vulnerabilities:
- CVE-2022-3996
- CVE-2022-43551
- CVE-2021-44716
- CVE-2022-2625
- CVE-2022-37454
- CVE-2022-42919
- CVE-2022-45061
- CVE-2022-43680
- CVE-2017-11468
- CVE-2022-21698
- CVE-2022-27664
- CVE-2022-46146
- CVE-2022-32149
- CVE-2022-27191
- CVE-2022-37601
- CVE-2021-43565
- CVE-2021-38561
0.30.4
Release date: November 3, 2022
Bug fixes
- Fixed an issue where
astronomer.houston.updateRuntimeCheck.url: true
was ignored when searching for new Astronomer Certified and Astro Runtime images.
Security fixes
Resolved the following vulnerabilities:
0.30.3
Release date: October 26, 2022
Additional improvements
- You can now configure custom Alertmanager receivers with their own rules and topics using
customReceiver
in the Alertmanager Helm chart. - You can now limit which Runtime versions are available for new Deployments using
astronomer.minAstroRuntimeVersion
andastronomer.airflowMinimumAstroRuntimeVersion
in yourvalues.yaml
file. - You can now configure a
livenessProbe
andreadinessProbe
specific to Prometheus in the Prometheus Helm chart. - You can now pass extra environment variables to logging sidecars using
global.loggingSidecar.extraEnv
in yourvalues.yaml
file. - You can now define resource requests for logging sidecars using
global.loggingSidecar.resources
in yourvalues.yaml
file. - You can now configure whether introspection APIs are available in GraphQL using
astronomer.apollo.introspection
in yourvalues.yaml
file.
Bug fixes
- Fixed an issue where upgrading Astronomer Software with a custom
houston.deployments.components
value in Helm could make the Software UI unavailable. - Fixed an issue where the Software UI didn't show the correct value for Extra Capacity.
- Fixed an issue where upgrading a Deployment from Airflow 1.10.15 to 2.3 prevented you from configuring Deployment resources in the Software UI.
- Added protections for using Arm-based Runtime images in Software Deployments.
- Fixed an issue where some Deployments failed when pulling secrets from a private Docker registry.
- Fixed an issue where some email alerts for unhealthy Deployments would not send if
namespaceFreeFormEntry: true
was set invalues.yaml
. - Fixed an issue where you could not view Deployment-level service accounts in the Software UI.
- Fixed an issue where token refreshing could break when the token didn't have a properly formatted date.
- Suppressed some extraneous ElasticSearch logs that made parsing logs for relevant information difficult.
Security fixes
Resolved the following vulnerabilities:
0.30.2
Release date: September 22, 2022
Additional improvements
- You can now use the Fluentd Helm chart to set a
securityContext
for Fluentd Pods and containers. - Improved the startup time for the platform NATS server.
- You can now configure external containers in the
astronomer.houston.config
section of the Astronomer Helm chart.
Bug fixes
- Fixed several CVEs as a result of updating images for system components.
0.30.1
Release date: September 12, 2022
Security fixes
Resolved the following vulnerabilities:
0.30.0
Release date: August 29, 2022
A change in 0.30 enabled the trgm
extension for PostgreSQL. If you use Azure Database for PostgreSQL as your database backend, you need to enable the pg_trgm
extension before upgrading to Software 0.30 using either Azure portal or the Azure CLI. See Azure documentation for configuration steps.
If you don't complete this setup before your upgrade, the upgrade will fail.
Improved token refreshing for IdP integrations
The Software UI now refreshes your JSON web token (JWT) based on the validity of your authentication token from your IdP. This means that as long as you stay logged in to your IdP, you no longer have to refresh the Software UI to continue accessing the Software UI, Astro CLI, and Houston API.
Additionally, if you change a user's access to Astronomer from your IdP, their permissions will be automatically updated in Astronomer after their current IdP token expires. If you remove a user completely from Astronomer, they are automatically logged out of the Software UI and CLI after their current IdP token expires.
As part of this change, you can now configure jwt.authDuration
in your Houston Helm configuration. If a user is logged on longer than authDuration
, they will be immediately logged out regardless of the status of their JWT or authentication token.
Additional improvements
- Workspace users are now paginated in the Software UI.
- You can now configure credentials for a private image registry by specifying a secret you create instead of a username and password. The secret is attached to any Pods that need to access the registry.
- You can now specify
authUrlParams
for your identity provider (IdP) invalues.yaml
. - System Editors can no longer manage Teams or users in a Workspace. These permissions are now available only at the System Admin level.
Bug fixes
- Fixed an issue where
updateRuntimeCheck.enabled:false
did not properly stop an Astronomer Software installation from checking for Runtime updates. - Fixed an issue where applying an IAM role to a Deployment would reset the Deployment's Extra Capacity setting back to the default of 0 AU.
- Fixed an issue where System Admins could receive an error when trying to view a Team imported from a different IdP than their current one.
- When a System Admin makes a change to a Team, that change now appears in the UI without needing to refresh the page.
- Configurations for disabling a specific executor type in
values.yaml
are now reflected in the Software UI. - Fixed an issue where Workspace-level service accounts could view Deployment information from Deployments outside of their Workspace.
- Fixed an issue where updating the role of a user in a Team using the Astro CLI would not throw an error as expected.
- Fixed an issue where JSON web tokens persisted after a user logged out if
idpGroupsRefreshEnabled
was set tofalse
. - Users authenticating with Google Direct are no longer automatically logged out of Astronomer Software after 1 hour.
0.29.5
Release date: October 11, 2022
(https://github.com/kubernetes/kubernetes/issues/65106) that occurs when you upgrade Helm charts that include duplicate keys in an env
array. If you have a Helm chart with duplicate keys and upgrade to Astronomer Software 0.29.3+, all key-value pairs with the duplicate key are removed from your environment.
To preserve duplicate keys in your Helm chart, you can either reapply the values after upgrading, or you can use the --reset-values
flag when you run the upgrade script as described in Upgrade Astronomer.
Additional improvements
- Improved the startup time for the platform NATS server.
- You can now configure a
livenessProbe
andreadinessProbe
specific to Prometheus in the Prometheus Helm chart. - You can now configure a specific
securityContext
for Fluentd Pods and containers in the Fluentd Helm chart.
Bug fixes
- Fixed an issue where upgrading Astronomer Software with a custom
houston.deployments.components
value in Helm could break the Software UI. - Fixed an issue where upgrading a Deployment from Airflow 1.10.15 to 2.3 can prevent you from configuring the Deployment's resources in the Software UI.
Security fixes
Resolved the following vulnerabilities:
0.29.4
Release date: September 13, 2022
Additional improvements
- You can now specify
authUrlParams
for your identity provider (IdP) invalues.yaml
- Added error handling for upgrading a Software installation on an unsupported upgrade path
Bug fixes
- Fixed an issue where you could not create Deployments with unsupported Airflow versions when
enableSystemAdminCanCreateDeprecatedAirflows: true
Security fixes
Resolved the following vulnerabilities:
0.29.3
Release date: August 5, 2022
Additional improvements
- Resolved several high and critical CVEs.
Bug fixes
- API requests to query the AUs allocated to Extra Capacity are now returning results for all Deployments in a Workspace. Previously, queries were only returning partial results.
0.29.2
Release date: July 18, 2022
Additional improvements
- You can now configure Vector on logging sidecars to send Airflow task logs to third-party log management systems.
- Resolved several high and critical CVEs.
- You can now assign System Viewer and System Editor permissions to a Team.
- You can now assign System Viewer and System Editor permissions to a user from the Software UI.
Bug fixes
- If you have
customLogging.enabled=true
andloggingSidecar.customConfig=false
in your Helm configuration, logs now appear in the Software UI as expected. - System Admins can no longer update their own role.
- The Software UI no longer counts inactive users in its user count figures.
- Fixed an issue where you could still access a Deployment using a URL after logging out of the Software UI.
- Fixed an issue where you could view Deployment information from a Workspace that was deleted with
astro workspace delete
. - Fixed an issue where you could not open Celery from the Software UI.
- Improved the reliability of upgrading Astronomer Software with 30+ Deployments when
upgradeDeployments=true
.
0.29.1
Release date: June 3, 2022
Bug fixes
- Fixed an issue where you couldn't run Houston API queries for Deployments using
releaseName
andlabel
- Fixed an issue where a user could not log in through Azure AD SSO if the user belonged to a group without a
displayName
0.29.0
Release date: June 1, 2022
Support for Astro Runtime images
You can now use Astro Runtime images in your Software Deployments. Additionally, you can now select Runtime images when setting Image Version for a Deployment in the Software UI.
Functionally, Runtime images are similar to Certified images. They both include:
- Same-day support for Apache Airflow releases
- Extended support lifecycles
- Regularly backported bug and security fixes
Astro Runtime includes additional features which are not available in Astronomer Certified images, including:
- The
astronomer-providers
package, which includes a set of operators that are built and maintained by Astronomer - Airflow UI improvements, such as showing your Deployment's Docker image tag in the footer
- Features that are exclusive to Astro Runtime and coming soon, such as new Airflow components and improvements to the DAG development experience
To upgrade a Deployment to Runtime, follow the steps in Upgrade Airflow, making sure to replace the Astronomer Certified image in your Dockerfile with an Astro Runtime version.
Use a custom container image registry to deploy code
You can now configure a custom container image registry in place of Astronomer's default registry. This option is best suited for mature organizations who require additional control for security and governance reasons. Using a custom registry provides your organization with the opportunity to scan images for CVEs, malicious code, and approved/ unapproved Python and OS-level dependencies prior to deploying code. To configure this feature, see Configure a custom image registry.
Export task logs using logging sidecars
You can now configure logging sidecar containers to collect and export task logs to ElasticSearch. This exporting approach is best suited for organizations that use Astronomer Software in a multi-tenant cluster where security is a concern, as well as for organizations running many small tasks using the Kubernetes executor. To configure this feature, see Export task logs.
Simplified configuration for namespace pools
The process for configuring namespace pools has been simplified. As an alternative to manually creating namespaces, you can now delegate the creation of each namespace, including roles and rolebindings, to Astronomer Software. While this feature is suitable for most use cases, you can still manually create namespaces if you want more fine-grained control over the namespace's resources and permissions. For more information, see Namespace pools.
Additional improvements
- Added support for Kubernetes 1.22
- Deprecated usage of kubed for security and performance improvements
- Redis containers can now run as non-root users
- Added minimum security requirements for user passwords when using local auth
- You can now use Azure DevOps repos in your Git sync configurations
- You can now disable all network policies for Airflow components using the Astronomer Helm chart
- System Admins can now view all Workspaces on their installation by default
- User auth tokens for the Software UI are now stored in httpOnly cookies
- When importing IdP groups as teams, you can now configure a
teamFilterRegex
invalues.yaml
to filter out IdP groups from being imported using regex - Added support for audit logging when a user interacts with the Houston API. This includes actions within the Software UI
Bug fixes
- Fixed an issue in Deployments running Airflow 2.3+ where logs for dynamically mapped tasks did not have a correct
log_id
- Fixed a typo in the
loadBalancerIP
key in the Nginx Helm chart - Fixed an issue where Azure AD connect sync did not work with Astronomer's Teams feature
- Fixed an issue where upgrades would fail if you had changed
networkNSLabels
fromtrue
tofalse
invalues.yaml
v0.28.8
Release date: January 26, 2023
Bug fixes
- Fixed an issue where service accounts with System Admin permissions couldn't create Deployments on deprecated Airflow versions.
- Fixed an issue where you could not upgrade a Deployment from an unsupported version of Astronomer Certified (AC) to another unsupported version of AC.
- Fixed an issue where Deployments with many DAGs could not be successfully upgraded due to a short timeout.
- Fixed an issue in the Software UI where an error message appeared after refreshing pages listing Workspace or Deployment service accounts.
- Fixed an issue where you could not view Deployment-level service accounts in the Software UI.
Security fixes
Resolved the following vulnerabilities:
- CVE-2021-44716
- CVE-2022-27664
- CVE-2022-2625
- CVE-2022-37454
- CVE-2022-42919
- CVE-2022-45061
- CVE-2022-46146
- CVE-2022-27191
- CVE-2022-32149
- CVE-2022-37601
- CVE-2022-43680
v0.28.7
Release date: October 14, 2022
Bug fixes
- Fixed the following vulnerabilities:
v0.28.6
Release date: September 21, 2022
Additional improvements
- You can now specify
authUrlParams
for your identity provider (IdP) invalues.yaml
- Added support for Kubernetes 1.21, 1.22, and 1.23
- Upgraded Prometheus to the LTS release of 2.37.0
Bug fixes
- Fixed several additional CVEs by upgrading images for system components
- Fixed an issue where custom authentication methods did not appear in the Software UI
Security fixes
Resolved the following vulnerabilities:
v0.28.5
Release date: June 23, 2022
Bug fixes
- Fixed several high level CVEs
- User auth tokens for the Software UI are now stored in httpOnly cookies
- Fixed an issue where Grafana dashboards were not accessible
- Fixed an issue where a user could not log in through Azure AD SSO if the user belonged to a group without a
displayName
v0.28.4
Release date: April 8, 2022
Additional Improvements
- Users added to Astronomer Software via an IDP group no longer need to be invited by email in order to join Astronomer.
- Teams now support Azure AD Connect sync for user groups.
- System admins can no longer remove the last user from an active Workspace or Deployment. This ensures that a given Workspace or Deployment can always be deleted by an existing member. Similarly, Workspace Admins can no longer remove a Team if doing so results in a Workspace having zero Admins.
- You can now map your IDP's groups claim to Astronomer's expected claim of
groups
via theastronomer.houston.config.auth.openidConnect.<idp>.claimsMapping
setting invalues.yaml
.
Bug Fixes
- Fixed an issue where deleted Teams did not disappear from the Software UI until you refreshed the page
- Fixed an issue where Teams were still available in the Software UI even when their underlying IDP group had been deleted from the IDP
- Fixed an issue where creating a Deployment with the default resource configuration would result in a Deployment having a Scheduler Count of 1 instead of the stated default of 2
- Fixed an issue where you could not deploy code to a Deployment that shared the release name of a previous Deployment which was hard deleted
- Fixed an issue where you could not create a Deployment with a numeric-only name in a pre-created namespace
v0.28.3
Release date: March 17, 2022
Bug Fixes
- Fixed an issue where airgapped upgrades and installations could fail due to a mismatched Airflow Helm chart between Astronomer components
v0.28.2
Release date: March 14, 2022
Additional Improvements
- System Admins can now update the name and description for any Workspace on their installation.
- You can now specify
global.external_labels
andremote_write
options for Prometheus through the Astronomer Helm chart. - You can now configure
nodeSelector
,tolerations
, andaffinity
in the STAN and NATS Helm charts.
Bug Fixes
- Fixed several CVEs
- Fixed a few issues where some buttons in the Software UI did not link to the appropriate page
- Fixed an issue where you could not install Astronomer Software 0.27 or 0.28 in an airgapped environment
- Fixed an issue where System and Workspace Admins were able to delete users that were part of an IDP team
v0.28.1
Release date: February 22, 2022
Bug fixes
- Fixed an issue where users could not successfully log in through Azure AD
v0.28.0
Release date: February 15, 2022
Import Identity Provider User Groups as Teams
You now can import existing identity provider (IDP) groups into Astronomer Software as Teams, which are groups of Astronomer users that have the same set of permissions to a given Workspace or Deployment. Importing existing IDP groups as Teams enables swift onboarding to Astronomer and better control over multiple user permissions.
Additional Improvements
- Astronomer now supports
prefer
andrequire
SSL modes for connecting to PGBouncer. You can set this SSL mode via theglobal.ssl.mode
value in yourvalues.yaml
file. Note that in v0.28.0, this feature works only with AWS and Azure. - You can now set Grafana environment variables using the
grafana.extraEnvVars
setting in yourvalues.yaml
file. - Added a new Ephemeral Storage Overwrite Gigabytes slider to the Git Sync configuration screen. You can configure this slider to allocate more memory for syncing larger Git repos.
- Added a new Sync Timeout slider to the Git Sync configuration screen. You can configure this slider to set a maximum allowed length of time for syncing a Git repo.
Bug Fixes
- Removed root user permissions for authSidecar
- Added AWS RDS certificates to list of trusted certificates
- Removed support for Kubernetes 1.18
- Fixed some confusing behavior with the Git-Sync SSH Key field in the UI
- Fixed an issue where the Astronomer platform and Airflow could not communicate in environments where inter-namespace communication is disabled
- Fixed an issue where users would frequently get 502 errors when logging in to the Astronomer UI
- Fixed an issue where users would get timeout issues when attempting to log in to an Astronomer installation on OpenShift
0.27.4
Release date: March 8, 2022
Bug Fixes
- Fixed an issue where you could not install Astronomer Software 0.27 or 0.28 in an airgapped environment
- Fixed an issue where new users without permissions to create Workspaces would be directed to the New Workspace page when they first log in
- Fixed several CVEs
0.27.3
Release date: Feb 22, 2022
Limit Workspace Creation to System Admins
To provide more control over resource usage across your organization, you can now limit the ability to create Workspaces only to users with System Admin permissions.
For more information about each role's permissions, including instructions for how to modify a role's permissions, see Platform User Management.
Additional improvements
- Added the
houston.config.deployments.enableSystemAdminCanCreateDeprecatedAirflows
configuration which enables System Admins to create Deployments with deprecated versions of Astronomer Certified
Bug Fixes
- Fixed documentation links in the Software UI
0.27.1
Release date: January 10, 2022
Bug Fixes
- Fixed an issue where users could not create Deployments via an IAM role
0.27.0
Release date: December 21, 2021
Custom OAuth Flows
You can now configure a custom OAuth flow as an alternative to Astronomer's default implicit flow. You can customize Astronomer's existing Okta, Google, and GitHub OAuth flows, or you can import an entirely custom OAuth flow. For more information, read Configure a Custom OAuth Flow.
Deploy DAGs via Git Sync
You can now configure a Git repo to continually push DAGs to an Astronomer Deployment via git-sync. DAGs deployed via git-sync automatically appear in the Airflow UI without requiring additional action or causing downtime. For more information, read Deploy DAGs via Git Sync.
External ElasticSearch Logging
Custom ElasticSearch logging tools are now supported via new values in your values.yaml
file:
# External ES logging
global:
customLogging:
enabled: true
scheme: https
host: ""
port: ""
secret: ""
#secretName: ~
#awsSecretName: ~
#awsIAMRole: ~
#awsServiceAccountAnnotation: ~
CLI Support for Podman
By default, the Astronomer CLI uses Docker to execute a few specific commands. As an alternative, you can now configure the Astronomer CLI to use Podman instead. For more information, read Run the CLI with Podman.
Bug Fixes
- Dropped support for Kubernetes 1.17
- Fixed an issue where redeployments could clobber existing annotations for namespaces
- Fixed an issue where new Deployments could potentially generate invalid usernames for Celery and the metadata DB
- Fixed an issue where scheduler, webserver, and worker logs were not accessible via the Astronomer CLI
- Fixed an issue where where setting extra volumes via
values.yaml
did not work when NFS DAG deploys were enabled.
0.26.7
Release date: March 1, 2022
Additional improvements
- Fixed several CVEs
- Updated documentation links in the UI to point to Software documentation
0.26.6
Release date: January 10, 2022
Bug Fixes
- Fixed an issue where users could not create Deployments via an IAM role
0.26.5
Release date: December 11, 2021
Security fixes
Resolved the following vulnerabilities:
- Remediated CVE-2021-44228 related to Log4J by setting ES_JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true at runtime for all ElasticSearch containers
0.26.4
Release date: November 22, 2021
Support for Airflow 2.2.0
Apache Airflow 2.2.0 is an exciting milestone in the open source project. Most notably, this release introduces custom timetables and deferrable operators.
Custom Timetables
Timetables are a powerful new framework that you can use to create custom schedules using Python. In an effort to provide more flexibility and address known limitations imposed by cron, timetables use an intuitive data_interval
that, for example, allows you to schedule a DAG to run daily on Monday through Friday, but not on the weekend. Timetables can be easily plugged into existing DAGs, which means that it's easy to create your own or use community-developed timetables in your project.
For more information on using timetables, read the Apache Airflow Documentation.
Deferrable Operators
Deferrable operators are a new type of Airflow operator that promises improved performance and lower resource costs. While standard operators and sensors take up a Worker or Scheduler slot even when they are waiting for an external trigger, deferrable operators are designed to suspend themselves and free up that Worker or Scheduler slot while they wait. This is made possible by a new, lightweight Airflow component called the Triggerer.
As part of supporting deferrable operators, you can provision multiple Triggerers on your Astronomer Deployments. By provisioning multiple Triggerers, you can ensure that tasks using Deferrable Operators are run even when one Triggerer goes down. For more information about configuring Triggerers and other resources, see Configure a Deployment.
CLI Verbosity Flag
You can now specify a --verbosity
flag for all Astronomer CLI commands. When you specify this flag with a CLI command, the CLI prints out Logrus logs as the command runs. This is useful for debugging any errors that might result from a CLI command.
The flag prints out different levels of logs depending on the value that you pass it. Each possible value (debug
, info
, warn
, error
, fatal
, and panic
) maps to a different Logrus logging level. For more information about these logging levels, read the Logrus documentation.
Minor Improvements
- You can now create a custom set of cluster-level permissions for the Astronomer Commander service by setting
astronomer.global.clusterRoles: false
in yourvalues.yaml
file and pushing a new RoleBinding to a pre-created Kubernetes namespace. - In the
astronomer.houston.config
section of yourvalues.yaml
file, you can now configure a list ofallowedSystemLevelDomains []
. If you configure this list, only users with emails from domains specified in the list (for example,<company>.com
) can be granted System Admin privileges. - Greatly improved load times for the System Admin page in the UI.
- You can now specify a node port for 3rd party ingress controllers with a service type of
nodePort
. - The naming format of service account pods has been changed from
<release-name>-dags-prod-worker-serviceaccount
torelease_name-dags-prod-airflow-worker
.
Bug Fixes
- Fixed an issue where you could not update an existing Deployment's IAM role via the Astronomer CLI
- Fixed an issue where Deployments would not work on clusters with custom domains
- Fixed error handling when interacting with a Deployment that wasn't fully spun up
- Added a new validation step for Airflow Helm chart values configured in the
astronomer.houston.config.deployments.helm.airflow
section ofvalues.yaml