Platform Observability

Git-sync relay metrics

For Deployments that use git-sync to deploy Dags, the git-sync relay emits operational metrics through StatsD. These metrics provide visibility into sync operations, repository size, and sync performance for each Deployment.

Git-sync relay metrics are disabled by default and require configuration at both the platform and cluster levels.

In APC 2.0, the strict schema validator doesn’t recognize git-sync relay metrics. You must disable the strict schema check to use this feature. See the following configuration steps.

Prerequisites

  • Astro Private Cloud 2.0 or later with Helm chart version 2.0.0 or later.
  • System Admin access to update the Astro Private Cloud Helm configuration.
  • One or more Deployments configured to use git-sync for Dag deployment.

Enable git-sync relay metrics

1

Disable the strict schema check

Because the strict schema validator in APC 2.0 doesn’t recognize git-sync relay metrics fields, you must disable it. Add the following to your platform-level values.yaml:

1astronomer:
2  strictSchemaCheck:
3    enabled: false
2

Enable metrics at the platform level

In the same platform-level values.yaml, enable git-sync relay metrics globally:

1global:
2  gitSyncRelay:
3    metrics:
4      enabled: true
3

Enable metrics in the cluster configuration

In your cluster configuration, enable git-sync relay metrics:

1helm:
2  gitSyncRelay:
3    metrics:
4      enabled: true
4

Apply the configuration

Run a Helm upgrade to apply the changes:

$helm upgrade -f values.yaml -n <your-namespace> <Release-Name> astronomer/astronomer
5

Verify the metrics

After a successful Helm upgrade, git-sync relay metrics are available through the Deployment’s StatsD exporter. Run the following command to confirm metrics are being emitted:

$curl -s http://localhost:9000/metrics | grep git_sync_relay

Available metrics

The following metrics are available with the git_sync_relay_ prefix. All metrics are emitted per Deployment.

Sync operation metrics

MetricTypeDescription
git_sync_relay_git_sync_countCounterTotal number of git sync operations initiated
git_sync_relay_git_sync_success_countCounterTotal number of successful git sync operations
git_sync_relay_git_sync_duration_secondsGaugeDuration of the last git sync operation in seconds
git_sync_relay_git_sync_interval_secondsGaugeGit sync polling interval in seconds
git_sync_relay_last_sync_timestampGaugeTimestamp of the last successful sync
git_sync_relay_git_sync_last_successful_sync_timestampGaugeTimestamp of the last successful sync operation

Git operation metrics

MetricTypeDescription
git_sync_relay_git_operations_totalCounterTotal git operations (clone, fetch, clean, reset, sync)
git_sync_relay_git_operations_duration_secondsSummaryDuration of individual git operations with quantile distribution (p50, p90, p99)
git_sync_relay_git_sync_clone_durationGaugeDuration of git clone operation in seconds

File and repository metrics

MetricTypeDescription
git_sync_relay_files_updated_totalCounterTotal files updated during sync operations
git_sync_relay_git_sync_file_changed_countGaugeNumber of files changed in the last sync
git_sync_relay_repo_size_bytesGaugeSize of the git repository in bytes
git_sync_relay_git_dir_size_bytesGaugeSize of the .git folder in bytes
git_sync_relay_repo_total_size_bytesGaugeTotal repository size including working tree in bytes
git_sync_relay_working_tree_size_bytesGaugeSize of the working tree in bytes
git_sync_relay_git_sync_repo_sizeGaugeGit repository size in bytes

Initialization metrics

MetricTypeDescription
git_sync_relay_git_sync_init_duration_secondsGaugeApplication initialization duration in seconds
git_sync_relay_git_sync_init_attemptGaugeNumber of initialization attempts
git_sync_relay_git_sync_light_init_duration_secondsGaugeLight initialization duration in seconds
git_sync_relay_git_sync_deferred_init_start_timestampGaugeDeferred initialization start timestamp

State metrics

MetricTypeDescription
git_sync_relay_git_sync_ready_stateGaugeWhether the git-sync relay is ready (1 = ready, 0 = not ready)
git_sync_relay_git_sync_shallow_stateGaugeWhether the repository is in a shallow clone state
git_sync_relay_git_sync_configured_depthGaugeConfigured git clone depth