Skip to content

Add Dagster exporter to list of miscellaneous exporters - #3117

Closed
HirofumiTsuda wants to merge 1 commit into
prometheus:mainfrom
HirofumiTsuda:add-dagster-exporter
Closed

HirofumiTsuda wants to merge 1 commit into
prometheus:mainfrom
HirofumiTsuda:add-dagster-exporter

Conversation

@HirofumiTsuda

Copy link
Copy Markdown

Re-submission of #3051, which was closed with the feedback that the exporter
had "limited community" and "seemingly low adoption" at the time, with an
invitation to try again later. Adoption has grown since then:

  • 11 stars, 1 fork (the repo was ~1 week old with no real track record when
    Add Dagster exporter to list of miscellaneous exporters #3051 was filed)
  • Actively maintained: Grafana dashboard, Docker image, and Helm chart
    included; a combined traces+metrics demo (via docker-compose) with a
    sibling OpenTelemetry tracing project
  • ~100 unique visitors and ~100 unique cloners over the last two weeks alone

Adds the Dagster exporter
to the Miscellaneous section, in alphabetical order (between c-lightning
exporter and DHCPD leases exporter).

It polls Dagster's GraphQL API directly and exposes run-state metrics (active
runs, completed runs, latest run status) as Prometheus metrics, avoiding the
blind spots of Dagster's official Pushgateway-based integration (e.g.
crashed/queued runs that never get a chance to push).

Sample output:

dagster_active_runs{job_name="heavy_job",location="dev-dagster-workspace",status="queued"} 0
dagster_active_runs{job_name="heavy_job",location="dev-dagster-workspace",status="started"} 1
dagster_active_runs{job_name="heavy_job",location="dev-dagster-workspace",status="starting"} 0

dagster_completed_runs_total{job_name="heavy_job",location="dev-dagster-workspace",status="failure"} 0
dagster_completed_runs_total{job_name="heavy_job",location="dev-dagster-workspace",status="success"} 12
dagster_completed_runs_total{job_name="failing_job",location="dev-dagster-workspace",status="failure"} 3

dagster_last_run_info{job_name="heavy_job",location="dev-dagster-workspace",status="success"} 1
dagster_last_run_info{job_name="failing_job",location="dev-dagster-workspace",status="failure"} 1

License: MIT

Signed-off-by: HirofumiTsuda <46987671+HirofumiTsuda@users.noreply.github.com>
@HirofumiTsuda

Copy link
Copy Markdown
Author

Closing in favor of a fresh PR from a new branch (same content) to avoid any confusion with the previously-closed #3051.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant