Skip to content

feat: implement execution of benchmark jobs on GitHub trigger #69

@michael-johnston

Description

@michael-johnston

Description

On a GitHub event, associated with a PR, we want benchmarks associated with the package in that PR to be executed on our infrastructure

Motivation

This featured is defined in the benchmark requirements doc (docs/requirements/benchmark.md)

Proposed Solution

  1. User opens PR
  2. At a point in PR reviewers request the benchmark via GitHub command e.g. /run-benchmark
  3. This triggers a GitHub action which creates a benchmark "job" specification, which then is pending
  4. A Github Runner, executing on our infrastructure, on a schedule, retrieves pending benchmark jobs
  5. The runner pulls the relevant PR and constructs using the job spec the ado explore operation and runs it

What needs to be placed in the job specification and what can be inferred at ado job creation time TBD

Alternatives Considered

An alternative which

  1. PR is merged
  2. PR merge GitHub action triggers - on of these is run-benchmark, which creates the task
  3. the GitHub runner on our arch is performing a long-polling on the benchmark tasks, it sees the task and pulls it
    • Tasks are queued so the runner will pull max one at each poll
  4. (As step 5 above)

The event could also be on deployment event, release event etc.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions