From e9bd5092dfdcb56237e6dd1e3b92a902a11250e9 Mon Sep 17 00:00:00 2001 From: Joao Viana Date: Mon, 27 Apr 2026 12:21:23 +0100 Subject: [PATCH 1/3] docs: add pre-aggregate CLI audit reference Document the lightdash pre-aggregate-audit CLI command in a dedicated page under the pre-aggregates section, covering when to use it, how each output status maps back to YAML edits, and CI integration via --fail-on-miss. Add the page to the navigation, link it from the dashboard audit tip in monitoring.mdx, and list the command in the CLI reference. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs.json | 3 +- references/lightdash-cli.mdx | 28 +++++ references/pre-aggregates/cli-audit.mdx | 141 +++++++++++++++++++++++ references/pre-aggregates/monitoring.mdx | 4 + references/pre-aggregates/overview.mdx | 3 + 5 files changed, 178 insertions(+), 1 deletion(-) create mode 100644 references/pre-aggregates/cli-audit.mdx diff --git a/docs.json b/docs.json index 0a56d6df..c9a12f6a 100644 --- a/docs.json +++ b/docs.json @@ -217,7 +217,8 @@ "pages": [ "references/pre-aggregates/overview", "references/pre-aggregates/getting-started", - "references/pre-aggregates/monitoring" + "references/pre-aggregates/monitoring", + "references/pre-aggregates/cli-audit" ] }, "references/lightdash-cli", diff --git a/references/lightdash-cli.mdx b/references/lightdash-cli.mdx index d218cc72..0e751469 100644 --- a/references/lightdash-cli.mdx +++ b/references/lightdash-cli.mdx @@ -134,6 +134,7 @@ For examples and command-specific options, click through the command in the tabl |[`lightdash diagnostics`](/references/lightdash-cli#lightdash-diagnostics) | Show diagnostic information about the CLI environment | |[`lightdash sql`](/references/lightdash-cli#lightdash-sql) | Run raw SQL query against the warehouse using project credentials | |[`lightdash lint`](/references/lightdash-cli#lightdash-lint) | Validate Lightdash Code files against JSON schemas | +|[`lightdash pre-aggregate-audit`](/references/lightdash-cli#lightdash-pre-aggregate-audit) | Audit pre-aggregate hit/miss coverage for one or every dashboard | --- ### `lightdash login` @@ -956,6 +957,33 @@ Output results in SARIF JSON format (useful for CI/CD integration): lightdash lint --format json ``` +### `lightdash pre-aggregate-audit` + +Reports pre-aggregate hit/miss coverage for one dashboard or every dashboard in the project. Use it to find gaps in your pre-aggregate YAML or to gate CI on dashboard coverage. + +```console +lightdash pre-aggregate-audit [options] +``` + +This command does not support using dbt options. + +**Options:** + +- `--dashboard ` + - Audit a specific dashboard. Required unless `--all` is set. +- `--all` + - Audit every dashboard in the project. +- `--project ` + - Override the active project. Falls back to `LIGHTDASH_PROJECT_UUID`, then to the CLI config. +- `--json` + - Emit machine-readable JSON instead of human-friendly text. +- `--fail-on-miss` + - Exit 1 if any eligible tile misses. Ineligible tiles do not trigger a failure. +- `--verbose` + - Include ineligible tiles (and their reason) in the human output. + +See [Auditing pre-aggregates from the CLI](/references/pre-aggregates/cli-audit) for the full guide, including miss-reason → YAML-fix mappings and CI examples. + --- ## dbt options diff --git a/references/pre-aggregates/cli-audit.mdx b/references/pre-aggregates/cli-audit.mdx new file mode 100644 index 00000000..3f8beae6 --- /dev/null +++ b/references/pre-aggregates/cli-audit.mdx @@ -0,0 +1,141 @@ +--- +title: "Auditing pre-aggregates from the CLI" +sidebarTitle: "CLI audit" +description: "Use lightdash pre-aggregate-audit to inspect coverage, find gaps in your YAML, and gate CI on dashboard hit rates." +--- + + + **Availability:** Pre-aggregates are an [Early Access](/references/workspace/feature-maturity-levels) feature available on **Enterprise plans** only. + + +`lightdash pre-aggregate-audit` reports, for one dashboard or every dashboard in a project, which tiles are served from a pre-aggregate, which ones miss (and why), and which ones are ineligible for pre-aggregates entirely. It calls the same audit data as the in-app **Pre-aggregation audit** menu (see [Dashboard pre-aggregate view](/references/pre-aggregates/monitoring#dashboard-pre-aggregate-view)) but exposes it for scripting, automation, and CI. + +## When to use it + +The CLI audit is the right tool when you want to: + +- Iterate on pre-aggregate YAML and verify that misses you've targeted are now hits, without clicking through dashboards in the UI +- Run a project-wide sweep to find dashboards that aren't benefitting from pre-aggregates yet +- Get machine-readable JSON for custom reporting (for example, exporting coverage to a BI tool or Slack) +- Block merges in CI when a critical dashboard regresses to warehouse queries + +For a quick spot-check on one dashboard, the in-app audit menu is usually faster. + +## Prerequisites + +- Lightdash Enterprise plan +- [Install the Lightdash CLI](/guides/cli/how-to-install-the-lightdash-cli) +- [Authenticate](/guides/cli/cli-authentication) with `lightdash login` + +## Auditing a single dashboard + +Pass the dashboard's UUID or its URL slug: + +```bash +lightdash pre-aggregate-audit --dashboard sales-overview +``` + +The output groups tiles by tab and tags each one with its status: + +```text +Dashboard: Sales Overview (sales-overview) 4 hit 2 miss — 1 ineligible + Tab: Q1 + HIT Total revenue (pre-aggregate: orders_daily_by_status) + HIT Average order value (pre-aggregate: orders_daily_by_status) + MISS New customers (Metric not in pre-aggregate) + MISS Revenue by hour (Granularity too fine) + 1 ineligible tile(s) hidden (pass --verbose to show) +``` + +Each tile is one of: + +- **Hit** — the chart is served from the named pre-aggregate +- **Miss** — the chart is eligible to use pre-aggregates but no pre-aggregate matches. The output prints the miss reason +- **Ineligible** — the tile cannot use pre-aggregates at all (markdown tile, SQL chart, orphaned chart, broken explore). Hidden by default; pass `--verbose` to expand + +## Auditing every dashboard + +```bash +lightdash pre-aggregate-audit --all +``` + +The CLI fetches every dashboard in the project and audits each one. This is the workflow for project-wide health checks or CI gates. + +## Selecting the project + +The CLI resolves the project in this order: + +1. `--project ` flag +2. `LIGHTDASH_PROJECT_UUID` environment variable +3. The project saved by your last `lightdash login` / `lightdash config set-project` + +```bash +lightdash pre-aggregate-audit --all --project +``` + +## JSON output for scripting + +```bash +lightdash pre-aggregate-audit --dashboard sales-overview --json +``` + +Returns the full audit object per dashboard (`{ "dashboards": [...] }` when combined with `--all`). Pipe into `jq` to extract what you care about — for example, every dashboard with at least one miss: + +```bash +lightdash pre-aggregate-audit --all --json \ + | jq '.dashboards[] | select(.summary.missCount > 0) | .dashboardName' +``` + +## Using audit results to improve your YAML + +Each miss reason maps to a specific YAML edit. The miss labels printed by the CLI are the same ones documented in [Why a query misses a pre-aggregate](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate). The most common mappings: + +| Miss output | YAML action | +|---|---| +| `Dimension not in pre-aggregate` | Add the dimension to the pre-aggregate's `dimensions` list | +| `Metric not in pre-aggregate` | Add the metric to the pre-aggregate's `metrics` list | +| `Filter dimension not in pre-aggregate` | Add the filter dimension to `dimensions` — filter-only fields still need to be listed | +| `Pre-aggregate filter not satisfied` | Add or narrow the dashboard filter to match the pre-aggregate, or define a separate pre-aggregate without the static filter | +| `Granularity too fine` | Lower the pre-aggregate's `granularity` (for example, `month` → `day`) | +| `No pre-aggregates defined` | Add a `pre_aggregates` block to the explore's source model. See [Getting started](/references/pre-aggregates/getting-started) | +| `No active materialization` | Trigger a refresh from **Project Settings > Pre-aggregates** or wait for the next scheduled run | +| `Non-additive metric`, `Custom SQL metric`, `Custom dimension detected`, `Custom metric detected`, `SQL Table calculation detected` | Not supported by pre-aggregates. See [supported metric types](/references/pre-aggregates/overview#supported-metric-types) and [current limitations](/references/pre-aggregates/overview#current-limitations) | +| `Bypassed by user` | Intentional cache bypass — no action needed | + +A typical iteration loop: + +1. `lightdash pre-aggregate-audit --dashboard ` to see the current misses +2. Edit your dbt model YAML to address each miss +3. `lightdash deploy` to push the change +4. Trigger a refresh from **Project Settings > Pre-aggregates** if the pre-aggregate definition changed +5. Re-run the audit to confirm the misses are now hits + +## CI integration with `--fail-on-miss` + +`--fail-on-miss` makes the command exit with code 1 if any tile is a miss, so CI can block changes that regress pre-aggregate coverage: + +```yaml +- name: Audit pre-aggregate coverage + run: lightdash pre-aggregate-audit --dashboard executive-summary --fail-on-miss + env: + LIGHTDASH_API_KEY: ${{ secrets.LIGHTDASH_API_KEY }} + LIGHTDASH_URL: ${{ vars.LIGHTDASH_URL }} + LIGHTDASH_PROJECT_UUID: ${{ vars.LIGHTDASH_PROJECT_UUID }} +``` + +Ineligible tiles do not count as misses — only eligible tiles that failed to match contribute to the failure. + + + Gating an entire project with `--all --fail-on-miss` is usually too strict, since most projects have at least one dashboard with unsupported features. Pin the gate to the high-traffic dashboards you care about with `--dashboard --fail-on-miss`, or use `--json` and filter the dashboards you want to enforce. + + +## Flag reference + +| Flag | Description | +|---|---| +| `--dashboard ` | Audit a specific dashboard. Required unless `--all` is set. | +| `--all` | Audit every dashboard in the project. | +| `--project ` | Override the active project. Falls back to `LIGHTDASH_PROJECT_UUID`, then to the CLI config. | +| `--json` | Emit machine-readable JSON instead of the human-friendly text output. | +| `--fail-on-miss` | Exit 1 if any eligible tile misses. Ineligible tiles do not trigger a failure. | +| `--verbose` | Include ineligible tiles (and their reason) in the human output. | diff --git a/references/pre-aggregates/monitoring.mdx b/references/pre-aggregates/monitoring.mdx index 6d2d12c8..04f4d8ad 100644 --- a/references/pre-aggregates/monitoring.mdx +++ b/references/pre-aggregates/monitoring.mdx @@ -67,3 +67,7 @@ You can also monitor and manage pre-aggregates directly from any dashboard. Open 2. **Rebuild pre-aggregates** — Triggers a manual re-materialization for the pre-aggregates used by this dashboard. Available to developers and admins only. Pre-aggregate audit view for a dashboard + + + The same audit data is available from the terminal via [`lightdash pre-aggregate-audit`](/references/pre-aggregates/cli-audit), which is useful for scripting, project-wide sweeps, and CI gates. + diff --git a/references/pre-aggregates/overview.mdx b/references/pre-aggregates/overview.mdx index 72b6b93a..e56324bf 100644 --- a/references/pre-aggregates/overview.mdx +++ b/references/pre-aggregates/overview.mdx @@ -21,6 +21,9 @@ Any query that goes through the Lightdash semantic layer can hit a pre-aggregate Track materialization status, debug query matching, and view hit/miss stats. + + Inspect dashboard coverage from the terminal and gate CI on hit rates. + ## How it works From 44c32056761fac35a9471bb08f2952660520ac71 Mon Sep 17 00:00:00 2001 From: Joao Viana Date: Mon, 27 Apr 2026 12:27:48 +0100 Subject: [PATCH 2/3] docs: highlight agentic-tool use case for pre-aggregate audit The structured miss output (failing field + human reason on every line) is exactly what coding agents like Cursor, Claude Code, and Codex need to iterate on pre-aggregate YAML autonomously, so call that out explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) --- references/pre-aggregates/cli-audit.mdx | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/references/pre-aggregates/cli-audit.mdx b/references/pre-aggregates/cli-audit.mdx index 3f8beae6..0a6a58db 100644 --- a/references/pre-aggregates/cli-audit.mdx +++ b/references/pre-aggregates/cli-audit.mdx @@ -21,6 +21,10 @@ The CLI audit is the right tool when you want to: For a quick spot-check on one dashboard, the in-app audit menu is usually faster. + + **Pairs well with agentic coding tools.** Cursor, Claude Code, Codex, and similar agents do really well with this command — every miss prints both the failing field and the reason, which is exactly the signal an agent needs to edit the right pre-aggregate in your dbt YAML. A typical loop is "run `lightdash pre-aggregate-audit --dashboard `, then update the model YAML to close the misses, then re-run until it's clean." The output is structured enough that the agent can fix coverage on its own without you having to translate misses into YAML edits manually. + + ## Prerequisites - Lightdash Enterprise plan From 2f3a0c5ea61512dd7f374b5a9a55b3cad733fb0e Mon Sep 17 00:00:00 2001 From: Joao Viana Date: Mon, 27 Apr 2026 12:31:56 +0100 Subject: [PATCH 3/3] docs: deduplicate miss-reason table on CLI audit page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The miss-reason → fix table already lives on the monitoring page, so link to it from the CLI page instead of restating the mapping. Keep the iteration loop, which is unique to the CLI workflow. Co-Authored-By: Claude Opus 4.7 (1M context) --- references/pre-aggregates/cli-audit.mdx | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/references/pre-aggregates/cli-audit.mdx b/references/pre-aggregates/cli-audit.mdx index 0a6a58db..61440961 100644 --- a/references/pre-aggregates/cli-audit.mdx +++ b/references/pre-aggregates/cli-audit.mdx @@ -92,24 +92,12 @@ lightdash pre-aggregate-audit --all --json \ ## Using audit results to improve your YAML -Each miss reason maps to a specific YAML edit. The miss labels printed by the CLI are the same ones documented in [Why a query misses a pre-aggregate](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate). The most common mappings: - -| Miss output | YAML action | -|---|---| -| `Dimension not in pre-aggregate` | Add the dimension to the pre-aggregate's `dimensions` list | -| `Metric not in pre-aggregate` | Add the metric to the pre-aggregate's `metrics` list | -| `Filter dimension not in pre-aggregate` | Add the filter dimension to `dimensions` — filter-only fields still need to be listed | -| `Pre-aggregate filter not satisfied` | Add or narrow the dashboard filter to match the pre-aggregate, or define a separate pre-aggregate without the static filter | -| `Granularity too fine` | Lower the pre-aggregate's `granularity` (for example, `month` → `day`) | -| `No pre-aggregates defined` | Add a `pre_aggregates` block to the explore's source model. See [Getting started](/references/pre-aggregates/getting-started) | -| `No active materialization` | Trigger a refresh from **Project Settings > Pre-aggregates** or wait for the next scheduled run | -| `Non-additive metric`, `Custom SQL metric`, `Custom dimension detected`, `Custom metric detected`, `SQL Table calculation detected` | Not supported by pre-aggregates. See [supported metric types](/references/pre-aggregates/overview#supported-metric-types) and [current limitations](/references/pre-aggregates/overview#current-limitations) | -| `Bypassed by user` | Intentional cache bypass — no action needed | +The miss labels printed by the CLI are the same ones documented in [Why a query misses a pre-aggregate](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate) — that table maps each reason to the YAML or configuration change that fixes it. A typical iteration loop: 1. `lightdash pre-aggregate-audit --dashboard ` to see the current misses -2. Edit your dbt model YAML to address each miss +2. Edit your dbt model YAML to address each miss (use the [miss-reason table](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate) to know which field to change) 3. `lightdash deploy` to push the change 4. Trigger a refresh from **Project Settings > Pre-aggregates** if the pre-aggregate definition changed 5. Re-run the audit to confirm the misses are now hits