diff --git a/docs.json b/docs.json index 0a56d6df..c9a12f6a 100644 --- a/docs.json +++ b/docs.json @@ -217,7 +217,8 @@ "pages": [ "references/pre-aggregates/overview", "references/pre-aggregates/getting-started", - "references/pre-aggregates/monitoring" + "references/pre-aggregates/monitoring", + "references/pre-aggregates/cli-audit" ] }, "references/lightdash-cli", diff --git a/references/lightdash-cli.mdx b/references/lightdash-cli.mdx index d218cc72..0e751469 100644 --- a/references/lightdash-cli.mdx +++ b/references/lightdash-cli.mdx @@ -134,6 +134,7 @@ For examples and command-specific options, click through the command in the tabl |[`lightdash diagnostics`](/references/lightdash-cli#lightdash-diagnostics) | Show diagnostic information about the CLI environment | |[`lightdash sql`](/references/lightdash-cli#lightdash-sql) | Run raw SQL query against the warehouse using project credentials | |[`lightdash lint`](/references/lightdash-cli#lightdash-lint) | Validate Lightdash Code files against JSON schemas | +|[`lightdash pre-aggregate-audit`](/references/lightdash-cli#lightdash-pre-aggregate-audit) | Audit pre-aggregate hit/miss coverage for one or every dashboard | --- ### `lightdash login` @@ -956,6 +957,33 @@ Output results in SARIF JSON format (useful for CI/CD integration): lightdash lint --format json ``` +### `lightdash pre-aggregate-audit` + +Reports pre-aggregate hit/miss coverage for one dashboard or every dashboard in the project. Use it to find gaps in your pre-aggregate YAML or to gate CI on dashboard coverage. + +```console +lightdash pre-aggregate-audit [options] +``` + +This command does not support using dbt options. + +**Options:** + +- `--dashboard ` + - Audit a specific dashboard. Required unless `--all` is set. +- `--all` + - Audit every dashboard in the project. +- `--project ` + - Override the active project. Falls back to `LIGHTDASH_PROJECT_UUID`, then to the CLI config. +- `--json` + - Emit machine-readable JSON instead of human-friendly text. +- `--fail-on-miss` + - Exit 1 if any eligible tile misses. Ineligible tiles do not trigger a failure. +- `--verbose` + - Include ineligible tiles (and their reason) in the human output. + +See [Auditing pre-aggregates from the CLI](/references/pre-aggregates/cli-audit) for the full guide, including miss-reason → YAML-fix mappings and CI examples. + --- ## dbt options diff --git a/references/pre-aggregates/cli-audit.mdx b/references/pre-aggregates/cli-audit.mdx new file mode 100644 index 00000000..61440961 --- /dev/null +++ b/references/pre-aggregates/cli-audit.mdx @@ -0,0 +1,133 @@ +--- +title: "Auditing pre-aggregates from the CLI" +sidebarTitle: "CLI audit" +description: "Use lightdash pre-aggregate-audit to inspect coverage, find gaps in your YAML, and gate CI on dashboard hit rates." +--- + + + **Availability:** Pre-aggregates are an [Early Access](/references/workspace/feature-maturity-levels) feature available on **Enterprise plans** only. + + +`lightdash pre-aggregate-audit` reports, for one dashboard or every dashboard in a project, which tiles are served from a pre-aggregate, which ones miss (and why), and which ones are ineligible for pre-aggregates entirely. It calls the same audit data as the in-app **Pre-aggregation audit** menu (see [Dashboard pre-aggregate view](/references/pre-aggregates/monitoring#dashboard-pre-aggregate-view)) but exposes it for scripting, automation, and CI. + +## When to use it + +The CLI audit is the right tool when you want to: + +- Iterate on pre-aggregate YAML and verify that misses you've targeted are now hits, without clicking through dashboards in the UI +- Run a project-wide sweep to find dashboards that aren't benefitting from pre-aggregates yet +- Get machine-readable JSON for custom reporting (for example, exporting coverage to a BI tool or Slack) +- Block merges in CI when a critical dashboard regresses to warehouse queries + +For a quick spot-check on one dashboard, the in-app audit menu is usually faster. + + + **Pairs well with agentic coding tools.** Cursor, Claude Code, Codex, and similar agents do really well with this command — every miss prints both the failing field and the reason, which is exactly the signal an agent needs to edit the right pre-aggregate in your dbt YAML. A typical loop is "run `lightdash pre-aggregate-audit --dashboard `, then update the model YAML to close the misses, then re-run until it's clean." The output is structured enough that the agent can fix coverage on its own without you having to translate misses into YAML edits manually. + + +## Prerequisites + +- Lightdash Enterprise plan +- [Install the Lightdash CLI](/guides/cli/how-to-install-the-lightdash-cli) +- [Authenticate](/guides/cli/cli-authentication) with `lightdash login` + +## Auditing a single dashboard + +Pass the dashboard's UUID or its URL slug: + +```bash +lightdash pre-aggregate-audit --dashboard sales-overview +``` + +The output groups tiles by tab and tags each one with its status: + +```text +Dashboard: Sales Overview (sales-overview) 4 hit 2 miss — 1 ineligible + Tab: Q1 + HIT Total revenue (pre-aggregate: orders_daily_by_status) + HIT Average order value (pre-aggregate: orders_daily_by_status) + MISS New customers (Metric not in pre-aggregate) + MISS Revenue by hour (Granularity too fine) + 1 ineligible tile(s) hidden (pass --verbose to show) +``` + +Each tile is one of: + +- **Hit** — the chart is served from the named pre-aggregate +- **Miss** — the chart is eligible to use pre-aggregates but no pre-aggregate matches. The output prints the miss reason +- **Ineligible** — the tile cannot use pre-aggregates at all (markdown tile, SQL chart, orphaned chart, broken explore). Hidden by default; pass `--verbose` to expand + +## Auditing every dashboard + +```bash +lightdash pre-aggregate-audit --all +``` + +The CLI fetches every dashboard in the project and audits each one. This is the workflow for project-wide health checks or CI gates. + +## Selecting the project + +The CLI resolves the project in this order: + +1. `--project ` flag +2. `LIGHTDASH_PROJECT_UUID` environment variable +3. The project saved by your last `lightdash login` / `lightdash config set-project` + +```bash +lightdash pre-aggregate-audit --all --project +``` + +## JSON output for scripting + +```bash +lightdash pre-aggregate-audit --dashboard sales-overview --json +``` + +Returns the full audit object per dashboard (`{ "dashboards": [...] }` when combined with `--all`). Pipe into `jq` to extract what you care about — for example, every dashboard with at least one miss: + +```bash +lightdash pre-aggregate-audit --all --json \ + | jq '.dashboards[] | select(.summary.missCount > 0) | .dashboardName' +``` + +## Using audit results to improve your YAML + +The miss labels printed by the CLI are the same ones documented in [Why a query misses a pre-aggregate](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate) — that table maps each reason to the YAML or configuration change that fixes it. + +A typical iteration loop: + +1. `lightdash pre-aggregate-audit --dashboard ` to see the current misses +2. Edit your dbt model YAML to address each miss (use the [miss-reason table](/references/pre-aggregates/monitoring#why-a-query-misses-a-pre-aggregate) to know which field to change) +3. `lightdash deploy` to push the change +4. Trigger a refresh from **Project Settings > Pre-aggregates** if the pre-aggregate definition changed +5. Re-run the audit to confirm the misses are now hits + +## CI integration with `--fail-on-miss` + +`--fail-on-miss` makes the command exit with code 1 if any tile is a miss, so CI can block changes that regress pre-aggregate coverage: + +```yaml +- name: Audit pre-aggregate coverage + run: lightdash pre-aggregate-audit --dashboard executive-summary --fail-on-miss + env: + LIGHTDASH_API_KEY: ${{ secrets.LIGHTDASH_API_KEY }} + LIGHTDASH_URL: ${{ vars.LIGHTDASH_URL }} + LIGHTDASH_PROJECT_UUID: ${{ vars.LIGHTDASH_PROJECT_UUID }} +``` + +Ineligible tiles do not count as misses — only eligible tiles that failed to match contribute to the failure. + + + Gating an entire project with `--all --fail-on-miss` is usually too strict, since most projects have at least one dashboard with unsupported features. Pin the gate to the high-traffic dashboards you care about with `--dashboard --fail-on-miss`, or use `--json` and filter the dashboards you want to enforce. + + +## Flag reference + +| Flag | Description | +|---|---| +| `--dashboard ` | Audit a specific dashboard. Required unless `--all` is set. | +| `--all` | Audit every dashboard in the project. | +| `--project ` | Override the active project. Falls back to `LIGHTDASH_PROJECT_UUID`, then to the CLI config. | +| `--json` | Emit machine-readable JSON instead of the human-friendly text output. | +| `--fail-on-miss` | Exit 1 if any eligible tile misses. Ineligible tiles do not trigger a failure. | +| `--verbose` | Include ineligible tiles (and their reason) in the human output. | diff --git a/references/pre-aggregates/monitoring.mdx b/references/pre-aggregates/monitoring.mdx index 6d2d12c8..04f4d8ad 100644 --- a/references/pre-aggregates/monitoring.mdx +++ b/references/pre-aggregates/monitoring.mdx @@ -67,3 +67,7 @@ You can also monitor and manage pre-aggregates directly from any dashboard. Open 2. **Rebuild pre-aggregates** — Triggers a manual re-materialization for the pre-aggregates used by this dashboard. Available to developers and admins only. Pre-aggregate audit view for a dashboard + + + The same audit data is available from the terminal via [`lightdash pre-aggregate-audit`](/references/pre-aggregates/cli-audit), which is useful for scripting, project-wide sweeps, and CI gates. + diff --git a/references/pre-aggregates/overview.mdx b/references/pre-aggregates/overview.mdx index 72b6b93a..e56324bf 100644 --- a/references/pre-aggregates/overview.mdx +++ b/references/pre-aggregates/overview.mdx @@ -21,6 +21,9 @@ Any query that goes through the Lightdash semantic layer can hit a pre-aggregate Track materialization status, debug query matching, and view hit/miss stats. + + Inspect dashboard coverage from the terminal and gate CI on hit rates. + ## How it works