feat(lineage): new CLL experience with impact-aware column annotations

[danyelf]

PR checklist

• Ensure you have added or ran the appropriate tests for your PR.
• DCO signed

What type of PR is this?

Feature (behind --new-cll-experience flag)

What this PR does / why we need it:

Adds a new column-level lineage (CLL) experience that treats column lineage as an annotation overlay on the impact radius view, rather than a separate mode. Behind the --new-cll-experience CLI flag.

Specifically:

• When running CLL (and at startup, if you're also using --impact_at_startup flag), draws nodes as "impacted" . Does NOT show specific impacted columns for any model
• Maintains positions of nodes as you click and move through the UI
• When examining any column's CLL, shows its ancestry on the current view.

Here's a brief table of how the previous model and the new CLL use the three API calls:

	Classic	New CLL
/lineage	lineage	Layer 1: lineage for layout
/cll?change_analysis=true	new layout; load CLL data; add impact marks	Layer 2: keep layout; colors all nodes by impact; prepares sidebar for coloring
/cll?column=[foo]	new layout; load column CLL data; add column annotations	Layer 3: keep layout; keep colors from layer 2; add column annotations

If you run with both --new-cli-experience and --impact-at-startup then you get layers 1&2 together immediately

Key behaviors:

• Impact radius view: Models downstream of changes get amber backgrounds; impacted columns highlighted in the sidebar schema view
• Column focus mode: Clicking a column shows its upstream ancestry chain as column nodes inside model nodes, with amber edges/backgrounds on impacted columns. Model-level coloring stays frozen — no visual mode switch
• Stable impact state: Impacted node/column sets are snapshot during impact analysis and remain stable across column selections
• Multi-column ancestry: A model can show multiple ancestor columns when a downstream column depends on several columns from the same upstream model
• Graph boundary aware: Ancestry columns on models not in the current view are filtered out (no floating nodes)

What's new:

• --new-cll-experience CLI flag + new_cll_experience server flag
• computeIsImpacted — three-signal impact detection (changeStatus, CLL node flag, parent_map walk)
• computeImpactedColumns — memoized DFS walk of parent_map for column-level impact
• computeColumnAncestry — traces selected column's upstream ancestry chain
• getStyleForImpacted — amber highlight style for impacted nodes/columns
• Frozen impactedNodeIds / impactedColumnIds refs for cross-mode stability

Special notes for your reviewer:

All behind --new-cll-experience flag — zero impact on existing behavior. Use recce server --new-cll-experience --impact-at-startup to test.

TODO noted in code: move isImpacted to per-model state on node data instead of a lookup set.

UX Nites:

• currently, you can exit CLL mode, and get back to a 'reset' state with the legacy Impact Radius buttons. I would like to make them go away in this new experience.
• the highlighting/hover behavior in the sidebar is annoying, as is the tooltip there.

Open question:

• Should we tweak column CLL to also give DOWNSTREAM effects of changes?
• is the yellow highlight the right choice?

Does this PR introduce a user-facing change?:

New --new-cll-experience flag for recce server that enables an improved column-level lineage visualization with impact-aware column annotations.

Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
recce/cli.py	66.66%	1 Missing ⚠️

Files with missing lines	Coverage Δ
recce/cli.py	61.64% <66.66%> (+0.01%)	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

Adds a new flagged column-level lineage (CLL) UX that overlays column ancestry/impact annotations on top of the existing impact-radius view, plus the supporting impact/ancestry computations and UI styling.

Changes:

• Introduces --new-cll-experience / new_cll_experience server flag wiring (CLI → /api/flag consumption).
• Adds impacted node/column detection + column-ancestry annotation rendering (ReactFlow) for the new CLL experience.
• Updates schema sidebar + lineage node/column styling and extends tests/storybook coverage for the new behavior.

Reviewed changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
recce/cli.py	Adds --new-cll-experience flag and includes new_cll_experience in server flag dict.
js/src/components/lineage/tests/LineageView.component.test.tsx	Updates mocks for new server-flag usage in lineage view tests.
js/src/components/lineage/tests/GraphNode.test.tsx	Extends contexts mock to include useRecceServerFlag.
js/packages/ui/src/lib/dataGrid/generators/toSchemaDataGrid.ts	Adds isImpacted to schema rows and marks impacted columns for styling.
js/packages/ui/src/contexts/lineage/types.ts	Extends lineage context/types with new CLL experience + impacted sets.
js/packages/ui/src/components/ui/dataGrid/dataGridFactory.tsx	Minor refactor to pass typed schema options into generator.
js/packages/ui/src/components/schema/style.css	Adds impacted-row background variables and classes for schema sidebar.
js/packages/ui/src/components/schema/SchemaView.tsx	Reads server flag and highlights impacted columns using frozen impacted set.
js/packages/ui/src/components/lineage/styles.tsx	Adds getStyleForImpacted amber background helper.
js/packages/ui/src/components/lineage/nodes/LineageNode.tsx	Supports new CLL props and impacted background; disables dimming in new mode.
js/packages/ui/src/components/lineage/LineageViewOss.tsx	Implements new CLL overlay flow: impacted sets, ancestry computation, layout preservation, cache patch helper.
js/packages/ui/src/components/lineage/lineage.ts	Updates toReactFlow for new experience and adds ancestry annotation node/edge generation.
js/packages/ui/src/components/lineage/GraphNodeOss.tsx	Passes newCllExperience and isImpacted down to node renderer.
js/packages/ui/src/components/lineage/GraphColumnNodeOss.tsx	Threads isImpacted through column-node data.
js/packages/ui/src/components/lineage/computeIsImpacted.ts	New helper to determine impacted nodes from changeStatus, node flag, and impacted columns.
js/packages/ui/src/components/lineage/computeImpactedColumns.ts	New memoized DFS to derive impacted columns from parent_map + change_status.
js/packages/ui/src/components/lineage/computeColumnAncestry.ts	New ancestry tracer for selected column producing per-model column annotations.
js/packages/ui/src/components/lineage/columns/LineageColumnNode.tsx	Applies impacted background styling for annotated column nodes.
js/packages/ui/src/components/lineage/tests/styles.test.ts	Adds unit tests for getStyleForImpacted.
js/packages/ui/src/components/lineage/tests/LineageNode.test.tsx	Adds tests for “no dimming” + impacted background behavior.
js/packages/ui/src/components/lineage/tests/lineage.test.ts	Adds test ensuring classic column nodes are skipped in new CLL experience.
js/packages/ui/src/components/lineage/tests/isImpacted.test.ts	New tests for computeIsImpacted.
js/packages/ui/src/components/lineage/tests/computeImpactedColumns.test.ts	New tests for computeImpactedColumns (chains, cycles, empties).
js/packages/ui/src/api/flag.ts	Extends RecceServerFlags with new_cll_experience.
js/packages/ui/src/advanced.ts	Exports new CLL helpers/types via @datarecce/ui/advanced.
js/packages/storybook/stories/lineage/CllExperience.stories.tsx	Adds storybook fixtures/demos for new CLL impacted/ancestry visuals.

Comments suppressed due to low confidence (1)

js/src/components/lineage/tests/LineageView.component.test.tsx:141

• The mocked module object defines useRecceServerFlag twice; in an object literal the later key silently overrides the earlier one, which can mask issues and make the test setup misleading. Remove the duplicate and return a single consistent shape for data (e.g. include new_cll_experience when needed).

vi.mock("@datarecce/ui/contexts", async () => {
  const React = await vi.importActual<typeof import("react")>("react");
  return {
    useRouteConfig: vi.fn(() => ({ basePath: "" })),
    useRecceServerFlag: vi.fn(() => ({ data: undefined })),
    useLineageGraphContext: vi.fn(() => mockLineageGraphContext),
    useRecceInstanceContext: vi.fn(() => mockRecceInstanceContext),
    useRecceServerFlag: vi.fn(() => ({ data: {} })),
    useRecceActionContext: vi.fn(() => ({

[Copilot]

computeImpactedSets calls computeIsImpacted for every node, and computeIsImpacted linearly scans the full impactedColumns set (via startsWith). This makes the overall computation O(numNodes × numImpactedColumns) and can get expensive on large graphs. Consider deriving nodeIds directly while computing impacted columns (e.g. parse the nodeId prefix from impacted column IDs) or maintaining a nodeId→impacted lookup map.

[wcchang1115]

keep as-is as this can be further perf enhancement.

[Copilot]

Inside addColumnAncestryNodes, expanding each model node uses nodes.find(...) inside the loop over columnAncestry, which becomes O(n²) as the node list grows. Consider building an id -> node map once (or tracking model nodes as you create them) and updating heights via that map.

[wcchang1115]

keep as-is as this can be further perf enhancement.

[wcchang1115]

js/packages/ui/src/components/lineage/nodes/LineageNode.tsx:338

newCllExperience has a triple fallback (prop ?? data.newCllExperience ?? false) but data.newCllExperience is never set in production — GraphNodeOss passes it as a direct prop only. The data path is only used by some Storybook stories via adaptForCanvas. Consider removing newCllExperience from LineageNodeData and keeping it as a direct prop only — one path, no fallback chain.

js/packages/ui/src/components/lineage/nodes/LineageNode.tsx:398

This simplification is correct — iconColor and titleColor had identical logic before this PR. But it's a drive-by refactor unrelated to the CLL feature. Consider separating it into its own commit or reverting to keep this PR focused.

[wcchang1115]

Review Fixes Summary

#	Source	Suggestion	Fixed?	Reason
1	Copilot	Missing impactedColumnIdsRef snapshot in refreshLayout	Yes	Real bug — sidebar highlighting goes stale
2	Copilot	O(n×m) startsWith scan performance	No	CLL data is scoped to a subgraph, not a real perf issue at current scale
3	Copilot	O(n²) nodes.find() in ancestry	No	Ancestry map is a handful of models, not a real perf issue
4	Copilot	Docstring says != false but code checks truthy	Yes	Misleading docs, one-word fix
5	Copilot	Duplicate useRecceServerFlag() call	Yes	Redundant hook, easy cleanup
6	Copilot (suppressed)	Duplicate useRecceServerFlag mock key	Yes	Breaks pnpm type:check
7	wcchang	Triple fallback for newCllExperience	No	Cannot fully remove data path — Storybook stories go through ReactFlow which requires it. Minimal improvement for the noise
8	wcchang	Drive-by iconColor = titleColor refactor	Yes (reverted)	Unrelated to CLL, reverted to pre-PR IIFE to keep PR focused

[wcchang1115]

LGTM, thanks!