diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ef0c5766..4ff48096 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -16,7 +16,10 @@ A devtools dashboard for end-to-end browser tests. Three test frameworks (Webdri │ ▼ [core] framework-agnostic capture/reporting library - │ + │ │ + │ ▼ + │ [trace] ◀── also used by backend + │ pure transforms: events → trace.zip ▼ (WS frames typed by shared) [backend] Fastify + WS gateway + baseline store + rerun spawner │ @@ -30,7 +33,7 @@ A separate piece, **`packages/script`**, is injected into the browser under test ## Packages -The workspace is a pnpm monorepo. Two of the packages (`shared`, `core`) are workspace-internal — they're marked `"private": true` and never published; consumers bundle their code into their own `dist/`. +The workspace is a pnpm monorepo. Three of the packages (`shared`, `trace`, `core`) are workspace-internal — they're marked `"private": true` and never published; consumers bundle their code into their own `dist/`. ### `packages/shared` @@ -48,10 +51,29 @@ Contains the canonical definitions for: Imports from: nothing. Imported by: every other package. +### `packages/trace` + +Trace-format transforms: captured events in, `trace.zip` out. Workspace-internal; inlined into each consumer at build time. + +Split out of `core` because **two layers need it and only one of them may import `core`**. Adapters build their own trace through `core`; the backend builds one on behalf of an adapter that cannot run Node-side trace code (the Python adapter), and §2.2 bars the backend from `core`. Everything here is a pure transform over `shared` types — no framework API, no driver, no capture, no I/O beyond writing the zip. + +Contains: + +- `writeTraceZip` / `buildTraceZip` — the zip writer and its resource model (`TraceZipResource`). +- `buildActionEvents` — commands → the ordered before/after event stream the player replays, including the `(startTime, sequence, index)` ordering key. +- `buildGroupPath` — the suite/test/step hierarchy a row is nested under. +- `buildImageFrameSnapshots` / `upsertRichestSnapshot` / `buildDenseScreencast` / `thinScreencastFrames` — filmstrip and per-action frame snapshots. +- `buildMutationsNdjson` / `reattributeDomAnchors` — the DOM mutation stream and its anchor repositioning. +- `buildConsoleEvents`, `networkRequestToHar`, `buildSourceResources` / `sourceResourceName` / `callSourceToStack`, `generateTranscript`, `sha1Hex`. + +Imports from: `shared`. Imported by: `core` (which re-exports it, so adapters reach it unchanged) and `backend`. + ### `packages/core` Framework-agnostic capture and reporting library. Workspace-internal; inlined into each adapter at build time. +Adapter-side trace *orchestration and policy* stay here — `trace-finalizer` (what to write and when), `spec-trace-helpers` (slice boundaries and per-test slicing), `trace-retention` (`shouldRetainTrace`, which also governs screenshot and video retention). The transforms they call live in `trace`. + Contains: - `SessionCapturerBase` — orchestrates per-session capture (console/stream patching, WS connection, command-id bookkeeping, upstream-send guard with `onUpstreamDrop` hook). @@ -60,7 +82,7 @@ Contains: - `resolveAdapterOutputDir` — the dir-resolution helper that picks where screencast/trace files land (test-file dir → config dir → cwd, with a `node_modules/` skip). - Pure helpers: `assert-patcher`, `bidi` (`attachBidiHandlers`, `loadSeleniumSubmodule`, `arrayHeadersToObject`), `console` (`stripAnsi`, `detectLogLevel`, `createConsoleLogEntry`, `mapChromeBrowserLogs`, `chromeLogLevelToLogLevel`), `error` (`serializeError`, `errorMessage`), `finalize-screencast`, `net` (`isPortInUse`, `findFreePort`, `getRequestType`), `performance-capture` (`CAPTURE_PERFORMANCE_SCRIPT`, `applyPerformanceData`), `retry-tracker`, `script-loader` (`loadInjectableScript`, `pollUntilReady`), `stack` (`isUserCodeFrame`, `normalizeFilePath`, `getCallSourceFromStack`), `suite-helpers`, `test-discovery` (`findTestDefinitions`, `extractTestMetadata`), `uid` (`generateStableUid`, `deterministicUid`, `resetSignatureCounters`), `video-encoder` (`encodeToVideo`). -Imports from: `shared`. Imported by: all three adapter packages. +Imports from: `shared`, `trace`. Imported by: all three adapter packages. ### `packages/service` — WebdriverIO adapter @@ -215,6 +237,7 @@ The repo has converged on a clear ownership story. When in doubt, the top-down d - A type, constant, enum, schema, or contract used by more than one package → **`shared`**. - Capture, parsing, normalization, sourcemap, UID, reporter, screencast, or WS-framing logic that doesn't depend on a specific framework's API → **`core`**. +- A pure transform that turns captured events into trace-zip content → **`trace`**. The test is whether the backend would ever need it: it builds traces for adapters that can't, and it may not import `core`. - A specific framework's hook, driver patch, or runner integration → the matching **adapter** package. Adapter code calls `core` for the actual work and only owns the hook registration. - A backend HTTP route, WS handler, or rerun behavior → **`backend`**, with the contract added to `shared` first. - UI → **`app`**, consuming `shared` contracts only. @@ -225,7 +248,7 @@ A few cross-cutting conventions follow from this layout: - Adapter packages don't import each other. Anything two adapters would both want lives in `core`. - Backend doesn't import adapter packages, and adapter packages don't import backend or app. - The script package is a leaf — adapters load its built bundle as a string and inject it; they don't import from it at runtime. -- `shared` and `core` are private workspace packages. Consumers bundle them. The bundler config has to inline them (not externalize) or the published artifact won't resolve — see the build-config notes in `CLAUDE.md`. +- `shared`, `trace`, and `core` are private workspace packages. Consumers bundle them. The bundler config has to inline them (not externalize) or the published artifact won't resolve — see the build-config notes in `CLAUDE.md`. --- diff --git a/CLAUDE.md b/CLAUDE.md index 6d2204f4..37505686 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,7 +10,7 @@ Anyone working in the repo, human or AI agent, can use this as the source of tru A devtools dashboard for end-to-end browser tests. Three test frameworks (WebdriverIO, Nightwatch, Selenium) push the same normalized event stream through a single backend into a single Lit-based browser UI. The adapters are deliberately thin — they translate framework hooks into calls on a shared core capture/reporting library and own only the framework-specific glue. -Package map and data flow are in [ARCHITECTURE.md](./ARCHITECTURE.md). The summary: `shared` for types and contracts, `core` for framework-agnostic capture, three adapters (`service`, `nightwatch-devtools`, `selenium-devtools`) for framework glue, `backend` for the server, `app` for the UI, `script` for the page-injected runtime. +Package map and data flow are in [ARCHITECTURE.md](./ARCHITECTURE.md). The summary: `shared` for types and contracts, `trace` for the event→zip transforms, `core` for framework-agnostic capture, three adapters (`service`, `nightwatch-devtools`, `selenium-devtools`) for framework glue, `backend` for the server, `app` for the UI, `script` for the page-injected runtime. --- @@ -48,6 +48,7 @@ Defined in root `tsconfig.json`: | `@wdio/selenium-devtools` / `*` | `packages/selenium-devtools/src/...` | | `@wdio/devtools-shared` / `*` | `packages/shared/src/...` | | `@wdio/devtools-core` / `*` | `packages/core/src/...` | +| `@wdio/devtools-trace` / `*` | `packages/trace/src/...` | | `@wdio/elements` / `*` | `packages/elements/src/...` | These exist so imports stay short and grep-able. Long relative paths (`../../../components/…`) aren't used. @@ -72,6 +73,14 @@ If the same logical change would land in two or more adapters, the logic belongs Some helpers are framework-agnostic by nature but used in only one adapter today (e.g. nightwatch's `parseNetworkFromPerfLogs` for CDP perf-log parsing, selenium's `detectRunner`/`captureLaunchCommand`). They stay in their adapter until a second consumer appears; at that point they move to core. +### Trace-format transforms live in `trace`, one layer below `core` + +`packages/trace` holds the pure transforms that turn captured events into trace-zip content — the zip writer, action events, group paths, frame snapshots, mutations, HAR, sources, transcript. `core` keeps the adapter-side *orchestration and policy* that calls them: `trace-finalizer`, `spec-trace-helpers`, `trace-retention`. + +The split is not aesthetic. **`backend` may not import `core`** (it would pull framework-adapter logic into the server), but it does need to build a trace on behalf of an adapter that can't — the Python adapter ships no Node. `trace` is the layer both can reach, so it may import `shared` and nothing else; a single import of `core` from it re-creates the cycle the split exists to remove, and ESLint enforces that. + +The test for a new helper: *would the backend ever need this to build a zip?* If yes, `trace`. If it needs a driver, a framework hook, or a capture session, `core`. + ### Adapters are thin and isolated Adapter packages own only: @@ -96,13 +105,16 @@ No `any` crosses a package boundary. When a framework API forces a loosely-typed ### Workspace-internal packages stay bundled -`packages/shared` and `packages/core` are `"private": true` and never published. Each consumer inlines their code into its own `dist/` at build time. +`packages/shared`, `packages/trace` and `packages/core` are `"private": true` and never published. Each consumer inlines their code into its own `dist/` at build time. -- Both deps are listed in `devDependencies` with `workspace:^`, never in `dependencies`. Vite and tsup both externalize anything in `dependencies` by default; `devDependencies` is what gets inlined. -- Neither is added to a bundler's `external` config. Vite's `external` callback receives both the bare package name *and* the resolved absolute path (e.g. `/Users/.../packages/core/src/index.ts`); a check for only one form silently externalizes the other. +- All three are listed in `devDependencies` with `workspace:^`, never in `dependencies`. Vite and tsup both externalize anything in `dependencies` by default; `devDependencies` is what gets inlined. +- None of them is added to a bundler's `external` config. Vite's `external` callback receives both the bare package name *and* the resolved absolute path (e.g. `/Users/.../packages/core/src/index.ts`); a check for only one form silently externalizes the other. +- That callback enumerates the private packages, so **adding a fourth one means editing it** — a package missing from the list falls through to the default and is externalized silently, producing a dist that dies at install with `ERR_MODULE_NOT_FOUND`. It is a `PRIVATE_WORKSPACE_PACKAGES` array rather than a chain of `||`s for exactly that reason. Adding a workspace package also means adding it to `pnpm-workspace.yaml`, whose `packages:` list is explicit rather than a glob. - The same callback receives bare relative imports (`./utils.js`, `../constants.js`). A check that allows only `./` will externalize `../`-style imports from subfolders and the dist crashes with `ERR_MODULE_NOT_FOUND` at install time. - `packages/service/vite.config.ts` is the canonical pattern for getting both right. -- After any change to a bundler config or build script, `grep -nE "(from|require\()\s*['\"](@wdio/devtools-(core|shared)|.*/packages/(core|shared)/)" packages//dist/*.js` should return nothing. That's how you catch the absolute-path leak. Match on the `from`/`require(` prefix, not the bare package name: `LIBRARY_NAME = "@wdio/devtools-core"` (written into the trace's `context-options`) and `Symbol.for("@wdio/devtools-core/assert-patched")` are inlined string *values* that legitimately survive bundling, so a bare-name grep always reports a false leak. +- After any change to a bundler config or build script, `grep -nE "(from|require\()\s*['\"](@wdio/devtools-(core|shared|trace)|.*/packages/(core|shared|trace)/)" packages//dist/*.js` should return nothing. That's how you catch the absolute-path leak. Match on the `from`/`require(` prefix, not the bare package name: `LIBRARY_NAME = "@wdio/devtools-core"` (written into the trace's `context-options`) and `Symbol.for("@wdio/devtools-core/assert-patched")` are inlined string *values* that legitimately survive bundling, so a bare-name grep always reports a false leak. + +- A **CJS-only dependency must be externalized, not inlined**, or esbuild rewrites its `require` into a shim that throws `Dynamic require of "fs" is not supported` the moment the module loads. Declaring it in `dependencies` is what externalizes it; that is why all three adapters — and now `backend` — list `yazl` there rather than in `devDependencies`. This is the opposite of the workspace-internal rule above, and for the same underlying reason: `dependencies` is externalized, `devDependencies` is inlined. Neither `pnpm build` nor `pnpm test` nor the leak grep notices — every one of them passes on a dist that dies on first import — so `packages/backend/tests/dist-bundling.test.ts` asserts the shim is absent. Bundlers in use: **vite** for `app`, `service`, `script`; **tsup** for `backend`, `nightwatch-devtools`, `selenium-devtools`. @@ -190,13 +202,14 @@ A handful of tests need `@wdio/devtools-script` to be built first (the browser-i The decision tree from [ARCHITECTURE.md "Where things live"](./ARCHITECTURE.md#where-things-live) is the starting point. The general shape: - Shared concept → `shared`. +- Pure transform producing trace-zip content → `trace`. - Framework-agnostic capture/reporting logic → `core`. - Framework-specific glue → the matching adapter. - Server route/WS handler → `backend` (contract in `shared` first). - UI → `app`. - Code that runs in the browser under test → `script`. -When the right place is ambiguous (something between `shared` and `core`, or between `core` and an adapter), the question that resolves it is: *who else would want this?* If the answer is "any future adapter would," it's `core`. If "only the framework with X-specific API does," it's the adapter. +When the right place is ambiguous (something between `shared` and `core`, or between `core` and an adapter), the question that resolves it is: *who else would want this?* If the answer is "any future adapter would," it's `core`. If "only the framework with X-specific API does," it's the adapter. If "the backend would, to build a zip for an adapter that can't," it's `trace`. ### While editing diff --git a/eslint.config.cjs b/eslint.config.cjs index 7ef666a2..3d4b0ab4 100644 --- a/eslint.config.cjs +++ b/eslint.config.cjs @@ -281,7 +281,7 @@ module.exports = [ { group: ['@wdio/devtools-core', '@wdio/devtools-core/*'], message: - 'Backend must not depend on core (CLAUDE.md §2.2). core is framework-agnostic adapter logic; backend only needs shared contracts.' + 'Backend must not depend on core (CLAUDE.md §2.2). core is framework-agnostic ADAPTER logic. For trace-format transforms use @wdio/devtools-trace, which sits below both; anything else belongs in shared.' } ] } @@ -326,6 +326,11 @@ module.exports = [ group: ['@wdio/devtools-core', '@wdio/devtools-core/*'], message: 'App must not import from core (CLAUDE.md §2.2). core is framework-agnostic adapter logic; the app receives normalized events over WS.' + }, + { + group: ['@wdio/devtools-trace', '@wdio/devtools-trace/*'], + message: + 'App must not import from trace (CLAUDE.md §2.2). trace WRITES the zip; the app reads one the backend has already parsed for it.' } ] } @@ -374,5 +379,54 @@ module.exports = [ } ] } + }, + + // CLAUDE.md §2.2 — trace sits below core and backend, and both import it. + // It holds pure transforms over shared types only: anything it pulled from + // core would re-create the dependency the split exists to remove. + { + files: ['packages/trace/**/*.{ts,tsx,js,mjs,cjs}'], + rules: { + 'no-restricted-imports': [ + 'error', + { + patterns: [ + { + group: ['@wdio/devtools-core', '@wdio/devtools-core/*'], + message: + 'trace must not depend on core (CLAUDE.md §2.2). core imports trace; keeping the arrow one-way is what lets backend use trace at all.' + }, + { + group: ['@wdio/devtools-backend', '@wdio/devtools-backend/*'], + message: + 'trace must not depend on backend (CLAUDE.md §2.2). trace is the lower layer.' + }, + { + group: ['@wdio/devtools-service', '@wdio/devtools-service/*'], + message: + 'trace must not depend on any adapter (CLAUDE.md §2.2). Adapters import trace, not the other way around.' + }, + { + group: [ + '@wdio/nightwatch-devtools', + '@wdio/nightwatch-devtools/*' + ], + message: + 'trace must not depend on any adapter (CLAUDE.md §2.2). Adapters import trace, not the other way around.' + }, + { + group: ['@wdio/selenium-devtools', '@wdio/selenium-devtools/*'], + message: + 'trace must not depend on any adapter (CLAUDE.md §2.2). Adapters import trace, not the other way around.' + }, + { + group: ['@/*', '@components/*'], + message: + 'trace must not depend on app (CLAUDE.md §2.2). trace is Node-side transform logic.' + } + ] + } + ] + } } ] diff --git a/packages/backend/package.json b/packages/backend/package.json index d5abd272..e43c208f 100644 --- a/packages/backend/package.json +++ b/packages/backend/package.json @@ -49,12 +49,14 @@ "import-meta-resolve": "^4.2.0", "shell-quote": "^1.8.4", "tree-kill": "^1.2.2", - "ws": "^8.21.0" + "ws": "^8.21.0", + "yazl": "^3.3.1" }, "devDependencies": { "@types/shell-quote": "^1.7.5", "@types/ws": "^8.18.1", "@wdio/devtools-shared": "workspace:^", + "@wdio/devtools-trace": "workspace:^", "nodemon": "^3.1.14", "tsup": "^8.5.1" } diff --git a/packages/backend/src/baseline/types.ts b/packages/backend/src/baseline/types.ts index a7211761..b361b883 100644 --- a/packages/backend/src/baseline/types.ts +++ b/packages/backend/src/baseline/types.ts @@ -1,9 +1,11 @@ import type { CommandLog, ConsoleLog, + Metadata, NetworkRequest, TestError, - TestStatus + TestStatus, + TraceMutation } from '@wdio/devtools-shared' // Backend storage uses the canonical shared types. The `*Like` aliases below @@ -13,13 +15,10 @@ export type CommandLogLike = CommandLog export type ConsoleLogLike = ConsoleLog export type NetworkRequestLike = NetworkRequest -// Mutations stay loose: the concrete shape (TraceMutation) lives in -// packages/script (browser-side, depends on DOM types) and isn't safe to -// import here. -export interface MutationLike { - timestamp: number - [key: string]: unknown -} +// `TraceMutation` in shared is deliberately the Node-safe version of the +// browser-side shape — string literals instead of DOM node types — so it flows +// here without dragging the DOM lib into shared's compilation. +export type MutationLike = TraceMutation export type NodeState = TestStatus export type NodeError = TestError @@ -48,4 +47,11 @@ export interface ActiveRun { sources: Record nodes: Map startedAt: number + /** Last `metadata` frame the worker sent. Preserve & Rerun never needed it; + * a trace artifact does — it carries the session identity and capabilities + * the viewer reads. */ + metadata?: Metadata + /** Raw `logs` frames, the trace's transcript source. Only the JS adapters + * send these, so this is routinely empty. */ + traceLogs: string[] } diff --git a/packages/backend/src/baseline/utils.ts b/packages/backend/src/baseline/utils.ts index 8ff8e5d1..6f5825cf 100644 --- a/packages/backend/src/baseline/utils.ts +++ b/packages/backend/src/baseline/utils.ts @@ -8,7 +8,8 @@ export function freshRun(): ActiveRun { mutations: [], sources: {}, nodes: new Map(), - startedAt: Date.now() + startedAt: Date.now(), + traceLogs: [] } } diff --git a/packages/backend/src/baselineStore.ts b/packages/backend/src/baselineStore.ts index e6ed8f25..c23420cf 100644 --- a/packages/backend/src/baselineStore.ts +++ b/packages/backend/src/baselineStore.ts @@ -3,12 +3,11 @@ * into an accumulator, then time-window-filters per test/suite on demand. */ import logger from '@wdio/logger' +import type { Metadata } from '@wdio/devtools-shared' import type { ActiveRun, CommandLogLike, - ConsoleLogLike, - MutationLike, NetworkRequestLike, NodeError, NodeState, @@ -23,6 +22,14 @@ export type { PreservedAttempt, PreservedStep } from './baseline/types.js' const log = logger('@wdio/devtools-baseline') +/** Append a wire payload that should be a list. A scope arriving as anything + * else is a malformed frame, not a reason to throw inside the message loop. */ +function appendArray(target: T[], data: unknown): void { + if (Array.isArray(data)) { + target.push(...(data as T[])) + } +} + class BaselineStore { #activeRun: ActiveRun = freshRun() #baselines = new Map() @@ -38,9 +45,7 @@ class BaselineStore { } switch (scope) { case 'commands': - if (Array.isArray(data)) { - this.#activeRun.commands.push(...(data as CommandLogLike[])) - } + appendArray(this.#activeRun.commands, data) return case 'replaceCommand': { // A command is sent first, then re-sent with late-attached fields @@ -56,9 +61,7 @@ class BaselineStore { return } case 'consoleLogs': - if (Array.isArray(data)) { - this.#activeRun.consoleLogs.push(...(data as ConsoleLogLike[])) - } + appendArray(this.#activeRun.consoleLogs, data) return case 'networkRequests': if (Array.isArray(data)) { @@ -66,19 +69,31 @@ class BaselineStore { } return case 'mutations': - if (Array.isArray(data)) { - this.#activeRun.mutations.push(...(data as MutationLike[])) - } + appendArray(this.#activeRun.mutations, data) return case 'sources': Object.assign(this.#activeRun.sources, data as Record) return + case 'metadata': + // Last one wins, mirroring the exporter: a run that replaces its + // session carries the latest session's identity. + this.#activeRun.metadata = data as Metadata + return + case 'logs': + appendArray(this.#activeRun.traceLogs, data) + return case 'suites': this.#ingestSuites(data) return } } + /** The run accumulated so far. Read-only by contract — the trace exporter + * reads it; nothing outside this class writes to it. */ + activeRun(): Readonly { + return this.#activeRun + } + // Mirrors the app's command replacement: match by stable `id`, then by the // old timestamp, appending only when neither locates the original. #replaceCommand(oldTimestamp: number | undefined, command: CommandLogLike) { diff --git a/packages/backend/src/index.ts b/packages/backend/src/index.ts index 9382eef0..3f7adca4 100644 --- a/packages/backend/src/index.ts +++ b/packages/backend/src/index.ts @@ -361,7 +361,12 @@ function registerWorkerWebSocket(s: FastifyInstance): void { testRunner, videoRegistry, broadcastToClients, - clientCount: () => clients.size + clientCount: () => clients.size, + replyToWorker: (message) => { + if (socket.readyState === WebSocket.OPEN) { + socket.send(message) + } + } }) ) } diff --git a/packages/backend/src/trace-export-message.ts b/packages/backend/src/trace-export-message.ts new file mode 100644 index 00000000..5be36407 --- /dev/null +++ b/packages/backend/src/trace-export-message.ts @@ -0,0 +1,93 @@ +/** + * Worker control frame for server-side trace export. Split from + * `worker-message-handler` so its dispatcher stays a dispatcher — this is the + * only control scope that does real work rather than routing. + */ + +import logger from '@wdio/logger' +import { + TRACE_EXPORT_SCOPE, + type TraceExportRequest, + type TraceExportResult +} from '@wdio/devtools-shared' +import type { ActiveRun } from './baseline/types.js' +import { exportActiveRunTrace } from './trace-export.js' + +const log = logger('@wdio/devtools-backend') + +export interface TraceExportDeps { + activeRun: () => Readonly + /** Back down the worker's own socket. Absent when the socket has already + * gone, which is ordinary at the end of a run. */ + replyToWorker?: (message: string) => void +} + +/** + * A request is only actionable with all three fields. A partial frame would + * otherwise write `trace-undefined` into a directory named `undefined`. + */ +export function asTraceExportRequest( + data: Record | undefined +): TraceExportRequest | undefined { + if ( + typeof data?.requestId !== 'string' || + typeof data.outputDir !== 'string' || + typeof data.sessionId !== 'string' + ) { + return undefined + } + return data as unknown as TraceExportRequest +} + +/** + * Build the artifact and answer on the worker's own socket. + * + * Deliberately never rejects: an adapter asks for this while finishing a run, + * and a failed export must report itself rather than take the run down. The + * reason travels in `error` so the adapter can log something actionable + * instead of the artifact silently not appearing. + */ +export async function runTraceExport( + request: TraceExportRequest, + deps: TraceExportDeps +): Promise { + const reply = (result: TraceExportResult) => + deps.replyToWorker?.( + JSON.stringify({ scope: TRACE_EXPORT_SCOPE.result, data: result }) + ) + try { + const path = await exportActiveRunTrace(deps.activeRun(), { + outputDir: request.outputDir, + sessionId: request.sessionId, + format: request.format, + fileStem: request.fileStem + }) + log.info(`Trace exported for session ${request.sessionId}: ${path}`) + reply({ requestId: request.requestId, path }) + } catch (err) { + const error = err instanceof Error ? err.message : String(err) + log.error(`Trace export failed for session ${request.sessionId}: ${error}`) + reply({ requestId: request.requestId, error }) + } +} + +/** + * Handle the frame if it is one, and report whether it was. Returns + * synchronously — the export runs detached, because the worker keeps streaming + * while it writes and blocking the message loop would stall the run. + */ +export function tryHandleTraceExportMessage( + parsed: { scope?: string; data?: Record }, + deps: TraceExportDeps +): boolean { + if (parsed.scope !== TRACE_EXPORT_SCOPE.request) { + return false + } + const request = asTraceExportRequest(parsed.data) + if (!request) { + log.error('Ignoring a trace export request missing required fields') + return true + } + void runTraceExport(request, deps) + return true +} diff --git a/packages/backend/src/trace-export.ts b/packages/backend/src/trace-export.ts new file mode 100644 index 00000000..872cc0f1 --- /dev/null +++ b/packages/backend/src/trace-export.ts @@ -0,0 +1,87 @@ +/** + * Builds a trace artifact from the run the backend is already accumulating. + * + * The transforms are `@wdio/devtools-trace`, the same code the JS adapters run + * in-process — the point of the split is that there is one implementation, not + * one per language. This module is only the adapter between the accumulator's + * shape and the exporter's, plus the two derivations the wire does not carry. + */ + +import type { TestMetadataMap, TraceExportRequest } from '@wdio/devtools-shared' +import { + writeTraceZip, + type TraceCapturer +} from '@wdio/devtools-trace/trace-exporter' +import type { ActiveRun, TimeWindowNode } from './baseline/types.js' + +/** + * Test titles for `Tracing.tracingGroup` events, derived from the suite tree + * the backend builds for Preserve & Rerun. `specFile` is required by the entry + * shape but genuinely unknown for a node that reported no file, and an empty + * string is what the exporter already tolerates from adapters that omit it. + */ +export function testMetadataFromNodes( + nodes: Map +): TestMetadataMap { + const metadata: TestMetadataMap = new Map() + for (const node of nodes.values()) { + if (node.kind !== 'test') { + continue + } + metadata.set(node.uid, { + title: node.title ?? node.fullTitle ?? node.uid, + specFile: node.file ?? '', + ...(node.state ? { state: node.state } : {}) + }) + } + return metadata +} + +/** + * Adapt the accumulator to the exporter's input. Only `sources` needs + * reshaping — the accumulator stores the canonical shared types for + * everything else, so nothing here is a cast. + */ +function toCapturer(run: Readonly): TraceCapturer { + return { + mutations: run.mutations, + traceLogs: run.traceLogs, + consoleLogs: run.consoleLogs, + networkRequests: run.networkRequests, + commandsLog: run.commands, + sources: new Map(Object.entries(run.sources)), + metadata: run.metadata, + startWallTime: run.startedAt + } +} + +/** Nothing worth writing. An empty artifact is worse than a clear decline — + * it reads in the viewer as a run that captured nothing. */ +export function hasExportableData(run: Readonly): boolean { + return ( + run.commands.length > 0 || + run.consoleLogs.length > 0 || + run.networkRequests.length > 0 + ) +} + +export async function exportActiveRunTrace( + run: Readonly, + request: Pick< + TraceExportRequest, + 'outputDir' | 'sessionId' | 'format' | 'fileStem' + > +): Promise { + if (!hasExportableData(run)) { + throw new Error('nothing captured for this run') + } + // `capabilities` is not passed separately: writeTraceZip spreads the + // capturer's own metadata, which already carries it. + return writeTraceZip(toCapturer(run), { + outputDir: request.outputDir, + sessionId: request.sessionId, + ...(request.format ? { format: request.format } : {}), + ...(request.fileStem ? { fileStem: request.fileStem } : {}), + testMetadata: testMetadataFromNodes(run.nodes) + }) +} diff --git a/packages/backend/src/trace-reader-utils.ts b/packages/backend/src/trace-reader-utils.ts index 72f2a709..5f59d8fb 100644 --- a/packages/backend/src/trace-reader-utils.ts +++ b/packages/backend/src/trace-reader-utils.ts @@ -1,8 +1,8 @@ // Pure helpers for reconstructing a player payload from trace.zip events. // No I/O — the reader pipeline (trace-reader.ts) composes these. -import { createHash } from 'node:crypto' import { strFromU8 } from 'fflate' +import { sourceResourceName } from '@wdio/devtools-trace/trace-sources' import { isTestRunnerId, TraceType, @@ -210,11 +210,6 @@ export function buildConsoleLogs( return logs.sort((a, b) => a.timestamp - b.timestamp) } -// Local copy of core's sha1 helper — the backend only imports from shared. -function sha1Hex(data: string): string { - return createHash('sha1').update(data).digest('hex') -} - // Older zips glued ':[:]' onto the frame's file (and shifted // line/column); peel up to two numeric suffixes — the innermost is the real // line. `at < 2` keeps bare Windows drive specs (`C:...`) intact. @@ -286,7 +281,7 @@ export function buildSources( if (file in sources) { continue } - const data = files[`resources/src@${sha1Hex(file)}.txt`] + const data = files[`resources/${sourceResourceName(file)}`] if (data) { sources[file] = strFromU8(data) } diff --git a/packages/backend/src/worker-message-handler.ts b/packages/backend/src/worker-message-handler.ts index ccb8f8b1..3f060d89 100644 --- a/packages/backend/src/worker-message-handler.ts +++ b/packages/backend/src/worker-message-handler.ts @@ -2,6 +2,7 @@ import logger from '@wdio/logger' import { WS_SCOPE } from '@wdio/devtools-shared' import type { baselineStore as BaselineStore } from './baselineStore.js' import type { testRunner as TestRunner } from './runner.js' +import { tryHandleTraceExportMessage } from './trace-export-message.js' const log = logger('@wdio/devtools-backend') @@ -11,6 +12,9 @@ export interface WorkerMessageContext { videoRegistry: Map broadcastToClients: (message: string) => void clientCount: () => number + /** Back down the worker's own socket. Absent when the socket has already + * gone, which is ordinary at the end of a run. */ + replyToWorker?: (message: string) => void } // Returns true if the message was fully handled and shouldn't be forwarded. @@ -90,6 +94,14 @@ export function createWorkerMessageHandler( if (tryHandleControlMessage(parsed, ctx)) { return } + if ( + tryHandleTraceExportMessage(parsed, { + activeRun: () => ctx.baselineStore.activeRun(), + replyToWorker: ctx.replyToWorker + }) + ) { + return + } // Tee the event into the baseline accumulator for time-window // partitioning at preserve time. After special-case handling so we // don't accumulate control frames (clearCommands, screencast). diff --git a/packages/backend/tests/baselineStore.test.ts b/packages/backend/tests/baselineStore.test.ts index dbd328aa..206c0d71 100644 --- a/packages/backend/tests/baselineStore.test.ts +++ b/packages/backend/tests/baselineStore.test.ts @@ -525,3 +525,34 @@ describe('baselineStore', () => { expect(snap.test.state).toBe('running') }) }) + +// Preserve & Rerun never needed either of these, so both were dropped on the +// floor. A trace built from the accumulated run does need them: `metadata` +// carries the session identity and capabilities the viewer reads, and `logs` +// is the transcript's source. +describe('baselineStore — scopes the trace export reads', () => { + beforeEach(() => { + baselineStore.resetActiveRun() + }) + + it('keeps the latest metadata frame', () => { + baselineStore.recordEvent('metadata', { sessionId: 'a' }) + baselineStore.recordEvent('metadata', { sessionId: 'b' }) + expect(baselineStore.activeRun().metadata).toEqual({ sessionId: 'b' }) + }) + + it('appends log frames in arrival order and ignores a non-array', () => { + baselineStore.recordEvent('logs', ['one']) + baselineStore.recordEvent('logs', ['two', 'three']) + baselineStore.recordEvent('logs', 'not-an-array') + expect(baselineStore.activeRun().traceLogs).toEqual(['one', 'two', 'three']) + }) + + it('starts a new run with neither carried over', () => { + baselineStore.recordEvent('metadata', { sessionId: 'a' }) + baselineStore.recordEvent('logs', ['one']) + baselineStore.resetActiveRun() + expect(baselineStore.activeRun().metadata).toBeUndefined() + expect(baselineStore.activeRun().traceLogs).toEqual([]) + }) +}) diff --git a/packages/backend/tests/dist-bundling.test.ts b/packages/backend/tests/dist-bundling.test.ts new file mode 100644 index 00000000..e9f235be --- /dev/null +++ b/packages/backend/tests/dist-bundling.test.ts @@ -0,0 +1,61 @@ +/** + * The backend ships `dist/server.js` and `dist/show-trace.js` as executables, + * so a bundling mistake here is a crash on the user's machine rather than a + * failing build. + * + * The failure this guards against: importing a value from a CJS-only package + * (yazl) pulls it into tsup's ESM output, where esbuild replaces `require` with + * a shim that throws `Dynamic require of "fs" is not supported` the moment the + * module is loaded. It cost `pnpm show-trace` entirely, and neither `pnpm + * build`, `pnpm test` nor the workspace-internal leak grep in CLAUDE.md §2.6 + * noticed — every one of them passes on a dist that dies on first import. + * + * The fix is always the same: declare the CJS package in `dependencies` so it + * is externalized and Node loads it natively, which is what the three adapters + * already do for yazl. + * + * Gated on the build having run — CI test jobs may execute before it. + */ + +import fs from 'node:fs' +import path from 'node:path' +import url from 'node:url' +import { describe, it, expect } from 'vitest' + +const distDir = path.resolve( + url.fileURLToPath(new URL('.', import.meta.url)), + '..', + 'dist' +) + +const bundles = fs.existsSync(distDir) + ? fs.readdirSync(distDir).filter((f) => f.endsWith('.js')) + : [] + +describe('backend dist bundling', () => { + it.skipIf(bundles.length === 0)( + 'bundles no CJS dependency that would need a require shim at runtime', + () => { + const offenders = bundles.filter((file) => + fs + .readFileSync(path.join(distDir, file), 'utf8') + .includes('Dynamic require of') + ) + expect(offenders).toEqual([]) + } + ) + + // Named explicitly because it is the one the trace writer pulls in, and the + // shim check above only fails once esbuild happens to emit a shim. + it.skipIf(bundles.length === 0)( + 'reaches yazl by import rather than inlining it — it is CJS', + () => { + const inlining = bundles.filter((file) => + fs + .readFileSync(path.join(distDir, file), 'utf8') + .includes('node_modules/yazl/index.js') + ) + expect(inlining).toEqual([]) + } + ) +}) diff --git a/packages/backend/tests/trace-export-message.test.ts b/packages/backend/tests/trace-export-message.test.ts new file mode 100644 index 00000000..b55bba34 --- /dev/null +++ b/packages/backend/tests/trace-export-message.test.ts @@ -0,0 +1,180 @@ +/** + * The worker control frame that asks the backend to build a trace. + * + * Two properties matter more than the happy path. An export happens while the + * adapter is finishing a run, so a failure has to come back as a message rather + * than an unhandled rejection; and a frame missing its fields must be refused + * outright, since the fields name the directory written to. + */ + +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { afterEach, describe, it, expect, vi } from 'vitest' +import { TRACE_EXPORT_SCOPE } from '@wdio/devtools-shared' +import { + asTraceExportRequest, + runTraceExport, + tryHandleTraceExportMessage +} from '../src/trace-export-message.js' +import { freshRun } from '../src/baseline/utils.js' +import type { ActiveRun } from '../src/baseline/types.js' + +const dirs: string[] = [] + +afterEach(async () => { + await Promise.all( + dirs.splice(0).map((d) => fs.rm(d, { recursive: true, force: true })) + ) +}) + +async function tmpDir(): Promise { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'trace-export-msg-')) + dirs.push(dir) + return dir +} + +function run(overrides: Partial = {}): ActiveRun { + return { + ...freshRun(), + commands: [{ command: 'click', args: ['#go'], timestamp: 1200 }], + ...overrides + } +} + +function deps(activeRun: ActiveRun = run()) { + const replyToWorker = vi.fn() + return { + replyToWorker, + deps: { activeRun: () => activeRun, replyToWorker } + } +} + +/** The single reply frame, parsed. */ +function reply(replyToWorker: ReturnType) { + expect(replyToWorker).toHaveBeenCalledTimes(1) + return JSON.parse(replyToWorker.mock.calls[0]![0] as string) +} + +describe('asTraceExportRequest', () => { + it('accepts a frame carrying all three required fields', () => { + expect( + asTraceExportRequest({ + requestId: 'r1', + outputDir: '/tmp/out', + sessionId: 's1' + }) + ).toEqual({ requestId: 'r1', outputDir: '/tmp/out', sessionId: 's1' }) + }) + + // Each of these would otherwise reach the writer as `undefined` and put a + // `trace-undefined.zip` into a directory literally named "undefined". + it('refuses a frame missing any required field, or carrying a non-string', () => { + const full = { requestId: 'r1', outputDir: '/tmp/out', sessionId: 's1' } + for (const key of ['requestId', 'outputDir', 'sessionId'] as const) { + const partial = { ...full } + delete partial[key] + expect(asTraceExportRequest(partial)).toBeUndefined() + expect(asTraceExportRequest({ ...full, [key]: 42 })).toBeUndefined() + } + expect(asTraceExportRequest(undefined)).toBeUndefined() + }) +}) + +describe('tryHandleTraceExportMessage', () => { + it('ignores frames of any other scope', () => { + const { deps: d, replyToWorker } = deps() + expect( + tryHandleTraceExportMessage({ scope: 'commands', data: {} }, d) + ).toBe(false) + expect(tryHandleTraceExportMessage({ scope: undefined }, d)).toBe(false) + expect(replyToWorker).not.toHaveBeenCalled() + }) + + // Claimed, not forwarded: a malformed export request is still an export + // request, and passing it on would broadcast it to every dashboard client. + it('claims a malformed request without exporting or replying', () => { + const { deps: d, replyToWorker } = deps() + expect( + tryHandleTraceExportMessage( + { scope: TRACE_EXPORT_SCOPE.request, data: { requestId: 'r1' } }, + d + ) + ).toBe(true) + expect(replyToWorker).not.toHaveBeenCalled() + }) + + it('claims a well-formed request', async () => { + const outputDir = await tmpDir() + const { deps: d } = deps() + expect( + tryHandleTraceExportMessage( + { + scope: TRACE_EXPORT_SCOPE.request, + data: { requestId: 'r1', outputDir, sessionId: 's1' } + }, + d + ) + ).toBe(true) + }) +}) + +describe('runTraceExport', () => { + it('replies with the artifact path, under the result scope', async () => { + const outputDir = await tmpDir() + const { deps: d, replyToWorker } = deps() + + await runTraceExport({ requestId: 'r1', outputDir, sessionId: 'sess-1' }, d) + + const frame = reply(replyToWorker) + expect(frame.scope).toBe(TRACE_EXPORT_SCOPE.result) + expect(frame.data.requestId).toBe('r1') + expect(frame.data.path).toBe(path.join(outputDir, 'trace-sess-1.zip')) + expect(frame.data.error).toBeUndefined() + // The path is only worth reporting if something is actually there. + await expect(fs.stat(frame.data.path)).resolves.toBeTruthy() + }) + + it('reports a failure as a reply rather than rejecting', async () => { + const { deps: d, replyToWorker } = deps(run({ commands: [] })) + + await expect( + runTraceExport( + { requestId: 'r2', outputDir: await tmpDir(), sessionId: 'sess-2' }, + d + ) + ).resolves.toBeUndefined() + + const frame = reply(replyToWorker) + expect(frame.data.requestId).toBe('r2') + expect(frame.data.error).toMatch(/nothing captured/) + expect(frame.data.path).toBeUndefined() + }) + + it('reports an unwritable directory instead of taking the run down', async () => { + const { deps: d, replyToWorker } = deps() + const missing = path.join(await tmpDir(), 'no', 'such', '\0bad') + + await expect( + runTraceExport( + { requestId: 'r3', outputDir: missing, sessionId: 'sess-3' }, + d + ) + ).resolves.toBeUndefined() + + expect(reply(replyToWorker).data.error).toBeTruthy() + }) + + // The socket closing before the artifact is written is ordinary at the end of + // a run; the export still has to complete rather than throw on the reply. + it('completes with no reply channel at all', async () => { + const outputDir = await tmpDir() + await expect( + runTraceExport( + { requestId: 'r4', outputDir, sessionId: 'sess-4' }, + { activeRun: () => run() } + ) + ).resolves.toBeUndefined() + expect(await fs.readdir(outputDir)).toEqual(['trace-sess-4.zip']) + }) +}) diff --git a/packages/backend/tests/trace-export.test.ts b/packages/backend/tests/trace-export.test.ts new file mode 100644 index 00000000..9e992a3f --- /dev/null +++ b/packages/backend/tests/trace-export.test.ts @@ -0,0 +1,271 @@ +/** + * Building a trace from the run the backend already accumulates. + * + * The transforms themselves are covered in packages/trace. What matters here is + * the adaptation: that every stream the accumulator holds reaches the artifact, + * that the suite tree becomes test metadata, and that a run with nothing in it + * declines rather than writing an artifact that reads as a run which captured + * nothing. + */ + +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' +import { afterEach, describe, it, expect } from 'vitest' +import { unzipSync, strFromU8 } from 'fflate' +import { + TRACE_EVENT_TYPES, + TRACE_ZIP_ENTRIES, + type Metadata +} from '@wdio/devtools-shared' +import { + exportActiveRunTrace, + hasExportableData, + testMetadataFromNodes +} from '../src/trace-export.js' +import { freshRun } from '../src/baseline/utils.js' +import type { ActiveRun, TimeWindowNode } from '../src/baseline/types.js' + +const dirs: string[] = [] + +afterEach(async () => { + await Promise.all( + dirs.splice(0).map((d) => fs.rm(d, { recursive: true, force: true })) + ) +}) + +async function tmpDir(): Promise { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), 'trace-export-')) + dirs.push(dir) + return dir +} + +function run(overrides: Partial = {}): ActiveRun { + return { + ...freshRun(), + commands: [ + { command: 'url', args: ['https://x/login'], timestamp: 1100 }, + { command: 'click', args: ['#go'], timestamp: 1200 } + ], + ...overrides + } +} + +function node(overrides: Partial = {}): TimeWindowNode { + return { uid: 't1', kind: 'test', childUids: [], ...overrides } +} + +/** Read `trace.trace` out of the written zip as parsed NDJSON lines. */ +async function traceEvents( + zipPath: string +): Promise[]> { + const files = unzipSync(new Uint8Array(await fs.readFile(zipPath))) + const entry = files[TRACE_ZIP_ENTRIES.trace] + if (!entry) { + throw new Error( + `no ${TRACE_ZIP_ENTRIES.trace} in zip; entries: ${Object.keys(files).join(', ')}` + ) + } + return strFromU8(entry) + .split('\n') + .filter(Boolean) + .map((line) => JSON.parse(line)) +} + +describe('testMetadataFromNodes', () => { + it('keeps tests and drops suites — only a test names a tracing group', () => { + const nodes = new Map([ + ['s1', node({ uid: 's1', kind: 'suite', title: 'Login' })], + ['t1', node({ uid: 't1', title: 'logs in', file: '/login.spec.py' })] + ]) + const meta = testMetadataFromNodes(nodes) + expect([...meta.keys()]).toEqual(['t1']) + expect(meta.get('t1')).toEqual({ + title: 'logs in', + specFile: '/login.spec.py' + }) + }) + + it('falls back through fullTitle to the uid so a group is never nameless', () => { + const nodes = new Map([ + ['t1', node({ uid: 't1', fullTitle: 'Login logs in' })], + ['t2', node({ uid: 't2' })] + ]) + const meta = testMetadataFromNodes(nodes) + expect(meta.get('t1')?.title).toBe('Login logs in') + expect(meta.get('t2')?.title).toBe('t2') + }) + + it('carries state when the node reported one, and omits the key otherwise', () => { + const nodes = new Map([ + ['t1', node({ uid: 't1', state: 'failed' })], + ['t2', node({ uid: 't2' })] + ]) + const meta = testMetadataFromNodes(nodes) + expect(meta.get('t1')?.state).toBe('failed') + expect(meta.get('t2')).not.toHaveProperty('state') + }) +}) + +describe('hasExportableData', () => { + it('is true when any of the three primary streams carries something', () => { + expect(hasExportableData(run())).toBe(true) + expect( + hasExportableData( + run({ + commands: [], + consoleLogs: [{ type: 'log', args: ['hi'], timestamp: 1 }] + }) + ) + ).toBe(true) + expect( + hasExportableData( + run({ + commands: [], + networkRequests: [ + { + id: 'r1', + url: 'https://x/a', + method: 'GET', + type: 'other', + startTime: 1, + timestamp: 1 + } + ] + }) + ) + ).toBe(true) + }) + + // Mutations and sources alone are not a run: the collector anchors a DOM on + // page load, so a session that opened a page and did nothing else would + // otherwise produce an artifact with no actions in it. + it('is false for an empty run and for one carrying only page-side noise', () => { + expect(hasExportableData(run({ commands: [] }))).toBe(false) + expect( + hasExportableData( + run({ + commands: [], + mutations: [ + { + type: 'childList', + addedNodes: [], + removedNodes: [], + timestamp: 1 + } + ], + sources: { '/a.py': 'x = 1' } + }) + ) + ).toBe(false) + }) +}) + +describe('exportActiveRunTrace', () => { + it('writes a zip whose actions are the accumulated commands', async () => { + const outputDir = await tmpDir() + const zipPath = await exportActiveRunTrace(run(), { + outputDir, + sessionId: 'sess-1' + }) + + expect(zipPath).toBe(path.join(outputDir, 'trace-sess-1.zip')) + const methods = (await traceEvents(zipPath)) + .filter((e) => e.type === 'before') + .map((e) => e.method) + // `url` normalizes to `navigate` through shared's action map — the + // artifact records what the action IS, not what the adapter called it. + expect(methods).toContain('navigate') + expect(methods).toContain('click') + }) + + it('carries console, network and sources into the artifact', async () => { + const outputDir = await tmpDir() + const zipPath = await exportActiveRunTrace( + run({ + consoleLogs: [ + { type: 'error', args: ['boom'], timestamp: 1150, source: 'browser' } + ], + networkRequests: [ + { + id: 'r1', + url: 'https://x/app.css', + method: 'GET', + type: 'stylesheet', + status: 200, + startTime: 1100, + timestamp: 1120 + } + ], + sources: { '/login.spec.py': 'def test_login(): pass' } + }), + { outputDir, sessionId: 'sess-2' } + ) + + const files = unzipSync(new Uint8Array(await fs.readFile(zipPath))) + const names = Object.keys(files) + expect(names).toContain(TRACE_ZIP_ENTRIES.network) + expect(strFromU8(files[TRACE_ZIP_ENTRIES.network]!)).toContain('app.css') + const trace = strFromU8(files[TRACE_ZIP_ENTRIES.trace]!) + expect(trace).toContain('boom') + }) + + it("names tracing groups from the run's suite tree", async () => { + const outputDir = await tmpDir() + const zipPath = await exportActiveRunTrace( + run({ + commands: [ + { command: 'click', args: ['#go'], timestamp: 1200, testUid: 't1' } + ], + nodes: new Map([ + ['t1', node({ uid: 't1', title: 'logs in', file: '/login.spec.py' })] + ]) + }), + { outputDir, sessionId: 'sess-3' } + ) + + const groups = (await traceEvents(zipPath)).filter( + (e) => e.method === 'tracingGroup' + ) + expect(JSON.stringify(groups)).toContain('logs in') + }) + + it('puts the metadata frame the worker sent into the artifact', async () => { + const outputDir = await tmpDir() + const metadata = { + sessionId: 'sess-4', + capabilities: { browserName: 'chrome' } + } as unknown as Metadata + const zipPath = await exportActiveRunTrace(run({ metadata }), { + outputDir, + sessionId: 'sess-4' + }) + + const options = (await traceEvents(zipPath)).filter( + (e) => e.type === TRACE_EVENT_TYPES.contextOptions + ) + expect(options).toHaveLength(1) + expect(JSON.stringify(options[0])).toContain('chrome') + }) + + it('honours fileStem so a per-test slice can name its own artifact', async () => { + const outputDir = await tmpDir() + const zipPath = await exportActiveRunTrace(run(), { + outputDir, + sessionId: 'sess-5', + fileStem: 'trace' + }) + expect(path.basename(zipPath)).toBe('trace.zip') + }) + + it('refuses a run with nothing captured rather than writing an empty artifact', async () => { + const outputDir = await tmpDir() + await expect( + exportActiveRunTrace(run({ commands: [] }), { + outputDir, + sessionId: 'sess-6' + }) + ).rejects.toThrow(/nothing captured/) + expect(await fs.readdir(outputDir)).toEqual([]) + }) +}) diff --git a/packages/backend/tests/worker-message-handler.test.ts b/packages/backend/tests/worker-message-handler.test.ts index 9b171d99..9584190f 100644 --- a/packages/backend/tests/worker-message-handler.test.ts +++ b/packages/backend/tests/worker-message-handler.test.ts @@ -1,5 +1,9 @@ +import fs from 'node:fs/promises' +import os from 'node:os' +import path from 'node:path' import { describe, it, expect, vi } from 'vitest' -import { WS_SCOPE } from '@wdio/devtools-shared' +import { TRACE_EXPORT_SCOPE, WS_SCOPE } from '@wdio/devtools-shared' +import { freshRun } from '../src/baseline/utils.js' import { createWorkerMessageHandler, type WorkerMessageContext @@ -156,3 +160,35 @@ describe('createWorkerMessageHandler — pass-through behavior', () => { expect(baselineStore.recordEvent).not.toHaveBeenCalled() }) }) + +// An export request is addressed to the backend, not to the dashboard. It has +// to be claimed like the other control scopes — forwarded, it would reach every +// open tab; accumulated, it would land in the run as a bogus event. +describe('createWorkerMessageHandler — traceExport', () => { + it('claims the frame: neither broadcast nor teed into the accumulator', async () => { + const outputDir = await fs.mkdtemp(path.join(os.tmpdir(), 'wmh-export-')) + try { + const { ctx, broadcastToClients, baselineStore } = makeCtx() + const activeRun = vi.fn(() => ({ + ...freshRun(), + commands: [{ command: 'click', args: ['#go'], timestamp: 1 }] + })) + ;(baselineStore as unknown as { activeRun: unknown }).activeRun = + activeRun + const handler = createWorkerMessageHandler(ctx) + + handler( + buf({ + scope: TRACE_EXPORT_SCOPE.request, + data: { requestId: 'r1', outputDir, sessionId: 's1' } + }) + ) + + expect(broadcastToClients).not.toHaveBeenCalled() + expect(baselineStore.recordEvent).not.toHaveBeenCalled() + expect(activeRun).toHaveBeenCalled() + } finally { + await fs.rm(outputDir, { recursive: true, force: true }) + } + }) +}) diff --git a/packages/core/package.json b/packages/core/package.json index 7e260900..03c62304 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -44,14 +44,12 @@ "license": "MIT", "devDependencies": { "@types/ws": "^8.18.1", - "@types/yazl": "^3.3.1", "@wdio/devtools-script": "workspace:*", "@wdio/devtools-shared": "workspace:^", + "@wdio/devtools-trace": "workspace:^", "@xmldom/xmldom": "^0.9.8", - "fflate": "^0.8.2", "stacktrace-parser": "^0.1.11", "ws": "^8.21.0", - "xpath": "^0.0.34", - "yazl": "^3.3.1" + "xpath": "^0.0.34" } } diff --git a/packages/core/src/assert-patcher.ts b/packages/core/src/assert-patcher.ts index 353c22b9..64457c0d 100644 --- a/packages/core/src/assert-patcher.ts +++ b/packages/core/src/assert-patcher.ts @@ -6,7 +6,7 @@ import { } from '@wdio/devtools-shared' import { getCallSourceFromStack, isAssertFromUserCode } from './stack.js' import { toError } from './error.js' -import { stripAnsi } from './console.js' +import { stripAnsi } from '@wdio/devtools-shared' export { TRACKED_ASSERT_METHODS } diff --git a/packages/core/src/bidi.ts b/packages/core/src/bidi.ts index 52786e7e..831449f2 100644 --- a/packages/core/src/bidi.ts +++ b/packages/core/src/bidi.ts @@ -8,7 +8,7 @@ import { LOG_SOURCES, chromeLogLevelToLogLevel, type LogSource -} from './console.js' +} from '@wdio/devtools-shared' import { errorMessage } from './error.js' import { getRequestType } from './net.js' diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index 345c9373..f27c20bc 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -1,7 +1,17 @@ // Framework-agnostic capture/reporter logic shared by @wdio/devtools-* // adapters. See ARCHITECTURE.md §2 and CLAUDE.md §2.2. -export * from './action-mapping.js' +// `action-mapping` and `console` moved to `shared` — the trace transforms need +// them and must not depend on adapter logic. Re-exported here, and only these +// names, so adapters keep importing them from `core` unchanged. +export { + ASSERT_ACTION_CLASS, + FILL_METHODS, + formatActionTitle, + mapAssertCommand, + mapCommandToAction, + type TraceAction +} from '@wdio/devtools-shared' export * from './action-snapshot.js' export * from './artifact-naming.js' export * from './artifacts-manifest.js' @@ -14,22 +24,40 @@ export * from './assert-patcher.js' export * from './element-snapshot.js' export * from './element-scripts.js' export * from './element-types.js' -export * from './sha1.js' -export * from './trace-action-events.js' -export * from './trace-console.js' -export * from './trace-hierarchy.js' -export * from './trace-exporter.js' +// The trace transforms moved to `@wdio/devtools-trace` so the backend can +// reach them without importing core. Re-exported per module rather than as one +// `export *` so core's surface stays exactly what it was — `trace-transcript` +// is deliberately absent here, as it was before the move. +export * from '@wdio/devtools-trace/sha1' +export * from '@wdio/devtools-trace/trace-action-events' +export * from '@wdio/devtools-trace/trace-console' +export * from '@wdio/devtools-trace/trace-hierarchy' +export * from '@wdio/devtools-trace/trace-exporter' export * from './trace-finalizer.js' -export * from './trace-frame-snapshots.js' +export * from '@wdio/devtools-trace/trace-frame-snapshots' export * from './trace-retention.js' -export * from './trace-sources.js' -export * from './trace-har.js' -export * from './trace-mutations.js' -export * from './trace-snapshots.js' -export * from './trace-zip-writer.js' +export * from '@wdio/devtools-trace/trace-sources' +export * from '@wdio/devtools-trace/trace-har' +export * from '@wdio/devtools-trace/trace-mutations' +export * from '@wdio/devtools-trace/trace-snapshots' +export * from '@wdio/devtools-trace/trace-zip-writer' export * from './bidi.js' export * from './bidi-preload.js' -export * from './console.js' +export { + ANSI_REGEX, + CONSOLE_METHODS, + ERROR_INDICATORS, + LOG_LEVEL_PATTERNS, + LOG_SOURCES, + SPINNER_RE, + chromeLogLevelToLogLevel, + createConsoleLogEntry, + detectLogLevel, + isInternalStreamLine, + mapChromeBrowserLogs, + stripAnsi, + type LogSource +} from '@wdio/devtools-shared' export * from './uid.js' export * from './net.js' export * from './request-type.js' @@ -44,7 +72,7 @@ export * from './read-value-locators.js' export * from './retry-tracker.js' export * from './run-id.js' export * from './screencast.js' -export * from './screencast-trace.js' +export * from '@wdio/devtools-trace/screencast-trace' export * from './script-loader.js' export * from './session-capturer.js' export * from './spec-trace-helpers.js' diff --git a/packages/core/src/session-capturer.ts b/packages/core/src/session-capturer.ts index c6aa219e..56b889bb 100644 --- a/packages/core/src/session-capturer.ts +++ b/packages/core/src/session-capturer.ts @@ -11,9 +11,9 @@ import type { TraceMutation } from '@wdio/devtools-shared' import { WORKER_WS_QUERY, WS_PATHS, WS_SCOPE } from '@wdio/devtools-shared' -import { mapCommandToAction } from './action-mapping.js' +import { mapCommandToAction } from '@wdio/devtools-shared' import { resolveRunId } from './run-id.js' -import { reattributeDomAnchors } from './trace-mutations.js' +import { reattributeDomAnchors } from '@wdio/devtools-trace/trace-mutations' import { CONSOLE_METHODS, LOG_SOURCES, @@ -22,7 +22,7 @@ import { detectLogLevel, isInternalStreamLine, stripAnsi -} from './console.js' +} from '@wdio/devtools-shared' import { TerminalLineThrottle } from './terminal-throttle.js' /** diff --git a/packages/core/src/spec-trace-helpers.ts b/packages/core/src/spec-trace-helpers.ts index 01cca374..51108c8b 100644 --- a/packages/core/src/spec-trace-helpers.ts +++ b/packages/core/src/spec-trace-helpers.ts @@ -15,8 +15,8 @@ import type { TraceFormat, TraceGranularity } from '@wdio/devtools-shared' -import type { TraceCapturer } from './trace-exporter.js' -import { writeTraceZip } from './trace-exporter.js' +import type { TraceCapturer } from '@wdio/devtools-trace/trace-exporter' +import { writeTraceZip } from '@wdio/devtools-trace/trace-exporter' import { deterministicUid, isStepUidOf } from './uid.js' import { trimChar } from './artifact-naming.js' diff --git a/packages/core/src/trace-finalizer.ts b/packages/core/src/trace-finalizer.ts index 84d54899..4630fbba 100644 --- a/packages/core/src/trace-finalizer.ts +++ b/packages/core/src/trace-finalizer.ts @@ -29,7 +29,10 @@ import { } from './spec-trace-helpers.js' import { shouldRetainTrace, type TestOutcome } from './trace-retention.js' import type { RetryOutcomeView } from './attempt-tracker.js' -import { writeTraceZip, type TraceCapturer } from './trace-exporter.js' +import { + writeTraceZip, + type TraceCapturer +} from '@wdio/devtools-trace/trace-exporter' /** One artifact produced (or, when `retained` is false, decided-against) by a * trace-mode run — a trace slice, a screencast video, or a per-test diff --git a/packages/core/tests/spec-trace-helpers.test.ts b/packages/core/tests/spec-trace-helpers.test.ts index 52f3d92e..474549aa 100644 --- a/packages/core/tests/spec-trace-helpers.test.ts +++ b/packages/core/tests/spec-trace-helpers.test.ts @@ -3,6 +3,7 @@ import os from 'node:os' import path from 'node:path' import { afterEach, beforeEach, describe, it, expect } from 'vitest' import { + buildGroupPath, buildSpecCapturer, buildSpecSessionId, buildTestSliceFolder, @@ -21,7 +22,11 @@ import { type TraceCapturer, type WriteSpecTraceInput } from '@wdio/devtools-core' -import { TraceType, type TestMetadataMap } from '@wdio/devtools-shared' +import { + TraceType, + type CommandLog, + type TestMetadataMap +} from '@wdio/devtools-shared' function capturer(): TraceCapturer { const cmd = (i: number) => ({ @@ -165,6 +170,35 @@ describe('filterTestMetadataByUid', () => { stepMetadataUid('u1', 1) ]) }) + + // The chain a per-test trace slice actually goes through, and the reason this + // one test spans core and trace: the two halves were individually defensible — + // the filter narrowed to one test, the path fell back to the uid — and only + // their composition showed the defect, every Gherkin step rendering as + // `stable-…:step:1` in the viewer. + it('names steps from a per-test-filtered metadata map', () => { + const all: TestMetadataMap = new Map([ + ['sc1', { title: 'Scenario', specFile: '/login.feature' }], + [ + stepMetadataUid('sc1', 1), + { title: 'When I log in', specFile: '/login.feature' } + ], + ['sc2', { title: 'Other', specFile: '/login.feature' }] + ]) + const command: CommandLog = { + command: 'click', + args: ['#go'], + timestamp: 1, + testUid: 'sc1', + stepUid: stepMetadataUid('sc1', 1) + } + expect( + buildGroupPath(command, filterTestMetadataByUid(all, 'sc1')) + ).toEqual([ + { uid: 'sc1', title: 'Scenario' }, + { uid: stepMetadataUid('sc1', 1), title: 'When I log in' } + ]) + }) }) describe('buildTestSliceSessionId', () => { diff --git a/packages/service/vite.config.ts b/packages/service/vite.config.ts index 05afcc84..730f53ad 100644 --- a/packages/service/vite.config.ts +++ b/packages/service/vite.config.ts @@ -5,6 +5,9 @@ import { defineConfig } from 'vite' const __dirname = url.fileURLToPath(new URL('.', import.meta.url)) +/** Workspace-internal `@wdio/devtools-*` packages, by directory name. */ +const PRIVATE_WORKSPACE_PACKAGES = ['core', 'shared', 'trace'] + // https://vitejs.dev/config/ export default defineConfig({ optimizeDeps: { @@ -33,20 +36,21 @@ export default defineConfig({ output: { entryFileNames: '[name].js' }, - // Inline private workspace packages (@wdio/devtools-core, - // @wdio/devtools-shared) — they are not published, so the dist must - // not contain runtime `import` statements for them. The `id` here can - // be EITHER the unresolved package name OR an already-resolved absolute - // path (vite resolves workspace symlinks before calling this), so we - // check for both forms. See CLAUDE.md §2.6. + // Inline private workspace packages — they are not published, so the + // dist must not contain runtime `import` statements for them. The `id` + // here can be EITHER the unresolved package name OR an already-resolved + // absolute path (vite resolves workspace symlinks before calling this), + // so both forms are checked. A package missing from this list is + // silently externalized and the dist then dies at install time with + // ERR_MODULE_NOT_FOUND, so it is a list rather than a chain of ors. + // See CLAUDE.md §2.6. external: (id) => { - const isPrivateWorkspaceDep = - id === '@wdio/devtools-core' || - id === '@wdio/devtools-shared' || - id.startsWith('@wdio/devtools-core/') || - id.startsWith('@wdio/devtools-shared/') || - id.includes('/packages/core/') || - id.includes('/packages/shared/') + const isPrivateWorkspaceDep = PRIVATE_WORKSPACE_PACKAGES.some( + (name) => + id === `@wdio/devtools-${name}` || + id.startsWith(`@wdio/devtools-${name}/`) || + id.includes(`/packages/${name}/`) + ) if (isPrivateWorkspaceDep) { return false } diff --git a/packages/core/src/action-mapping.ts b/packages/shared/src/action-mapping.ts similarity index 90% rename from packages/core/src/action-mapping.ts rename to packages/shared/src/action-mapping.ts index eb31c2b2..2f739494 100644 --- a/packages/core/src/action-mapping.ts +++ b/packages/shared/src/action-mapping.ts @@ -1,6 +1,6 @@ -// Helpers over the trace action vocabulary. ACTION_MAP itself lives in -// @wdio/devtools-shared so the reader (backend) can derive its inverse from the -// same source. Ported from Vince Graics' PR #209 (`@wdio/tracing-service`); the +// Helpers over the trace action vocabulary, beside the ACTION_MAP they read — +// here rather than in `core` because both the capture side and the trace +// transforms need them, and the transforms must not depend on adapter logic. Ported from Vince Graics' PR #209 (`@wdio/tracing-service`); the // existing devtools UI uses its own denylist (`INTERNAL_COMMANDS`) — this map // is for the trace.zip exporter to filter + rename in one step. @@ -9,7 +9,7 @@ import { ASSERT_ACTION_CLASS, mapAssertCommand, type TraceAction -} from '@wdio/devtools-shared' +} from './trace-actions.js' export type { TraceAction } export { ASSERT_ACTION_CLASS, mapAssertCommand } diff --git a/packages/core/src/console.ts b/packages/shared/src/console.ts similarity index 97% rename from packages/core/src/console.ts rename to packages/shared/src/console.ts index 96d572ee..1bbfb1fa 100644 --- a/packages/core/src/console.ts +++ b/packages/shared/src/console.ts @@ -1,4 +1,4 @@ -import type { ConsoleLog, LogLevel, LogSource } from '@wdio/devtools-shared' +import type { ConsoleLog, LogLevel, LogSource } from './types.js' /** * Console methods we intercept to forward test/runner-process output into the @@ -75,8 +75,6 @@ export const LOG_SOURCES = { TERMINAL: 'terminal' } as const satisfies Record -export type { LogSource } from '@wdio/devtools-shared' - /** * Classify a line of unstructured terminal output by scanning for log-level * keywords. Falls back to `'log'` when no pattern matches. diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index 2baf14c6..176fa159 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -1,7 +1,9 @@ // Single source of truth for types, constants, and HTTP/WS contracts shared // across @wdio/devtools-* packages. See ARCHITECTURE.md §2 and CLAUDE.md §2.1. +export * from './action-mapping.js' export * from './baseline.js' +export * from './console.js' export * from './collector.js' export * from './files.js' export * from './locator-dialect.js' @@ -10,6 +12,7 @@ export * from './runner.js' export * from './snapshot-format.js' export * from './timing.js' export * from './trace-actions.js' +export * from './trace-export.js' export * from './trace-file.js' export * from './trace-player.js' export * from './types.js' diff --git a/packages/shared/src/trace-export.ts b/packages/shared/src/trace-export.ts new file mode 100644 index 00000000..6a12622a --- /dev/null +++ b/packages/shared/src/trace-export.ts @@ -0,0 +1,54 @@ +/** + * Worker↔backend contract for building a trace artifact server-side. + * + * An adapter that can run the trace transforms in-process never uses this — + * the three JS adapters call them directly, which is what keeps trace mode + * working with no backend at all. It exists for an adapter that streams but + * cannot transform, today the Python one: it already sends every frame the + * exporter needs, so the backend can assemble the artifact from the run it is + * accumulating anyway. + * + * This travels over the WORKER socket rather than an HTTP route on purpose. + * `outputDir` is an absolute path the backend writes to, and the worker socket + * is the adapter's own channel — the same one that already hands over absolute + * video paths for the video registry. An HTTP endpoint taking a path would be + * reachable from any page the browser has open. + */ + +import type { TraceFormat } from './types.js' + +export const TRACE_EXPORT_SCOPE = { + /** Worker → backend: build an artifact from the accumulated run. */ + request: 'traceExport', + /** Backend → worker: where it landed, or why it did not. */ + result: 'traceExported' +} as const + +export type TraceExportScope = + (typeof TRACE_EXPORT_SCOPE)[keyof typeof TRACE_EXPORT_SCOPE] + +/** Payload sent under {@link TRACE_EXPORT_SCOPE.request}. */ +export interface TraceExportRequest { + /** Correlates the result frame. One export may be in flight per request. */ + requestId: string + /** Absolute directory to write into. The adapter decides where its + * artifacts live, exactly as it does when it exports in-process. */ + outputDir: string + /** Names the artifact and identifies the session inside it. */ + sessionId: string + /** `zip` (default) or an unpacked directory. */ + format?: TraceFormat + /** Artifact base name. Defaults to `trace-`. */ + fileStem?: string +} + +/** Payload sent under {@link TRACE_EXPORT_SCOPE.result}. Exactly one of + * `path` / `error` is set. */ +export interface TraceExportResult { + requestId: string + /** Absolute path of the artifact written. */ + path?: string + /** Why nothing was written. The adapter logs this; a failed export must not + * fail the user's test run. */ + error?: string +} diff --git a/packages/core/tests/action-mapping.test.ts b/packages/shared/tests/action-mapping.test.ts similarity index 100% rename from packages/core/tests/action-mapping.test.ts rename to packages/shared/tests/action-mapping.test.ts diff --git a/packages/core/tests/console-helpers.test.ts b/packages/shared/tests/console-helpers.test.ts similarity index 100% rename from packages/core/tests/console-helpers.test.ts rename to packages/shared/tests/console-helpers.test.ts diff --git a/packages/trace/package.json b/packages/trace/package.json new file mode 100644 index 00000000..f6d62129 --- /dev/null +++ b/packages/trace/package.json @@ -0,0 +1,35 @@ +{ + "name": "@wdio/devtools-trace", + "version": "1.0.0", + "private": true, + "author": "Vishnu Vardhan", + "description": "Trace-format transforms — turns captured events into the trace zip. Split out of core so the backend can build a trace for adapters that cannot (CLAUDE.md §2.2 bars backend from core). Workspace-internal, never published — code is inlined into each consuming package at build time.", + "repository": { + "type": "git", + "url": "git+https://github.com/webdriverio/devtools.git", + "directory": "packages/trace" + }, + "type": "module", + "sideEffects": false, + "exports": { + ".": { + "types": "./src/index.ts", + "default": "./src/index.ts" + }, + "./*": { + "types": "./src/*.ts", + "default": "./src/*.ts" + } + }, + "types": "./src/index.ts", + "scripts": { + "lint": "eslint ." + }, + "license": "MIT", + "devDependencies": { + "@types/yazl": "^3.3.1", + "@wdio/devtools-shared": "workspace:^", + "fflate": "^0.8.2", + "yazl": "^3.3.1" + } +} diff --git a/packages/trace/src/index.ts b/packages/trace/src/index.ts new file mode 100644 index 00000000..22b9c385 --- /dev/null +++ b/packages/trace/src/index.ts @@ -0,0 +1,24 @@ +// Trace-format transforms: captured events in, trace zip out. See +// ARCHITECTURE.md §2 and CLAUDE.md §2.2. +// +// Split out of `core` because two layers need it and `core` is reachable from +// only one of them: adapters build their own trace, while the backend builds +// one on behalf of an adapter that cannot (the Python adapter ships no Node), +// and CLAUDE.md §2.2 bars the backend from importing `core`. Everything here +// is a pure transform over shared types — no framework API, no driver, no +// capture. Adapter-side policy and orchestration stay in `core` +// (`trace-finalizer`, `spec-trace-helpers`, `trace-retention`). + +export * from './sha1.js' +export * from './screencast-trace.js' +export * from './trace-action-events.js' +export * from './trace-console.js' +export * from './trace-exporter.js' +export * from './trace-frame-snapshots.js' +export * from './trace-har.js' +export * from './trace-hierarchy.js' +export * from './trace-mutations.js' +export * from './trace-snapshots.js' +export * from './trace-sources.js' +export * from './trace-transcript.js' +export * from './trace-zip-writer.js' diff --git a/packages/core/src/screencast-trace.ts b/packages/trace/src/screencast-trace.ts similarity index 100% rename from packages/core/src/screencast-trace.ts rename to packages/trace/src/screencast-trace.ts diff --git a/packages/core/src/sha1.ts b/packages/trace/src/sha1.ts similarity index 100% rename from packages/core/src/sha1.ts rename to packages/trace/src/sha1.ts diff --git a/packages/core/src/trace-action-events.ts b/packages/trace/src/trace-action-events.ts similarity index 99% rename from packages/core/src/trace-action-events.ts rename to packages/trace/src/trace-action-events.ts index 3f8119e0..dc16777d 100644 --- a/packages/core/src/trace-action-events.ts +++ b/packages/trace/src/trace-action-events.ts @@ -13,7 +13,7 @@ import { mapCommandToAction, FILL_METHODS, type TraceAction -} from './action-mapping.js' +} from '@wdio/devtools-shared' import { callSourceToStack, type StackFrame } from './trace-sources.js' import type { FrameSnapshotIndex } from './trace-frame-snapshots.js' import { buildGroupPath } from './trace-hierarchy.js' diff --git a/packages/core/src/trace-console.ts b/packages/trace/src/trace-console.ts similarity index 100% rename from packages/core/src/trace-console.ts rename to packages/trace/src/trace-console.ts diff --git a/packages/core/src/trace-exporter.ts b/packages/trace/src/trace-exporter.ts similarity index 99% rename from packages/core/src/trace-exporter.ts rename to packages/trace/src/trace-exporter.ts index 41c1c3b9..083a5745 100644 --- a/packages/core/src/trace-exporter.ts +++ b/packages/trace/src/trace-exporter.ts @@ -17,7 +17,7 @@ import type { TraceLog, TraceMutation } from '@wdio/devtools-shared' -import { mapCommandToAction } from './action-mapping.js' +import { mapCommandToAction } from '@wdio/devtools-shared' import { buildConsoleEvents, type ConsoleEvent, diff --git a/packages/core/src/trace-frame-snapshots.ts b/packages/trace/src/trace-frame-snapshots.ts similarity index 100% rename from packages/core/src/trace-frame-snapshots.ts rename to packages/trace/src/trace-frame-snapshots.ts diff --git a/packages/core/src/trace-har.ts b/packages/trace/src/trace-har.ts similarity index 100% rename from packages/core/src/trace-har.ts rename to packages/trace/src/trace-har.ts diff --git a/packages/core/src/trace-hierarchy.ts b/packages/trace/src/trace-hierarchy.ts similarity index 100% rename from packages/core/src/trace-hierarchy.ts rename to packages/trace/src/trace-hierarchy.ts diff --git a/packages/core/src/trace-mutations.ts b/packages/trace/src/trace-mutations.ts similarity index 100% rename from packages/core/src/trace-mutations.ts rename to packages/trace/src/trace-mutations.ts diff --git a/packages/core/src/trace-snapshots.ts b/packages/trace/src/trace-snapshots.ts similarity index 100% rename from packages/core/src/trace-snapshots.ts rename to packages/trace/src/trace-snapshots.ts diff --git a/packages/core/src/trace-sources.ts b/packages/trace/src/trace-sources.ts similarity index 100% rename from packages/core/src/trace-sources.ts rename to packages/trace/src/trace-sources.ts diff --git a/packages/core/src/trace-transcript.ts b/packages/trace/src/trace-transcript.ts similarity index 97% rename from packages/core/src/trace-transcript.ts rename to packages/trace/src/trace-transcript.ts index d3a5161a..0d56c728 100644 --- a/packages/core/src/trace-transcript.ts +++ b/packages/trace/src/trace-transcript.ts @@ -7,8 +7,8 @@ import { mapCommandToAction, FILL_METHODS, type TraceAction -} from './action-mapping.js' -import { stripAnsi } from './console.js' +} from '@wdio/devtools-shared' +import { stripAnsi } from '@wdio/devtools-shared' /** Render `text` as one numbered markdown list item, indenting every line after * the first to the marker's content column (`'1. '` → 3, `'10. '` → 4 — an diff --git a/packages/core/src/trace-zip-writer.ts b/packages/trace/src/trace-zip-writer.ts similarity index 100% rename from packages/core/src/trace-zip-writer.ts rename to packages/trace/src/trace-zip-writer.ts diff --git a/packages/core/tests/screencast-trace.test.ts b/packages/trace/tests/screencast-trace.test.ts similarity index 100% rename from packages/core/tests/screencast-trace.test.ts rename to packages/trace/tests/screencast-trace.test.ts diff --git a/packages/core/tests/trace-assertions.test.ts b/packages/trace/tests/trace-assertions.test.ts similarity index 99% rename from packages/core/tests/trace-assertions.test.ts rename to packages/trace/tests/trace-assertions.test.ts index 1f32f697..c25eea7a 100644 --- a/packages/core/tests/trace-assertions.test.ts +++ b/packages/trace/tests/trace-assertions.test.ts @@ -4,7 +4,7 @@ import { mapAssertCommand, type CommandLog } from '@wdio/devtools-shared' -import { formatActionTitle, mapCommandToAction } from '../src/action-mapping.js' +import { formatActionTitle, mapCommandToAction } from '@wdio/devtools-shared' import { buildActionEvents, type AfterEvent, diff --git a/packages/core/tests/trace-console.test.ts b/packages/trace/tests/trace-console.test.ts similarity index 98% rename from packages/core/tests/trace-console.test.ts rename to packages/trace/tests/trace-console.test.ts index 5bb6ee59..b87a7ed7 100644 --- a/packages/core/tests/trace-console.test.ts +++ b/packages/trace/tests/trace-console.test.ts @@ -1,5 +1,5 @@ import { describe, it, expect } from 'vitest' -import { buildConsoleEvents } from '@wdio/devtools-core' +import { buildConsoleEvents } from '@wdio/devtools-trace' import type { ConsoleLog } from '@wdio/devtools-shared' const WALL_TIME = 1000 diff --git a/packages/core/tests/trace-exporter.test.ts b/packages/trace/tests/trace-exporter.test.ts similarity index 99% rename from packages/core/tests/trace-exporter.test.ts rename to packages/trace/tests/trace-exporter.test.ts index f91a3169..7c642e42 100644 --- a/packages/core/tests/trace-exporter.test.ts +++ b/packages/trace/tests/trace-exporter.test.ts @@ -10,7 +10,7 @@ import { type AfterEvent, type BeforeEvent, type TraceCapturer -} from '@wdio/devtools-core' +} from '@wdio/devtools-trace' import { TraceType, type CommandLog } from '@wdio/devtools-shared' const isBefore = (event: ActionEvent): event is BeforeEvent => diff --git a/packages/core/tests/trace-frame-snapshots.test.ts b/packages/trace/tests/trace-frame-snapshots.test.ts similarity index 99% rename from packages/core/tests/trace-frame-snapshots.test.ts rename to packages/trace/tests/trace-frame-snapshots.test.ts index b5c9b06a..26a87eac 100644 --- a/packages/core/tests/trace-frame-snapshots.test.ts +++ b/packages/trace/tests/trace-frame-snapshots.test.ts @@ -5,7 +5,7 @@ import { FrameSnapshotIndex, type AfterEvent, type FrameSnapshotRef -} from '@wdio/devtools-core' +} from '@wdio/devtools-trace' import type { ActionSnapshot, CommandLog } from '@wdio/devtools-shared' function snap(overrides: Partial = {}): ActionSnapshot { diff --git a/packages/core/tests/trace-hierarchy.test.ts b/packages/trace/tests/trace-hierarchy.test.ts similarity index 76% rename from packages/core/tests/trace-hierarchy.test.ts rename to packages/trace/tests/trace-hierarchy.test.ts index 1f162265..2afa6d2f 100644 --- a/packages/core/tests/trace-hierarchy.test.ts +++ b/packages/trace/tests/trace-hierarchy.test.ts @@ -1,9 +1,5 @@ import { describe, it, expect } from 'vitest' -import { - buildGroupPath, - filterTestMetadataByUid, - stepMetadataUid -} from '@wdio/devtools-core' +import { buildGroupPath } from '@wdio/devtools-trace' import type { CommandLog, TestMetadataMap } from '@wdio/devtools-shared' function cmd(overrides: Partial = {}): CommandLog { @@ -65,30 +61,9 @@ describe('buildGroupPath', () => { ]) }) - // The chain a per-test trace slice actually goes through. Asserted end to end - // because the two halves were individually defensible — the filter narrowed to - // one test, the path fell back to the uid — and only their composition showed - // the defect: every Gherkin step rendered as `stable-…:step:1` in the viewer. - it('names steps from a per-test-filtered metadata map', () => { - const all: TestMetadataMap = new Map([ - ['sc1', { title: 'Scenario', specFile: '/login.feature' }], - [ - stepMetadataUid('sc1', 1), - { title: 'When I log in', specFile: '/login.feature' } - ], - ['sc2', { title: 'Other', specFile: '/login.feature' }] - ]) - const sliceMeta = filterTestMetadataByUid(all, 'sc1') - expect( - buildGroupPath( - cmd({ testUid: 'sc1', stepUid: stepMetadataUid('sc1', 1) }), - sliceMeta - ) - ).toEqual([ - { uid: 'sc1', title: 'Scenario' }, - { uid: stepMetadataUid('sc1', 1), title: 'When I log in' } - ]) - }) + // The composition of this with core's per-test metadata filter is asserted in + // core's spec-trace-helpers.test.ts — it spans both packages, so it can only + // live in the one that sees both. }) // Some runners report a test's FULL title (` `), so nesting it diff --git a/packages/core/tests/trace-mutations.test.ts b/packages/trace/tests/trace-mutations.test.ts similarity index 99% rename from packages/core/tests/trace-mutations.test.ts rename to packages/trace/tests/trace-mutations.test.ts index fdbf6141..a1fd2fdf 100644 --- a/packages/core/tests/trace-mutations.test.ts +++ b/packages/trace/tests/trace-mutations.test.ts @@ -2,7 +2,7 @@ import { describe, it, expect } from 'vitest' import { buildMutationsNdjson, reattributeDomAnchors -} from '@wdio/devtools-core' +} from '@wdio/devtools-trace' import { isMutationsTruncationMarker, type TraceMutation diff --git a/packages/core/tests/trace-network-bodies.test.ts b/packages/trace/tests/trace-network-bodies.test.ts similarity index 100% rename from packages/core/tests/trace-network-bodies.test.ts rename to packages/trace/tests/trace-network-bodies.test.ts diff --git a/packages/core/tests/trace-snapshots.test.ts b/packages/trace/tests/trace-snapshots.test.ts similarity index 99% rename from packages/core/tests/trace-snapshots.test.ts rename to packages/trace/tests/trace-snapshots.test.ts index 4a95ddbe..3339d7f4 100644 --- a/packages/core/tests/trace-snapshots.test.ts +++ b/packages/trace/tests/trace-snapshots.test.ts @@ -6,7 +6,7 @@ import { upsertRichestSnapshot, writeTraceZip, type TraceCapturer -} from '@wdio/devtools-core' +} from '@wdio/devtools-trace' import { TraceType, type ActionSnapshot } from '@wdio/devtools-shared' const BLANK = 'AA' diff --git a/packages/core/tests/trace-sources.test.ts b/packages/trace/tests/trace-sources.test.ts similarity index 98% rename from packages/core/tests/trace-sources.test.ts rename to packages/trace/tests/trace-sources.test.ts index 90b6640d..57cc04ef 100644 --- a/packages/core/tests/trace-sources.test.ts +++ b/packages/trace/tests/trace-sources.test.ts @@ -4,7 +4,7 @@ import { callSourceToStack, sha1Hex, sourceResourceName -} from '@wdio/devtools-core' +} from '@wdio/devtools-trace' describe('callSourceToStack', () => { it('parses : into a single frame', () => { diff --git a/packages/core/tests/trace-transcript.test.ts b/packages/trace/tests/trace-transcript.test.ts similarity index 100% rename from packages/core/tests/trace-transcript.test.ts rename to packages/trace/tests/trace-transcript.test.ts diff --git a/packages/core/tests/trace-zip-writer.test.ts b/packages/trace/tests/trace-zip-writer.test.ts similarity index 99% rename from packages/core/tests/trace-zip-writer.test.ts rename to packages/trace/tests/trace-zip-writer.test.ts index 71d76ab0..95614430 100644 --- a/packages/core/tests/trace-zip-writer.test.ts +++ b/packages/trace/tests/trace-zip-writer.test.ts @@ -3,7 +3,7 @@ // writer's output to `TRACE_ZIP_ENTRIES` in shared, so renaming an entry on one // side fails here rather than when someone opens a real trace. -import { buildTraceZip, type TraceZipInputs } from '@wdio/devtools-core' +import { buildTraceZip, type TraceZipInputs } from '@wdio/devtools-trace' import { TRACE_ZIP_ENTRIES } from '@wdio/devtools-shared' import { unzipSync } from 'fflate' import { describe, expect, it } from 'vitest' diff --git a/packages/trace/tsconfig.json b/packages/trace/tsconfig.json new file mode 100644 index 00000000..a5cb75c5 --- /dev/null +++ b/packages/trace/tsconfig.json @@ -0,0 +1,4 @@ +{ + "extends": "../../tsconfig.json", + "include": ["src/**/*.ts"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 8b1feb3d..d10043d1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -310,6 +310,9 @@ importers: ws: specifier: ^8.21.0 version: 8.21.0 + yazl: + specifier: ^3.3.1 + version: 3.3.1 devDependencies: '@types/shell-quote': specifier: ^1.7.5 @@ -320,6 +323,9 @@ importers: '@wdio/devtools-shared': specifier: workspace:^ version: link:../shared + '@wdio/devtools-trace': + specifier: workspace:^ + version: link:../trace nodemon: specifier: ^3.1.14 version: 3.1.14 @@ -332,21 +338,18 @@ importers: '@types/ws': specifier: ^8.18.1 version: 8.18.1 - '@types/yazl': - specifier: ^3.3.1 - version: 3.3.1 '@wdio/devtools-script': specifier: workspace:* version: link:../script '@wdio/devtools-shared': specifier: workspace:^ version: link:../shared + '@wdio/devtools-trace': + specifier: workspace:^ + version: link:../trace '@xmldom/xmldom': specifier: ^0.9.8 version: 0.9.10 - fflate: - specifier: ^0.8.2 - version: 0.8.3 stacktrace-parser: specifier: ^0.1.11 version: 0.1.11 @@ -356,9 +359,6 @@ importers: xpath: specifier: ^0.0.34 version: 0.0.34 - yazl: - specifier: ^3.3.1 - version: 3.3.1 packages/elements: dependencies: @@ -665,6 +665,21 @@ importers: packages/shared: {} + packages/trace: + devDependencies: + '@types/yazl': + specifier: ^3.3.1 + version: 3.3.1 + '@wdio/devtools-shared': + specifier: workspace:^ + version: link:../shared + fflate: + specifier: ^0.8.2 + version: 0.8.3 + yazl: + specifier: ^3.3.1 + version: 3.3.1 + packages: '@alloc/quick-lru@5.2.0': diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 038df802..670ede36 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,6 +1,7 @@ # pnpm-workspace.yaml packages: - 'packages/shared' + - 'packages/trace' - 'packages/core' - 'packages/elements' - 'packages/backend' diff --git a/tsconfig.json b/tsconfig.json index 79b23784..5f4bb283 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -28,6 +28,8 @@ "@wdio/devtools-shared/*": ["./packages/shared/src/*"], "@wdio/devtools-core": ["./packages/core/src/index.ts"], "@wdio/devtools-core/*": ["./packages/core/src/*"], + "@wdio/devtools-trace": ["./packages/trace/src/index.ts"], + "@wdio/devtools-trace/*": ["./packages/trace/src/*"], "@wdio/elements": ["./packages/elements/src/index.ts"], "@wdio/elements/*": ["./packages/elements/src/*"], "@wdio/devtools-backend": ["./packages/backend/src/index.ts"],