ci(e2e): catch published exports.types→source regression in released bit#10474
Conversation
Temporary experiment to learn whether the existing e2e suite, run against the released binary, catches the exports.types->source regression. - Pin bbit to the known-broken 2.0.10 in bvm_upgrade (was 'bvm upgrade' -> latest). - Run e2e_test_bbit on this PR branch (normally nightly-only). - Fix the --bit_bin flag (was transposed as --bin_bit, so e2e_test_bbit and e2e_test_bundle_simulation silently fell back to the local binary instead of the released one). This part is a real fix and should stay. The pin + workflow entry are temporary, to be reverted after the experiment.
PR Summary by QodoCI e2e: pin bbit 2.0.10 and run released-binary suite to probe exports regression
AI Description
Diagram
High-Level Assessment
Files changed (1)
|
Code Review by Qodo
1. Env gate mis-parses false
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
…bundle scan - scripts/verify-released-exports.js: fails if any published @teambit package points exports.types at source (.ts) instead of dist/*.d.ts. Wired into e2e_test_bbit (fails fast). Locally: 183 offenders on broken 2.0.10, 0 on fixed 2.0.11. - e2e/harmony/released-bit-core-exports.e2e.ts: builds a component importing @teambit/application and asserts TypescriptCompile succeeds. Gated to released binaries only (E2E_RELEASED_BINARY), since local dev links core packages to source. Verified: fails on 2.0.10, passes on 2.0.11. - filesystem-read.e2e.ts: bump status file-count threshold 1500 -> 1510. - remove flawed aspect-package-exports.e2e.ts (asserted on local pack output, which never reflects published exports). Note: the 2.0.10 pin + per-PR e2e_test_bbit are still temporary (to validate the scan on CI).
|
Code review by qodo was updated up to the latest commit 188e367 |
…tly-only Reverts the temporary experiment scaffolding now that both guards are validated on CI (scan + consume test both caught the pinned 2.0.10 regression; e2e_test/local passed): - bvm_upgrade: restore 'bvm upgrade' (latest) instead of the 2.0.10 pin. - remove e2e_test_bbit from the per-PR build_and_test workflow (it stays nightly-only), so a bad release fails the nightly rather than blocking every PR. Kept (permanent): the --bit_bin flag fix, the E2E_RELEASED_BINARY gate, and the verify-released-exports.js scan step in e2e_test_bbit.
davidfirst
changed the title
|
Code review by qodo was updated up to the latest commit f36aaed |
| const runAgainstReleasedBinary = !!process.env.E2E_RELEASED_BINARY; | ||
|
|
||
| // A bit-core aspect that carried the regression in 2.0.10; importing it forces its types to resolve. | ||
| const CORE_ASPECT_PKG = '@teambit/application'; | ||
| // Any TS env that honors package `exports` reproduces it; react-env is the version verified against | ||
| // bbit 2.0.10 (fails) and 2.0.11 (passes). | ||
| const ENV = 'bitdev.react/react-env@6.0.20'; | ||
|
|
||
| (runAgainstReleasedBinary ? describe : describe.skip)( | ||
| 'released binary: bit-core aspects type-check from dist, not source', |
There was a problem hiding this comment.
1. Env gate mis-parses false 🐞 Bug ≡ Correctness
released-bit-core-exports.e2e.ts enables the suite for any non-empty E2E_RELEASED_BINARY value (e.g. "false"/"0"), which can unexpectedly run released-binary-only tests and cause false failures outside the intended CI job.
Agent Prompt
### Issue description
The suite gate is computed via JS truthiness (`!!process.env.E2E_RELEASED_BINARY`), so values like `"false"` still enable the test.
### Issue Context
Other e2e gates in this repo use explicit string comparison for env vars to avoid this surprise.
### Fix Focus Areas
- e2e/harmony/released-bit-core-exports.e2e.ts[23-33]
### Suggested change
Replace the truthy check with an explicit allowlist, e.g.:
- `const runAgainstReleasedBinary = process.env.E2E_RELEASED_BINARY === 'true' || process.env.E2E_RELEASED_BINARY === '1';`
(or normalize case and accept `True` if desired).
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| * Point it at a released bundle's @teambit dir, e.g. in the e2e_test_bbit CI job: | ||
| * BV=$(bbit --version | tr -d '[:space:]') | ||
| * node scripts/verify-released-exports.js "$HOME/.bvm/versions/$BV/bit-$BV/node_modules/@teambit" | ||
| * |
There was a problem hiding this comment.
2. Verifier comment path mismatch 🐞 Bug ⚙ Maintainability
verify-released-exports.js documents running node scripts/verify-released-exports.js ..., but the CircleCI job runs it from the parent workspace as node bit/scripts/...; the comment can mislead anyone trying to reproduce the CI step locally in the same directory layout.
Agent Prompt
### Issue description
The usage example in the script assumes the current working directory is the repo root, but in CircleCI the workspace root contains the repo under `bit/`, so the invoked path differs.
### Issue Context
In CircleCI, `checkout_code` persists a `bit` directory to the workspace, and `e2e_test_bbit` attaches it at `./`.
### Fix Focus Areas
- scripts/verify-released-exports.js[7-10]
### Suggested change
Update the comment to include both valid invocations, e.g.:
- From within the repo (`cd bit`): `node scripts/verify-released-exports.js ...`
- From the workspace root (as CI does): `node bit/scripts/verify-released-exports.js ...`
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
Code review by qodo was updated up to the latest commit 69c2019 |
- verify-released-exports.js: fail (not silently skip) on unreadable/corrupt package.json; use plain ASCII markers instead of Unicode symbols. - e2e_test_bbit: run the released-bundle scan once (node 0) instead of on all 32 parallel nodes.
|
Code review by qodo was updated up to the latest commit efe369a |
2 participants
Adds two guards (in the nightly
e2e_test_bbitjob, against the bvm-released binary) that catch a released bit whose core@teambit/*packages pointexports.typesat source (./index.ts) instead of./dist/index.d.ts— the regression that broke client builds in 2.0.10 (fixed in 2.0.11).Guards
scripts/verify-released-exports.js— fails the job if any published@teambitpackage pointsexports.typesat a.tssource. Deterministic; runs early ine2e_test_bbit.e2e/harmony/released-bit-core-exports.e2e.ts— builds a component importing@teambit/applicationand assertsTypescriptCompilepasses. Gated onE2E_RELEASED_BINARY(set only ine2e_test_bbit), because local dev links core packages to source and would always fail otherwise.Both live in
e2e_test_bbit, which is nightly-only — a bad release fails the nightly (before promoting to stable / before clients), without blocking every PR.Also
--bin_bit→--bit_bin. The helper readsnpm_config_bit_bin, soe2e_test_bbitande2e_test_bundle_simulationwere silently running the local binary instead of the released/cleaned one.filesystem-readstatus file-count threshold 1500 → 1510.Validated on CI
Temporarily pinning
bbitto the broken 2.0.10 (reverted in the final commit): the scan failed with183 packages point exports.types at SOURCEand the consume test failed onTypescriptCompile— whilee2e_test(local dev) passed and the consume test correctly skipped there.