Support TypeScript 7.1 API - #1704
johnnyreilly with Copilot wants to merge 129 commits into
Conversation
|
@copilot update ts-loader/.github/workflows/push.yml Line 52 in 53082fb ts-loader/.github/workflows/push.yml Line 84 in 53082fb We expect execution-tests to pass after this change |
Updated in |
|
@copilot great work! We would like the following to succeed:
Remember backwards compatibility is not required - feel free to make changes |
Verified on |
|
@copilot we have explictly removed the historic typescript version from dependencies and moved to use 7.1 directly. This has broken compilation. Please fix it. You will likely need to use new APIs in TypeScript 7 to achieve this. No cheating. |
Updated in |
|
@copilot make |
Addressed in |
- Use interface instead of type for the plain object shape TypeScriptApiModule, matching AGENTS.md's convention. - Extract reportConfigFileParsingErrors, deduplicating the ~20-line broken-tsconfig error-reporting block shared by getTypeScriptEmit and getTranspileOnlyEmit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TypeScriptInstance.resolvedFilePathCache duplicated the exact same function reference already stored on the owning TSInstance's own resolvedPathCache field. Drop the stored field; thread it as an explicit parameter through prepareSnapshotForFile/ensureSyntheticConfigForFile instead, sourced from instance.resolvedPathCache at the getTypeScriptEmit call site - matching the convention already used by getProjectDtsFileNames, registerResolvedImportDependencies, getCachedDirectResolvedImports, and findTransitiveDependants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…opilot/implement-new-tsgo-api-support
The tsgo sync API spawns a native child process per ts-loader instance, making a "cold build" iteration ~10-20x more expensive than the classic API's cheap in-process instantiation; touching a widely-imported (hub) file is similarly ~35x more expensive per incremental rebuild due to per-dependant recheck round trips over the sync RPC channel. The fixed iteration counts (tuned for the classic API's cost profile) made the cold-typeCheck and hub-touch scenarios alone take ~28 minutes combined, blowing the 20-minute CI job timeout before a single scenario finished. Cap each scenario's wall-clock time in run-side.mts instead of guessing a smaller fixed iteration count, so cheap scenarios keep their full sample size while expensive ones stop once they've collected enough measured samples. Confirmed locally: full default run (300 files) went from never finishing to completing in ~4 minutes.
Benchmark (Ubuntu)
PR branch = |
Benchmark (Windows)
PR branch = |
…per file Profiling the benchmark's slowness (tsgo branch showing 10-20x worse numbers than the classic branch, contrary to tsgo's own "faster compiler" characteristic) traced 83-91% of both cold-build and hub-touch-rebuild time to recheckTransitiveDependants: it ran once per file webpack compiled rather than once per build, and each call does a full O(project size) dependant search plus two diagnostic calls per dependant found. For a wide-fanout change (a file most of the project depends on) that's close to O(n²) work in a single build. The raw tsgo API itself opens a 300-file project and double-diagnoses every file in ~25ms - it's not the bottleneck. Batches the recheck into one pass per build instead: getTypeScriptEmit now just records which files it compiled (changedFilesThisBuild), and a new recheckAllTransitiveDependants runs once from the existing postCompile hook, searching dependants of the whole changed-file set in a single pass. Getting this right needed two follow-up fixes surfaced by comparison tests, both around same-build ordering: - A changed file can itself import another changed file compiled later in the same build (e.g. an entry file and the dependency it just changed). Its own diagnostics may have been computed before that other file's compile updated the shared file cache the API's readFile override serves from. findTransitiveDependants no longer excludes changed files from being found as dependants of each other, so this gets caught and rechecked too. - The recheck's own snapshot needs a forced full rescan (pendingInvalidation = true) rather than reusing the arbitrary anchor file's incremental view, otherwise it can still read a stale copy of a same-build sibling. Only costs one extra rescan per build (not per file), so it's affordable now. Measured on a synthetic 300-file fixture: cold build ~9.1s -> ~1.0s, hub-touch incremental rebuild ~4.2s -> ~300-380ms. Full comparison-test suite (including all watch-mode tests) still passes.
…opilot/implement-new-tsgo-api-support
…opilot/implement-new-tsgo-api-support
|
The code in this branch will likely be need to be refactored as a result of: |
|
We should experiment with nightlies again now the API refactoring has landed: microsoft/TypeScript#64204 Here are the benchmark results prior to moving to the new API: Benchmark (Ubuntu)
PR branch = Benchmark (Windows)
PR branch = |
|
Tested with new API - mostly good but one regression that is breaking comparison tests. See microsoft/TypeScript#64204 (comment) for details |
Two related "no project found for opened file" Go-side panics surfaced against a virtual (never-on-disk) identity while migrating to the new snapshot API: 1. `toApiFacingFileName` aliases a file with an unrecognized extension (e.g. a Vue SFC's extracted `<script>` block, `App.vue` -> `App.vue.ts`) for every API-facing call, including `openFiles`. But the `fs.fileExists`/ `readFile` host overrides looked up content keyed only by the real name, never by that alias, so the API's callback for `App.vue.ts` came up empty on both the virtual map and real disk - from the host's perspective the file genuinely didn't exist. Fixed by having those overrides fall back to the pre-alias name (`toRealFacingFileName`) on a miss. 2. Separately, on Windows only, `appendSuffixTo` (the older `appendTsSuffixTo` option) hit the same panic for a different reason: its aliased entry file is the very first file webpack compiles, so the very first `createSnapshot` call has to open the project and that not-yet-a-member file together, in one request - a combination the API panics on regardless of whether create or update does it. Fixed by catching that failure in `updateSnapshot` and retrying without `openFiles`, letting the existing synthetic-config fallback in `prepareSnapshotForFile` take over exactly as it already does for any later never-before-seen file. Verified via `yarn comparison-tests` (50/50, both fixes needed for sourceMapsShouldConsiderInputSourceMap) and via .github/workflows/windows-test-probe.yml (50/50 on Windows, where the second bug only reproduces). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
990d5ad to
693fd7c
Compare
…" panic A minimal, ts-loader/webpack-free repro of the second panic shape described in the PR discussion: a fresh API instance's very first createSnapshot() call opens a project and, in the same request, opens a file that isn't part of that project (real, inferred, or synthetic) - which should fall back to the inferred project per SnapshotRequestChangesParams.openFiles's own doc comment, but instead panics on Windows. Doesn't reproduce on macOS/Linux. Also extends windows-test-probe.yml with a repro_script input so this (or any other standalone script) can be run on the Windows runner on-demand, for verifying platform-specific upstream issues like this one. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Copilot didn't write this - I did! The below can also be found in the CHANGELOG.md.
This is a ground-up rewrite of ts-loader's compilation engine. Instead of driving TypeScript's classic
LanguageService/Program/ watch APIs, ts-loader now compiles exclusively through TypeScript's new nativetypescript/unstable/syncAPI (the tsgo-powered engine) - the legacy compiler API integration has been removed entirely.Not supported yet
getCustomTransformers,resolveModuleNameandresolveTypeReferenceDirectiveare still accepted for backwards compatibility but are now inert - the native API doesn't expose equivalent extension points, so custom transformers and custom module/type-reference resolution are no longer applied. It is possible that the API will support these in future, and so the options have been left in place for now, but they will be removed if the API never exposes them.Breaking changes:
nextprerelease that exposes this native API ahead of a stable 7.1 release.compileroption must now resolve to a package exposing a<compiler>/unstable/syncentry point (the TypeScript native API). Drop-in classic-API compilers (e.g.ttypescript) are no longer supported.compilerOptionsloader option; the native API resolves a project's compiler options purely from its on-disk tsconfig.json, with no per-loader-instance override hook. Set compiler options intsconfig.jsoninstead.contextloader option; the native API always resolves relative paths against the config file's own directory, with no basePath override exposed to let a tsconfig live outside the project root.happyPackModeloader option - HappyPack / thread-loader based parallelisation is no longer supported this way.experimentalFileCachingloader option; there's no equivalent under the native API's own caching model.experimentalWatchApiloader option, now that the native API supersedes TypeScript's classic watch API.onlyCompileBundledFilesloader option; the native API always resolves a project's own root files and offers no hook to restrict them to what webpack actually bundles.errorFormatter'scolorsargument is now a smallpicocolors-backed helper object instead of achalkinstance;chalkhas been dropped as a runtime dependency in favour ofpicocolors.