Prioritize visible thumbnails and expose real model variants

AGENTS.md

@@ -91,7 +91,7 @@ Do not duplicate decoders across features.
 - **Game-data detection is async and cached once:** `cascStorage.getGameDataRoot()` returns a shared promise (registry queries, drive probing and directory walks all run asynchronously). Never add synchronous `existsSync`/`execFileSync` probing on the game-data path — it runs on the extension host while the language client is booting. `wurst.wc3path` changes reset it via `registerGameDataSettingsWatcher`. Basename fallbacks (`findPathByBasenameAsync`) are index lookups in both casc-ts and the MPQ storage; keep them O(1).
 - **Inline model preview:** to embed a 3D model render in a webview (not a separate window), load `dist/webview/mdxViewer.js` via `webview.asWebviewUri` (requires `extensionUri` on the provider, `dist/webview` in `localResourceRoots`, and `${webview.cspSource}` in `script-src`). A page that still has un-nonced inline scripts (objmod, the code asset picker) must keep `'unsafe-inline'` and must not add a nonce: a nonce in `script-src` makes the browser ignore `'unsafe-inline'`. `blpPreview.ts` loads the viewer under a nonce because all its scripts carry it. Host side: `preview/modelPreviewHost.ts` `postModelToWebview` / `postTexturesToWebview`; the viewer posts `requestTextures` back. Route the icon/thumbnail/texture messages (`loadObjectIcon`, `loadModelThumb`, `requestTextures`, `modelThumbRendered`, `modelThumbFailed`) through `handleModelThumbMessage` rather than re-dispatching them per page. The objmod editor's `#mpv-box` is the reference implementation.
 - **Thumbnail tracing is opt-in:** `[wurst-model-thumb]` console lines and `~/.wurst/model_thumbs/thumbnail-diagnostics.jsonl` are only written with `WURST_MODEL_THUMB_DEBUG=1` (or the e2e cache switch below). Do not add always-on per-thumbnail logging or disk writes.
-- **Objmod asset-browser model thumbnails:** visible model cards should enter a pending/spinner state immediately and stay there until the thumbnail is either loaded or decisively marked missing (`?`). Generation must drain visible thumbnails in DOM order, one complete thumbnail lifecycle at a time: host resolve -> render in the `mdxThumbnailWorker` bundle (a Blob worker; there is no main-thread fallback) -> cache/write or missing decision -> next item. Do not pre-resolve/render later visible models in parallel, and do not add fixed inter-thumbnail idle delays after a thumbnail has finished. Cancel queued work only when a thumbnail scrolls out of view before it starts; when it returns, re-observe/requeue it. Large models are not skipped by size (a guard in `scripts/test-webview.js` enforces this); keep them renderable by bounding texture size instead. Use `WURST_MODEL_THUMB_DISABLE_CACHE=1` for local validation so tests measure actual generation rather than cached webps.
+- **Objmod asset-browser model thumbnails:** visible model cards should enter a pending/spinner state immediately and stay there until the thumbnail is either loaded or decisively marked missing (`?`). Generation must drain visible thumbnails in DOM order, one complete thumbnail lifecycle at a time: host resolve -> render in the `mdxThumbnailWorker` bundle (a Blob worker; there is no main-thread fallback) -> cache/write or missing decision -> next item. Do not pre-resolve/render later visible models in parallel, and do not add fixed inter-thumbnail idle delays after a thumbnail has finished. Request only thumbnails currently in view. Cancel queued work and active worker texture waits when their cards leave view; let host resolution already in flight settle and ignore its stale reply before rendering. When a card returns, re-observe/requeue it. Large models are not skipped by size (a guard in `scripts/test-webview.js` enforces this); keep them renderable by bounding texture size instead. Use `WURST_MODEL_THUMB_DISABLE_CACHE=1` for local validation so tests measure actual generation rather than cached webps.
 - **Local-only thumbnail validation:** use `npm run test:e2e:objmod-thumbs:local` with `WURST_OBJMOD_E2E=1` to launch VS Code against the checked-in `e2e/war3map.w3u` fixture, open the objmod asset browser, disable thumbnail cache, and assert visible FIFO order plus per-thumbnail timing (default max 200ms). Override `WURST_OBJMOD_E2E_PROJECT` and `WURST_OBJMOD_E2E_FILE` for a real map/project. This is intentionally not a CI test because it depends on local WC3 data and VS Code/Electron.
 
 ## Testing tiers

e2e/specs/objmod-browse.spec.js

@@ -6,6 +6,40 @@
 
 const { test, expect } = require('../fixtures');
 
+for (const filter of ['search', 'source', 'empty']) {
+    test(`asset ${filter} filter cancels a removed thumbnail texture wait`, async ({ openObjMod }) => {
+        const { page, host } = await openObjMod();
+        const received = [];
+        const originalReceive = host.receive.bind(host);
+        host.receive = message => {
+            if (message.type === 'loadModelThumb' || message.type === 'requestAssetCatalog' || message.type === 'requestTextures') {
+                received.push(message);
+                return;
+            }
+            return originalReceive(message);
+        };
+        await page.evaluate(() => {
+            window.__wurstModelThumbDebug.openModelAssetBrowser();
+            window.postMessage({ type: 'assetCatalog', models: [
+                { value: 'old.mdx', label: 'Old', source: 'wc3' },
+                { value: 'new.mdx', label: 'New', source: 'import' }], icons: [], sounds: [],
+            }, '*');
+        });
+        await expect.poll(() => received.filter(m => m.type === 'loadModelThumb' && m.key.startsWith('ab-model:')).length).toBe(1);
+        const first = received.find(m => m.type === 'loadModelThumb' && m.key.startsWith('ab-model:'));
+        const bytes = require('fs').readFileSync(require('path').join(__dirname, '../../wc3data/melon.mdx')).toString('base64');
+        await page.evaluate(({ key, bytes }) => window.postMessage({ type: 'modelThumbRender', key,
+            cacheKey: 'filter-test', mdxBase64: bytes, format: 'mdx' }, '*'), { key: first.key, bytes });
+        await expect.poll(() => received.some(m => m.type === 'requestTextures' && m.thumbKey === first.key)).toBe(true);
+        if (filter === 'source') await page.locator('#ab-source').selectOption('import');
+        else await page.locator('#ab-search').fill(filter === 'empty' ? 'no-such-asset' : 'New');
+        await expect.poll(() => page.evaluate(() => window.__wurstModelThumbDebug.state().activeJob)).not.toBe(first.key);
+        if (filter === 'empty') await expect(page.locator('#ab-grid')).toContainText('No matching assets');
+        else await expect.poll(async () => ({ requests: received.filter(m => m.type === 'loadModelThumb').map(m => m.path),
+            state: await page.evaluate(() => window.__wurstModelThumbDebug.state()) }), { timeout: 7000 }).toMatchObject({ requests: expect.arrayContaining(['new.mdx']) });
+    });
+}
+
 /** Rows only exist for expanded branches, so "all objects" means expanding everything first. */
 async function expandAll(page) {
     for (let i = 0; i < 6; i++) {

e2e/specs/standalone-assets.spec.js

@@ -135,7 +135,104 @@ test('standalone browser uses the thumbnail service, waits for textures and fram
     expect(result.cacheKey).toBe('v12s-abc');
 });
 
-for (const cacheVersion of ['v10s', 'v11s', 'v12s']) {
+test('restored viewport gets the first thumbnail request', async ({ page, server }) => {
+    const vscode = createVscodeMock();
+    let html;
+    vscode.window.createWebviewPanel = () => ({ webview: {
+        cspSource: server.origin,
+        asWebviewUri: uri => ({ toString: () => server.origin + '/dist/webview/' + path.basename(uri.fsPath) }),
+        set html(value) { html = value; }, onDidReceiveMessage() {},
+    } });
+    const load = createTsLoader({ mocks: { vscode,
+        'src/features/objModPreview.ts': { loadObjValueCatalog: async () => ({
+            models: Array.from({ length: 120 }, (_, index) => ({ value: `model-${index}.mdx` })), icons: [], sounds: [],
+        }) },
+        'src/features/imageAssetSupport.ts': { getCandidateRoots: async () => [], gatherImportedAssets: async () => ({ model: [], icon: [], sound: [] }) },
+    } });
+    await load('src/features/assetLinks.ts').openAssetBrowser({ extensionUri: fileUri(root) });
+    await page.addInitScript(() => {
+        window.messages = [];
+        window.acquireVsCodeApi = () => ({
+            postMessage: message => window.messages.push(message),
+            getState: () => ({ activeTab: 'model', scrollTop: 1000 }),
+        });
+    });
+    await page.goto(server.publish(html));
+    await expect.poll(() => page.evaluate(() => window.messages.filter(m => m.type === 'loadModelThumb').length)).toBe(1);
+    const result = await page.evaluate(() => {
+        const grid = document.querySelector('#grid');
+        const viewport = grid.getBoundingClientRect();
+        const visible = [...grid.querySelectorAll('.model-thumb')].find(el => {
+            const rect = el.getBoundingClientRect();
+            return rect.bottom > viewport.top && rect.top < viewport.bottom;
+        });
+        return { scrollTop: grid.scrollTop, visibleKey: visible.getAttribute('data-key'),
+            requestKey: window.messages.find(m => m.type === 'loadModelThumb').key };
+    });
+    // The saved position may clamp to the grid's maximum scroll offset.
+    expect(result.scrollTop).toBeGreaterThan(500);
+    expect(result.requestKey).toBe(result.visibleKey);
+    expect(result.requestKey).not.toContain('model-0.mdx');
+});
+
+test('scrolling defers old thumbnails and requests the current viewport first', async ({ page, server }) => {
+    const vscode = createVscodeMock();
+    let html;
+    vscode.window.createWebviewPanel = () => ({ webview: {
+        cspSource: server.origin,
+        asWebviewUri: uri => ({ toString: () => server.origin + '/dist/webview/' + path.basename(uri.fsPath) }),
+        set html(value) { html = value; }, onDidReceiveMessage() {},
+    } });
+    const load = createTsLoader({ mocks: { vscode,
+        'src/features/objModPreview.ts': { loadObjValueCatalog: async () => ({
+            models: Array.from({ length: 120 }, (_, index) => ({ value: `model-${index}.mdx` })), icons: [], sounds: [],
+        }) },
+        'src/features/imageAssetSupport.ts': { getCandidateRoots: async () => [], gatherImportedAssets: async () => ({ model: [], icon: [], sound: [] }) },
+    } });
+    await load('src/features/assetLinks.ts').openAssetBrowser({ extensionUri: fileUri(root) });
+    await page.addInitScript(() => {
+        window.messages = [];
+        window.acquireVsCodeApi = () => ({ postMessage: message => window.messages.push(message) });
+    });
+    await page.goto(server.publish(html));
+    const requests = () => page.evaluate(() => window.messages.filter(m => m.type === 'loadModelThumb'));
+    await expect.poll(async () => (await requests()).length).toBe(1);
+    const first = (await requests())[0];
+    await page.locator('#grid').evaluate(grid => { grid.scrollTop = grid.scrollHeight / 2; });
+    // Let the offscreen host response arrive after the scroll. No texture/render work may start.
+    await page.evaluate(key => window.postMessage({ type: 'modelThumbRender', key,
+        cacheKey: 'unused', mdxBase64: 'AA==' }, '*'), first.key);
+    await expect.poll(async () => (await requests()).length).toBe(2);
+    const second = (await requests())[1];
+    const firstVisibleKey = await page.evaluate(() => {
+        const viewport = document.querySelector('#grid').getBoundingClientRect();
+        return [...document.querySelectorAll('.model-thumb')].find(el => {
+            const rect = el.getBoundingClientRect();
+            return rect.bottom > viewport.top && rect.top < viewport.bottom;
+        }).getAttribute('data-key');
+    });
+    expect(second.key).toBe(firstVisibleKey);
+    expect(second.key).not.toBe(first.key);
+    expect(await page.evaluate(() => window.messages.some(m => m.type === 'requestTextures' || m.type === 'modelThumbFailed'))).toBe(false);
+    // A stale acknowledgement must not unlock the current request.
+    await page.evaluate(key => window.postMessage({ type: 'modelThumbLoaded', key, uri: 'data:image/png;base64,AA==' }, '*'), first.key);
+    await expect.poll(async () => (await requests()).length).toBe(2);
+    const bytes = fs.readFileSync(path.join(root, 'wc3data/melon.mdx'));
+    await page.evaluate(({ key, bytes }) => window.postMessage({ type: 'modelThumbRender', key,
+        cacheKey: 'current', mdxBase64: bytes, format: 'mdx' }, '*'), { key: second.key, bytes: bytes.toString('base64') });
+    await expect.poll(() => page.evaluate(() => window.messages.some(m => m.type === 'requestTextures' && m.thumbKey === window.messages.filter(m => m.type === 'loadModelThumb')[1].key))).toBe(true);
+    // Scrolling away while the worker waits for textures cancels that render immediately.
+    await page.locator('#grid').evaluate(grid => { grid.scrollTop = 0; });
+    await expect.poll(async () => (await requests()).length).toBe(3);
+    const third = (await requests())[2];
+    expect(third.path).toBe('model-1.mdx');
+    await page.evaluate(key => window.postMessage({ type: 'modelThumbMissing', key }, '*'), second.key);
+    await expect.poll(async () => (await requests()).length).toBe(3);
+    expect(await page.evaluate(() => window.messages.some(m => m.type === 'modelThumbFailed'))).toBe(false);
+    await page.screenshot({ path: test.info().outputPath('visible-thumbnail-queue.png') });
+});
+
+for (const cacheVersion of ['v10s', 'v11s', 'v12s', 'v13s', 'v14s']) {
     test(`rendered model thumbnails stay loaded after saving ${cacheVersion}`, async ({ page, server }) => {
         const cacheDir = fs.mkdtempSync(path.join(os.tmpdir(), 'wurst-thumb-cache-'));
         try {

package.json

@@ -966,6 +966,8 @@
         "vscode:prepublish": "npm run package-web && npm run test:vsix-contents",
         "compile": "tsc -watch -p ./",
         "lint": "eslint .",
+        "precompile-web": "node ./scripts/build-local-model-library.js",
+        "prewatch-web": "node ./scripts/build-local-model-library.js",
         "compile-web": "webpack",
         "watch-web": "webpack --watch",
         "test": "npm run test:fuzzy && npm run test:image-decoders && npm run test:diagnostics && npm run test:language-features && npm run test:editor-work && npm run test:webview && npm run test:private-libraries",
@@ -986,6 +988,7 @@
         "test:e2e:objmod-clipboard:local": "playwright test --project=local-vscode objmod-clipboard",
         "test:wc3-previews": "node ./scripts/wc3-preview-smoke.js",
         "test:fuzzy": "node ./scripts/test-fuzzy.js",
+        "prepackage-web": "node ./scripts/build-local-model-library.js",
         "package-web": "webpack --mode production",
         "publish": "vsce publish --no-dependencies"
     },

scripts/build-local-model-library.js

@@ -0,0 +1,14 @@
+'use strict';
+
+// file: dependencies export dist, so webpack otherwise silently bundles an old renderer.
+const fs = require('fs');
+const path = require('path');
+const { spawnSync } = require('child_process');
+
+const library = path.dirname(path.dirname(require.resolve('war3-model')));
+if (fs.existsSync(path.join(library, 'mdx', 'parse.ts'))) {
+    if (!process.env.npm_execpath) throw new Error('Run this build hook through npm.');
+    const result = spawnSync(process.execPath, [process.env.npm_execpath, '--prefix', library, 'run', 'build-lib'], { stdio: 'inherit' });
+    if (result.error) throw result.error;
+    if (result.status !== 0) process.exit(result.status || 1);
+}

scripts/test-webview.js

@@ -502,6 +502,17 @@ async function testFolderModeMapAssetResolution() {
     );
 
     const reforgedVariants = mod.assetPathVariants('Units\\Creeps\\ArachnathidWarrior\\ArachnathidWarrior_Diffuse.tif', 'texture');
+    const doodadStem = 'Doodads\\Undercity\\Props\\AlchemySet\\AlchemySet';
+    const doodadPath = path.join(imported, 'Doodads', 'Undercity', 'Props', 'AlchemySet');
+    fs.mkdirSync(doodadPath, { recursive: true });
+    const numberedModel = path.join(doodadPath, 'AlchemySet0.mdx');
+    fs.writeFileSync(numberedModel, Buffer.from('MDLX'));
+    assert.equal(await mod.resolveAssetPathWithCasc(doodadStem + '.mdl', roots, 'model'), undefined,
+        'explicit model requests must not silently resolve a different numbered file');
+    const exactModel = path.join(doodadPath, 'AlchemySet.mdx');
+    fs.writeFileSync(exactModel, Buffer.from('MDLX'));
+    assert.equal(await mod.resolveAssetPathWithCasc(doodadStem + '.mdl', roots, 'model'), exactModel,
+        'an exact model takes precedence over numbered variations');
     assert.ok(
         reforgedVariants.includes('Units\\Creeps\\ArachnathidWarrior\\ArachnathidWarrior_Diffuse.dds'),
         'Reforged .tif material references should probe .dds by replacing the extension'
@@ -929,7 +940,7 @@ async function testModelThumbnailRequestsTexturesByDefault() {
 
     const render = posted.find((message) => message.type === 'modelThumbRender');
     assert.ok(render, 'uncached model thumbnails should render regardless of model byte size');
-    assert.equal(render.cacheKey, 'v12s-abc123', 'older framing captures must be regenerated');
+    assert.equal(render.cacheKey, 'v14s-abc123', 'older renderer captures must be regenerated');
     assert.equal(render.skipTextures, undefined, 'model thumbnail renders must load textures by default');
     assert.ok(render.mdxBase64, 'model bytes should still be sent for thumbnail rendering');
 
@@ -1036,7 +1047,7 @@ function testThumbnailLifecycleGuards() {
     assert.ok(host.includes("if (ext === 'blp')"), 'BLP thumbnails should retain the renderer decoder rather than using the generic preview decoder');
     assert.ok(viewer.includes('downscaleTextureImageData'), 'decoded BLP thumbnail textures should be reduced before GPU upload');
     assert.ok(thumbnailWorker.includes('MAX_TEXTURE_DIMENSION'), 'worker thumbnail renders should bound browser-side texture uploads');
-    assert.ok(host.includes('return `v12s-'), 'the cache version must invalidate thumbnails captured before the current consumer renderer build');
+    assert.ok(host.includes('return `v14s-'), 'the cache version must invalidate thumbnails captured before the current consumer renderer build');
     assert.ok(!objmod.includes('capture-dark-accepted'), 'dark frames must never be persisted as successful thumbnails');
     assert.ok(objmod.includes('Array.from(new Set((texturePaths || [])'), 'thumbnail capture must wait for every referenced material texture');
     assert.ok(!objmod.includes('(?:normal|orm)'), 'thumbnail loading must not omit HD material textures');
@@ -1432,6 +1443,20 @@ function testMpqReextractUsesFreshUriAfterDeletedOutput() {
 }
 
 async function main() {
+    const { expandModelVariants } = loadTsModule('src/features/preview/modelVariants.ts');
+    const stem = 'Doodads\\Undercity\\Props\\AlchemySet\\AlchemySet';
+    const options = [{ value: `${stem}.mdl`, label: 'Alchemy Set', detail: 'UOal' },
+        { value: 'units\\hero.mdl', label: 'Hero' }];
+    const actualPaths = [4, 0, 2, 1, 3].map((i) => `${stem}${i}.mdx`);
+    const variants = expandModelVariants(options, [...actualPaths, `${stem}0.mdl`, 'Other\\AlchemySet9.mdx']);
+    assert.deepStrictEqual(variants.slice(0, 5).map((option) => option.value),
+        [0, 1, 2, 3, 4].map((i) => `${stem}${i}.mdx`), 'all existing variations replace the unresolved metadata stem');
+    assert.strictEqual(variants[5].label, 'Hero', 'unrelated catalog entries are preserved');
+    assert.strictEqual(variants[4].label, 'Alchemy Set (variation 4)');
+    assert.ok(variants.slice(0, 5).every((option) => option.detail.includes('UOal')),
+        'expanded variants preserve the owning rawcode for asset searches');
+    assert.strictEqual(expandModelVariants(options, [...actualPaths, `${stem}.mdx`])[0].value,
+        `${stem}.mdl`, 'a real unsuffixed model is retained alongside its variants');
     testAssetPathNormalization();
     testSignals();
     testObjModTreeSelectionStaysUntracked();