Skip to content

Type after as or satisfies at end of line is tokenized as an expression #1079

Description

When a line ends with as or satisfies, the type on the next line gets expression scopes instead of type scopes.

JavaScript and TypeScript Nightly version: not tested in the extension. Reproduced with TypeScript.tmLanguage at eeeb0dc (current master), vscode-textmate 9.3.2, vscode-oniguruma 2.0.1, Node 26.7.0, macOS arm64.

Code

const a = b as A | undefined

const a = b as
  A | undefined

const a = b satisfies
  A | undefined

Prettier 3.9.8 emits this layout when an as expression exceeds the print width, e.g.:

const inlineSourceMap = convertSourceMap.fromSource(content)?.toObject() as
  SourceMap | undefined

Actual

Token Same line Next line
A entity.name.type.ts variable.other.constant.ts
| keyword.operator.type.ts keyword.operator.bitwise.ts
undefined support.type.builtin.ts constant.language.undefined.ts

A type on the line after satisfies gets the same scopes as a type on the line after as.

Expected

The same scopes on both layouts, because TypeScript parses both as a type assertion with the type A | undefined.

Repro script
npm install vscode-textmate@9.3.2 vscode-oniguruma@2.0.1
curl -sLO https://raw.githubusercontent.com/microsoft/TypeScript-TmLanguage/eeeb0dc4daa8793b8227bb3b34ddfc267c85fd81/TypeScript.tmLanguage
node repro.mjs
// repro.mjs
import { readFileSync } from "node:fs";
import { createRequire } from "node:module";
import vsctm from "vscode-textmate";
import oniguruma from "vscode-oniguruma";

const require = createRequire(import.meta.url);
await oniguruma.loadWASM(
  readFileSync(require.resolve("vscode-oniguruma/release/onig.wasm")).buffer,
);
const registry = new vsctm.Registry({
  onigLib: Promise.resolve({
    createOnigScanner: (patterns) => new oniguruma.OnigScanner(patterns),
    createOnigString: (s) => new oniguruma.OnigString(s),
  }),
  loadGrammar: async () =>
    vsctm.parseRawGrammar(
      readFileSync("TypeScript.tmLanguage", "utf8"),
      "TypeScript.tmLanguage",
    ),
});
const grammar = await registry.loadGrammar("source.ts");

const tokenize = (code) => {
  let state = vsctm.INITIAL;
  for (const line of code.split("\n")) {
    const { tokens, ruleStack } = grammar.tokenizeLine(line, state);
    state = ruleStack;
    for (const { startIndex, endIndex, scopes } of tokens) {
      const text = line.slice(startIndex, endIndex);
      if (text.trim())
        console.log(JSON.stringify(text).padEnd(10), scopes.at(-1));
    }
  }
  console.log();
};

tokenize("const a = b as A | undefined");
tokenize("const a = b as\n  A | undefined");
tokenize("const a = b satisfies\n  A | undefined");

Possible cause (unverified)

The as/satisfies rule's end includes $ through lookAheadEndOfType, so the type ends at the line break right after the keyword:

- begin: '{{startOfIdentifier}}(?:(as)|(satisfies))\s+'
beginCaptures:
'1': { name: keyword.control.as.ts }
'2': { name: keyword.control.satisfies.ts }
end: (?=^|{{lookAheadEndOfType}}|({{startOfIdentifier}}(as|satisfies)\s+)|(\s+\<))
patterns:
- include: '#type'

lookAheadEndOfType: '[;),}\]:?\-\+\>]|\|\||\&\&|\!\=\=|$'

The $ is presumably what ends the type at the end of a line without a semicolon (x as T followed by a new statement), so dropping it could break that case.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions