feat: add hover documentation provider with offline SQLite cache and background indexing

[rgunindi]

Summary

Add comprehensive, high-performance LSP Hover (textDocument/hover) support to matlab-language-server across all LSP clients (VS Code, Helix, Neovim, Emacs, etc.).

Key Features

1. Multi-Tier Dynamic Hover Resolution (HoverSupportProvider):

   • Online Mode (MVM Engine): When MATLAB is connected, dynamically queries help(topic) via matlabls.handlers.hover.getHover.
   • Offline Fast Mode (SQLite Cache): Sub-millisecond (~0.05ms) local documentation lookups from ~/.cache/matlabls/matlab_docs.db using Node 22 native node:sqlite (zero third-party npm dependencies).
   • Workspace M-File Docstrings: Resolves user-defined functions and scripts within workspace folders, extracting header comments.
   • Local Variable Resolver: Detects local variable definitions and assignment contexts in the active buffer.

2. Structured Markdown Formatter (HoverMarkdownUtils):

   • Losslessly transforms official MATLAB help text into clean, high-fidelity Markdown.
   • Wraps function calling signatures in structured matlab syntax blocks.
   • Formats parameters and descriptions into organized bulleted sections.
   • Highlights See Also references with appropriate markup.

3. Hybrid Background Indexing Architecture (DocumentationIndexer):

   • Added matlab.indexDocumentation setting and --indexDocumentation CLI option ('onMissing' [default], 'never', 'always').
   • On first launch with missing cache, spawns a non-blocking background indexer without delaying server readiness or editor responsiveness.
   • Dynamically reloads the SQLite database connection upon index completion without requiring a language server restart.
   • Registered matlabls.indexDocumentation workspace command for manual / on-demand triggers.
   • Multi-core parallel indexer tool (tools/indexer/index_docs.js & tools/indexer/worker.m) automatically scaling worker pool to host CPU cores with quiet/headless mode.

4. Testing & Verification:

   • Added unit test suites for HoverSupportProvider, HoverMarkdownUtils, DocumentationIndexer, and ExecuteCommandProvider.
   • All 281 test suite cases passing (npm test).
   • Verified production packaging (npm run package) and strict linting.

Motivation: Why is this needed?

Currently, matlab-language-server does not implement LSP Hover (textDocument/hover). Developers editing MATLAB code in modern editors (Helix, Neovim, VS Code, Emacs, Sublime) are left without inline function signatures, argument definitions, or documentation.

Furthermore, connecting to MATLAB via the Language Server (matlabConnectionTiming = "onStart" or "onDemand") spawns a full MATLAB runtime process (consuming 1.5 GB – 3 GB of RAM, locking license seats, taking 15–30 seconds to boot, and draining battery). Because of this overhead, many developers explicitly configure:

# Helix / Neovim / VS Code configuration
matlabConnectionTiming = "never"

Under this lightweight configuration—or in headless/remote environments (SSH, containers, CI)—developers previously had no documentation access whatsoever.

There was a critical need for an offline-first, zero-overhead, sub-millisecond Hover documentation engine that delivers rich, authentic MATLAB documentation without requiring a running MATLAB GUI/daemon.

---

Solution: How did we solve it?

Currently, matlab-language-server does not implement LSP Hover (textDocument/hover). Developers editing MATLAB code outside the official MATLAB desktop are left without inline signatures, argument descriptions, or function docs.

To query documentation dynamically via MATLAB:

1. Persistent Daemon Requirement: The developer must keep a full MATLAB runtime process running continuously in the background for their entire editing session just to answer hover queries.
2. Massive Idle Resource Waste: That persistent daemon consumes ~2.1 GB of RAM, drains laptop battery, takes 15–30 seconds to boot, and locks floating license seats even when the developer is simply reading or writing code.
3. No Offline Support: Because of this overhead, developers in lightweight, remote (SSH), or containerized environments explicitly configure:

    # Helix / Neovim / VS Code configuration
    matlabConnectionTiming = "never"

Under this lightweight configuration, developers previously had no documentation access whatsoever.

There was a critical need to decouple documentation from the live MATLAB runtime lifecycle, providing an offline-first, zero-overhead, sub-millisecond Hover documentation engine that eliminates the need to keep MATLAB running in the background.

We implemented a comprehensive, multi-tier Hover architecture with an offline SQLite cache and parallel multi-core indexer:

1. Sub-Millisecond Offline SQLite Lookup (~0.05ms):

   • Uses Node 22 native node:sqlite (DatabaseSync), requiring zero third-party npm dependencies.
   • Queries ~/.cache/matlabls/matlab_docs.db instantly with virtually 0 MB RAM footprint and 0% idle CPU usage.

2. Zero Hardcoding — 100% Data-Driven Extraction:

   • MathWorks documentation evolves with every release. Hardcoding static dictionaries is fragile and unmaintainable.
   • We extract documentation directly from the user's installed MATLAB catalog (helpfuncbycat.xml) via an automated parallel indexer.

3. High-Performance Multi-Core Parallel Indexer (tools/indexer/index_docs.js):

   • Dynamically scales to available host CPU cores (e.g., 12 workers on Apple Silicon / 14-core machines).
   • Indexes all 2,400+ canonical MATLAB functions in ~15–20 seconds with live progress metrics and quiet/headless support.

4. Multi-Tier Fallback Hierarchy:

   • Tier 1 (Online MVM Engine): If MATLAB is connected, dynamically queries help(topic) via matlabls.handlers.hover.getHover.
   • Tier 2 (Offline SQLite Cache): If MATLAB is offline (never), retrieves authentic documentation in <0.1ms.
   • Tier 3 (Workspace M-Files): Parses user-defined functions and scripts in project directories to show docstrings.
   • Tier 4 (Buffer Variables): Resolves local variable assignments and contextual types within the active document.

5. High-Fidelity Markdown Formatter (HoverMarkdownUtils):

   • Losslessly parses raw monospace MATLAB help text into structured Markdown.
   • Formats syntax signatures inside matlab blocks.
   • Converts parameters and descriptions into clean bullet lists.
   • Formats See Also links into highlighted cross-references.

6. Non-Blocking Hybrid Lifecycle (DocumentationIndexer):

   • Configured via matlab.indexDocumentation setting and --indexDocumentation CLI option ('onMissing' [default], 'never', 'always').
   • If the database is missing on first launch, it triggers a non-blocking background indexer without delaying server readiness.
   • Re-attaches database connection dynamically upon completion without requiring an LSP restart.
   • Provides LSP workspace command matlabls.indexDocumentation for on-demand re-indexing.

---

Verification (Proof of Work)

First-indexing:

After indexing:

---

Key Changes Summary

• src/providers/hover/HoverSupportProvider.ts: Multi-tier hover provider (MVM feval, SQLite, workspace files, local variables).
• src/providers/hover/HoverMarkdownUtils.ts: Authentic MATLAB help-to-Markdown parser.
• src/indexing/DocumentationIndexer.ts: Background non-blocking indexer service with setting callbacks.
• src/lifecycle/ConfigurationManager.ts & src/utils/CliUtils.ts: Added indexDocumentation configuration and CLI option.
• src/providers/lspCommands/ExecuteCommandProvider.ts: Added matlabls.indexDocumentation command.
• tools/indexer/index_docs.js & tools/indexer/worker.m: Dynamic multi-core parallel indexer.
• matlab/+matlabls/+handlers/+hover/getHover.m: MVM engine help evaluator.
• tests/: 281 passing tests (100% test suite pass rate).

[rgunindi]

Follow-up Capability: Offline Auto-Completion & Signature Help

Hi,

Just wanted to share an exciting follow-up capability: I've also implemented and locally verified daemon-independent offline auto-completion (textDocument/completion) and signature help (textDocument/signatureHelp) built directly on top of this PR's SQLite documentation cache.

As shown in the preview below, it delivers sub-millisecond function suggestions (plot, plot3, etc.) with authentic documentation summaries (2-D line plot) and local buffer variable completions—even when MATLAB is completely offline (matlabConnectionTiming = "never").

To keep this PR focused and easy to review, I kept the completion feature on a separate branch. Once this Hover PR is reviewed and merged, I would be very happy to submit the offline completion PR as a clean follow-up!

[dklilley]

Hi @rgunindi, thanks for working on this and opening the PR!

It will take me a few days before I have an opportunity to take a more in-depth look at this, but wanted to let you know it's on my radar.

[rgunindi]

Thanks for letting me know! I'll be around whenever you have the time to dive into it. Have a great week, @dklilley