Hello, I'm running into issues trying to use this extension in NodeJS. I identified multiple problems:
1. The sqlite-vec package has broken platform/arch checks
The sqlite-vec package acting as the "router" for the platform-specific ones currently checks these combinations:
const supportedPlatforms = [["macos","aarch64"],["linux","aarch64"],["macos","x86_64"],["windows","x86_64"],["linux","x86_64"]];
This is checked against Node's process.platform and process.arch respectively. Sadly, all of these are wrong... As a result, an error is thrown before any attempts at loading the suitable package are even made. First I thought maybe these were Deno or Bun specifics, but they all seem to match Node:
To address this:
- "windows" should be "win32"
- "macos" should be "darwin"
- "aarch64" should be "arm64"
- "x86_64" should be "x64"
2. The sqlite-vec package manually composes the path to vec0.* file
While this may work in classic setups where everything is unpacked into node_modules, in stricter setups like Plug'n'Play (Yarn, pnpm, Bun does something similar, ...), this is very likely to fail and compose a path to nowhere.
The correct way, I believe, would be to ask the module resolver itself to provide the path via APIs like import.meta.resolve or require.resolve, which should abstract away all of the unknowns.
3. The platform-specific packages declare non-existent exports
E.g. the sqlite-vec-windows-x64 package declares:
"main": "./index.cjs",
"module": "./index.mjs",
"types": "./index.d.ts",
"exports": {
".": {
"require": "./index.cjs",
"import": "./index.mjs",
"types": "./index.d.ts"
}
}...and none of these files actually exist within the published NPM package. What's worse, the vec0.* file is not declared anywhere, so resolvers refuse to see it. I believe the package should simply declare this:
"exports": {
"./vec0.dll": {
"default": "./vec0.dll"
}
}With the above change, calling import.meta.resolve("sqlite-vec-windows-x64/vec0.dll") (ESM) or require.resolve("sqlite-vec-windows-x64/vec0.dll") (CJS) cleanly resolves the path to the binary, wherever the package manager decided to put it.
After applying the mentioned changes via yarn patch I'm now up and running.
I noticed similar open issues here already, but with next to no information, so I decided to open one with some research. Hope this can help improve the NPM bundles, cheers!
Hello, I'm running into issues trying to use this extension in NodeJS. I identified multiple problems:
1. The
sqlite-vecpackage has broken platform/arch checksThe
sqlite-vecpackage acting as the "router" for the platform-specific ones currently checks these combinations:This is checked against Node's
process.platformandprocess.archrespectively. Sadly, all of these are wrong... As a result, an error is thrown before any attempts at loading the suitable package are even made. First I thought maybe these were Deno or Bun specifics, but they all seem to match Node:platformandarchshould stick to the specTo address this:
2. The
sqlite-vecpackage manually composes the path tovec0.*fileWhile this may work in classic setups where everything is unpacked into
node_modules, in stricter setups like Plug'n'Play (Yarn, pnpm, Bun does something similar, ...), this is very likely to fail and compose a path to nowhere.The correct way, I believe, would be to ask the module resolver itself to provide the path via APIs like
import.meta.resolveorrequire.resolve, which should abstract away all of the unknowns.3. The platform-specific packages declare non-existent
exportsE.g. the
sqlite-vec-windows-x64package declares:...and none of these files actually exist within the published NPM package. What's worse, the
vec0.*file is not declared anywhere, so resolvers refuse to see it. I believe the package should simply declare this:With the above change, calling
import.meta.resolve("sqlite-vec-windows-x64/vec0.dll")(ESM) orrequire.resolve("sqlite-vec-windows-x64/vec0.dll")(CJS) cleanly resolves the path to the binary, wherever the package manager decided to put it.After applying the mentioned changes via
yarn patchI'm now up and running.I noticed similar open issues here already, but with next to no information, so I decided to open one with some research. Hope this can help improve the NPM bundles, cheers!