npm: support GitHub Packages SHA tarball URLs

[ksivamuthu-cei]

Summary

Support npm registries that publish content-addressed tarball URLs, including GitHub Packages URLs shaped like:

https://npm.pkg.github.com/download/@scope/package/1.0.0/SHA-1

Previously, the proxy used the final URL segment as the filename. Because GitHub's final segment is a SHA-1 rather than {package}-{version}.tgz, downloads failed before cache lookup with could not determine version from filename.

This change:

• rewrites content-addressed tarballs to a conventional proxy .tgz path with a private git-pkgs-shasum query parameter;
• resolves that request by reloading trusted upstream metadata and verifying the requested version's dist.shasum;
• fetches the original upstream dist.tarball URL rather than reconstructing GitHub's content-addressed path; and
• accepts that tarball URL only when its scheme and host match the configured npm upstream.

Regular npm tarball paths retain their existing rewrite and download behavior.

Tests

• go test ./internal/handler -run 'TestNPM'
• go test ./...
• go vet ./...
• go build ./cmd/proxy

Added coverage for GitHub Packages metadata rewriting, fetching the original content-addressed tarball, checksum mismatch rejection, and cross-host tarball rejection.

[Copilot]

Pull request overview

Adds support in the npm proxy handler for registries that publish content-addressed tarball URLs (e.g., GitHub Packages), by rewriting metadata tarball URLs to a conventional *.tgz proxy path and resolving downloads via a shasum-backed lookup against upstream metadata.

Changes:

• Rewrites content-addressed tarball URLs to {name}-{version}.tgz plus a git-pkgs-shasum query parameter.
• Resolves shasum-based download requests by re-fetching upstream metadata, verifying dist.shasum, and using dist.tarball (with upstream-origin validation).
• Adds tests covering GitHub Packages-style rewriting, successful fetch, checksum mismatch rejection, and cross-host tarball rejection.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
internal/handler/npm.go	Adds shasum-aware tarball rewrite + download resolution/validation for content-addressed tarball URLs.
internal/handler/npm_test.go	Adds test coverage for GitHub Packages tarball rewriting and shasum-based download validation paths.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[andrew]

Thanks for this — GitHub Packages as an npm upstream is currently broken exactly as you describe, and it's worth fixing.

I'd like to avoid the git-pkgs-shasum query parameter though. At download time the handler already has packageName and version from the request path, so it can look up versions[version].dist.tarball from cached metadata directly without the client carrying a hint back. A couple of specific concerns with the current shape:

• The shasum comparison doesn't verify anything: both the client-supplied value and dist.shasum come from the same upstream packument (one via the rewritten metadata, one via the re-fetch), so it only confirms the metadata agrees with itself.
• npm records resolved verbatim in package-lock.json, so ?git-pkgs-shasum=... ends up committed in users' lockfiles, baking a proxy-internal detail into files the proxy doesn't control.

A simpler approach that covers GitHub Packages and any other registry with non-standard tarball paths:

• In rewriteTarballURLs, when the upstream filename doesn't parse as <shortName>-<version>.tgz, synthesize that filename (as you already do) but with no query string.
• In handleDownload, resolve the upstream URL by reading versions[version].dist.tarball from FetchOrCacheMetadata, keeping your host/scheme/base-path check in validateUpstreamTarballURL. Fall back to the current fmt.Sprintf construction only if metadata is unavailable.

This will also need a rebase onto main: #241 removed npmAbbreviatedCT (which tarballURLForShasum references, so it won't compile as-is) and #240 added a cooldown check in handleDownload right where this PR changes the downloadURL assignment.

[andrew]

Thanks for the update. The query parameter is gone and the base-path check is present, but I found three remaining issues:

1. validateUpstreamTarballURL accepts dot-dot path segments. For an upstream rooted at /root, a tarball URL under /root/../outside passes the prefix check. An upstream or reverse proxy can normalize that to /outside, while URL-prefix authentication may still attach credentials based on the original string. Reject traversal segments before accepting the URL.

2. handleDownload resolves metadata before the artifact cache lookup. Since metadata caching is disabled by default, every tarball cache hit now fetches a full packument and can wait for the upstream timeout. With cooldown enabled, the handler fetches the packument twice. Check the artifact cache before resolving metadata, or resolve the URL lazily after a cache miss.

3. If a cached packument lacks the requested version or tarball, the handler returns 400 without trying the conventional URL. This blocks direct lockfile downloads of newly published versions until the metadata TTL expires. Use the constructed URL fallback when metadata cannot provide the requested version or tarball, not only when the metadata fetch itself fails.