Auth metadata discovery: fallback only triggers on 404, misses 401/403

[sahil7886]

Bug Description

In the OAuth metadata discovery code (discoverOAuthProtectedResourceMetadata / discoverAuthorizationServerMetadata in client/auth.ts), the client tries the path-inserted well-known URL first, then falls back to the root well-known URL — but the fallback only triggers on HTTP 404.

Servers behind CDNs / bot protection often answer unknown paths with 401 or 403 instead of 404. In those cases discovery hard-fails even though the root metadata document exists and returns 200.

Steps to Reproduce

Repro A (Retool): resource https://hashnote.retool.com/mcp

1. GET https://hashnote.retool.com/.well-known/oauth-protected-resource/mcp → 401
2. GET https://hashnote.retool.com/.well-known/oauth-protected-resource → 200
3. Client throws "HTTP 401 trying to load well-known OAuth metadata" instead of falling back.

Repro B (Superhuman): resource https://docs.superhuman.com/apis/mcp

1. GET https://id.superhuman.com/.well-known/oauth-authorization-server/apis/mcp → 403
2. GET https://id.superhuman.com/.well-known/oauth-authorization-server → 200
3. Same hard-fail ("HTTP 403 trying to load well-known OAuth metadata").

Expected Behavior

The fallback should trigger on any 4xx (or at least 401/403 in addition to 404).

Actual Behavior

Discovery throws on 401/403 instead of trying the root well-known URL.

Environment

• Discovered via mcp-hub 4.2.1 (npm, node 24, macOS) in front of remote OAuth MCP servers.
• Local workaround confirmed: relaxing the fallback predicate from status === 404 to include 401/403 fixes both servers.

Related

Possibly related to #1946 (401 with non-Bearer WWW-Authenticate) and #2126 (non-JSON 200 fallback) — this report is about plain 401/403 statuses where the root document is valid.

[tiagovilasboas]

Taking a look at this (and #2784) — planning a small fix for 4xx well-known fallback + AS-host base for authorization-server metadata.