diff --git a/unstructured2graph/spec/sso-design.md b/unstructured2graph/spec/sso-design.md new file mode 100644 index 00000000..f07e56ce --- /dev/null +++ b/unstructured2graph/spec/sso-design.md @@ -0,0 +1,127 @@ +# SSO (Single Sign-On) Design for unstructured2graph + +**Status:** Discovery / Requirements Gathering +**Date:** 2026-03-31 + +## 1. System Landscape + +The SSO solution must unify authentication across three components: + +| Component | Protocol/Interface | Current Auth | +|---|---|---| +| **Memgraph** (database) | Bolt (port 7687) | Username/password via env vars (`MEMGRAPH_URL`, `MEMGRAPH_USER`, `MEMGRAPH_PASSWORD`). Enterprise supports OIDC (Entra ID, Okta), SAML, and LDAP via external auth modules. | +| **Memgraph Lab** (UI) | HTTP (browser) | Enterprise Docker deployment supports OIDC and SAML SSO with Entra ID and Okta. Desktop version has no SSO. | +| **LightRAG** (AI/RAG API) | REST/FastAPI (port 9621) | API key (`X-API-Key` header) and/or JWT tokens (OAuth2 password flow at `/login`). Supports bcrypt-hashed passwords, token expiry, auto-renewal, and path whitelisting. | +| **unstructured2graph** (this project) | Python library | No auth layer — inherits Memgraph connection credentials from env vars and LightRAG wrapper config. | + +## 2. Approach Options + +### Option A: Centralized Identity Provider (IdP) with OIDC + +A single IdP (e.g., Entra ID, Okta, Keycloak, Auth0) issues tokens consumed by all components. + +- Memgraph Enterprise: native OIDC support via external auth module +- Memgraph Lab Enterprise (Docker): native OIDC support +- LightRAG: would need a custom auth dependency or reverse-proxy layer that validates OIDC tokens +- unstructured2graph: token-passing middleware or service account credentials + +### Option B: Token Broker / API Gateway + +An API gateway (e.g., Kong, APISIX, Nginx + OAuth2-proxy) sits in front of LightRAG and unstructured2graph, handling authentication centrally. Memgraph uses its own OIDC/LDAP integration. + +### Option C: Shared JWT with a Lightweight Auth Service + +A custom auth service issues JWTs after authenticating against the IdP. All services validate the same JWT. Memgraph's external auth module is extended to validate these JWTs on Bolt connections. + +### Option D: Service Accounts + User SSO (Hybrid) + +User-facing SSO via OIDC for Memgraph Lab and a web frontend, while backend services (unstructured2graph, LightRAG) use service account credentials or short-lived tokens minted from the IdP's client-credentials flow. + +--- + +## 3. Token Type Considerations + +| Token Type | Pros | Cons | +|---|---|---| +| **OIDC ID Token (JWT)** | Standard claims (`sub`, `email`, `roles`), widely supported, Memgraph native support | Short-lived, needs refresh flow, not designed for API authorization | +| **OIDC Access Token (opaque or JWT)** | Designed for API auth, scoped, Memgraph accepts it | May require introspection endpoint if opaque | +| **Custom JWT** | Full control over claims, single validation across services | Must build issuance and rotation; not natively recognized by Memgraph | +| **API Key** | Simple, LightRAG supports it natively | No identity, no expiry by default, hard to revoke at scale | +| **SAML Assertion** | Enterprise-friendly, Memgraph supports it | XML-based, not practical for API-to-API calls | + +--- + +## 4. Client Connection Patterns + +| Client | Connects To | Mechanism | +|---|---|---| +| Browser user | Memgraph Lab | OIDC redirect flow (authorization code + PKCE) | +| Browser user | LightRAG WebUI | OIDC redirect or JWT from `/login` | +| Python SDK (unstructured2graph) | Memgraph (Bolt) | Token passed via Neo4j driver auth (`access_token=...;id_token=...`) | +| Python SDK (unstructured2graph) | LightRAG API | Bearer token or API key in HTTP header | +| CI/CD / batch jobs | Memgraph + LightRAG | Client-credentials (machine-to-machine) tokens | +| Memgraph Lab | Memgraph (Bolt) | Forwards user's SSO token to database | + +--- + +## 5. LightRAG AI Authentication + +LightRAG's API provides RAG queries, document management, and an Ollama-compatible chat interface. Authentication options: + +1. **Extend LightRAG's FastAPI auth dependency** — replace `get_combined_auth_dependency()` in `lightrag/api/utils_api.py` with OIDC token validation (e.g., using `python-jose` or `authlib`). +2. **Reverse proxy with token validation** — Nginx/Envoy validates OIDC tokens before forwarding to LightRAG; LightRAG runs unauthenticated behind the proxy. +3. **Keep LightRAG's JWT but federate login** — LightRAG's `/login` endpoint delegates to the IdP instead of local username/password. The `AuthHandler` class in `lightrag/api/auth.py` is the extension point. + +--- + +## 6. Follow-Up Questions + +Before designing the solution, I need clarity on the following: + +### Deployment & Infrastructure + +1. **Are you running Memgraph Enterprise or Community?** OIDC/SAML SSO and RBAC are Enterprise-only features. If Community, we're limited to basic username/password on Bolt, and the SSO scope shrinks to LightRAG + any web layer. + +2. **How is the stack deployed?** Docker Compose? Kubernetes? Bare metal? This affects whether an API gateway or sidecar proxy approach is practical. + +3. **Is Memgraph Lab deployed via Docker or used as a desktop app?** SSO in Lab requires the Docker deployment. + +4. **Is there an existing Identity Provider (IdP)?** If so, which one — Entra ID, Okta, Keycloak, Google Workspace, or something else? Or do we need to stand one up (e.g., Keycloak)? + +### Users & Access Patterns + +5. **Who are the users?** Internal team only, or external/customer-facing? This affects the complexity of the auth flow and whether self-registration is needed. + +6. **Do you need role-based access control (RBAC)?** For example: some users can only query (read), others can ingest documents (write), admins manage the system. If so, where should roles be enforced — at the IdP level, Memgraph level, LightRAG level, or all? + +7. **Are there machine-to-machine (M2M) clients?** For example, CI/CD pipelines, cron jobs, or other services calling unstructured2graph programmatically. These typically need client-credentials flow rather than user login. + +8. **Is multi-tenancy a concern?** Should different teams/orgs see different subsets of the knowledge graph? + +### LightRAG Specifics + +9. **Are you running LightRAG as a standalone server (`lightrag-server`) or only as an embedded Python library?** The SSO approach differs significantly — a server has HTTP endpoints to protect, while a library delegates auth to the calling application. + +10. **Do you need to secure the LightRAG WebUI separately from the API?** The WebUI currently falls back to guest access if no auth accounts are configured. + +11. **Which LightRAG endpoints need protection?** All of them, or only write operations (document upload, graph mutation) while queries remain open? + +### Token & Session Management + +12. **What token lifetime is acceptable?** Short-lived (minutes) with refresh, or longer-lived (hours)? This affects UX for interactive users vs. batch processes. + +13. **Do you need token revocation?** For example, if someone leaves the team, can you wait for tokens to expire, or must you revoke immediately? + +14. **Should the Python SDK (unstructured2graph) handle token refresh transparently**, or is it acceptable to require callers to manage their own tokens? + +### Scope & Priority + +15. **Is this for a production deployment or a development/POC setup?** This determines how much infrastructure (IdP, gateway, TLS) to invest in upfront. + +16. **What's the priority order?** If we can't do everything at once, which is most important: + - Securing Memgraph Bolt connections? + - SSO for Memgraph Lab? + - Protecting LightRAG API/WebUI? + - Unified identity across all three? + +17. **Are there compliance requirements** (SOC2, HIPAA, GDPR) that constrain token storage, session length, or audit logging? diff --git a/unstructured2graph/spec/sso-shared-token-patch.md b/unstructured2graph/spec/sso-shared-token-patch.md new file mode 100644 index 00000000..ce3ba319 --- /dev/null +++ b/unstructured2graph/spec/sso-shared-token-patch.md @@ -0,0 +1,111 @@ +# Patch: Hot-Reloadable API Key for LightRAG + +**Status:** Proposal +**Date:** 2026-04-01 +**Targets:** [LightRAG](https://github.com/HKUDS/LightRAG) upstream + +## Problem + +`LIGHTRAG_API_KEY` is read once at startup and captured in a closure. Rotating the key requires restarting the server, which drops in-flight requests and interrupts users. + +## Root Cause + +In `lightrag/api/utils_api.py`, `get_combined_auth_dependency(api_key)` receives the key as a parameter and closes over it: + +```python +# Current code (simplified from utils_api.py) +def get_combined_auth_dependency(api_key: Optional[str] = None): + api_key_configured = api_key is not None + + async def combined_dependency( + request: Request, + api_key_header_value: Optional[str] = Security(api_key_header), + token: Optional[str] = Depends(oauth2_scheme_optional), + ): + # ... + if api_key_configured and api_key_header_value and api_key_header_value == api_key: + return # API key validation successful + # ... + + return combined_dependency +``` + +The `api_key` variable is bound at dependency-creation time, not at request time. + +## Fix + +Re-read `os.environ` on each request instead of comparing against the closure-captured value. + +### Patch for `lightrag/api/utils_api.py` + +```diff + def get_combined_auth_dependency(api_key: Optional[str] = None): +- api_key_configured = api_key is not None ++ # Store the initial key as fallback, but prefer the live env var ++ initial_api_key = api_key + + async def combined_dependency( + request: Request, + api_key_header_value: Optional[str] = Security(api_key_header), + token: Optional[str] = Depends(oauth2_scheme_optional), + ): ++ # Re-read env var on each request to support hot-reload ++ current_api_key = os.environ.get("LIGHTRAG_API_KEY") or initial_api_key ++ api_key_configured = current_api_key is not None ++ + # ... (whitelist check unchanged) ... + + if api_key_configured and api_key_header_value and api_key_header_value == current_api_key: + return # API key validation successful + + # ... (rest of auth logic unchanged) ... + + return combined_dependency +``` + +### Patch for `lightrag/api/lightrag_server.py` + +No change needed. `create_app()` still passes `api_key` to `get_combined_auth_dependency()` — it just becomes the fallback if the env var is unset. + +## Rotation flow after patching + +```bash +# No restart needed — just update the env var in the running process +# Option 1: If running via a process manager that supports env updates (e.g., systemd, Docker) +docker exec sh -c 'export LIGHTRAG_API_KEY=' + +# Option 2: Add a small admin endpoint (optional, see below) +curl -X POST http://localhost:9621/admin/rotate-key \ + -H "X-API-Key: $OLD_KEY" \ + -d '{"new_key": ""}' +``` + +### Optional: Admin endpoint for key rotation + +If you can't easily update env vars in the running process (e.g., Docker without exec), add a small admin endpoint: + +```python +# Add to lightrag/api/lightrag_server.py in create_app() + +@app.post("/admin/rotate-key") +async def rotate_key( + request: Request, + body: dict, + _=Depends(combined_auth), # must authenticate with current key +): + new_key = body.get("new_key") + if not new_key or len(new_key) < 32: + raise HTTPException(400, "new_key must be at least 32 characters") + os.environ["LIGHTRAG_API_KEY"] = new_key + return {"status": "rotated"} +``` + +This is self-securing: you need the current key to set a new one. + +## Performance Impact + +`os.environ.get()` is a dict lookup in CPython — negligible cost per request (nanoseconds). No caching or locking needed. + +## Contribution Path + +This is a small, backwards-compatible change suitable for an upstream PR to [HKUDS/LightRAG](https://github.com/HKUDS/LightRAG). If upstream doesn't accept it, it can be monkey-patched in `lightrag-memgraph` or applied as a fork patch. diff --git a/unstructured2graph/spec/sso-shared-token.md b/unstructured2graph/spec/sso-shared-token.md new file mode 100644 index 00000000..18f22dbe --- /dev/null +++ b/unstructured2graph/spec/sso-shared-token.md @@ -0,0 +1,120 @@ +# Shared Token Auth (Quick & Dirty) + +**Status:** Proposal +**Date:** 2026-03-31 + +## Goal + +Minimal auth that protects LightRAG WebUI and API with a single shared token. No IdP, no user accounts, no infrastructure changes. Something you can set up in 10 minutes. + +## How It Works + +LightRAG already supports API key auth natively. The flow: + +``` + User (browser/SDK) + | + | X-API-Key: + v + LightRAG Server (port 9621) + | + | bolt:// (MEMGRAPH_USER / MEMGRAPH_PASSWORD) + v + Memgraph (port 7687) +``` + +1. Generate a random token (e.g., `openssl rand -hex 32`) +2. Set `LIGHTRAG_API_KEY=` on the LightRAG server +3. Share the token with authorized users +4. Users pass it via `X-API-Key` header (API) or configure it in the WebUI + +Memgraph stays on username/password over Bolt — no changes needed there. + +## Configuration + +### LightRAG Server + +```bash +# .env or environment +LIGHTRAG_API_KEY=a3f8c1...your-random-token +``` + +Or via CLI: + +```bash +lightrag-server --key a3f8c1...your-random-token +``` + +### Python SDK (unstructured2graph callers) + +```python +import httpx + +headers = {"X-API-Key": os.environ["LIGHTRAG_API_KEY"]} +response = httpx.post("http://localhost:9621/query", headers=headers, json={...}) +``` + +### WebUI + +LightRAG's built-in WebUI checks for the API key. When `LIGHTRAG_API_KEY` is set, unauthenticated requests get a 401. The WebUI should prompt for the key — but note the caveat below. + +### curl + +```bash +curl -H "X-API-Key: $LIGHTRAG_API_KEY" http://localhost:9621/query -d '{"query": "..."}' +``` + +## Caveats + +1. **WebUI guest fallback** — LightRAG's WebUI falls back to guest access if no `AUTH_ACCOUNTS` are configured, even when `LIGHTRAG_API_KEY` is set. To fully lock down the WebUI, you may also need to set `AUTH_ACCOUNTS` or put a reverse proxy in front. + +2. **No identity** — everyone shares the same token. You can't distinguish who did what in logs or enforce per-user permissions. + +3. **No expiry** — the token lives until you rotate it manually. If it leaks, you must regenerate and redistribute. + +4. **No revocation per user** — rotating the token locks out everyone; you can't revoke one person's access. + +5. **HTTP in cleartext** — without TLS, the token travels in plaintext. In production, either: + - Enable LightRAG's built-in SSL (`--ssl --ssl-certfile cert.pem --ssl-keyfile key.pem`) + - Put it behind a TLS-terminating reverse proxy (Nginx, Caddy) + +6. **Memgraph and Memgraph Lab are separate** — this only covers LightRAG. Memgraph Bolt stays on its own username/password. Memgraph Lab has no shared-token mechanism (it uses SSO or its own login). + +## Token Rotation + +### Current limitation: restart required + +The API key **cannot be hot-reloaded** in upstream LightRAG. It is read once at startup and captured in a closure across three layers: + +1. `config.py` — `parse_args()` reads `LIGHTRAG_API_KEY` env var once into `global_args.key` +2. `lightrag_server.py` — `create_app()` captures it into a local `api_key` via `os.getenv()` once +3. `utils_api.py` — `get_combined_auth_dependency(api_key)` stores it in a closure compared on every request + +Even mutating `os.environ` at runtime has no effect — the closure still holds the old value. + +### Rotation without hot-reload + +```bash +# 1. Generate new token +NEW_TOKEN=$(openssl rand -hex 32) + +# 2. Update LightRAG server env and restart +export LIGHTRAG_API_KEY=$NEW_TOKEN +# restart lightrag-server + +# 3. Distribute new token to users +``` + +### Proposed patch: hot-reloadable API key + +A small change to `lightrag/api/utils_api.py` allows the key to be rotated by updating the env var without restarting the server. See `sso-shared-token-patch.md` for the full patch. + +## When to Outgrow This + +Move to the full SSO design (`sso-design.md`) when any of these become true: + +- You need to know *who* is making requests (audit trail) +- Different users need different permissions (RBAC) +- You have more than ~10 users sharing the token +- Compliance requires individual credentials +- You need to revoke a single user's access without disrupting others