PromptMan

PromptMan is a FastAPI application with a conversation-first API, RBAC, plugin support, and pluggable runtime caching.

The current product surface is focused on conversation threads and messages. Prompt chain versioning, analysis, and orchestration are part of the active API/UI.

Key Capabilities

Conversation thread CRUD with project-scoped access control.
Message append and message history listing.
Conversation import from JSON and plain text chain formats.
Thread-level lightweight analysis (message counts and content stats).
Prompt chain versioning with chain-level and version-level analysis.
Prompt Orchestrator preview with dual mode execution: try configured live LLM, fallback to heuristic output when unavailable.
Prompt version test-runs with RAG enrichment and non-failing LLM fallback mode.
Security metrics with RU/EN marker detection (injection_risk, contradiction_risk, ambiguity_risk) and marker logs.
RBAC (admin, developer, viewer) with project-level access assignment.
Runtime config management through admin endpoints.
Pluggable runtime cache backends: memory, redis, garnet, none.
Recursive plugin discovery and plugin modal sessions.
JWT auth with refresh flow.

Quick Start

Requirements

Python 3.11+
uv (recommended) or pip

Setup (uv)

uv sync --extra dev
.\.venv\Scripts\Activate.ps1
alembic upgrade head

Setup (pip)

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
alembic upgrade head

Run

uvicorn main:app --reload

UI: http://127.0.0.1:8000
API docs: http://127.0.0.1:8000/docs

Optional OpenTelemetry + SigNoz

OpenTelemetry support is optional and uses lazy import. Without OTel dependencies, PromptMan keeps running in standard mode.

SigNoz is expected to run separately (for example, official SigNoz Docker deployment on http://localhost:8080).

Install with optional OTel dependencies:

uv sync --extra otel

Enable OTel (host runtime):

$env:ENABLE_OTEL = "true"
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "localhost:4317"
$env:OTEL_SERVICE_NAME = "promptman"
$env:OTEL_SERVICE_NAMESPACE = "prompt-stack"
$env:OTEL_DEPLOYMENT_ENVIRONMENT = "dev"

PromptMan initializes and shuts down OTel inside FastAPI lifespan context manager (@asynccontextmanager).

Default state:

OpenTelemetry is disabled by default (ENABLE_OTEL=false).

Run PromptMan with local OTel Collector:

docker compose -f docker-compose.yml -f docker-compose.otel.yml up --build

Disable OTel (host runtime):

$env:ENABLE_OTEL = "false"

Stop local OTel Collector:

docker compose -f docker-compose.yml -f docker-compose.otel.yml down

Files used:

docker-compose.otel.yml
observability/otel-collector-config.yaml

Default endpoints:

SigNoz UI (external): http://localhost:8080
OTLP gRPC ingest (local collector): http://localhost:4317
OTLP HTTP ingest (local collector): http://localhost:4318

Exposed telemetry (when enabled):

traces: HTTP request spans and app lifecycle spans
metrics: HTTP request count, latency, 5xx errors, startup/shutdown durations
logs: PromptMan logs are mirrored to OTLP logs pipeline

OpenTelemetry Payload Reference

Название метрики, трейса, лога	Его описание
`promptman_http_requests_total`	Counter. Total HTTP requests by attributes: `http.method`, `http.route`, `http.status_code`.
`promptman_http_errors_total`	Counter. Error count. Increments for HTTP 5xx responses and explicit backend error events (`operation`, `error.type`).
`promptman_http_latency_ms`	Histogram (ms). Per-request HTTP latency with `http.method`, `http.route`, `http.status_code`.
`promptman_lifecycle_duration_ms`	Histogram (ms). App lifecycle durations with attribute `phase` (`startup`/`shutdown`).
`http.request`	Trace span created around each request in middleware; contains attributes like `http.method`, `http.route`, `http.client_ip`.
`promptman.otel` (logger stream)	OTLP log stream used by telemetry bridge. Receives Loguru records via telemetry sink (`level>=INFO`).
`request.start` / `request.end` / `request.exception`	Key structured application log events exported when OTel is enabled (request ingress/egress/errors with method/path/status/duration).

How To Enable / Disable OpenTelemetry

Host runtime:

Enable: set ENABLE_OTEL=true before app start.
Disable: set ENABLE_OTEL=false (or unset it).

Docker compose override (docker-compose.otel.yml):

Default is disabled: ENABLE_OTEL=${ENABLE_OTEL:-false}.
Enable explicitly for a run:

$env:ENABLE_OTEL = "true"
docker compose -f docker-compose.yml -f docker-compose.otel.yml up --build

Disable explicitly for a run:

$env:ENABLE_OTEL = "false"
docker compose -f docker-compose.yml -f docker-compose.otel.yml up --build

Dashboard template blueprints:

observability/signoz-dashboard-promptman.yaml
observability/signoz-dashboard-unified-stack.yaml

Use the PromptMan dashboard blueprint for API-only monitoring. Use the unified stack blueprint when PromptMan, PromptOrchestrator, and RagflowOrchestrator all export into the same collector and you want one cross-service view in SigNoz.

Unified Telemetry for PromptMan + PromptOrchestrator + RagflowOrchestrator

All three projects can push telemetry into one collector/sink. To split data in SigNoz, set unique service names:

PromptMan: OTEL_SERVICE_NAME=promptman
PromptOrchestrator: OTEL_SERVICE_NAME=prompt-orchestrator
RagflowOrchestrator: OTEL_SERVICE_NAME=ragflow-orchestrator

Keep the same OTLP endpoint for all three projects and use OTEL_SERVICE_NAMESPACE/OTEL_DEPLOYMENT_ENVIRONMENT for additional filtering.

Authentication

On an empty database, bootstrap the first admin:

POST /v1/auth/bootstrap-admin

Then sign in:

POST /v1/auth/login
POST /v1/auth/refresh
GET /v1/auth/status
GET /v1/auth/me
POST /v1/auth/me/password

API Surface

Version

GET /v1/version

Roles (admin)

GET /v1/roles

Users (admin)

GET /v1/users
POST /v1/users
GET /v1/users/{user_id}
PUT /v1/users/{user_id}
PUT /v1/users/{user_id}/projects
DELETE /v1/users/{user_id}

Projects (admin)

GET /v1/projects
GET /v1/projects/{project_id}
POST /v1/projects
PUT /v1/projects/{project_id}
DELETE /v1/projects/{project_id}

Admin Global Config (admin)

GET /v1/admin/config/
GET /v1/admin/config/{key}
PUT /v1/admin/config/{key} with query param value
GET /v1/admin/config/meta/providers
GET /v1/admin/config/meta/providers/{provider}/models
POST /v1/admin/config/llm/autoconfigure/preview
POST /v1/admin/config/llm/autoconfigure/apply

Conversations

POST /v1/conversations/threads
GET /v1/conversations/threads
GET /v1/conversations/threads/{thread_id}
DELETE /v1/conversations/threads/{thread_id}
POST /v1/conversations/threads/{thread_id}/messages
GET /v1/conversations/threads/{thread_id}/messages
POST /v1/conversations/import/json
POST /v1/conversations/import/text
GET /v1/conversations/import/{import_id}
POST /v1/conversations/analyze/{thread_id}

Prompt Versions

POST /v1/prompt-versions/chains
GET /v1/prompt-versions/chains
GET /v1/prompt-versions/chains/{chain_id}
GET /v1/prompt-versions/chains/{chain_id}/versions
GET /v1/prompt-versions/chains/{chain_id}/versions/{version_no}
POST /v1/prompt-versions/chains/{chain_id}/versions
POST /v1/prompt-versions/chains/{chain_id}/analyze
POST /v1/prompt-versions/chains/{chain_id}/versions/{version_no}/analyze
POST /v1/prompt-versions/chains/{chain_id}/versions/{version_no}/orchestrate
POST /v1/prompt-versions/chains/{chain_id}/versions/{version_no}/test-runs
GET /v1/prompt-versions/chains/{chain_id}/versions/{version_no}/test-runs
GET /v1/prompt-versions/test-runs

Plugins

GET /v1/plugins
POST /v1/plugins/_rescan
POST /v1/plugins/{plugin_name}/_load
POST /v1/plugins/{plugin_name}/_reload
DELETE /v1/plugins/{plugin_name}
POST /v1/plugins/{plugin_name}/health

Modal session endpoints:

GET /v1/plugins/{plugin_name}/modals
POST /v1/plugins/{plugin_name}/modals
GET /v1/plugins/{plugin_name}/modals/{session_id}
PATCH /v1/plugins/{plugin_name}/modals/{session_id}/controls/{control_name}
POST /v1/plugins/{plugin_name}/modals/{session_id}/stop
DELETE /v1/plugins/{plugin_name}/modals/{session_id}

Environment Variables

Core

DATABASE_URL (default local SQLite)
PROMPTMAN_KEY (required stable key for persistent deployments)
PROMPTMAN_KEY_PREVIOUS (optional key rotation support)
BOOTSTRAP_ADMIN_USERNAME (optional)
BOOTSTRAP_ADMIN_PASSWORD (optional)
LOG_LEVEL (optional)
SHOW_CONSOLE_SOURCE (optional)

Runtime Cache / Config Keys

Managed via app_settings and admin config API:

PROMPTMAN_CACHE_ENABLED
PROMPTMAN_CACHE_MAX_ENTRIES
PROMPTMAN_CACHE_PERSISTENCE_ENABLED
PROMPTMAN_CACHE_PERSISTENCE_LIMIT
PROMPTMAN_RUNTIME_CACHE_BACKEND
PROMPTMAN_RUNTIME_CACHE_URL
PROMPTMAN_RUNTIME_CACHE_NAMESPACE
PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL
PROMPTMAN_PLUGINS_SIGNED_ONLY

Simulation Test Keys

Managed via app_settings and admin config API:

TEST_LLM_PROVIDER
TEST_LLM_MODEL
TEST_LLM_BASE_URL
TEST_LLM_API_TOKEN
TEST_LLM_TIMEOUT_SECONDS
TEST_LLM_USE_OPTIMIZER_FALLBACK
TEST_RAG_ENABLED
TEST_RAG_SOURCE_PATH
TEST_RAG_TOP_K

Optimizer / Compression LLM Keys

Managed via app_settings and admin config API:

OPTIMIZER_PROVIDER
OPTIMIZER_MODEL
OPTIMIZER_BASE_URL
OPTIMIZER_API_TOKEN
OPTIMIZER_BACKEND
OPTIMIZER_TIMEOUT_SECONDS
PROMPT_COMPRESSION_PROVIDER
PROMPT_COMPRESSION_MODEL
PROMPT_COMPRESSION_BASE_URL
PROMPT_COMPRESSION_API_TOKEN
PROMPT_COMPRESSION_BACKEND

Notes:

TEST_LLM_* are used by prompt-version simulation test runs.
If TEST_LLM_USE_OPTIMIZER_FALLBACK=true, missing test LLM fields fall back to OPTIMIZER_*.
RAG chunks are loaded from TEST_RAG_SOURCE_PATH and appended to the test prompt when enabled.
Prompt Orchestrator preview uses dedicated profiles for optimizer/compression and falls back to heuristic output if live invocation is unavailable.
The UI displays explicit Optimizer/Compression mode badges (LIVE, FALLBACK, UNKNOWN) in Prompt Orchestrator preview.

Simulation Testing (Logs + RAG + Security)

PromptMan now includes a simulation runner similar to PromptOrchestrator style logs:

Script: simulations/conversation_simulation_test.py
Log file: simulations/conversation_simulation_test.log
Default RAG knowledge file: simulations/rag_knowledge.md

Run one simulation cycle:

.\.venv\Scripts\python.exe .\simulations\conversation_simulation_test.py --chain-id 51 --version-no 5 --cycles 1

Reset database and seed explicit test data before simulation:

.\.venv\Scripts\python.exe .\simulations\conversation_simulation_test.py --seed --reset-db --scale small --chain-id 1 --version-no 1 --cycles 3

Direct seed command (without simulation):

.\.venv\Scripts\python.exe .\scripts\seed_demo_data.py --reset-db --scale small

Generate Russian-only synthetic data:

.\.venv\Scripts\python.exe .\scripts\seed_demo_data.py --reset-db --scale medium --lang-ru

Generate English-only synthetic data:

.\.venv\Scripts\python.exe .\scripts\seed_demo_data.py --reset-db --scale medium --lang-en

Generate mixed RU/EN synthetic data:

.\.venv\Scripts\python.exe .\scripts\seed_demo_data.py --reset-db --scale medium --lang-mix

More varied demo data for analysis-heavy runs:

.\.venv\Scripts\python.exe .\scripts\seed_demo_data.py --reset-db --scale large

The seeded demo set is intentionally mixed:

threads have different lengths, and most are longer than a simple 2-question / 2-answer exchange
prompt versions are written to produce different analysis signals, including tokens, reliability, cacheability, and selected security markers
synthetic thread messages and chain versions periodically include deterministic threat snippets (injection/contradiction/ambiguity) for benchmark-style security checks

UI Notes

Prompt versions metrics table includes full risk and delta columns:
- Version, Tokens, Reliability, Cache hit %
- Injection risk, Contradiction risk, Ambiguity risk
- Delta tokens, Delta reliability, Delta cache %
- Delta injection risk, Delta contradiction risk, Delta ambiguity risk
Prompt Orchestrator preview shows mode badges for optimizer and compression (LIVE/FALLBACK/UNKNOWN) and detailed mode reason text.

Complex simulation example with seeded data and repeated analysis cycles:

.\.venv\Scripts\python.exe .\simulations\conversation_simulation_test.py --seed --reset-db --scale large --chain-id 1 --version-no 1 --cycles 5

What is logged per cycle:

Full prompt
Prompt with RAG context
Fixed / semi-fixed / variable prompt parts
LLM snapshot (provider/model/backend/base_url)
LLM response and errors
Token usage
Security metrics (injection_risk, contradiction_risk, ambiguity_risk, markers)
Prompt analysis metrics

Runtime Cache Backends

PromptMan supports four runtime cache backend modes:

memory
- local in-process cache
redis
- external Redis via RESP
garnet
- external Garnet via RESP (same URL format and client path as Redis)
none
- disables runtime cache reads/writes

Implementation note:

redis and garnet use the same RESP backend client (redis Python package).
If external backend initialization fails, runtime cache falls back to memory.
PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL=true forces none behavior regardless of backend name.

Quick Config Examples

Set via environment variables before app start.

1. Local in-memory cache (`memory`)

$env:PROMPTMAN_RUNTIME_CACHE_BACKEND = "memory"
$env:PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL = "false"
uvicorn main:app --reload

2. Redis cache (`redis`)

$env:PROMPTMAN_RUNTIME_CACHE_BACKEND = "redis"
$env:PROMPTMAN_RUNTIME_CACHE_URL = "redis://127.0.0.1:6379/0"
$env:PROMPTMAN_RUNTIME_CACHE_NAMESPACE = "promptman"
$env:PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL = "false"
uvicorn main:app --reload

3. Garnet cache (`garnet`)

$env:PROMPTMAN_RUNTIME_CACHE_BACKEND = "garnet"
$env:PROMPTMAN_RUNTIME_CACHE_URL = "redis://127.0.0.1:6379/0"
$env:PROMPTMAN_RUNTIME_CACHE_NAMESPACE = "promptman"
$env:PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL = "false"
uvicorn main:app --reload

Use the same RESP URL scheme as Redis (redis://host:port/db).

4. Disable runtime cache (`none`)

$env:PROMPTMAN_RUNTIME_CACHE_BACKEND = "none"
$env:PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL = "true"
uvicorn main:app --reload

Change Cache Backend At Runtime (Admin API)

You can switch backend without restart through admin config endpoints.

PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_BACKEND?value=redis
PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_URL?value=redis://127.0.0.1:6379/0
PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_NAMESPACE?value=promptman
PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL?value=false

Garnet runtime switch example:

PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_BACKEND?value=garnet
PUT /v1/admin/config/PROMPTMAN_RUNTIME_CACHE_URL?value=redis://127.0.0.1:6379/0

Docker Compose Examples

Below are practical compose snippets for each runtime cache mode.

Redis + PromptMan

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      PROMPTMAN_RUNTIME_CACHE_BACKEND: redis
      PROMPTMAN_RUNTIME_CACHE_URL: redis://redis:6379/0
      PROMPTMAN_RUNTIME_CACHE_NAMESPACE: promptman
      PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL: "false"
    depends_on:
      - redis

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"

Garnet + PromptMan

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      PROMPTMAN_RUNTIME_CACHE_BACKEND: garnet
      PROMPTMAN_RUNTIME_CACHE_URL: redis://garnet:6379/0
      PROMPTMAN_RUNTIME_CACHE_NAMESPACE: promptman
      PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL: "false"
    depends_on:
      - garnet

  garnet:
    image: ghcr.io/microsoft/garnet:latest
    ports:
      - "6379:6379"

Local In-Process Cache (Memory)

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      PROMPTMAN_RUNTIME_CACHE_BACKEND: memory
      PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL: "false"

No Runtime Cache

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    environment:
      PROMPTMAN_RUNTIME_CACHE_BACKEND: none
      PROMPTMAN_RUNTIME_CACHE_DISABLE_INTERNAL: "true"

Quick start for any snippet:

docker compose up --build

Database Notes

Current active domain model includes users, roles, projects, project access, conversations, and imports.
Legacy prompt/optimizer tables were removed by Alembic migration 20260603_0015.

Verification Snapshot (2026-06-03)

Automated tests run from project root:

.\.venv\Scripts\python.exe -m pytest -q

Load & Stress Benchmark (PostgreSQL Memory vs Redis)

The benchmark script was run twice on 2026-06-03:

.\.venv\Scripts\python.exe scripts\run_db_concurrency_benchmark.py

Scenario profile:

load_low: 16 concurrent users, 12s
load_high: 48 concurrent users, 12s
stress: 96 concurrent users, 16s

Run A (generated at UTC 14:18:09):

Mode	Scenario	Users	RPS	P95 (ms)	Failure %
postgres_memory	load_low	16	169.67	131.67	0.0
postgres_memory	load_high	48	164.43	387.18	0.0
postgres_memory	stress	96	157.30	811.67	0.0
postgres_redis	load_low	16	174.05	129.57	0.0
postgres_redis	load_high	48	164.63	386.39	0.05
postgres_redis	stress	96	40.80	10002.16	15.79

Run B (generated at UTC 14:20:33):

Mode	Scenario	Users	RPS	P95 (ms)	Failure %
postgres_memory	load_low	16	171.40	130.87	0.0
postgres_memory	load_high	48	165.88	386.56	0.0
postgres_memory	stress	96	17.90	10014.31	49.87
postgres_redis	load_low	16	173.07	130.18	0.0
postgres_redis	load_high	48	164.99	383.78	0.0
postgres_redis	stress	96	27.30	10013.92	30.52

Conclusions:

For low/high load scenarios, memory and redis are nearly identical.
In stress, both modes showed instability across runs (timeouts and higher failure rate), which indicates a broader system bottleneck, not only runtime cache backend behavior.
Based on current data, switching to Garnet is optional and should be treated as a later optimization after stabilizing stress-path bottlenecks (DB capacity, worker model, and request concurrency profile).

Detailed artifacts:

Run A report: loadtests/results/cache_compare/concurrency_20260603_171630/db_concurrency_report.md
Run A JSON: loadtests/results/cache_compare/concurrency_20260603_171630/db_concurrency_results.json
Run B report: loadtests/results/cache_compare/concurrency_20260603_171853/db_concurrency_report.md
Run B JSON: loadtests/results/cache_compare/concurrency_20260603_171853/db_concurrency_results.json

Plugins

See plugins/README.md for plugin lifecycle, endpoint contracts, signatures, and modal support.

Development

ruff check .
ruff format .
mypy .

License

MIT. See LICENSE.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
alembic		alembic
app_core		app_core
auth		auth
cache		cache
crud		crud
database		database
loadtests		loadtests
middleware		middleware
models		models
observability		observability
plugin_engine		plugin_engine
plugins		plugins
plugins_signin		plugins_signin
routes		routes
runtime_cache		runtime_cache
schemas		schemas
scripts		scripts
simulations		simulations
tests		tests
ui		ui
.dockerignore		.dockerignore
.gitignore		.gitignore
1.png		1.png
2.png		2.png
3.png		3.png
Dockerfile		Dockerfile
Dockerfile.dev		Dockerfile.dev
LICENSE		LICENSE
P_240x240.png		P_240x240.png
PromptMan_240x240.png		PromptMan_240x240.png
README.md		README.md
alembic.ini		alembic.ini
app_settings.py		app_settings.py
database.py		database.py
docker-compose-dev.yml		docker-compose-dev.yml
docker-compose.otel.yml		docker-compose.otel.yml
docker-compose.yml		docker-compose.yml
main.py		main.py
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
ruff.toml		ruff.toml
screen2.png		screen2.png
security.py		security.py
test_imports.py		test_imports.py

Latest commit

History

Folders and files

Repository files navigation

PromptMan

Key Capabilities

Quick Start

Requirements

Setup (uv)

Setup (pip)

Run

Optional OpenTelemetry + SigNoz

OpenTelemetry Payload Reference

How To Enable / Disable OpenTelemetry

Unified Telemetry for PromptMan + PromptOrchestrator + RagflowOrchestrator

Authentication

API Surface

Version

Roles (admin)

Users (admin)

Projects (admin)

Admin Global Config (admin)

Conversations

Prompt Versions

Plugins

Environment Variables

Core

Runtime Cache / Config Keys

Simulation Test Keys

Optimizer / Compression LLM Keys

Simulation Testing (Logs + RAG + Security)

UI Notes

Runtime Cache Backends

Quick Config Examples

1. Local in-memory cache (memory)

2. Redis cache (redis)

3. Garnet cache (garnet)

4. Disable runtime cache (none)

Change Cache Backend At Runtime (Admin API)

Docker Compose Examples

Redis + PromptMan

Garnet + PromptMan

Local In-Process Cache (Memory)

No Runtime Cache

Database Notes

Verification Snapshot (2026-06-03)

Load & Stress Benchmark (PostgreSQL Memory vs Redis)

Plugins

Development

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

1. Local in-memory cache (`memory`)

2. Redis cache (`redis`)

3. Garnet cache (`garnet`)

4. Disable runtime cache (`none`)