Skip to content

# Security Report: Sensitive Configuration Values Leaked via PrometheusPropertiesException to HTTP Clients #2286

Open
#2291
Open
# Security Report: Sensitive Configuration Values Leaked via PrometheusPropertiesException to HTTP Clients#2286
#2291

Description

@manqingzhou
manqingzhou
opened on Jul 8, 2026

Security Report: Sensitive Configuration Values Leaked via PrometheusPropertiesException to HTTP Clients

1. Vulnerability Summary

Field Value
Product prometheus/client_java
Affected Version 1.8.0 (main branch); all prior 1.x releases using prometheus-metrics-config are likely affected
Component prometheus-metrics-config (root cause), prometheus-metrics-exporter-httpserver (disclosure vector)
Vulnerable File prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/Util.java
Vulnerable Lines 26-27, 90-91, 132-133, 148-149, 164-165, 196-197
CWE CWE-209: Generation of Error Message Containing Sensitive Information
CVSS 3.1 Score 5.3 (Medium)
CVSS 3.1 Vector CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N
Discoverer Security Research

2. Vulnerability Description

The Util class in the prometheus-metrics-config module embeds raw, unredacted configuration property values directly into PrometheusPropertiesException error messages. When a property value cannot be parsed as the expected type (integer, double, long, boolean, or double list), the exception message includes the full property key and its verbatim value using the pattern fullKey + "=" + property + ": Expecting <type> value".

PrometheusPropertiesException extends RuntimeException (unchecked), which means it propagates freely through the call stack without requiring explicit handling. When this exception reaches the HttpExchangeAdapter in the prometheus-metrics-exporter-httpserver module, the sendErrorResponseWithStackTrace() method serializes the complete exception -- including the exception message containing the raw property value and the full Java stack trace -- into an HTTP 500 response body and sends it directly to the requesting client.

This creates an information disclosure vulnerability: if a configuration property inadvertently contains sensitive data (API keys, database passwords, tokens) and that value fails type parsing, the sensitive value is leaked verbatim to any unauthenticated HTTP client that triggers the error path. The /metrics endpoint served by the built-in HTTPServer is accessible without authentication by default.

3. Affected Component Architecture

The vulnerability involves a two-stage pipeline: (1) raw values are embedded in exception messages at the configuration layer, and (2) those messages are forwarded unredacted to HTTP clients at the exporter layer.

Environment Variable / Properties File
        |
        |  raw property value read
        v
Util.loadInteger() / loadDouble() / loadLong() / loadBoolean() / loadDoubleList()
        |
        |  parse fails -> PrometheusPropertiesException("key=RAW_VALUE: Expecting ...")
        v
PrometheusPropertiesException (RuntimeException, unchecked)
        |
        |  propagates unchecked through call stack
        v
PrometheusScrapeHandler.handleRequest()
        |
        |  catch (RuntimeException e) -> exchange.handleException(e)
        v
HttpExchangeAdapter.sendErrorResponseWithStackTrace()
        |
        |  exception.printStackTrace() -> HTTP 500 response body
        v
HTTP Client receives full exception message + stack trace
        (including the raw property value)

Key Design Flaws

  1. Root cause (Util.java): Raw property values are concatenated into exception messages instead of being redacted or truncated.
  2. Propagation enabler (PrometheusPropertiesException): Being an unchecked RuntimeException, the exception bypasses all intermediate error handling.
  3. Disclosure vector (HttpExchangeAdapter): The full stack trace, including the exception message with raw property values, is written to the HTTP response body.

4. Vulnerable Code

4.1 Root Cause: Raw Property Values in Exception Messages

File: prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/Util.java

All type-parsing methods in Util.java include the verbatim property value in exception messages. There are six instances:

Instance 1 -- loadBoolean() (lines 26-27):

throw new PrometheusPropertiesException(
    String.format("%s: Expecting 'true' or 'false'. Found: %s", fullKey, property));

Instance 2 -- loadDoubleList() (lines 90-91):

throw new PrometheusPropertiesException(
    fullKey + "=" + property + ": Expecting comma separated list of double values");

Instance 3 -- loadInteger() (lines 132-133):

throw new PrometheusPropertiesException(
    fullKey + "=" + property + ": Expecting integer value");

Instance 4 -- loadDouble() (lines 148-149):

throw new PrometheusPropertiesException(
    fullKey + "=" + property + ": Expecting double value");

Instance 5 -- loadLong() (lines 164-165):

throw new PrometheusPropertiesException(
    fullKey + "=" + property + ": Expecting long value");

Instance 6 -- assertValue() (lines 196-197):

String fullMessage = String.format("%s: %s Found: %s", fullKey, message, number);
throw new PrometheusPropertiesException(fullMessage);

In every case, the raw, unredacted value of property is interpolated directly into the exception message string.

4.2 Exception Type: Unchecked RuntimeException

File: prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/PrometheusPropertiesException.java

@StableApi
public class PrometheusPropertiesException extends RuntimeException {

  private static final long serialVersionUID = 0L;

  public PrometheusPropertiesException(String msg) {
    super(msg);
  }

  public PrometheusPropertiesException(String msg, Exception cause) {
    super(msg, cause);
  }
}

Because PrometheusPropertiesException extends RuntimeException, it is not required to be declared in method signatures and is not required to be caught. This allows it to propagate unchecked through the entire call stack to the HTTP handler layer.

4.3 Propagation Path: PrometheusScrapeHandler

File: prometheus-metrics-exporter-common/src/main/java/io/prometheus/metrics/exporter/common/PrometheusScrapeHandler.java (lines 99-105)

public void handleRequest(PrometheusHttpExchange exchange) throws IOException {
    try {
      // ... scrape metrics ...
    } catch (IOException e) {
      exchange.handleException(e);
    } catch (RuntimeException e) {
      exchange.handleException(e);       // <-- PrometheusPropertiesException caught here
    } finally {
      exchange.close();
    }
}

The catch (RuntimeException e) clause catches any PrometheusPropertiesException and forwards it to the exchange adapter's handleException() method without sanitizing the exception message.

4.4 Disclosure Vector: HTTP Response with Stack Trace

File: prometheus-metrics-exporter-httpserver/src/main/java/io/prometheus/metrics/exporter/httpserver/HttpExchangeAdapter.java (lines 98-114)

@Override
public void handleException(RuntimeException e) {
    sendErrorResponseWithStackTrace(e);
}

private void sendErrorResponseWithStackTrace(Exception requestHandlerException) {
    if (!responseSent) {
      responseSent = true;
      try {
        StringWriter stringWriter = new StringWriter();
        PrintWriter printWriter = new PrintWriter(stringWriter);
        printWriter.write("An Exception occurred while scraping metrics: ");
        requestHandlerException.printStackTrace(new PrintWriter(printWriter));
        byte[] stackTrace = stringWriter.toString().getBytes(StandardCharsets.UTF_8);
        httpExchange.getResponseHeaders().set("Content-Type", "text/plain; charset=utf-8");
        httpExchange.sendResponseHeaders(500, stackTrace.length);
        httpExchange.getResponseBody().write(stackTrace);
      } catch (IOException errorWriterException) {
        // ...
      }
    }
}

The requestHandlerException.printStackTrace() call serializes the full exception chain -- including the PrometheusPropertiesException message containing the raw property value -- into a StringWriter, which is then written verbatim to the HTTP response body.

4.5 Call Sites: Properties Loaded During Initialization

File: prometheus-metrics-config/src/main/java/io/prometheus/metrics/config/ExporterHttpServerProperties.java (lines 34-44)

static ExporterHttpServerProperties load(PropertySource propertySource)
    throws PrometheusPropertiesException {
  Integer port = Util.loadInteger(PREFIX, PORT, propertySource);  // <-- can throw with raw value
  Util.assertValue(port, t -> t > 0, "Expecting value > 0.", PREFIX, PORT);

  Boolean preferUncompressedResponse =
      Util.loadBoolean(PREFIX, PREFER_UNCOMPRESSED_RESPONSE, propertySource);

  return new ExporterHttpServerProperties(
      port, preferUncompressedResponse != null && preferUncompressedResponse);
}

The Util.loadInteger() call parses the io.prometheus.exporter.http_server.port property. If the value is not a valid integer, the exception message includes the raw value.

5. CVSS 3.1 Detailed Breakdown

Metric Value Justification
Attack Vector (AV) Network (N) The /metrics HTTP endpoint is network-accessible. The HTTPServer binds to a network socket (default: all interfaces).
Attack Complexity (AC) Low (L) No special conditions required beyond network access to the metrics port. The attacker only needs to send a standard HTTP request to the endpoint during a configuration error condition.
Privileges Required (PR) None (N) The HTTPServer does not require authentication by default. The Authenticator is an optional builder parameter that is null unless explicitly configured.
User Interaction (UI) None (N) No user interaction is needed. The attacker sends an HTTP request directly to the endpoint.
Scope (S) Unchanged (U) The vulnerability affects only the confidentiality of the vulnerable component; it does not cross trust boundaries to impact other components.
Confidentiality (C) Low (L) The disclosed information consists of raw configuration property values. If those values happen to contain credentials, API keys, or tokens (due to misconfiguration), the impact escalates, but the base vulnerability provides limited information.
Integrity (I) None (N) No data modification occurs.
Availability (A) None (N) No denial-of-service impact.

CVSS 3.1 Vector String: CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:L/I:N/A:N
CVSS 3.1 Base Score: 5.3 (Medium)

6. Attack Chain Analysis

6.1 Primary Attack Chain: Lazy Servlet Initialization

The most realistic exploitation scenario involves servlet-based deployments where initialization is deferred until the first HTTP request:

  1. A web application registers PrometheusMetricsServlet with load-on-startup unset or set to a negative value.
  2. The servlet container defers servlet initialization until the first request to the servlet's URL pattern.
  3. An environment variable or system property intended for a different purpose (e.g., IO_PROMETHEUS_EXPORTER_HTTP_SERVER_PORT) is set to a non-numeric sensitive value.
  4. On the first request, the servlet container calls PrometheusMetricsServlet() constructor, which calls PrometheusProperties.get(), triggering PrometheusPropertiesLoader.load().
  5. ExporterHttpServerProperties.load() calls Util.loadInteger(), which throws PrometheusPropertiesException with the raw value in the message.
  6. For the httpserver module, HttpExchangeAdapter.sendErrorResponseWithStackTrace() writes the full exception to the HTTP response. For servlet modules, the exception propagates to the container, which may display it in its default error page.

6.2 Secondary Attack Chain: Static Initializer Wrapping

When PrometheusProperties class initialization fails:

  1. PrometheusProperties.instance is initialized via static field initializer: PrometheusPropertiesLoader.load().
  2. If load() throws PrometheusPropertiesException, the JVM wraps it in ExceptionInInitializerError.
  3. The original PrometheusPropertiesException (containing the raw property value) is preserved as the cause.
  4. When printStackTrace() is called, the full cause chain is printed, including the PrometheusPropertiesException message with the raw value.

6.3 Tertiary Attack Chain: Custom Collectors Triggering Config Reload

If application code calls PrometheusProperties.get() or accesses configuration lazily within a custom Collector.collect() method, any resulting PrometheusPropertiesException will propagate through PrometheusScrapeHandler.handleRequest() to HttpExchangeAdapter.sendErrorResponseWithStackTrace(), exposing the raw property value in the HTTP response.

7. Proof of Concept

7.1 Prerequisites

  • A Java application using prometheus-metrics-exporter-httpserver with the built-in HTTPServer
  • Network access to the metrics HTTP port

7.2 Reproduction Steps

Step 1: Set a configuration property to a value that will fail type parsing.

The value should represent sensitive data that an operator might accidentally place in a Prometheus configuration property. For example, set the histogram upper bounds to a value containing a credential:

# Simulates an operator who accidentally assigns a database password
# to a numeric configuration property
export IO_PROMETHEUS_METRICS_HISTOGRAM_CLASSIC_UPPER_BOUNDS="db_pass:S3cretK3y!@host:5432"

Alternatively, to target the HTTP server port property:

export IO_PROMETHEUS_EXPORTER_HTTP_SERVER_PORT="my_secret_api_key_12345"

Step 2: Start the application with a Prometheus HTTP server.

import io.prometheus.metrics.core.metrics.Histogram;
import io.prometheus.metrics.exporter.httpserver.HTTPServer;

public class VulnerableApp {
    public static void main(String[] args) throws Exception {
        // Register a histogram to ensure histogram config properties are read
        Histogram histogram = Histogram.builder()
            .name("example_histogram")
            .help("An example histogram")
            .register();

        HTTPServer server = HTTPServer.builder()
            .port(9400)
            .buildAndStart();

        System.out.println("Server started on port " + server.getPort());
        Thread.currentThread().join();
    }
}

Step 3: Request the metrics endpoint.

curl -s http://localhost:9400/metrics

Step 4: Observe the raw property value in the HTTP response.

Expected HTTP 500 response body:

An Exception occurred while scraping metrics: io.prometheus.metrics.config.PrometheusPropertiesException:
  io.prometheus.metrics.histogram.classic_upper_bounds=db_pass:S3cretK3y!@host:5432: Expecting comma separated list of double values
    at io.prometheus.metrics.config.Util.loadDoubleList(Util.java:90)
    at io.prometheus.metrics.config.MetricsProperties.load(MetricsProperties.java:...)
    at io.prometheus.metrics.config.PrometheusPropertiesLoader.load(PrometheusPropertiesLoader.java:25)
    at io.prometheus.metrics.config.PrometheusProperties.<clinit>(PrometheusProperties.java:17)
    at io.prometheus.metrics.exporter.common.PrometheusScrapeHandler.<init>(PrometheusScrapeHandler.java:37)
    at io.prometheus.metrics.exporter.httpserver.MetricsHandler.<init>(MetricsHandler.java:18)
    at io.prometheus.metrics.exporter.httpserver.HTTPServer.<init>(HTTPServer.java:88)
    ...

The raw value db_pass:S3cretK3y!@host:5432 is fully visible in both the exception message and the HTTP response body.

7.3 Servlet-Based Reproduction

For servlet containers with lazy initialization:

web.xml (no load-on-startup):

<servlet>
    <servlet-name>prometheus</servlet-name>
    <servlet-class>
        io.prometheus.metrics.exporter.servlet.jakarta.PrometheusMetricsServlet
    </servlet-class>
    <!-- No load-on-startup: servlet initialized on first request -->
</servlet>
<servlet-mapping>
    <servlet-name>prometheus</servlet-name>
    <url-pattern>/metrics</url-pattern>
</servlet-mapping>

Step 1: Set an invalid property with a sensitive value.

export IO_PROMETHEUS_EXPORTER_HTTP_SERVER_PORT="Bearer_eyJhbGciOiJIUzI1NiJ9.secret"

Step 2: Deploy the application and send the first request.

curl -s http://localhost:8080/metrics

Step 3: The servlet container receives the PrometheusPropertiesException.

The Jakarta and Javax servlet adapters re-throw the exception (throw e; in handleException()), delegating error handling to the servlet container. Many servlet containers (Tomcat, Jetty) include the exception message in their default error page, disclosing the raw property value.

7.4 Integer Port Property Example

# Set port to a value containing an OAuth token
export IO_PROMETHEUS_EXPORTER_HTTP_SERVER_PORT="ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

When ExporterHttpServerProperties.load() calls Util.loadInteger(), the thrown exception message will be:

io.prometheus.exporter.http_server.port=ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: Expecting integer value

This message, containing the full GitHub Personal Access Token, is then written to the HTTP response body via sendErrorResponseWithStackTrace().

8. Impact Analysis

8.1 Direct Impact: Sensitive Configuration Value Disclosure

Raw configuration property values are exposed to unauthenticated HTTP clients. The severity depends on what those values contain:

Scenario Leaked Data Severity
Database password in wrong property Full connection string/password Critical
API key/token in wrong property Full API key or bearer token Critical
Internal URL with embedded credentials Host, port, username, password High
Non-sensitive misconfiguration Configuration value Low

8.2 Root Cause vs. Symptom Distinction

This vulnerability is distinct from the general stack trace disclosure issue (CVE Report #2). While stack trace disclosure reveals internal implementation details (class names, method names, line numbers), this vulnerability specifically leaks the content of configuration property values. The distinction matters because:

  1. Stack traces reveal structural information about the application (reconnaissance value).
  2. Property values may contain credentials, tokens, or secrets (direct exploitation value).

The two vulnerabilities compound: the stack trace provides the delivery mechanism, and the raw property value in the exception message provides the sensitive payload.

8.3 Why Misconfiguration Happens

Configuration errors that trigger this vulnerability are not hypothetical:

  • Environment variable collisions: In containerized environments, environment variables from different services may overlap. An environment variable like IO_PROMETHEUS_EXPORTER_HTTP_SERVER_PORT might be set by an orchestration tool to a value containing metadata or credentials.
  • Copy-paste errors: Operators copying configuration blocks may accidentally assign a credential value to a Prometheus property.
  • Configuration management tools: Templating errors in tools like Ansible, Terraform, or Helm can substitute incorrect values into property placeholders.
  • Twelve-factor app patterns: Applications following the twelve-factor methodology store all configuration in environment variables, increasing the chance of sensitive values being present in the environment.

8.4 Deployment Context

The HTTPServer component is commonly used in:

  • Standalone Java applications without a servlet container
  • Java agents for JVM instrumentation
  • Microservices exposing metrics alongside application traffic

In Kubernetes deployments, the metrics port is frequently reachable from any pod in the cluster via the pod IP or service IP, and Prometheus scrape targets are often discoverable through service discovery. An attacker with network access to the cluster can enumerate and scrape all metrics endpoints.

9. Scope Clarification

Affected Modules

Module Role Impact
prometheus-metrics-config Root cause -- raw values embedded in exception messages All 6 throw sites in Util.java include verbatim property values
prometheus-metrics-exporter-httpserver Direct disclosure vector -- sendErrorResponseWithStackTrace() writes full exception to HTTP response Raw values visible in HTTP 500 response body
prometheus-metrics-exporter-servlet-jakarta Indirect disclosure vector -- handleException() re-throws to servlet container Raw values may appear in container error pages (depends on container configuration)
prometheus-metrics-exporter-servlet-javax Indirect disclosure vector -- handleException() re-throws to servlet container Same as Jakarta variant

Comparison: httpserver vs. Servlet Adapters

Behavior httpserver servlet-jakarta / servlet-javax
handleException(RuntimeException e) Calls sendErrorResponseWithStackTrace(e) -- writes full stack trace to response throw e; -- re-throws to servlet container
Disclosure guaranteed? Yes -- always writes stack trace to response body Depends on container error page configuration
Authentication by default? None (optional Authenticator) Depends on servlet container and web.xml security constraints

10. Recommended Remediation

10.1 Primary Fix: Redact Property Values in Exception Messages (Util.java)

Replace raw property value interpolation with a redacted form that indicates the value type and length without disclosing its content:

// BEFORE (vulnerable):
throw new PrometheusPropertiesException(
    fullKey + "=" + property + ": Expecting integer value");

// AFTER (remediated):
throw new PrometheusPropertiesException(
    fullKey + "=<redacted " + property.length() + " chars>: Expecting integer value");

Apply this pattern to all six throw sites in Util.java:

  • loadBoolean() line 27
  • loadDoubleList() line 91
  • loadInteger() line 133
  • loadDouble() line 149
  • loadLong() line 165
  • assertValue() line 197

10.2 Secondary Fix: Sanitize Error Responses in HttpExchangeAdapter

In HttpExchangeAdapter.sendErrorResponseWithStackTrace(), replace the full stack trace with a generic error message and log the details server-side:

private void sendErrorResponseWithStackTrace(Exception requestHandlerException) {
    if (!responseSent) {
      responseSent = true;
      try {
        // Log full details server-side only
        Logger.getLogger(this.getClass().getName())
            .log(Level.SEVERE,
                "An Exception occurred while scraping metrics.",
                requestHandlerException);

        // Send generic message to client -- no exception details
        byte[] message = "An internal error occurred while scraping metrics.\n"
            .getBytes(StandardCharsets.UTF_8);
        httpExchange.getResponseHeaders().set("Content-Type", "text/plain; charset=utf-8");
        httpExchange.sendResponseHeaders(500, message.length);
        httpExchange.getResponseBody().write(message);
      } catch (IOException errorWriterException) {
        // ... existing fallback logging ...
      }
    }
}

10.3 Structural Fix: Fail-Fast at Startup

Move all configuration validation to application startup time rather than allowing lazy initialization to defer errors to scrape time:

  1. Call PrometheusProperties.get() eagerly during HTTPServer.builder() (already done) and during servlet init().
  2. Validate all properties during PrometheusPropertiesLoader.load() (already done for known properties).
  3. Document that applications should call PrometheusProperties.get() during startup to surface configuration errors early.

10.4 Defense in Depth: Make PrometheusPropertiesException Checked

Consider changing PrometheusPropertiesException to extend Exception instead of RuntimeException. This would force all callers to explicitly handle or declare the exception, preventing accidental propagation to HTTP clients. Note: this is a breaking API change and would require a major version bump.

10.5 Remediation Priority

Fix Priority Breaking Change Effort
10.1 Redact values in exception messages P0 -- Critical No Low
10.2 Sanitize HTTP error responses P0 -- Critical No Low
10.3 Fail-fast at startup P1 -- High No Medium
10.4 Make exception checked P2 -- Medium Yes High

Fixes 10.1 and 10.2 provide independent layers of defense and should both be applied. Fix 10.1 eliminates the root cause (sensitive values in exception messages). Fix 10.2 eliminates the disclosure vector (stack traces in HTTP responses). Together they provide defense in depth.

11. References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions