Skip to content

OTLP HTTP exporter (OkHttp sender) silently drops client mTLS certificate when no ca certificates are configured #8562

Description

@anoopjb

Description

When configuring client mTLS on an OTLP HTTP exporter via setClientTls(privateKeyPem, certificatePem) (or the equivalent otel.exporter.otlp.client.key / otel.exporter.otlp.client.certificate autoconfigure properties) without also configuring trusted certificates (a CA via setTrustedCertificates(...)), the client key/certificate is silently dropped when the default OkHttp HttpSender is used.

The exporter still connects over TLS, but presents an empty client certificate during the handshake (the JSSE DummyX509KeyManager is used), so an mTLS-requiring collector rejects the request (in our case, HTTP 400 Bad Request).

The exact same configuration works when the JDK HttpSender is on the classpath, which makes the behavior sender-dependent and surprising.

Root cause

TlsConfigHelper.getSslContext() correctly builds a non-null SSLContext that includes the client key manager, even when no trust manager was configured:

// exporters/common/src/main/java/io/opentelemetry/exporter/internal/TlsConfigHelper.java
public SSLContext getSslContext() {
  if (sslContext != null) {
    return sslContext;
  }
  try {
    SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(
        keyManager == null ? null : new KeyManager[] {keyManager},
        trustManager == null ? null : new TrustManager[] {trustManager},
        null);
    return sslContext; // non-null, and it CONTAINS the client keyManager
  } catch (NoSuchAlgorithmException | KeyManagementException e) {
    throw new IllegalArgumentException(e);
  }
}

HttpExporterBuilder.build() then passes the SSL context and the trust manager to the sender config independently:

// exporters/otlp/all/src/main/java/io/opentelemetry/exporter/otlp/internal/HttpExporterBuilder.java
ImmutableHttpSenderConfig.create(
    endpoint,
    ...
    isPlainHttp ? null : tlsConfigHelper.getSslContext(),   // non-null (contains keyManager)
    isPlainHttp ? null : tlsConfigHelper.getTrustManager(), // null when no CA configured
    executorService,
    ...);

The OkHttp sender only installs the client SSLSocketFactory when both the SSL context and the trust manager are non-null:

// exporters/sender/okhttp/src/main/java/io/opentelemetry/exporter/sender/okhttp/internal/OkHttpHttpSender.java
boolean isPlainHttp = endpoint.getScheme().equals("http");
if (isPlainHttp) {
  builder.connectionSpecs(Collections.singletonList(ConnectionSpec.CLEARTEXT));
} else if (sslContext != null && trustManager != null) {   // <-- requires BOTH
  builder.sslSocketFactory(sslContext.getSocketFactory(), trustManager);
}

Because trustManager is null when no CA is configured (system-default trust is intended), the else if is false, OkHttp falls back to the JDK default SSLSocketFactory, and the client key manager built from setClientTls(...) is never applied → the client sends an empty certificate.

By contrast, the JDK sender applies the SSL context whenever it is non-null (and never receives a trust manager at all), so client-only mTLS works there:

// exporters/sender/jdk/src/main/java/io/opentelemetry/exporter/sender/jdk/internal/JdkHttpSender.java
private static HttpClient configureClient(
    @Nullable SSLContext sslContext, Duration connectTimeout, @Nullable ProxyOptions proxyOptions) {
  HttpClient.Builder builder = HttpClient.newBuilder().connectTimeout(connectTimeout);
  if (sslContext != null) {          // <-- only requires the SSLContext
    builder.sslContext(sslContext);
  }
  ...
}

Steps to reproduce

  1. Stand up an OTLP/HTTP endpoint that requires a client certificate (mTLS) but whose server certificate is publicly trusted (e.g. Let's Encrypt), so the client needs no custom CA.
  2. Build an OTLP HTTP exporter with the default OkHttp sender and only a client key + cert:
    OtlpHttpLogRecordExporter.builder()
        .setEndpoint("https://collector.example:4318/v1/logs")
        .setClientTls(privateKeyPem, clientCertPem) // no setTrustedCertificates(...)
        .build();
    (or, via autoconfigure, set only otel.exporter.otlp.logs.client.key + otel.exporter.otlp.logs.client.certificate).
  3. Export, with -Djavax.net.debug=ssl:handshake.

Expected result

Configuring a client key/certificate causes the client certificate to be presented during the TLS handshake, regardless of whether custom trusted certificates (a CA) are also configured — matching the JDK sender's behavior.

Actual result

With the OkHttp sender the client certificate is silently omitted unless a trust manager is also configured. The handshake shows DummyX509KeyManager and an empty client Certificate message, and the mTLS collector rejects the request (HTTP 400). Adding an (otherwise unnecessary) CA via setTrustedCertificates(...) makes it start working, which is a confusing coupling.

Workaround

Provide a full SSLContext (client key manager + a non-null trust manager) via setSslContext(sslContext, trustManager). When the server certificate is publicly trusted, the JDK system-default X509TrustManager is a sufficient (non-null) trust manager:

TrustManagerFactory tmf =
    TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init((KeyStore) null); // system default trust
X509TrustManager systemTm =
    (X509TrustManager) Arrays.stream(tmf.getTrustManagers())
        .filter(X509TrustManager.class::isInstance).findFirst().orElseThrow();

SSLContext ctx = SSLContext.getInstance("TLS");
ctx.init(new KeyManager[] {clientKeyManager}, new TrustManager[] {systemTm}, null);

builder.setSslContext(ctx, systemTm); // now OkHttp installs the client SSLSocketFactory

Suggested fix

Make the OkHttp sender consistent with the JDK sender: install the client SSLSocketFactory whenever sslContext != null, deriving the trust manager from the platform default when one was not explicitly provided. For example, when trustManager == null, resolve the system-default X509TrustManager (as TlsUtil/JSSE already do internally) rather than skipping the custom socket factory entirely.

Environment

  • opentelemetry-java: 1.58.0 (via opentelemetry-spring-boot-starter 2.28.1)
  • Sender: opentelemetry-exporter-sender-okhttp (default)
  • Signal: logs (the code path is shared, so traces/metrics are affected identically)
  • JDK: SapMachine 25.0.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions