feat(openfeature): add direct flagevaluation fallback by leoromanovsky · Pull Request #12204 · DataDog/dd-trace-java

leoromanovsky · 2026-08-13T15:07:19Z

Motivation

The base PR gives exposure events direct EVP capability. Agentless Feature Flags also need direct EVP capability for aggregate flagevaluation events when no compatible local receiver exists.

flowchart TD
    A[Agentless Feature Flags] --> E[Exposure writer from PR 12195]
    A --> F[Flagevaluation writer in this PR]
    E --> ER{Compatible local EVP route?}
    F --> FR{Compatible local EVP route?}
    ER -->|Yes| L[Local EVP proxy]
    FR -->|Yes| L
    ER -->|No, API key available| DE[Direct EVP<br/>/api/v2/exposures]
    FR -->|No, API key available| DF[Direct EVP<br/>/api/v2/flagevaluation]
    L --> P[Event Platform]
    DE --> P
    DF --> P

Changes and Decisions

This stacked PR retains direct EVP exposure delivery from PR #12195 and adds direct EVP flagevaluation delivery.

Use one Feature Flag transport selector for both exposures and flagevaluations.
Preserve exposure response compression and disable response compression for flagevaluations.
Prefer a compatible local EVP proxy for both event types.
Use authenticated direct EVP intake when no compatible local route exists.
Switch from local to direct intake only after connection refusal or HTTP 403, 404, or 405.
Do not replay after timeouts, resets, rate limits, or server errors. This prevents possible duplicate events.
Keep /api/v2/exposures and /api/v2/flagevaluation as separate direct routes.
Cover the shared fallback behavior for both event routes.

dd-octo-sts · 2026-08-13T15:42:47Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.94 s	13.92 s	[-0.6%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent	12.98 s	12.99 s	[-0.8%; +0.7%] (no difference)
startup:petclinic:appsec:Agent	17.46 s	17.33 s	[-0.2%; +1.7%] (no difference)
startup:petclinic:iast:Agent	17.38 s	17.56 s	[-1.8%; -0.3%] (maybe better)
startup:petclinic:profiling:Agent	17.50 s	17.44 s	[-0.8%; +1.5%] (no difference)
startup:petclinic:sca:Agent	17.36 s	17.37 s	[-1.0%; +0.9%] (no difference)
startup:petclinic:tracing:Agent	16.49 s	16.63 s	[-1.9%; +0.2%] (no difference)

Commit: 6616b0c1 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

datadog-official

Direct flag-evaluation delivery still negotiates gzip because IntakeApi leaves Accept-Encoding unset when compression is disabled, allowing OkHttp to add gzip automatically. This defeats the explicit no-compression contract on every new direct or fallback route, although the local EVP path correctly sends identity.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 5916539 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-official · 2026-08-13T15:52:53Z

+      return null;
+    }
+    try {
+      return backendApiFactory.createDirectIntakeApi(Intake.EVENT_PLATFORM, responseCompression);


Direct intake ignores disabled response compression

Every agentless flag-evaluation request using the new direct or fallback route still negotiates gzip, defeating the route’s stated no-compression compatibility contract.

Assertion details

Input: Agentless feature flagging with an API key when the local EVP proxy is unavailable or definitively rejects flag-evaluation delivery.

Expected: Direct flag-evaluation requests should send Accept-Encoding: identity so response compression is genuinely disabled on both local and direct routes.

Actual: The new direct path passes responseCompression=false to createDirectIntakeApi, but IntakeApi only adds Accept-Encoding when the value is true. OkHttp therefore automatically adds Accept-Encoding: gzip. The sibling EvpProxyApi explicitly sends identity on its false path to prevent exactly this behavior. FeatureFlagBackendApiFactoryTest only verifies the boolean argument, while the direct writer test does not inspect Accept-Encoding. A complete fix must update IntakeApi to send Accept-Encoding: identity when compression is disabled and cover that shared-client behavior; changing this factory line alone is insufficient.

Was this helpful? React 👍 or 👎
_{🤖 Datadog Autotest · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

dd-oleksii · 2026-08-13T18:27:54Z

+        capacity,
+        flushInterval,
+        timeUnit,
+        new FeatureFlagBackendApiFactory(config, sco, "exposure", true),


nitpick: reading the code, I've no idea what true means here. Not sure what java conventions are but an enum or a builder would help. (although builder for a factory is a bit 🤪.) A builder-like setter also works

dd-oleksii · 2026-08-13T18:35:02Z

+      final Config config,
+      final SharedCommunicationObjects sharedCommunicationObjects,
+      final String eventType,
+      final boolean responseCompression) {


Q: What is "response compression?" Why is it disabled for flag evaluations?

dd-oleksii · 2026-08-13T18:44:03Z

+  private static final Logger LOGGER =
+      LoggerFactory.getLogger(AgentlessFeatureFlagBackendApi.class);

  private final BackendApi localApi;


aside: probably not in scope of this PR but it took me a bit to figure out what "local API" means. I think that "proxy" is the more important part of the name, and "direct vs. proxy" makes more sense as a dichotomy

I guess "local" part might also be false when agent is deployed to a different node in a cluster

…ct-flagevaluation-egress # Conflicts: # products/feature-flagging/feature-flagging-lib/src/main/java/com/datadog/featureflag/AgentlessFeatureFlagBackendApi.java # products/feature-flagging/feature-flagging-lib/src/main/java/com/datadog/featureflag/ExposureBackendApiFactory.java # products/feature-flagging/feature-flagging-lib/src/test/java/com/datadog/featureflag/AgentlessFeatureFlagBackendApiTest.java # products/feature-flagging/feature-flagging-lib/src/test/java/com/datadog/featureflag/FeatureFlagBackendApiFactoryTest.java

dd-octo-sts · 2026-08-13T21:11:20Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

…ct-flagevaluation-egress # Conflicts: # products/feature-flagging/feature-flagging-lib/src/test/java/com/datadog/featureflag/AgentlessFeatureFlagBackendApiTest.java # products/feature-flagging/feature-flagging-lib/src/test/java/com/datadog/featureflag/FeatureFlagBackendApiFactoryTest.java

…ct-flagevaluation-egress

sarahchen6

A few small clean-up comments, but otherwise looks good!

sarahchen6 · 2026-08-14T18:06:32Z

side comment: Github UI didn't recognize ExposureBackendApiFactory->FeatureFlagBackendApiFactory as a ~rename, but did for AgentlessExposureBackendApi->AgentlessFeatureFlagBackendApi......booo

sarahchen6 · 2026-08-14T18:14:52Z

 * EVP flagevaluation writer for Java.
 *
 * <p>Uses the same EVP publisher path as ExposureWriterImpl, with two-tier aggregation replacing
 * the single-exposure buffer. Routes to the Agent-advertised EVP proxy endpoint for


now direct EVP instead of just Agent (local) EVP is possible?

sarahchen6 · 2026-08-14T18:18:42Z

+              API_KEY,
+              "123",
+              HttpRetryPolicy.Factory.NEVER_RETRY,
+              new OkHttpClient.Builder().build(),


nit: should clean up this client like done in IntakeApiTest

sarahchen6 · 2026-08-14T18:25:43Z

    private final AtomicBoolean shutdownRequested = new AtomicBoolean(false);
    private final CountDownLatch finalFlushDone = new CountDownLatch(1);

    FlagEvaluationSerializingHandler(


Can we reduce the amount of constructors for FlagEvaluationWriterImpl and FlagEvaluationSerializingHandler?

I checked this quickly by Codex, and it gave the following suggestion:

The writer ultimately converts everything into Supplier<BackendApi> at products/feature-flagging/feature-flagging-lib/src/main/java/com/datadog/ featureflag/FlagEvaluationWriterImpl.java:149. The handler does the same at products/feature-flagging/feature-flagging-lib/src/main/java/com/datadog/ featureflag/FlagEvaluationWriterImpl.java:409. That should be the shared boundary. A cleaner shape would be: public FlagEvaluationWriterImpl( SharedCommunicationObjects sco, Config config) { this( DEFAULT_CAPACITY, FLUSH_INTERVAL_SECONDS, SECONDS, new FeatureFlagBackendApiFactory( config, sco, FeatureFlagEventType.FLAG_EVALUATION) ::create, config); } FlagEvaluationWriterImpl( int capacity, long flushInterval, TimeUnit timeUnit, Supplier<BackendApi> backendApiSupplier, Config config) { // Actual initialization } Then the handler needs only its full constructor: FlagEvaluationSerializingHandler( Supplier<BackendApi> backendApiSupplier, MessagePassingBlockingQueue<FlagEvalEvent> queue, long flushInterval, TimeUnit timeUnit, Map<String, String> context, AtomicLong droppedQueueOverflow, ConcurrentHashMap<String, AtomicLong> contextTruncatedCounts, Runnable errorCallback, int payloadSizeLimitBytes) { // Actual initialization } Production passes FeatureFlagBackendApiFactory::create; tests and benchmarks pass a lambda or mocked supplier. That would allow removing: - The sized SharedCommunicationObjects writer constructor at line 108. - The BackendApiFactory writer constructor at line 123. - The FeatureFlagBackendApiFactory writer constructor at line 140. - Both handler constructors accepting BackendApiFactory at lines 345 and 366. - Potentially the default-size Supplier handler constructor at line 388. So the practical minimum is two writer constructors and one handler constructor. The current nine combined overloads aren’t necessary.

feat(openfeature): add direct flagevaluation fallback

44c4896

leoromanovsky requested review from a team, AlexeyKuznetsov-DD, dd-oleksii, mhlidd, sabrenner and vjfridge August 13, 2026 15:07

test(openfeature): cover both direct EVP signals

5916539

This comment has been minimized.

Sign in to view

leoromanovsky marked this pull request as ready for review August 13, 2026 15:28

leoromanovsky requested a review from a team as a code owner August 13, 2026 15:28

datadog-official Bot reviewed Aug 13, 2026

View reviewed changes

dd-oleksii approved these changes Aug 13, 2026

View reviewed changes

leoromanovsky added the comp: openfeature OpenFeature label Aug 14, 2026

leoromanovsky added 4 commits August 13, 2026 21:28

fix(openfeature): clarify feature flag transport policy

855b295

Merge branch 'agent/java-direct-exposure-egress' into agent/java-dire…

06d4ee2

…ct-flagevaluation-egress

Merge branch 'agent/java-direct-exposure-egress' into agent/java-dire…

7ec5079

…ct-flagevaluation-egress

Merge branch 'agent/java-direct-exposure-egress' into agent/java-dire…

6616b0c

…ct-flagevaluation-egress

leoromanovsky added the type: feature Enhancements and improvements label Aug 14, 2026

sarahchen6 approved these changes Aug 14, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(openfeature): add direct flagevaluation fallback - #12204

feat(openfeature): add direct flagevaluation fallback#12204
leoromanovsky wants to merge 8 commits into
agent/java-direct-exposure-egressfrom
agent/java-direct-flagevaluation-egress

leoromanovsky commented Aug 13, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Aug 13, 2026 •

edited

Loading

Uh oh!

datadog-official Bot left a comment

Uh oh!

datadog-official Bot Aug 13, 2026

Uh oh!

dd-oleksii Aug 13, 2026

Uh oh!

dd-oleksii Aug 13, 2026

Uh oh!

dd-oleksii Aug 13, 2026

Uh oh!

dd-octo-sts Bot commented Aug 13, 2026

Uh oh!

sarahchen6 left a comment

Uh oh!

sarahchen6 Aug 14, 2026

Uh oh!

sarahchen6 Aug 14, 2026

Uh oh!

sarahchen6 Aug 14, 2026

Uh oh!

sarahchen6 Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

leoromanovsky commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes and Decisions

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

dd-oleksii Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

dd-oleksii Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

dd-oleksii Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

dd-octo-sts Bot commented Aug 13, 2026

Uh oh!

sarahchen6 left a comment

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

sarahchen6 Aug 14, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

leoromanovsky commented Aug 13, 2026 •

edited

Loading

dd-octo-sts Bot commented Aug 13, 2026 •

edited

Loading