Skip to content

fix(sync/grpc): track readiness per Sync instance#1985

Open
anxkhn wants to merge 1 commit into
open-feature:mainfrom
anxkhn:fix/grpc-sync-per-instance-readiness
Open

fix(sync/grpc): track readiness per Sync instance#1985
anxkhn wants to merge 1 commit into
open-feature:mainfrom
anxkhn:fix/grpc-sync-per-instance-readiness

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 5, 2026

Copy link
Copy Markdown
## This PR

The gRPC sync guards its one-shot readiness with a package-level `sync.Once`:

```go
var once msync.Once
// ...
func (g *Sync) handleFlagSync(...) error {
    once.Do(func() {
        g.ready = true
    })
    // ...
}

Because that Once is shared across every grpc.Sync in the process, only the
first instance whose stream connects runs the closure and sets its own ready
flag. Every other grpc.Sync instance stays not-ready forever, so its
IsReady() keeps returning false.

flagd builds a distinct grpc.Sync per configured grpc source
(SyncBuilder.newGRPC in core/pkg/sync/builder/syncbuilder.go), and the
runtime only reports ready once every sync is ready (Runtime.isReady in
flagd/pkg/runtime/runtime.go returns false if any Syncs[].IsReady() is
false). The net effect is that any flagd configured with two or more grpc://
sync sources never passes /readyz, even after all of its gRPC streams are
healthy, so the pod stays NotReady.

The fix

Set g.ready = true directly in handleFlagSync and drop the package-level
once (and its now-unused sync import alias). The assignment is idempotent,
so this is a no-op for the single-source case and makes readiness per instance
for the multi-source case. This also matches the other sync implementations:
the blob, file, and http syncs already set a plain ready bool directly, and
kubernetes tracks it per instance with an atomic.Bool. The package-level
Once was the outlier.

The write still happens-before the update pushed onto the dataSync channel,
so no new concurrent read path is introduced and the change stays race-clean.

Tests

Adds Test_MultipleSyncsBecomeReady, which starts two independent grpc.Sync
instances (each backed by its own bufconn server), drives each one, and asserts
both report IsReady() == true. The test synchronizes by receiving a payload
from the sync channel before reading IsReady(), so it establishes a
happens-before edge and is race-free by construction rather than by timing.

Before this change the test fails on the second instance
("sync 1 should be ready after its stream connects"); after it, both pass.
Readiness was previously untested in this file, so this also closes that
coverage gap.

Verified with:

  • go build ./...
  • go test -race -covermode=atomic -cover -short ./pkg/sync/grpc/ (from core/)
  • golangci-lint run ./pkg/sync/grpc/...

### Notes on the clean body
- No upstream issue exists for this bug, so the PR does not claim to close one.
  Before opening, do a final search for a matching readiness/`/readyz` issue; if
  none, file a short issue describing the multi-grpc-source NotReady symptom and
  link it, or open the PR as a standalone fix (both are acceptable here).
- Diff is 2 files, +60/-6 (source: -6, test: +59). Surgical.

The gRPC sync guarded its one-shot readiness with a package-level
sync.Once. Because that Once is shared across every grpc.Sync in the
process, only the first instance whose stream connects ran the closure
and set its own ready flag; all other instances stayed not-ready
forever, and IsReady() kept returning false for them.

flagd builds a distinct grpc.Sync per configured grpc source and the
runtime only reports ready once every sync is ready, so any flagd with
two or more grpc sources never passed /readyz even after all streams
were healthy.

Set g.ready directly in handleFlagSync (the assignment is idempotent),
which makes readiness per instance, matching the other sync
implementations. Add a regression test that starts two independent
grpc.Sync instances and asserts both become ready.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn
anxkhn requested review from a team as code owners July 5, 2026 09:13
@netlify

netlify Bot commented Jul 5, 2026

Copy link
Copy Markdown

Deploy Preview for polite-licorice-3db33c canceled.

Name Link
🔨 Latest commit 20e0398
🔍 Latest deploy log https://app.netlify.com/projects/polite-licorice-3db33c/deploys/6a4a205760d76f00086e3977

@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jul 5, 2026
@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b69fca4a-158e-4949-a3ee-367a7266090c

📥 Commits

Reviewing files that changed from the base of the PR and between d99a3e3 and 20e0398.

📒 Files selected for processing (2)
  • core/pkg/sync/grpc/grpc_sync.go
  • core/pkg/sync/grpc/grpc_sync_test.go

📝 Walkthrough

Walkthrough

The sync.Once-based readiness guard in the gRPC sync handler is removed, so g.ready is now set unconditionally each time the stream handler runs instead of only once. A new test validates that readiness is tracked independently across multiple Sync instances.

Changes

Readiness Flag Fix and Test

Layer / File(s) Summary
Remove shared Once guard for readiness
core/pkg/sync/grpc/grpc_sync.go
Removes the sync import alias and package-level once variable, and changes handleFlagSync to set g.ready = true unconditionally instead of via once.Do(...).
Test multi-instance readiness
core/pkg/sync/grpc/grpc_sync_test.go
Adds Test_MultipleSyncsBecomeReady with a newReadySync helper creating isolated bufconn-based Sync instances, and asserts both become ready independently after streaming payloads.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
    participant Test as Test_MultipleSyncsBecomeReady
    participant Sync1 as Sync Instance 1
    participant Sync2 as Sync Instance 2
    participant Server as bufconn gRPC Server

    Test->>Sync1: Sync(ctx1, dataSync)
    Test->>Sync2: Sync(ctx2, dataSync)
    Sync1->>Server: stream request (handleFlagSync)
    Server-->>Sync1: SyncFlags payload
    Sync1->>Sync1: set ready = true
    Sync2->>Server: stream request (handleFlagSync)
    Server-->>Sync2: SyncFlags payload
    Sync2->>Sync2: set ready = true
    Test->>Sync1: IsReady()
    Test->>Sync2: IsReady()
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main fix: gRPC sync readiness is now tracked per Sync instance.
Description check ✅ Passed The description accurately explains the readiness bug, the fix, and the added regression test.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

sonarqubecloud Bot commented Jul 5, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant