WPB-22963 migrate domain registration data to postgres by battermann · Pull Request #5195 · wireapp/wire-server

battermann · 2026-04-21T06:35:02Z

Checklist

Add a new entry in an appropriate subdirectory of changelog.d
Read and follow the PR guidelines

store

Copilot

Pull request overview

This PR extends the existing Cassandra→PostgreSQL migration framework to cover domain registration data, including new Postgres-backed store interpreters, a dual-write migration mode, a background-worker backfill job, and config/chart/doc updates to support operating and rolling out the migration safely.

Changes:

Add Postgres + dual-write interpreters for DomainRegistrationStore and DomainVerificationChallengeStore, and wire them into Brig via postgresMigration.domainRegistration.
Add a background-worker migration loop for backfilling domain registration data from Cassandra to Postgres, plus Prometheus metrics.
Update Helm/config/docs to add the new postgresMigration.domainRegistration field and make Galley’s postgresMigration values the single source of truth for Brig/background-worker.

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
services/galley/galley.integration.yaml	Adds `domainRegistration` to integration `postgresMigration` settings.
services/brig/src/Brig/Options.hs	Plumbs `PostgresMigrationOpts` into Brig options.
services/brig/src/Brig/CanonicalInterpreter.hs	Selects Cassandra/Postgres/dual-write interpreters for domain registration + challenge stores; adds migration lock error mapping and Cassandra client input.
services/brig/src/Brig/App.hs	Adds `postgresMigration` into Brig runtime environment.
services/brig/brig.integration.yaml	Adds `postgresMigration.domainRegistration` integration config.
services/background-worker/test/Test/Wire/Util.hs	Updates test env `PostgresMigrationOpts` with `domainRegistration`.
services/background-worker/test/Test/Wire/BackendNotificationPusherSpec.hs	Updates test env `PostgresMigrationOpts` with `domainRegistration`.
services/background-worker/src/Wire/PostgresMigrations.hs	Adds background-worker migration entrypoint for domain registration.
services/background-worker/src/Wire/BackgroundWorker/Options.hs	Adds `migrateDomainRegistration` flag to options.
services/background-worker/src/Wire/BackgroundWorker.hs	Runs domain registration migration loop when enabled and adds it to cleanup concurrency.
services/background-worker/background-worker.integration.yaml	Adds `migrateDomainRegistration` and `postgresMigration.domainRegistration` integration config.
postgres-schema.sql	Adds new Postgres tables/indexes/constraints for domain registration + updates dump version.
libs/wire-subsystems/wire-subsystems.cabal	Exposes newly added store/migration modules in the subsystems library.
libs/wire-subsystems/src/Wire/PostgresMigrationOpts.hs	Adds `domainRegistration :: StorageLocation` to migration settings parsing.
libs/wire-subsystems/src/Wire/MigrationLock.hs	Adds `MigrationLockError`→`HttpError` mapping helper used by Brig.
libs/wire-subsystems/src/Wire/DomainVerificationChallengeStore/Postgres.hs	New Postgres interpreter for domain verification challenges.
libs/wire-subsystems/src/Wire/DomainVerificationChallengeStore/DualWrite.hs	New Cassandra→Postgres dual-write interpreter for challenges.
libs/wire-subsystems/src/Wire/DomainVerificationChallengeStore/Cassandra.hs	Refactors Cassandra interpreter to use `Input ClientState` via `embedClientInput`.
libs/wire-subsystems/src/Wire/DomainRegistrationStore/Postgres.hs	New Postgres interpreter for domain registrations (+ `exists`).
libs/wire-subsystems/src/Wire/DomainRegistrationStore/Migration.hs	New backfill migration loop from Cassandra to Postgres with locking.
libs/wire-subsystems/src/Wire/DomainRegistrationStore/DualWrite.hs	New dual-write interpreter (Cassandra source-of-truth, mirror writes to Postgres under locks).
libs/wire-subsystems/src/Wire/DomainRegistrationStore.hs	Adds Postgres marshalling/unmarshalling + migration-lock keying; exports internal store ops.
libs/wire-subsystems/postgres-migrations/20260420134603-domain_registration.sql	Adds schema migration for domain registration tables and indexes.
libs/wire-api/src/Wire/API/PostgresMarshall.hs	Adds Postgres marshalling/unmarshalling instances needed by the new stores.
integration/test/Testlib/ModService.hs	Ensures dynamic backend wiring sets background-worker’s `cassandraBrig.keyspace`.
integration/test/Test/Migration/DomainRegistration.hs	New integration test that exercises migration phases and verifies behavior across cutover.
integration/integration.cabal	Registers the new migration integration test module.
hack/helm_vars/wire-server/values.yaml.gotmpl	Adds `domainRegistration` store preference and moves background-worker migration config to Galley source-of-truth.
hack/helm_vars/common.yaml.gotmpl	Adds `domainRegistration` preferred store value.
docs/src/developer/reference/config-options.md	Updates docs to describe `domainRegistration` and the new “Galley as source-of-truth” pattern.
charts/wire-server/values.yaml	Adds `domainRegistration` to `galley.config.postgresMigration` and adds migration flag docs.
charts/wire-server/templates/brig/configmap.yaml	Injects `postgresMigration` into Brig from Galley chart values.
charts/wire-server/templates/background-worker/configmap.yaml	Injects `postgresMigration` into background-worker from Galley chart values; adds `migrateDomainRegistration`.
changelog.d/0-release-notes/WPB-22963	Release notes for new migration setting and Helm/value implications.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

fisx · 2026-05-07T09:49:13Z

            setField "cassandra.keyspace" resource.berGundeckKeyspace
-              >=> setField "cassandraGalley.keyspace" resource.berGalleyKeyspace,
+              >=> setField "cassandraGalley.keyspace" resource.berGalleyKeyspace
+              >=> setField "cassandraBrig.keyspace" resource.berBrigKeyspace,


this used to be a thing already in background-worker's options, but i guess it hasn't had any integration tests so there was no need to set it here, is that correct?

fisx · 2026-05-07T15:38:30Z

+  eithErr <- runError (runError action)
+  case eithErr of
+    Right (Right _) -> pure ()
+    Right (Left e) -> logError (show e)
+    Left e -> logError (show e)


could be done with join, but maybe your way's prettier?

IIRC, only if we convert both lefts to text, because they do not have the same type. Which makes this less nice.

fisx · 2026-05-07T15:47:00Z

-interpretDomainVerificationChallengeStoreToCassandra casClient ttl =
-  runInputConst ttl
-    . runEmbedded (runClient casClient)
-    . interpret
-      ( \case
-          Insert challenge -> insertImpl challenge
-          Lookup challengeId -> lookupImpl challengeId
-          Delete challengeId -> deleteImpl challengeId
-      )
-    . raiseUnder2
+interpretDomainVerificationChallengeStoreToCassandra ttl =
+  interpret
+    ( \case
+        Insert challenge -> embedClientInput $ insertImpl ttl challenge
+        Lookup challengeId -> embedClientInput $ lookupImpl challengeId
+        Delete challengeId -> embedClientInput $ deleteImpl challengeId
+    )


i'm not opposed, but why bother?

can't remember, but there was a reason 😄

zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Apr 21, 2026

battermann force-pushed the WPB-22963-migrate-domain-registration-data-to-postgres branch from e83f366 to 41254ac Compare April 21, 2026 16:04

battermann added 12 commits April 29, 2026 06:52

postgres schema migration

615b0fe

postgres interpreter

80def52

dual write interpreter and wiring

507bbf2

single source of truth for the postgresMigration settings

6e59916

linter

ee614f2

migration and postgres interpreter for domain verification challenge

23c4bba

store

updated config

90373b7

the actual migration

2295248

the migration job runner

6df82df

wip: integrationtest

97e6a81

wip: migration test for on prem flow

2133e57

test team flow

cb09d5a

battermann force-pushed the WPB-22963-migrate-domain-registration-data-to-postgres branch from 2ee5b64 to cb09d5a Compare April 30, 2026 15:22

use locks for the migration interpreter and the migration

dfe6436

battermann marked this pull request as ready for review May 6, 2026 15:05

battermann requested review from a team as code owners May 6, 2026 15:05

battermann added 3 commits May 6, 2026 15:37

used type aliases to make code more readable

b254dac

moved migration error converter

8e4c333

lint

71208fb

battermann requested a review from Copilot May 7, 2026 06:38

Copilot started reviewing on behalf of battermann May 7, 2026 06:39 View session

Copilot AI reviewed May 7, 2026

View reviewed changes

Comment thread postgres-schema.sql

Comment thread libs/wire-subsystems/postgres-migrations/20260420134603-domain_registration.sql Outdated

Comment thread libs/wire-subsystems/src/Wire/DomainRegistrationStore/Migration.hs Outdated

battermann and others added 2 commits May 7, 2026 06:57

removed unused index

4d31157

Potential fix for pull request finding

00d0d4e

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

fisx approved these changes May 7, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WPB-22963 migrate domain registration data to postgres#5195

WPB-22963 migrate domain registration data to postgres#5195
battermann wants to merge 18 commits intodevelopfrom
WPB-22963-migrate-domain-registration-data-to-postgres

battermann commented Apr 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fisx May 7, 2026

Uh oh!

battermann May 7, 2026

Uh oh!

fisx May 7, 2026

Uh oh!

battermann May 7, 2026

Uh oh!

fisx May 7, 2026

Uh oh!

battermann May 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

battermann commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fisx May 7, 2026

Choose a reason for hiding this comment

Uh oh!

battermann May 7, 2026

Choose a reason for hiding this comment

Uh oh!

fisx May 7, 2026

Choose a reason for hiding this comment

Uh oh!

battermann May 7, 2026

Choose a reason for hiding this comment

Uh oh!

fisx May 7, 2026

Choose a reason for hiding this comment

Uh oh!

battermann May 7, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

battermann commented Apr 21, 2026 •

edited

Loading