Skip to content

feat: signup corporate-only experiment with exposure and new_signup events#7979

Open
Zaimwa9 wants to merge 10 commits into
mainfrom
feat/signup-corporate-only-experiment
Open

feat: signup corporate-only experiment with exposure and new_signup events#7979
Zaimwa9 wants to merge 10 commits into
mainfrom
feat/signup-corporate-only-experiment

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Sets up our first dogfood experiment, Signup corporate domains only.

The experiment tests whether requiring a work email address at signup affects conversion. Control keeps today's behaviour (an informational message for free email domains); the signup_corporate_only variant disables the Create Account button for free email domains and shows "Please use your work email address to create your account."

  • Upgrade @flagsmith/flagsmith to 12.1.0 for trackEvent, trackExposureEvent, getExperimentFlag and the variant key on flags.
  • New useSignupExperiment hook: identifies anonymous signup visitors with a transient identity (persistent signup_anonymous_id so returning visitors keep their variant), resolves the signup_corporate_only flag's variant, and records a $flag_exposure. Takes a boolean to fall back to environment flags — no identify, no exposure, control experience — for non-participants (invitees, login views, logged-in users, preventEmailPassword).
  • new_signup conversion event on successful registration, attributed to the same anonymous identity, with free_email_domain, invite, signup_method and utm_source metadata.
  • Signup button now also requires non-blank first and last names.

How did you test this code?

Manually, against the staging configuration with the flag and experiment set up:

  1. Open /signup logged out — the network tab shows identities/?identifier=<uuid>&transient=true, followed (≤10s) by a POST /v1/events containing a $flag_exposure with the variant as value (verified 202).
  2. In the treatment variant, entering a gmail.com address disables Create Account and shows the work-email message; a work domain enables it. Control behaviour is unchanged.
  3. Completing a signup sends new_signup with the same identifier and metadata.
  4. Invite links and the login view trigger no identify/exposure and always get control.
  5. Variant bucketing verified ~50/50 across identifiers against the identities API; unit tests (236) and typecheck pass.

@Zaimwa9 Zaimwa9 requested a review from a team as a code owner July 10, 2026 11:07
@Zaimwa9 Zaimwa9 requested review from talissoncosta and removed request for a team July 10, 2026 11:07
@vercel

vercel Bot commented Jul 10, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
flagsmith-frontend-preview Ready Ready Preview, Comment Jul 15, 2026 9:55am
flagsmith-frontend-staging Ready Ready Preview, Comment Jul 15, 2026 9:55am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview Jul 15, 2026 9:55am

Request Review

@github-actions github-actions Bot added the front-end Issue related to the React Front End Dashboard label Jul 10, 2026
@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b8218748-2df1-4c14-adf4-15717ef70854

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The signup flow now records Flagsmith new_signup events with registration metadata. A new hook persists an anonymous identifier and reads the signup_corporate_only experiment variant. The homepage uses that variant to block free email domains, display work-email guidance, and disable account creation. The Flagsmith frontend dependency is updated to version 12.1.0.

Estimated code review effort: 3 (Moderate) | ~20 minutes


Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the feature New feature or request label Jul 10, 2026
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-api-test:pr-7979 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-e2e:pr-7979 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api:pr-7979 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-7979 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-7979 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-frontend:pr-7979 Finished ✅ Results

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 3 suites
duration  35.1 seconds
commit  0f99759
info  🔄 Run: #18155 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  46.5 seconds
commit  71683f0
info  🔄 Run: #18154 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  4 passed

Details

stats  4 tests across 3 suites
duration  46 seconds
commit  0f99759
info  🔄 Run: #18155 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  5 passed

Details

stats  5 tests across 4 suites
duration  47.1 seconds
commit  71683f0
info  🔄 Run: #18154 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  7 passed

Details

stats  7 tests across 5 suites
duration  12 seconds
commit  0f99759
info  🔄 Run: #18155 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  47.4 seconds
commit  0f99759
info  🔄 Run: #18155 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  59.3 seconds
commit  71683f0
info  🔄 Run: #18154 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  4 passed

Details

stats  4 tests across 3 suites
duration  41.4 seconds
commit  5fc7532
info  🔄 Run: #18248 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  35.3 seconds
commit  5fc7532
info  🔄 Run: #18248 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  3 passed

Details

stats  3 tests across 3 suites
duration  12 seconds
commit  5fc7532
info  🔄 Run: #18248 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  5 passed

Details

stats  5 tests across 3 suites
duration  22.7 seconds
commit  5fc7532
info  🔄 Run: #18248 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 3 suites
duration  17.5 seconds
commit  017cd9e
info  🔄 Run: #18270 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  5 passed

Details

stats  5 tests across 3 suites
duration  41.3 seconds
commit  017cd9e
info  🔄 Run: #18270 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  7 passed

Details

stats  7 tests across 5 suites
duration  33.8 seconds
commit  017cd9e
info  🔄 Run: #18270 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  5 passed

Details

stats  5 tests across 4 suites
duration  44 seconds
commit  017cd9e
info  🔄 Run: #18270 (attempt 1)

@github-actions

Copy link
Copy Markdown
Contributor

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 3 suites
duration  34.1 seconds
commit  71683f0
info  🔄 Run: #18154 (attempt 1)

@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a04fcc89-95a6-4ac2-aec1-a2a6a17d5749

📥 Commits

Reviewing files that changed from the base of the PR and between a7b0d21 and 0f99759.

⛔ Files ignored due to path filters (1)
  • frontend/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (4)
  • frontend/common/stores/account-store.js
  • frontend/common/useSignupExperiment.ts
  • frontend/package.json
  • frontend/web/components/pages/HomePage.tsx

Comment thread frontend/common/useSignupExperiment.ts
Comment thread frontend/common/useSignupExperiment.ts Outdated
Comment thread frontend/common/useSignupExperiment.ts Outdated
Comment thread frontend/web/components/pages/HomePage.tsx
Comment thread frontend/web/components/pages/HomePage.tsx

@talissoncosta talissoncosta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just two small comments. Do you mind checking them?

Comment thread frontend/common/stores/account-store.js Outdated
Comment thread frontend/common/useSignupExperiment.ts Outdated
@github-actions github-actions Bot added the feature New feature or request label Jul 14, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
frontend/common/useSignupExperiment.ts (2)

9-11: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Clear the variant when the hook is disabled.

If useEnvironmentFlag changes from false to true after a variant has been loaded, this early return leaves the old variant in state. HomePage can then apply corporate-only blocking to excluded invite/login flows. Clear the state and return undefined while disabled.

Proposed fix
   if (useEnvironmentFlag) {
+    setVariant(undefined)
     return
   }
...
-  return variant
+  return useEnvironmentFlag ? undefined : variant

Also applies to: 25-26


12-22: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Handle flagsmith.identify failures

If flagsmith.identify rejects, this async call is left unhandled and variant never gets set. Catch the error here and make the fail-open/fail-closed behaviour explicit.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: cdc0e798-7e26-40be-af09-442fd68a0cad

📥 Commits

Reviewing files that changed from the base of the PR and between 0f99759 and 5fc7532.

📒 Files selected for processing (1)
  • frontend/common/useSignupExperiment.ts

talissoncosta
talissoncosta previously approved these changes Jul 14, 2026

@talissoncosta talissoncosta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking the comments!

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
frontend/common/useSignupExperiment.ts (1)

15-33: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Retry-on-failure is a no-op.

identifyAndExpose().catch(() => { started.current = false }) only mutates a ref; it doesn't trigger a re-render or change the effect's dependencies (useEnvironmentFlag, email), so the effect will not re-run after a failed identify/expose unless the user subsequently edits email (or the other dependency changes). A transient identify failure silently drops that visitor from the experiment for the remainder of the render cycle.

♻️ Possible fix: force a retry via state instead of a ref-only flag
-  const [variant, setVariant] = useState<string>()
-  const started = useRef(false)
+  const [variant, setVariant] = useState<string>()
+  const [retryToken, setRetryToken] = useState(0)
+  const started = useRef(false)

   useEffect(() => {
     if (useEnvironmentFlag || started.current || !isFreeEmailDomain(email)) {
       return
     }
     started.current = true
     const identifyAndExpose = async () => {
       ...
     }
     identifyAndExpose().catch(() => {
       started.current = false
+      setTimeout(() => setRetryToken((t) => t + 1), 2000)
     })
-  }, [useEnvironmentFlag, email])
+  }, [useEnvironmentFlag, email, retryToken])

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: fa4c7104-a304-4965-83b8-4d2013c98677

📥 Commits

Reviewing files that changed from the base of the PR and between 5fc7532 and 017cd9e.

📒 Files selected for processing (3)
  • frontend/common/stores/account-store.js
  • frontend/common/useSignupExperiment.ts
  • frontend/web/components/pages/HomePage.tsx

@Zaimwa9

Zaimwa9 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request front-end Issue related to the React Front End Dashboard

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants