chore: new env var USE_ROOM_SEARCH_INDEX to scope messages text index by room#40397
Open
sampaiodiego wants to merge 1 commit intodevelopfrom
Open
chore: new env var USE_ROOM_SEARCH_INDEX to scope messages text index by room#40397sampaiodiego wants to merge 1 commit intodevelopfrom
USE_ROOM_SEARCH_INDEX to scope messages text index by room#40397sampaiodiego wants to merge 1 commit intodevelopfrom
Conversation
Contributor
|
Looks like this PR is not ready to merge, because of the following issues:
Please fix the issues and try again If you have any trouble, please check the PR guidelines |
🦋 Changeset detectedLatest commit: 3f10240 The changes in this PR will be included in the next version bump. This PR includes changesets to release 42 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
WalkthroughAdds a runtime-controlled option (USE_ROOM_SEARCH_INDEX) to change the Messages collection text-index shape between ChangesText Index Lifecycle Management
Sequence DiagramsequenceDiagram
participant Startup as Startup Process
participant Logic as ensureMessagesTextIndex
participant Env as Environment
participant MongoDB as MongoDB (Messages)
Startup->>Logic: invoke ensureMessagesTextIndex()
Logic->>Env: read USE_ROOM_SEARCH_INDEX
Logic->>Logic: determine desired index shape
Logic->>MongoDB: list indexes on Messages
MongoDB-->>Logic: return index list
Logic->>Logic: classify text indexes (room-scoped/default/other)
alt stale indexes found
Logic->>MongoDB: drop stale text indexes
MongoDB-->>Logic: drop results
end
alt no matching index exists
Logic->>MongoDB: create desired text index
MongoDB-->>Logic: creation result
end
Logic-->>Startup: reconciliation complete
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Warning Review ran into problems🔥 ProblemsErrors were encountered while retrieving linked issues. Errors (1)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
coderabbitai
Bot
added
the
type: feature
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
packages/models/src/models/Messages.ts (1)
55-57: 💤 Low valueRemove inline comment block from implementation.
The three-line comment violates the project's coding guideline to avoid code comments in implementations. The
githistory and the startup file itself serve as the authoritative explanation.♻️ Proposed change
- // The text index on `msg` is managed at startup by `ensureMessagesTextIndex` - // because its shape is controlled by the `USE_ROOM_SEARCH_INDEX` env var - // (default `{ msg: 'text' }` vs. room-scoped `{ rid: 1, msg: 'text' }`). { key: { 'file._id': 1 }, sparse: true },As per coding guidelines:
**/*.{ts,tsx,js}— "Avoid code comments in the implementation".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/models/src/models/Messages.ts` around lines 55 - 57, Remove the three-line inline comment block in Messages.ts that explains the text index shape and references ensureMessagesTextIndex and the USE_ROOM_SEARCH_INDEX env var; simply delete that comment so implementation contains no explanatory comments (the startup file and git history remain the source of truth) while leaving the ensureMessagesTextIndex function and any code using USE_ROOM_SEARCH_INDEX unchanged.apps/meteor/server/startup/ensureMessagesTextIndex.ts (1)
7-9: 💤 Low valueRemove implementation comment block.
Lines 7–9 are an inline comment block explaining MongoDB's internal text-index key format, which violates the project guideline to avoid code comments in implementations.
♻️ Proposed change
-// MongoDB stores a text index's key with `_fts: 'text'` / `_ftsx: 1` placeholders -// and tracks the original text fields in `weights`. Classify by looking at the -// non-placeholder prefix fields plus weights. const classifyTextIndex = (idx: { key: Record<string, unknown>; weights?: Record<string, number> }) => {As per coding guidelines:
**/*.{ts,tsx,js}— "Avoid code comments in the implementation".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/meteor/server/startup/ensureMessagesTextIndex.ts` around lines 7 - 9, Remove the inline implementation comment block in ensureMessagesTextIndex.ts that explains MongoDB text-index internals (the `_fts: 'text'` / `_ftsx: 1` and `weights` explanation); locate the comment near the top of apps/meteor/server/startup/ensureMessagesTextIndex.ts (around the ensureMessagesTextIndex logic) and delete those lines so the implementation contains no explanatory code comments per guidelines—if the rationale must be preserved, move it to external docs or test descriptions rather than in the source file.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/meteor/server/startup/ensureMessagesTextIndex.ts`:
- Line 82: The current compound text index created by Messages.col.createIndex({
rid: 1, msg: 'text' }) breaks global $text searches because prefix field rid
cannot be queried with $in; update ensureMessagesTextIndex.ts to detect the
unsafe configuration and abort/warn: when USE_ROOM_SEARCH_INDEX is enabled and
Search.defaultProvider.GlobalSearchEnabled (or equivalent config) is true, log a
clear error and refuse to create the room-scoped compound text index (or create
only the global { msg: 'text' } index) so that messageSearch.ts queries using
query.rid = { $in: [...] } won’t fail; alternatively add a defensive try/catch
around Messages.find(...) in messageSearch.ts to catch error code 17007 and
return an empty result with a logged warning — implement one of these fixes and
reference Messages.col.createIndex and messageSearch.ts to locate the code.
---
Nitpick comments:
In `@apps/meteor/server/startup/ensureMessagesTextIndex.ts`:
- Around line 7-9: Remove the inline implementation comment block in
ensureMessagesTextIndex.ts that explains MongoDB text-index internals (the
`_fts: 'text'` / `_ftsx: 1` and `weights` explanation); locate the comment near
the top of apps/meteor/server/startup/ensureMessagesTextIndex.ts (around the
ensureMessagesTextIndex logic) and delete those lines so the implementation
contains no explanatory code comments per guidelines—if the rationale must be
preserved, move it to external docs or test descriptions rather than in the
source file.
In `@packages/models/src/models/Messages.ts`:
- Around line 55-57: Remove the three-line inline comment block in Messages.ts
that explains the text index shape and references ensureMessagesTextIndex and
the USE_ROOM_SEARCH_INDEX env var; simply delete that comment so implementation
contains no explanatory comments (the startup file and git history remain the
source of truth) while leaving the ensureMessagesTextIndex function and any code
using USE_ROOM_SEARCH_INDEX unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 55662408-51ea-4a45-b988-35467c4ad290
📒 Files selected for processing (4)
.changeset/room-search-index.mdapps/meteor/server/startup/ensureMessagesTextIndex.tsapps/meteor/server/startup/index.tspackages/models/src/models/Messages.ts
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: 📦 Build Packages
- GitHub Check: CodeQL-Build
- GitHub Check: CodeQL-Build
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js}
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
**/*.{ts,tsx,js}: Write concise, technical TypeScript/JavaScript with accurate typing in Playwright tests
Avoid code comments in the implementation
Files:
packages/models/src/models/Messages.tsapps/meteor/server/startup/index.tsapps/meteor/server/startup/ensureMessagesTextIndex.ts
🧠 Learnings (3)
📚 Learning: 2026-03-16T21:50:37.589Z
Learnt from: amitb0ra
Repo: RocketChat/Rocket.Chat PR: 39676
File: .changeset/migrate-users-register-openapi.md:3-3
Timestamp: 2026-03-16T21:50:37.589Z
Learning: For changes related to OpenAPI migrations in Rocket.Chat/OpenAPI, when removing endpoint types and validators from rocket.chat/rest-typings (e.g., UserRegisterParamsPOST, /v1/users.register) document this as a minor changeset (not breaking) per RocketChat/Rocket.Chat-Open-API#150 Rule 7. Note that the endpoint type is re-exposed via a module augmentation .d.ts in the consuming package (e.g., packages/web-ui-registration/src/users-register.d.ts). In reviews, ensure the changeset clearly states: this is a non-breaking change, the major version should not be bumped, and the changeset reflects a minor version bump. Do not treat this as a breaking change during OpenAPI migrations.
Applied to files:
.changeset/room-search-index.md
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In the Rocket.Chat repository, do not reference Biome lint rules in code review feedback. Biome is not used even if biome.json exists; only reference Biome rules if there is explicit, project-wide usage documented. For TypeScript files, review lint implications without Biome guidance unless the project enables Biome rules.
Applied to files:
packages/models/src/models/Messages.tsapps/meteor/server/startup/index.tsapps/meteor/server/startup/ensureMessagesTextIndex.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In this repository (RocketChat/Rocket.Chat), Biome lint rules are not used even if a biome.json exists. When reviewing TypeScript files (e.g., packages/ui-voip/src/providers/useMediaSession.ts), ensure lint suggestions do not reference Biome-specific rules. Rely on general ESLint/TypeScript lint rules and project conventions instead.
Applied to files:
packages/models/src/models/Messages.tsapps/meteor/server/startup/index.tsapps/meteor/server/startup/ensureMessagesTextIndex.ts
🪛 LanguageTool
.changeset/room-search-index.md
[style] ~6-~6: As an alternative to the over-used intensifier ‘very’, consider replacing this phrase.
Context: ...e source of repeated MongoDB outages on very large databases. The index is reconciled on ...
(EN_WEAK_ADJECTIVE)
🔇 Additional comments (2)
.changeset/room-search-index.md (1)
1-9: LGTM!Changeset accurately captures the behavior and startup reconciliation semantics.
apps/meteor/server/startup/index.ts (1)
21-22: LGTM!Consistent with the existing
startCronJobspattern; internal try/catch inensureMessagesTextIndexprevents unhandled rejections.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #40397 +/- ##
===========================================
+ Coverage 70.06% 70.08% +0.01%
===========================================
Files 3301 3302 +1
Lines 120778 120819 +41
Branches 21608 21583 -25
===========================================
+ Hits 84627 84677 +50
+ Misses 32869 32853 -16
- Partials 3282 3289 +7
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
sampaiodiego
changed the title
USE_ROOM_SEARCH_INDEX env var to scope messages text index by roomUSE_ROOM_SEARCH_INDEX to scope messages text index by room
sampaiodiego
force-pushed
the
new-envvar-room-search-index
branch
from
79627e4 to
ae5368e
Compare
…y room
When `USE_ROOM_SEARCH_INDEX=true`, the `rocketchat_message` text index is
built as `{ rid: 1, msg: 'text' }` so per-room `$text` searches use `rid`
as a prefix and scan a much smaller portion of the index on large
workspaces. Reverts to the default `{ msg: 'text' }` shape when unset.
The index is reconciled on every boot in a non-blocking startup task.
Refs CORE-2154.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sampaiodiego
force-pushed
the
new-envvar-room-search-index
branch
from
ae5368e to
3f10240
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/meteor/server/startup/ensureMessagesTextIndex.ts`:
- Around line 7-9: Remove the inline explanatory comments about MongoDB
text-index internals (the lines referencing `_fts: 'text'`, `_ftsx: 1`, and
`weights`) from the top of the TypeScript file so the implementation is
self-descriptive; leave the code, function and variable names (e.g., any
function that ensures the messages text index and usages of `weights`, `_fts`,
`_ftsx`) intact and ensure no other implementation comments remain in this file.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 34e93818-e2ce-4c18-9a41-f4ebedb25fae
📒 Files selected for processing (4)
.changeset/room-search-index.mdapps/meteor/server/startup/ensureMessagesTextIndex.tsapps/meteor/server/startup/index.tspackages/models/src/models/Messages.ts
✅ Files skipped from review due to trivial changes (3)
- apps/meteor/server/startup/index.ts
- packages/models/src/models/Messages.ts
- .changeset/room-search-index.md
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: CodeQL-Build
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js}
📄 CodeRabbit inference engine (.cursor/rules/playwright.mdc)
**/*.{ts,tsx,js}: Write concise, technical TypeScript/JavaScript with accurate typing in Playwright tests
Avoid code comments in the implementation
Files:
apps/meteor/server/startup/ensureMessagesTextIndex.ts
🧠 Learnings (2)
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In the Rocket.Chat repository, do not reference Biome lint rules in code review feedback. Biome is not used even if biome.json exists; only reference Biome rules if there is explicit, project-wide usage documented. For TypeScript files, review lint implications without Biome guidance unless the project enables Biome rules.
Applied to files:
apps/meteor/server/startup/ensureMessagesTextIndex.ts
📚 Learning: 2026-02-26T19:25:44.063Z
Learnt from: gabriellsh
Repo: RocketChat/Rocket.Chat PR: 38778
File: packages/ui-voip/src/providers/useMediaSession.ts:192-192
Timestamp: 2026-02-26T19:25:44.063Z
Learning: In this repository (RocketChat/Rocket.Chat), Biome lint rules are not used even if a biome.json exists. When reviewing TypeScript files (e.g., packages/ui-voip/src/providers/useMediaSession.ts), ensure lint suggestions do not reference Biome-specific rules. Rely on general ESLint/TypeScript lint rules and project conventions instead.
Applied to files:
apps/meteor/server/startup/ensureMessagesTextIndex.ts
Comment on lines
+7
to
+9
| // MongoDB stores a text index's key with `_fts: 'text'` / `_ftsx: 1` placeholders | ||
| // and tracks the original text fields in `weights`. Classify by looking at the | ||
| // non-placeholder prefix fields plus weights. |
Contributor
There was a problem hiding this comment.
Remove inline implementation comments in this TypeScript file
Please drop these explanatory comments and keep the implementation self-descriptive via naming/types, per repo standards.
Suggested diff
-// MongoDB stores a text index's key with `_fts: 'text'` / `_ftsx: 1` placeholders
-// and tracks the original text fields in `weights`. Classify by looking at the
-// non-placeholder prefix fields plus weights.
const classifyTextIndex = (idx: { key: Record<string, unknown>; weights?: Record<string, number> }) => {As per coding guidelines: "**/*.{ts,tsx,js} ... Avoid code comments in the implementation".
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| // MongoDB stores a text index's key with `_fts: 'text'` / `_ftsx: 1` placeholders | |
| // and tracks the original text fields in `weights`. Classify by looking at the | |
| // non-placeholder prefix fields plus weights. | |
| const classifyTextIndex = (idx: { key: Record<string, unknown>; weights?: Record<string, number> }) => { |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/meteor/server/startup/ensureMessagesTextIndex.ts` around lines 7 - 9,
Remove the inline explanatory comments about MongoDB text-index internals (the
lines referencing `_fts: 'text'`, `_ftsx: 1`, and `weights`) from the top of the
TypeScript file so the implementation is self-descriptive; leave the code,
function and variable names (e.g., any function that ensures the messages text
index and usages of `weights`, `_fts`, `_ftsx`) intact and ensure no other
implementation comments remain in this file.
| return; | ||
| } | ||
|
|
||
| const textIndexes = existing.filter((idx) => Object.values(idx.key).includes('text')); |
Contributor
There was a problem hiding this comment.
if mongo allows a single text index per collection, is it needed to look for many here?
Contributor
|
LGTM, tested locally with 10 million messages, it doesn't block while the index is being created, indexes are correctly reused or created in case the env variable is used, and the winningPlan uses it. Should we add this env var to the docs? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
USE_ROOM_SEARCH_INDEXenv var. When set to"true", therocketchat_messagetext index is built as{ rid: 1, msg: 'text' }so per-room$textsearches useridas a prefix and scan a much smaller portion of the index. When unset (or anything other than"true"), reverts to the default{ msg: 'text' }.setImmediate(() => ensureMessagesTextIndex())inapps/meteor/server/startup/index.ts). If the existing text index already matches the desired shape it is a no-op; otherwise the stale text index is dropped and the desired one is created.{ msg: 'text' }entry fromMessagesRaw.modelIndexes(). MongoDB allows only one text index per collection, so leaving it there would have collided with the env-driven swap on startup. The text index is now owned exclusively by the startup script.SystemLogger.startup(drop/create operations, with a duration warning on large databases) andSystemLogger.debug(no-op steady-state checks). Failures go throughSystemLogger.error.Refs CORE-2154. Context: repeated MongoDB outages on large SaaS workspaces (BBTS, Cruzeiro do Sul) were traced to
$textqueries againstrocketchat_messagescanning the whole text index even when constrained to a single room. SRE has been manually patching the index on the largest databases per affected customer; this PR makes the room-scoped shape opt-in via env var so it can be applied without manual DB intervention.Test plan
checking messages text index ... desired: defaultlog fires and thatdb.rocketchat_message.getIndexes()shows{ msg: 'text' }unchanged.USE_ROOM_SEARCH_INDEX=trueon the same workspace — verify the dropping/creating logs fire and the resulting index has key{ rid: 1, _fts: 'text', _ftsx: 1 }withweights: { msg: 1 }.USE_ROOM_SEARCH_INDEX=true(no DB change) — verify only the debug "already matches" log fires and no drop/create happens.USE_ROOM_SEARCH_INDEXand reboot — verify the room-scoped index is dropped and the default{ msg: 'text' }is recreated.$textsearch constrained to a single room with the room-scoped index in place and confirm the explain plan uses theridprefix.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Improvements