Skip to content

ENG-3250 - Add group_id to MonitorTask#8115

Open
vcruces wants to merge 13 commits into
mainfrom
ENG-3250
Open

ENG-3250 - Add group_id to MonitorTask#8115
vcruces wants to merge 13 commits into
mainfrom
ENG-3250

Conversation

@vcruces
Copy link
Copy Markdown
Contributor

@vcruces vcruces commented May 6, 2026

Ticket ENG-3250

Description Of Changes

Add group_id column to MonitorTask model and is_monitor_task_paused() function to support task stopping in fidesplus.

  • group_id: nullable indexed column that groups tasks spawned from a single classify operation, enabling batch stopping
  • is_monitor_task_paused(): checks if a task's status is awaiting_processing (the paused/stopped state)
  • Frontend: stopped tasks show as "Paused" instead of a loading spinner in the activity tab

This is the fides-side companion to fidesplus PR #3525.

Code Changes

  • Added group_id column to MonitorTask model
  • Added is_monitor_task_paused() standalone function
  • Added Alembic migration 9f21507db078
  • Frontend: removed paused from isInProgress statuses, simplified formatStatusForDisplay, removed paused from default status filters
  • Added tests for group_id and is_monitor_task_paused()
  • Updated db_dataset.yml with group_id data category annotation

Steps to Confirm

  1. Run migration: verify group_id column is added to monitortask table
  2. Run migration downgrade: verify column is removed
  3. Verify stopped tasks show as "Paused" with timestamp (not loading spinner) in the activity tab

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • UX feedback:
    • All UX related changes have been reviewed by a designer
    • No UX review needed
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 6, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments
Project Deployment Actions Updated (UTC)
fides-plus-nightly Ignored Ignored Preview May 12, 2026 5:47pm
fides-privacy-center Ignored Ignored May 12, 2026 5:47pm

Request Review

@codecov
Copy link
Copy Markdown

codecov Bot commented May 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.47%. Comparing base (2f9f5e5) to head (08c2bb1).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8115   +/-   ##
=======================================
  Coverage   85.47%   85.47%           
=======================================
  Files         650      650           
  Lines       42416    42422    +6     
  Branches     4980     4981    +1     
=======================================
+ Hits        36253    36259    +6     
  Misses       5055     5055           
  Partials     1108     1108           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vcruces vcruces changed the title Eng 3250 ENG-3250 - Add group_id to MonitorTask May 6, 2026
@vcruces vcruces force-pushed the ENG-3250 branch 2 times, most recently from a06b423 to f9a34f7 Compare May 6, 2026 14:38
@vcruces
Copy link
Copy Markdown
Contributor Author

vcruces commented May 6, 2026

/code-review

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

Title Lines Statements Branches Functions
admin-ui Coverage: 8%
6.67% (3082/46179) 6.03% (1604/26600) 4.65% (636/13649)
fides-js Coverage: 78%
79.51% (2019/2539) 66.24% (1248/1884) 73.31% (349/476)
privacy-center Coverage: 85%
82.53% (364/441) 79.74% (189/237) 74.07% (60/81)

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: Add group_id to MonitorTask (#8115)

The overall shape of the change is clean and focused: add a group_id column + index to MonitorTask, introduce an is_cancelled helper that maps awaiting_processing status to cancellation semantics, and stop showing paused tasks in the in-progress list on the frontend.

Must Fix

is_cancelled is mistyped as @classmethod (monitor_task.py:66): cls is unused in the method body — the method takes a db: Session + celery_id and performs a concrete record lookup. The existing module-level function pattern (create_monitor_task_with_execution_log, update_monitor_task_with_execution_log) is a better fit for this kind of named DB query. Using @classmethod here obscures intent and breaks the semantic contract of classmethods.

Suggestions

  • isInProgress uses raw string literals instead of ExecutionLogStatus enum values (InProgressMonitorTaskItem.tsx:97): the enum is already imported in the hook layer; using it here prevents silent breakage if values change.
  • formatStatusForDisplay is duplicated in two co-located files (InProgressMonitorTaskItem.tsx:35 and InProgressMonitorTasksList.tsx:21): this PR simplified both copies simultaneously, which is a natural moment to extract it into a shared utils.ts in the same directory.
  • Test cleanup uses db.delete() + db.commit() instead of task.delete(db=db) (test_monitor_task.py:545–547, 579–580): inconsistent with the rest of the test file.

Looks Good

  • group_id column definition (String(255), nullable, indexed) is consistent with celery_id.
  • Alembic migration down_revision = 'd76443a8a2e3' correctly extends the current head — no fork introduced.
  • Removing ExecutionLogStatus.PAUSED from defaultStatusFilters and isInProgress is the right approach; the loader-for-paused-tasks behaviour was the stated problem.
  • The is_cancelled docstring clearly explains the awaiting_processing ↔ "paused/cancelled" mapping — good documentation for a non-obvious semantic.
  • Parametrized test_is_cancelled covers all relevant status branches cleanly.

🔬 Codegraph: unavailable


💡 Write /code-review in a comment to re-run this review.

Comment thread src/fides/api/models/detection_discovery/monitor_task.py Outdated
Comment thread tests/ops/models/detection_discovery/test_monitor_task.py
Copy link
Copy Markdown
Contributor

@speaker-ender speaker-ender left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nits but approving as is

const isInProgress = [
"pending",
"in_processing",
"paused",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the paused state here because it was showing a spinner, which made it look like the process was still running

Copy link
Copy Markdown
Contributor

@adamsachs adamsachs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BE looks straightforward here, just a small nit for consideration

Comment thread src/fides/api/models/detection_discovery/monitor_task.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants