Skip to content

feat(importers): apply import-time tags per batch before post-processing, do not tag old findings#14839

Merged
valentijnscholten merged 1 commit intoDefectDojo:devfrom
valentijnscholten:apply-import-tags-per-batch
May 8, 2026
Merged

feat(importers): apply import-time tags per batch before post-processing, do not tag old findings#14839
valentijnscholten merged 1 commit intoDefectDojo:devfrom
valentijnscholten:apply-import-tags-per-batch

Conversation

@valentijnscholten
Copy link
Copy Markdown
Member

Summary

  • Import-time tags (apply_tags_to_findings / apply_tags_to_endpoints) were previously applied after all findings were processed and post-processing tasks (deduplication, rules) had already been dispatched — those tasks could not see the tags on the findings.
  • Tags are now applied per batch inside process_findings(), immediately after parser tags and location persistence, before dojo_dispatch_task() — the same pattern already used by bulk_apply_parser_tags.
  • Closed findings are intentionally excluded: they are absent from the current report and should not receive the import tags.
  • Renames the reimporter's ambiguous batch_findings (match-batch slice of unsaved parser objects) to unsaved_findings_batch, freeing batch_findings for the new dedupe-batch accumulator and fixing a latent iterator-mutation bug where appending to the iterated list caused extra loop iterations.
  • Consolidates apply_import_tags() into apply_import_tags_for_batch(), removing a redundant Finding.objects.filter() re-query and an unused Iterable import.
  • Updates performance test query counts accordingly (each import loses 1 query from the removed post-loop apply_import_tags call).

Previously, import-time tags (apply_tags_to_findings / apply_tags_to_endpoints)
were applied after all findings were processed and post-processing tasks
(deduplication, rules) had already been dispatched, so those tasks could not
see the tags.

Apply import-time tags per batch in process_findings(), immediately after
parser tags and location persistence, before dojo_dispatch_task() — the same
pattern already used by bulk_apply_parser_tags.

Closed findings are intentionally excluded: they are absent from the current
report and should not receive the import tags.

Also renames the reimporter's ambiguous batch_findings (match-batch slice of
unsaved parser objects) to unsaved_findings_batch, freeing batch_findings for
the new dedupe-batch accumulator and preventing an iterator-mutation bug.

Consolidates apply_import_tags() into apply_import_tags_for_batch(), removing
a redundant DB re-query and unused Iterable import.

Update performance test query counts accordingly (each import loses 1 query
from the removed post-loop apply_import_tags call).
@valentijnscholten valentijnscholten changed the title feat(importers): apply import-time tags per batch before post-processing feat(importers): apply import-time tags per batch before post-processing, do not tag old findings May 8, 2026
@valentijnscholten valentijnscholten added this to the 2.59.0 milestone May 8, 2026
@Maffooch Maffooch requested review from Jino-T, blakeaowens and dogboat May 8, 2026 16:55
Copy link
Copy Markdown
Contributor

@mtesauro mtesauro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

@valentijnscholten valentijnscholten added the affects_pro PRs that affect Pro and need a coordinated release/merge moment. label May 8, 2026
@valentijnscholten valentijnscholten merged commit 558a3d2 into DefectDojo:dev May 8, 2026
157 checks passed
Maffooch added a commit to devGregA/django-DefectDojo that referenced this pull request May 8, 2026
Resolve conflicts from dev's removal of Credential Manager (DefectDojo#14836),
Stub Findings (DefectDojo#14837), deprecated questionnaire API (DefectDojo#14835), plus
Xygeni parser (DefectDojo#14769) and import-time tag batching (DefectDojo#14839).

Accepted dev's deletions: cred module, stub findings, deprecated
viewsets, and their UI sections in view_eng/view_finding/view_test.
Kept tailwind's refactored auth (api_permissions shim, action-string
roles, _user_authorized_for) over dev's legacy Permissions-enum code.
Trimmed cred/Stub_Finding refs from authorization/{api_permissions,
query_registrations,url_permissions}.py and the legacy auth tests.

Note: dojo/templates_classic/ still references removed URL names
(new_cred_*, promote_to_finding, delete_stub_finding) — follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Maffooch pushed a commit to devGregA/django-DefectDojo that referenced this pull request May 8, 2026
Resolve conflicts from dev's removal of Credential Manager (DefectDojo#14836),
Stub Findings (DefectDojo#14837), deprecated questionnaire API (DefectDojo#14835), plus
Xygeni parser (DefectDojo#14769) and import-time tag batching (DefectDojo#14839).

Accepted dev's deletions: cred module, stub findings, deprecated
viewsets, and their UI sections in view_eng/view_finding/view_test.
Kept tailwind's refactored auth (api_permissions shim, action-string
roles, _user_authorized_for) over dev's legacy Permissions-enum code.
Trimmed cred/Stub_Finding refs from authorization/{api_permissions,
query_registrations,url_permissions}.py and the legacy auth tests.

Note: dojo/templates_classic/ still references removed URL names
(new_cred_*, promote_to_finding, delete_stub_finding) — follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Maffooch pushed a commit to devGregA/django-DefectDojo that referenced this pull request May 8, 2026
Resolve conflicts from dev's removal of Credential Manager (DefectDojo#14836),
Stub Findings (DefectDojo#14837), deprecated questionnaire API (DefectDojo#14835), plus
Xygeni parser (DefectDojo#14769) and import-time tag batching (DefectDojo#14839).

Accepted dev's deletions: cred module, stub findings, deprecated
viewsets, and their UI sections in view_eng/view_finding/view_test.
Kept tailwind's refactored auth (api_permissions shim, action-string
roles, _user_authorized_for) over dev's legacy Permissions-enum code.
Trimmed cred/Stub_Finding refs from authorization/{api_permissions,
query_registrations,url_permissions}.py and the legacy auth tests.

Note: dojo/templates_classic/ still references removed URL names
(new_cred_*, promote_to_finding, delete_stub_finding) — follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

affects_pro PRs that affect Pro and need a coordinated release/merge moment. unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants