feat(incidents): Create postmortem doc at incident declaration

[rgibert]

Move Notion postmortem page creation from the dumpslack flow (triggered on status change to mitigated/done/postmortem) to on_incident_created so the doc exists from the start of the incident.

The page is added as a Slack channel bookmark without posting a message to the channel. When dumpslack runs later on status change, it finds the existing page via the ExternalLink(type=NOTION) record and populates it with Slack channel content and AI timeline -- no duplicate page creation occurs.

The new _create_postmortem_doc function follows the same DB-dedup pattern as _create_troubleshooting_doc: SELECT FOR UPDATE to claim the ExternalLink row, Notion API call outside the transaction, race-condition guard on re-lock, and placeholder cleanup on failure.

Resolves RELENG-32

Agent transcript: https://claudescope.sentry.dev/share/UOIi3zUcU_qU3uMtvatWsPs__r3nGeTIqoLl5MUiFnI

[linear-code]

RELENG-32

[cursor]

Bookmark failure never retried later

Medium Severity

Postmortem Slack bookmarks are now added in _create_postmortem_doc, while dumpslack still only calls add_bookmark when it created the page (notion_page_created). If early creation saves the Notion link but add_bookmark fails, a later dumpslack run uses the existing link and never retries the bookmark, so the channel can lack a postmortem bookmark permanently.

Additional Locations (1)

• src/firetower/incidents/hooks.py#L828-L835

 

^{Reviewed by Cursor Bugbot for commit 6c93a66. Configure here.}

[rgibert]

False positive for this PR. The bookmark retry gap is pre-existing -- before this PR, if dumpslack's own add_bookmark call failed, there was no retry mechanism either.

[cursor]

Race loser drops orphan Notion page

Medium Severity

When _create_postmortem_doc re-locks and finds link.url already set, it logs and returns without using the stored URL. If dumpslack (or another racer) saved first, this path leaves the Notion page it just created unused.

 

^{Reviewed by Cursor Bugbot for commit 84673d7. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 4 total unresolved issues (including 3 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 48ea6dc. Configure here.}

[sentry]

Bug: In a race condition, if the winner's Notion URL is unparseable, the function returns silently without notifying the user, unlike other similar error paths.
_{Severity: MEDIUM}

Suggested Fix

Add error handling to the race condition loser path. When _extract_notion_page_id returns None, post an error message to the Slack channel, similar to the handling on lines 99-113. This will inform the user about the failure to parse the Notion page URL.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/firetower/slack_app/handlers/dumpslack.py#L145-L146

Potential issue: In `_trigger_slack_dump`, when a race condition loser attempts to adopt
the winner's Notion page, it calls `_extract_notion_page_id` on the winner's URL. If
this URL is malformed and a page ID cannot be extracted, the function silently returns.
This leaves the user with an empty, unpopulated postmortem page and no notification
explaining the failure. This behavior is inconsistent with a similar check for existing
URLs earlier in the function, which correctly posts an error message to the Slack
channel. While a malformed URL is an edge case, it could occur due to database
corruption or an unexpected API response.

[sentry-warden]

Postmortem template markdown silently skipped for incidents using the new early-creation flow

_create_postmortem_doc creates the Notion postmortem page via create_postmortem_page and saves the URL, but unlike create_troubleshooting_page (which applies troubleshooting_template_markdown), it never renders the configured postmortem template_markdown. When dumpslack later runs on a status change, _trigger_slack_dump finds the pre-existing URL (db_record_created=False, existing_url set) and unconditionally sets update_slack=True, so apply_template's if not update_slack and self.template_markdown: branch is skipped. The result is that the postmortem template is never populated for any incident that goes through the normal flow — a behavioral regression, since previously dumpslack created the page itself with update_slack=False and applied the template.

Evidence

• _create_postmortem_doc (hooks.py:802) calls notion.create_postmortem_page(...) and saves page['url'], but never calls apply_template/_send_markdown; create_postmortem_page (notion.py:162) only sets page properties and does not render template_markdown.
• Contrast create_troubleshooting_page (notion.py), which sends troubleshooting_template_markdown after page creation — the postmortem path has no equivalent.
• In dumpslack _trigger_slack_dump, when the row already exists with a URL, line ~98 enters if not db_record_created and existing_url: and sets update_slack = True (dumpslack.py:113).
• apply_template (notion.py:256) gates the template on if not update_slack and self.template_markdown:, so with update_slack=True the postmortem template_markdown is never applied.

_{Identified by Warden code-review · KG8-CZB}

[sentry-warden]

notion.archive_page() called while holding SELECT FOR UPDATE lock inside transaction

Move notion.archive_page(page["id"]) outside the transaction.atomic() block — the _create_troubleshooting_doc sibling explicitly documents this rule: "Notion API calls happen outside the transaction to avoid holding the SELECT FOR UPDATE lock while making slow external requests."

Evidence

• _create_troubleshooting_doc (hooks.py ~l.726) has an explicit comment: "Notion API calls happen outside the transaction to avoid holding the SELECT FOR UPDATE lock while making slow external requests" and never calls Notion inside transaction.atomic().
• In _create_postmortem_doc, the second with transaction.atomic(): block issues ExternalLink.objects.select_for_update().get(...) then, on the race-condition branch, calls notion.archive_page(page["id"]) at line 833 before the transaction closes — the SELECT FOR UPDATE row lock is held for the full duration of the HTTP call to Notion.
• NotionService.archive_page calls self.client.pages.update(...) (notion.py:158), a synchronous network request; if slow or hanging, it blocks the lock for all other writers on this ExternalLink row.
• The _create_troubleshooting_doc race-condition branch (hooks.py ~l.744) simply logs and returns without any Notion call inside the transaction, confirming the intended pattern.

Also found at 1 additional location

• src/firetower/slack_app/handlers/dumpslack.py:142-142

_{Identified by Warden find-bugs · 8LG-TC8}