fix(connectors): purge archived/deleted source items in KB connectors (#5880)

* fix(confluence): exclude archived pages from KB connector listings so reconciliation purges them

* fix(connectors): purge archived/deleted source items across seven more KB connectors

The sync engine only purges a knowledge-base document when its source item is
absent from a full-sync listing, so any connector that keeps listing
archived/trashed/canceled items never drops them. An audit of all 51 connectors
found seven with this bug:

- asana: list only non-archived projects (the API returns both when `archived`
  is omitted), so tasks under archived projects stop being re-listed
- google-sheets: skip a spreadsheet Drive reports as trashed, which stays
  readable by id for 30 days before the Sheets call starts 404ing
- incidentio: exclude canceled incidents by default (cancelling is incident.io's
  documented stand-in for deletion), with an explicit opt-in to sync them
- outlook: exclude Deleted Items from the all-mail listing, which Graph
  otherwise includes
- servicenow: drop retired knowledge articles, which the Table API returns with
  no implicit state filter
- webflow: drop archived CMS items, which the staged items endpoint always
  returns and offers no way to filter
- youtube: drop playlist entries whose video was deleted or made private, which
  the API keeps returning as placeholder items

Every exclusion keys off an explicit non-current signal and fails open on a
missing field or a failed metadata read, since wrongly excluding a live item
would hard-delete it. Explicit user filter selections are still honoured
verbatim; the new defaults apply only when nothing is configured.

Also flag truncated listings as capped in asana, outlook, and servicenow. All
three silently cut a listing short at their configured item cap without setting
`syncContext.listingCapped`, so reconciliation read the untraversed tail as
deleted at the source and hard-deleted it.

* fix(asana): honour the pinned-project exception on the task rehydrate path

listDocuments deliberately keeps syncing a project the user pinned via the
`project` config field even once it is archived, but getDocument ignored
sourceConfig and applied the all-parents-archived exclusion unconditionally.
For a pinned archived project the listing kept emitting its tasks while every
hydration returned null, so new tasks were dropped as empty and already-indexed
ones were frozen at their last content.

isTaskUnderActiveProject now takes the pinned project gid and keeps any task
reachable through it, matching the listing exactly. The unpinned path is
unchanged and still fails open on missing/non-boolean archived values.

* fix(connectors): key removal on explicit source signals, never on absence

Follow-up to the connector purge fixes, from an independent audit.

YouTube inferred deletion from absence: a playlist entry whose id was missing
from a `videos.list` response was dropped, so a well-formed 200 that returned 49
of 50 requested ids hard-deleted the 50th. Playlist items instead carry a
documented `status.privacyStatus`, available as a free part on a call the
connector already makes, so the extra `videos.list` request is gone along with
its quota-failure and pagination-wedge risks. An item is now excluded only on an
explicit `private`; missing, empty, or unrecognized values keep it.

ServiceNow read every record through a guard requiring a string `sys_id`, but
the listing requests `sysparm_display_value=all`, under which every field —
`sys_id` included — comes back as `{display_value, value}`. The guard rejected
every record, so the retired-article filter was unreachable and the sys_id
object would have leaked into `externalId` and `title` had it not been. Records
are now read through the existing `rawValue` normalizer, which accepts both wire
shapes, and the fixtures use the shape the API actually returns.

Also: resolve the ServiceNow cap ambiguity with `X-Total-Count` so a table that
ends exactly on a page boundary is not read as truncated; stop the Google Sheets
comment claiming a purge the engine's zero-document guard prevents; and assert
the Outlook junk-mail invariant instead of comparing a constant to itself.

Document the behavior change: content archived, retired, or trashed at the
source is now removed from the knowledge base, and restoring it re-ingests it.

Author: waleedlatif1

apps/docs/content/docs/en/knowledgebase/connectors.mdx

@@ -162,6 +162,12 @@ To reverse an exclusion, switch to the **Excluded** tab and click **Restore** 
 
 On each run the connector fetches documents from the source and compares them against what's already stored. Only changed documents are reprocessed — new content is added, updated content is re-chunked and re-embedded, deleted content is removed. A connector syncing thousands of documents will only do real work when something actually changes.
 
+### Content Removed From View
+
+Content that still exists at the source but is no longer current is treated the same as deleted, and is removed from your knowledge base. This covers an archived Confluence page, a task in an archived Asana project, an archived Webflow CMS item, a retired ServiceNow knowledge article, a spreadsheet in the Google Drive trash, a message moved to Outlook's Deleted Items, and a cancelled incident.io incident.
+
+This keeps agents from citing content your team has already retired. Restoring the item at the source brings it back on the next sync as a new document.
+
 ### Connector Status
 
 | Status | Meaning |
@@ -196,7 +202,7 @@ You can add as many connectors as you need to a single knowledge base. Each mana
 
 <FAQ items={[
   { question: "How often do connectors sync?", answer: "You choose from hourly, every 6 hours, daily (default), weekly, or manual-only. Sub-hourly frequencies require a Max or Enterprise plan. Each connector has its own schedule." },
-  { question: "What happens if a source document is deleted?", answer: "On the next sync the connector detects the document is gone and removes it from your knowledge base automatically." },
+  { question: "What happens if a source document is deleted?", answer: "On the next sync the connector detects the document is gone and removes it from your knowledge base automatically. Content that is archived, retired, trashed, or otherwise removed from view at the source is treated the same way — restoring it at the source brings it back on the next sync." },
   { question: "What happens when I delete a connector?", answer: "The connector is removed and future syncs stop. You're given the option to also delete all documents that were synced by that connector. If you don't check that option, they stay in the knowledge base as-is." },
   { question: "What does the Disabled status mean?", answer: "After 10 consecutive full-sync failures, the connector is automatically disabled to stop retrying. Reconnect the OAuth account or click Resume to re-enable it." },
   { question: "Do metadata tags count against a limit?", answer: "Yes. Tag slots are shared across all documents in a knowledge base — 17 slots total. Multiple connectors draw from the same pool, so plan accordingly if several connectors each auto-populate tags." },