Skip to content

Two environment-document rebuilds enqueued per Feature Versioning v2 publish #7492

@gagantrivedi

Description

@gagantrivedi

Bug

Every time an EnvironmentFeatureVersion.publish() runs, two separate task-processor tasks end up calling Environment.write_environment_documents for the same environment_id:

Path Aenvironment_feature_version_published signal receiver:

api/features/versioning/receivers.py:42-47

@receiver(environment_feature_version_published, sender=EnvironmentFeatureVersion)
def update_environment_document(instance, **kwargs):
    rebuild_environment_document.delay(
        kwargs={"environment_id": instance.environment_id},
        delay_until=instance.live_from,
    )

rebuild_environment_document (api/environments/tasks.py:26-28) calls Environment.write_environment_documents(environment_id=environment_id).

Path B — audit-log AFTER_CREATE hook fired when the EF_VERSION audit row is written:

api/audit/models.py:141-168

@hook(AFTER_CREATE, priority=priority.HIGHEST_PRIORITY,
      when="environment_document_updated", is_now=True)
def process_environment_update(self) -> None:
    ...
    process_environment_update.delay(args=(self.id,))

process_environment_update (api/environments/tasks.py:31-44) calls Environment.write_environment_documents(environment_id=audit_log.environment_id, project_id=audit_log.project_id) and then broadcasts the SSE update message.

The EF_VERSION audit log row is created by the third receiver on the same signal (create_environment_feature_version_published_audit_log, receivers.py:58-64), which calls create_environment_feature_version_published_audit_log_task to insert the row. That insert triggers Path B.

Both paths end up writing the same engine document to DynamoDB for the same environment.

Reproduction

  1. Enable Feature Versioning v2 on an environment with at least one feature and Dynamo writes configured.
  2. Edit a flag value through any of the supported v2 write paths (e.g. update-flag-v2 or EnvironmentFeatureVersionViewSet), or commit a change request.
  3. Tail the task processor / DynamoDB write metrics.
  4. Observe two Environment.write_environment_documents calls for the same environment within milliseconds: one from process_environment_update (TaskPriority.HIGHEST, fires immediately on audit row creation), and one from rebuild_environment_document (TaskPriority.HIGH, fires at live_from).

For scheduled changes (live_from > now), Path B still fires at publish time and produces a Dynamo write that does not yet include the future version (the SQL in get_latest_versions.sql filters by live_from <= now), then Path A fires again at live_from with the correct content. The Dynamo write at publish time is wasted but not incorrect.

Impact

  • Two Dynamo write-units per v2 publish where one would suffice. For organisations with high publish throughput and Dynamo-backed Edge proxy, this doubles the write-cost component attributable to v2 flag changes.
  • Task processor queue depth grows faster than necessary on bursty publish workloads.
  • No behavioural breakage — both writes produce the same engine document, and the SSE broadcast is only triggered from Path B.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions