perf: filter source networks using overrides in network mapping

[albertoflorez]

Description

This PR optimizes the network map creation process by filtering the source networks (vSphere/Ovirt) based on the provided overrides before entering the task loop.

Motivation

Beyond performance, this change addresses a critical scalability issue. In large-scale vSphere environments (e.g., thousands of PortGroups), attempting to map every available network results in an oversized NetworkMap Custom Resource.

This often leads to:

1. API Server Rejection: The generated resource exceeds the standard 1MB limit for objects in etcd, causing the k8s module to fail with a "Request entity too large" error.

2. Controller Timeout: The MTV controller struggles to reconcile objects with massive spec.map arrays, leading to timeouts and instability.

By filtering the networks at the source, we ensure that the resulting NetworkMap object only contains relevant entries, keeping the resource footprint small and within Kubernetes API limits.

Changes

• Added a mtv_allowed_ids fact to capture required network IDs from overrides.
• Updated the loop logic in Process VMware Networks and Process Ovirt Networks to only iterate over the filtered list when overrides are defined.
• Maintained backward compatibility: if no overrides are provided, it still processes all networks.

Impact

Significant reduction in playbook execution time for large-scale environments.

[stevefulme1]

Great optimization — the etcd 1MB limit issue in large vSphere environments is a real problem and this is the right approach to fix it.

One issue I spotted: exclude-only overrides will break with this change.

If a user provides overrides with only exclude: true entries (no include keys), the "Extract allowed network IDs" task runs but produces an empty list (no include attributes to select). The loop condition then sees mtv_management_network_map_overrides | length > 0 is true, so it filters mtv_networks against the empty mtv_allowed_ids — resulting in zero networks processed. The downstream exclude logic in _mtv_network_map_process_network.yml never gets a chance to run.

The fix is to key the loop condition on whether include IDs were actually extracted, not on whether overrides exist:

loop: >-
  {{
    (mtv_networks | selectattr('id', 'in', mtv_allowed_ids) | list)
    if (mtv_allowed_ids | default([]) | length > 0)
    else mtv_networks
  }}

This way the filter only kicks in when there are actual include IDs, and exclude-only overrides still iterate the full list (letting the existing downstream task handle the exclusion as it does today).

Minor nit: might also be worth initializing mtv_allowed_ids: [] in the "Initialize data structures" task at the top to prevent fact leakage if the role is called twice in one play (once with overrides, once without).

[albertoflorez]

Great catch! You are absolutely right about the exclude-only edge case. I've pushed a new commit that updates the loop condition to rely on mtv_allowed_ids | length > 0 so the filter only kicks in when there are actual include IDs. I also initialized mtv_allowed_ids: [] at the top to prevent any fact leakage. Thanks for the review!

[stevefulme1]

Thanks Alberto — the exclude-only fix and initialization look correct, exactly what I had in mind.

Two minor nits before I approve:

1. Comment language — Line 4 has a Spanish comment (# <--- AÑADIDO: Inicialización limpia). The rest of the codebase is in English, so this should be changed or removed for consistency.

2. Missing trailing newline — The file lost its trailing newline on the last line (...). Most linters and POSIX tools expect a newline at end of file.

Once those are cleaned up I'll approve.

[albertoflorez]

ouch! you are right! sorry going so fast :)