feat(vm): memory hotplug (phase 1) by diafour · Pull Request #2110 · deckhouse/virtualization

diafour · 2026-03-16T11:44:58Z

Description

Change memory setting from domain.resources to domain.memory.guest.
Support old VMs to not require reboot on module update.

Why do we need it, and what problem does it solve?

It is a phase 1 of implementation of the memory hotplug support.

Limitations:

No VMClass settings, only spec.memory.size in the VM is supported for now.
maxGuest is hardcoded to 256Gi (we choose conservative 38 bit physical address).
Memory reducing forced to require reboot.

What is the expected result?

VM is migrating and memory is changed when spec.memory field is changed.

384Mi -> 1384Mi (Crossing 1Gi threshold upward will enable hotplug and requires a reboot)

Restart initiated

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-6tfh8 1/1 Terminating 0 5m6s 10.66.10.4 virtlab-mi-1 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Stopped 1 20% 384Mi True virtlab-mi-1 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 5m6s Succeeded virtlab-mi-1 False True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Stopped False

After restart

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 1384Mi maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-npxkw 1/1 Running 0 54s 10.66.10.4 virtlab-mi-2 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Running 1 20% 1384Mi True True virtlab-mi-2 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 54s Running 10.66.10.4 virtlab-mi-2 True True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Running True

1384Mi -> 2384Mi (Hotplug enabled: add memory with live migration)

Migration initiated

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 2384Mi maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-s2cgs 1/1 Running 0 117m 10.66.10.4 virtlab-mi-1 1/1
pod/d8v-vm-vm-alpine-uefi-wnptq 0/1 ContainerCreating 0 6s virtlab-mi-3 0/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Running 1 20% 2384Mi True True virtlab-mi-1 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 117m Running 10.66.10.4 virtlab-mi-1 True True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Running True

NAME PHASE TYPE VIRTUALMACHINE AGE
virtualmachineoperation.virtualization.deckhouse.io/hotplug-resources-46c8z InProgress Evict vm-alpine-uefi 6s

NAME PHASE VMI
internalvirtualizationvirtualmachineinstancemigration.internal.virtualization.deckhouse.io/vmop-hotplug-resources-46c8z Scheduling vm-alpine-uefi

After migrating

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 2384Mi maxGuest: 256Gi

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-s2cgs 0/1 Completed 0 119m 10.66.10.4 virtlab-mi-1 1/1
pod/d8v-vm-vm-alpine-uefi-wnptq 1/1 Running 0 2m20s 10.66.10.4 virtlab-mi-3 1/1

2384Mi -> 384Mi (Crossing 1Gi threshold downward will disable hotplug and requires reboot)

Reboot started

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 2384Mi maxGuest: 256Gi

$ k -n vm get all,po,intvirt,vmop -owide

NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-wnptq 1/1 Terminating 0 3m5s 10.66.10.4 virtlab-mi-3 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Stopped 1 20% 2384Mi True virtlab-mi-3 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 120m Succeeded virtlab-mi-3 False True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Stopped False

After reboot

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-x2xvj 1/1 Running 0 26s 10.66.10.4 virtlab-mi-3 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Running 1 20% 384Mi False True virtlab-mi-3 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 27s Running 10.66.10.4 virtlab-mi-3 True True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Running True

384Mi -> 512Mi (Hotplug disabled: memory size changing requires reboot)

Reboot started

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 384Mi

$ k -n vm get all,po,intvirt,vmop -owide
Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2.
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-x2xvj 1/1 Terminating 0 118s 10.66.10.4 virtlab-mi-3 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Stopped 1 20% 384Mi True virtlab-mi-3 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 118s Succeeded virtlab-mi-3 False True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Stopped False

After reboot

$ k -n vm get internalvirtualizationvirtualmachines.internal.virtualization.deckhouse.io vm-alpine-uefi -oyaml | grep ' memory:' -A 10 memory: guest: 512Mi

$ k -n vm get all,po,intvirt,vmop -owide
Warning: kubevirt.io/v1 VirtualMachineInstancePresets is now deprecated and will be removed in v2.
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pod/d8v-vm-vm-alpine-uefi-grqdn 1/1 Running 0 33s 10.66.10.4 virtlab-mi-1 1/1

NAME PHASE CORES COREFRACTION MEMORY NEED RESTART AGENT MIGRATABLE NODE IPADDRESS AGE
virtualmachine.virtualization.deckhouse.io/vm-alpine-uefi Running 1 20% 512Mi False True virtlab-mi-1 10.66.10.4 10d

NAME AGE PHASE IP NODENAME READY LIVE-MIGRATABLE PAUSED
internalvirtualizationvirtualmachineinstance.internal.virtualization.deckhouse.io/vm-alpine-uefi 33s Running 10.66.10.4 virtlab-mi-1 True True

NAME AGE STATUS READY
internalvirtualizationvirtualmachine.internal.virtualization.deckhouse.io/vm-alpine-uefi 10d Running True

Checklist

The code is covered by unit tests.
e2e tests passed.
Documentation updated according to the changes.
Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: vm
type: feature
summary: "Initial implementation of memory hotplug support: change spec.memory will set new memory size with live migration."

diafour · 2026-03-20T13:29:28Z

images/virt-artifact/werf.inc.yaml

 shell:
  install:
  - |
+    echo rebuild 1


will be removed after merging PR in 3p-kubevirt

Simple implementation of the memory hotplug support. - Change memory configuration: set it in domain.memory.guest instead of setting requests and limits in domain.resources. - Support old VMs to not require reboot on module update. - Support for enabling feature gates explicitly in ModuleConfig. Dev notes: - Adding detector of kvvm.spec changes to require reboot if kvbuilder changes something. - New on-demand vmop migration prefixed as "hotplug-resource-". Signed-off-by: Ivan Mikheykin <ivan.mikheykin@flant.com>

LopatinDmitr · 2026-04-04T19:03:50Z

images/virtualization-artifact/pkg/controller/reconciler/reconciler.go

 		case err == nil: // OK.
 		case errors.Is(err, ErrStopHandlerChain):
-			log.Debug("Handler chain execution stopped")
+			msg := fmt.Sprintf("Handler %s stopped chain execution", name)


Why do we need the msg variable?

LopatinDmitr

in sync_kvvm.go: Patch → immediately Update the same object (The problem is that after Patch(...) the server object gets a new resourceVersion, and the local newKVVM can stay with the old one. Then the next Update(...)can catch a conflict on a real API.)
Inconsistency of the hotplug' threshold: builder and comparator are considered differently (in builder if memorySize.Cmp(*hotplugThreshold) >= 0, and comparator isHotpluggable := current.Memory.Size.Cmp(*hotplugThreshold) > 0.
For cases like:
- 1Gi -> 2Gi
- 1Gi -> 1.5Gi
The system can:
- generate a hotplug-ready spec,
- but it still requires a restart when comparing.
kvbuilder.SetMemory() feature gate is taken globally, and comparator receives it by injection (Two sources of truth are obtained:
- compare uses the passed feature gate;
- build uses a global singleton.)
Support for “old VMs" is tied only to PrintableStatus == Running. Does it take into account that the VM can migrate?

diafour added this to the v1.8.0 milestone Mar 16, 2026

diafour requested review from Isteb4k and yaroslavborbat as code owners March 16, 2026 11:44

diafour force-pushed the feat/vm/hotplug-memory branch from 0f99db6 to 8ebdbe2 Compare March 16, 2026 11:46

diafour requested review from nevermarine and universal-itengineer as code owners March 17, 2026 09:03

diafour self-assigned this Mar 17, 2026

diafour modified the milestones: v1.8.0, v1.7.0 Mar 17, 2026

diafour force-pushed the feat/vm/hotplug-memory branch from fb903a5 to e174fb5 Compare March 17, 2026 18:40

diafour marked this pull request as draft March 17, 2026 18:40

diafour force-pushed the feat/vm/hotplug-memory branch 2 times, most recently from 6c201f9 to 4af4c78 Compare March 20, 2026 13:17

diafour marked this pull request as ready for review March 20, 2026 13:18

diafour requested a review from LopatinDmitr March 20, 2026 13:28

diafour commented Mar 20, 2026

View reviewed changes

diafour force-pushed the feat/vm/hotplug-memory branch from 4af4c78 to 4a4baaf Compare March 24, 2026 19:44

nevermarine modified the milestones: v1.7.0, v1.8.0 Mar 31, 2026

diafour force-pushed the feat/vm/hotplug-memory branch from 4a4baaf to fb28e31 Compare April 2, 2026 06:38

diafour requested review from fl64, goganat and prismagod as code owners April 2, 2026 06:38

diafour removed request for fl64 and goganat April 2, 2026 07:13

diafour force-pushed the feat/vm/hotplug-memory branch from 177f431 to 3a30485 Compare April 2, 2026 12:38

diafour force-pushed the feat/vm/hotplug-memory branch from 31b93a2 to cc31668 Compare April 2, 2026 16:15

LopatinDmitr reviewed Apr 4, 2026

View reviewed changes

LopatinDmitr approved these changes Apr 4, 2026

View reviewed changes

LopatinDmitr self-requested a review April 4, 2026 20:02

LopatinDmitr reviewed Apr 4, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(vm): memory hotplug (phase 1)#2110

feat(vm): memory hotplug (phase 1)#2110
diafour wants to merge 1 commit intomainfrom
feat/vm/hotplug-memory

diafour commented Mar 16, 2026 •

edited

Loading

Uh oh!

diafour Mar 20, 2026

Uh oh!

LopatinDmitr Apr 4, 2026 •

edited

Loading

Uh oh!

LopatinDmitr left a comment •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

diafour commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Why do we need it, and what problem does it solve?

What is the expected result?

Checklist

Changelog entries

Uh oh!

diafour Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

LopatinDmitr Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LopatinDmitr left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

diafour commented Mar 16, 2026 •

edited

Loading

LopatinDmitr Apr 4, 2026 •

edited

Loading

LopatinDmitr left a comment •

edited

Loading