balloons: IRQ CPU affinity controls - #722
Conversation
Add pkg/irq for reading interrupts from /proc/interrupts and reading and writing CPU affinities through /proc/irq/NUMBER/smp_affinity_list. Wire the proc root to the plugin host root and mount the host proc filesystem into the balloons daemonset so that affinities of host IRQs can be updated (write access needed). Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Add the balloon type options irqClaim and irqMode to the balloons policy configuration. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Set CPU affinities of system IRQs according to the irqClaim and irqMode balloon type options. For each IRQ, the logic is: find balloons that want exactly this IRQ (irqClaim). If nobody, then find balloons that want all IRQs (irqMode: sink). If no volunteers, then list balloons that do not want any IRQs (irqMode: isolate), and set IRQs affinity to all those who do not care. The default is nobody cares (no irq options present), meaning that the balloons policy will not touch IRQ CPU affinities at all, keeping the original behavior. Affinities are updated after balloon changes. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Add an end-to-end test that verifies irqClaim, irqMode sink and irqMode isolate by inspecting /proc/irq affinities on the test VM. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
Describe the irqClaim and irqMode balloon type options and their common use cases. Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
f80e862 to
a93ae5f
Compare
| irqModeSink = "sink" | ||
| // irqModeIsolate removes CPUs of a balloon from the affinity | ||
| // of IRQs that are neither claimed nor sinked. | ||
| irqModeIsolate = "isolate" |
There was a problem hiding this comment.
nit: IMO it would be better to define these constants in the previous commit under pkg/apis/config/v1alpha1/resmgr/policy/balloons/config.go (since that is that where it is effectively defined that these plus "" are the valid modes) and then alias them here with a const-assignment.
| p.BlockMeters() | ||
| defer p.UnblockMeters() | ||
| defer p.commitCpuClasses() | ||
| defer p.applyIrqAffinities() |
There was a problem hiding this comment.
Related to configuration and reconfiguration: is my understanding correct that an irqMode: sink, irqClaim: [ $IRQ_CLAIMS ] is not sensible/meaningful combination for any single balloon type. If it is, should we verify against it in pkg/apis/config/*/resmgr/policy/balloons.Config.Validate() ?
|
|
||
| cfgapi "github.com/containers/nri-plugins/pkg/apis/config/v1alpha1/resmgr/policy/balloons" | ||
| "github.com/containers/nri-plugins/pkg/cpuallocator" | ||
| "github.com/containers/nri-plugins/pkg/irq" |
There was a problem hiding this comment.
Related to the commit message of commit "balloons: implement IRQ affinity control", which AFAICT github does not allow to comment on. Is the following description of sink mode
"""
...
For each IRQ, the logic is: find balloons that want exactly this
IRQ (irqClaim). If nobody, then find balloons that want all
IRQs (irqMode: sink).
...
"""
accurate ? Or should it be "... If nobody, then find balloons that want all
otherwise unclaimed IRQs (irqMode: sink). ..." ?
klihub
left a comment
klihub
left a comment
There was a problem hiding this comment.
I have a few minor questions and comments.
2 participants
Control IRQ CPU affinities using new
irqClaimandirqModeballoon type options.For each IRQ, the logic is:
Combining irqClaim with "irqMode: isolate" in the same balloon type creates balloons that handle only claimed IRQs and no others.