kipmi0 IPMI Kernel Helper 99% / 100% High CPU Usage (But Low Load Average)

In CentOS 7 or Red Hat Enterprise Linux (RHEL) 7, a process called “kipmi0” may show that it’s utilizing unusually high percentage of CPU processing power, e.g. 99.7%, 99.8%, 99.9% or even 100%, even though kipmi0 process is using 0.0% memory and does not actually increase the load average, which continue to hover at a low nearing 1 or 2 (provided the server is not under heavy load at the time).

The problematic kipmi0 can be shown by running the “top” command. The kipmi0 process remains at 100% for the entire lifetime of the server uptime.

Sometimes, a system reboot may make kipmi0 goes away for a little moment, but soon it’s roaring back to life with 100% CPU usage. To make matter worse, the kipmi0 process cannot be killed and terminated, where the process keeps running and stayed.

According to IBM support, the kipmi0 process may show increased CPU utilization in Linux. The utilization may increase up to 100% when the IPMI (Intelligent Platform Management Interface) device, such as a BMC (Baseboard Management Controller) or IMM (Integrated Management Controller) is busy or non-responsive.

The suggested resolution is, surprisingly, takes no action. No fix required and increased CPU utilization should be ignored as it has no impact on actual system performance since kipmiN kernel helper threads are executed at low priority. Other workarounds include reset the BMC or reboot the system if using an IPMI device (which does not solve the issue), or stop the IPMI service if not using IPMI device (where the IPMI service shouldn’t even started if it’s not in use).

Red Hat Enterprise Linux (RHEL) / CentOS 6 and Red Hat Enterprise Linux / CentOS 7 has built Linux IPMI ipmi_si drivers and its related handles and modules into the kernel by default. RedHat explains why this kipmi0 is generating high CPU load:

kipmi0 is a kernel helper process/thread involved in handling IPMI interfaces. Within IPMI, there are several standard classes of interfaces. Some of these classes, like KCS (Keyboard Control Style) and SMIC (System Management Interface Chip) do not use interrupt requests (IRQs) to signal changes, and thus require polling to obtain command results. The kipmiN kernel helper threads perform this polling. Thus, it is normal for these threads to consume significant CPU time while an IPMI operation is in progress.

In this case, there is a problem in the interaction between the driver and the hardware/firmware which leads the driver to believe that an operation is still in progress, causing the high CPU load to continue until the system is rebooted.

To solve the issue that kipmiN helper threads, there are plenty of workarounds, such as commenting out ipmi-si and ipmisensors in /etc/sysconfig/lm_sensors. If you’re certain that IPMI service is not required on the server, or there are no IPMI devices installed on the computer, you can disable the disable the kernel service for IPMI.

To disable the kernel service for IPMI completely, we can blacklist the IPMI related modules in modprobe utility to prevent the system from using IPMI kernel modules. To do this, edit the blacklist.conf in the /etc/modprobe.d/ directory and add the following line as root:

blacklist ipmi_si
blacklist ipmi_ssif
blacklist ipmi_devintf
blacklist ipmi_msghandler

The list above includes most major IPMI modules started on boot, but not all of them are started by the system. If any of them doesn’t exist on the system, the line can be excluded.