C-states not fuctioning on Ubuntu Server 22.04. Please help

Question

I’m putting together a home server running Ubuntu 22.04 LTS and I’m having a hell of a time with the c-states. Current power draw of the system is 33 watts but I’d like to lower that as far as possible. At the moment I can only get a C2 c-state. I’ve tried updating the kernel with HWE, I’ve tried turning off the sound and LAN in the bios, I’ve made sure the C packages are enabled in the bios, I’ve run a Ubuntu live USB with powertop –auto-tune and so far nothing has worked. My system is as follows:

Motherboard: Asrock B660M-ITX/ac
CPU: Intel i3-12100
Storage: Intel 660p 1 TB M.2-2280 PCIe 3.0 X4 NVME Solid State Drive
SATA Storage: Seagate Ironwolf NAS 6TB
RAM: Corsair Vengeance LPX 16 GB CMK16GX4M2Z3200C16
OS: Ubuntu 22.04.03 LTS, Kernel 6.2.0-26 generic

Some more info, don’t know if it will help

hsadmin@homeserver:~$ grep . /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver
/sys/devices/system/cpu/cpu0/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu1/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu2/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu3/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu4/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu5/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu6/cpufreq/scaling_driver:intel_pstate
/sys/devices/system/cpu/cpu7/cpufreq/scaling_driver:intel_pstate

hsadmin@homeserver:~$ grep . /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu1/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu2/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu3/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu4/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu5/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu6/cpufreq/scaling_governor:powersave
/sys/devices/system/cpu/cpu7/cpufreq/scaling_governor:powersave
hsadmin@homeserver:~$ grep . /sys/devices/system/cpu/cpuidle/*
/sys/devices/system/cpu/cpuidle/available_governors:ladder menu teo
/sys/devices/system/cpu/cpuidle/current_driver:intel_idle
/sys/devices/system/cpu/cpuidle/current_governor:menu
/sys/devices/system/cpu/cpuidle/current_governor_ro:menu
/sys/devices/system/cpu/cpuidle/low_power_idle_cpu_residency_us:0
/sys/devices/system/cpu/cpuidle/low_power_idle_system_residency_us:0
hsadmin@homeserver:~$ grep . /sys/devices/system/cpu/cpu0/cpuidle/state*/name
/sys/devices/system/cpu/cpu0/cpuidle/state0/name:POLL
/sys/devices/system/cpu/cpu0/cpuidle/state1/name:C1E
/sys/devices/system/cpu/cpu0/cpuidle/state2/name:C6
/sys/devices/system/cpu/cpu0/cpuidle/state3/name:C8
/sys/devices/system/cpu/cpu0/cpuidle/state4/name:C10
hsadmin@homeserver:~$ grep . /sys/devices/system/cpu/cpu0/cpuidle/state*/desc
/sys/devices/system/cpu/cpu0/cpuidle/state0/desc:CPUIDLE CORE POLL IDLE
/sys/devices/system/cpu/cpu0/cpuidle/state1/desc:MWAIT 0x01
/sys/devices/system/cpu/cpu0/cpuidle/state2/desc:MWAIT 0x20
/sys/devices/system/cpu/cpu0/cpuidle/state3/desc:MWAIT 0x40
/sys/devices/system/cpu/cpu0/cpuidle/state4/desc:MWAIT 0x60

In my bios I have enabled:

Package C State Support
CFG Lock
C6DRAM

Prior to enabling them powertop showed no c-states being used, after enabling them C2 was being used.

So I have no idea where to go from here. Perhaps the issue is with my NVME drive.

Also, there are some other settings in the motherboard that are disabled, could they be relevant?

PCI Express Native Control - Select Enable for enhanced PCI Express power saving in OS.
PCIE ASPM Support - This option enables/disables the ASPM support for all CPU downstream devices.
PCH PCIE ASPM Support - This option enables/disables the ASPM support for all PCH PCIE devices.
DMI ASPM Support -This option enables/disables the control of ASPM on CPU side of the DMI Link.
PCH DMI ASPM Support - This option enables/disables the ASPM support for all PCH DMI devices.

Edit: I enabled the ASPM options and now powertop is showing that I am getting down to C3. Any idea what would be preventing me from getting to C6?

Edit2: I Decided to order a new SSD and RAM that are on the motherboard's list of compatible parts.

Edit3: New parts did not help. Have contacted Asrock to see what they say.

Individual CPUs are getting to C10. Myself, I would not expect your system to get to package level C10. Mains power of 33 watts is pretty good. My previous computer (i5-9600K) was about 33 watts, and that was the best I could achieve. My current test server (i5-10600K) draws 41 watts at idle, but it has multiple 2 TB HDD drives attached. — Doug Smythies, Sep 05 '23 at 13:50

bleptoga · Answer 1 · 2023-09-05T13:07:17.383

1

For anyone with the same issue I heard back from Asrock and they said that the reason I can't achieve any better than C3 package state is because my particular board does not support modern standby.

I believe this intel page confirms their response

https://edc.intel.com/content/www/us/en/design/ipla/software-development-platforms/client/platforms/alder-lake-desktop/12th-generation-intel-core-processors-datasheet-volume-1-of-2/001/package-c-states/

Modern Standby is a platform state. On display time out the OS requests the processor to enter package C10 and platform devices at RTD3 (or disabled) in order to attain low power in idle. Modern Standby requires proper BIOS and OS configuration.

So looks like I am out of luck.

edited Sep 05 '23 at 13:07

answered Sep 05 '23 at 13:06

bleptoga

61

This cannot be the correct answer. Modern Standby is basically a feature to explicitly request the package C10 state. Regardless, the system should still easily be able to reach the C6 and C8 states. – Gil Nov 09 '23 at 10:39
1

Yes Gil you are correct. I meant to make another post saying this was incorrect. After about 3 weeks of further corrospondece with Asrock I eventually got them to talk to their engineers in detail, and the engineers said that the actual problem was that the board was not made to go into better than a C3 state. They were clear that there was nothing I could do to make it do otherwise. To anyone else who reads this confirm in writing before buying. I have unselected this post as the answer now, I meant to do that before, but, life. – bleptoga Nov 11 '23 at 06:50
Thanks for the followup. I have the same issue with a same generation ASRock motherboard. I guess I will just return it and try another vendor. – Gil Nov 16 '23 at 12:57
1

Yeah good call. In case you're not aware of it there is a spreadsheet online that shows lots of different systems and the c-states they can acheive and their power output. Might be useful for you. https://docs.google.com/spreadsheets/u/0/d/1LHvT2fRp7I6Hf18LcSzsNnjp10VI-odvwZpQZKv_NCI/htmlview – bleptoga Nov 18 '23 at 03:41

voido · Answer 2 · 2024-01-17T19:37:55.787

I managed to get C10 pkg C-State (previously I get no higher than C3) on Asrock LGA1700 mobo and you can too. Yay!

Motherboard: Asrock H610M-ITX/ac
CPU: i5-12500 NVME: Samsung 970 EVO 500Gb
SSD: PLEXTOR PX-128M (only used on Windows) / 2x2.5" HDD: 250GB Samsung HM250HI + 4TB Seagate ST4000LM016 (on Proxmox)
RAM: 2x32Gb Samsung DDR4 3200 PSU: Corsair RM650x 2021

So you have to enable/change hidden BIOS menus by using AMISCE (AMI Setup Control Environment) utility v5.03 or 5.05 for Windows (it can easily be found on the internet). So you have to install Windows and to enable Administrator password in your BIOS.

Run Powershell as admin and cd to folder where your AMISCE extracted when run this command

.\SCEWIN_64.exe /o /s '.\setup_script_file.txt' /a

In the setup_script_file.txt current values is marked with asterisk “*”. Our goal is to change “Lower Power S0 Idle Capability” from 0x0 (Disabled) to 0x1 (Enabled). From the command line you can check value/status by this command:

.\SCEWIN_64.exe /o /lang 'en-US' /ms "Low Power S0 Idle Capability" /hb

“*” next to “[00]Disabled” indicates it currently disabled. Then change it:

.\SCEWIN_64.exe /i /lang 'en-US' /ms "Low Power S0 Idle Capability" /qv 0x1 /cpwd YOUR-BIOS-ADMIN-PASSWORD /hb

Check again:

.\SCEWIN_64.exe /o /lang 'en-US' /ms "Low Power S0 Idle Capability" /hb

I also changed this settings because I wanted to :)

.\SCEWIN_64.exe /i /lang 'en-US' /ms "LED MCU" /qv 0x0 /hb
.\SCEWIN_64.exe /i /lang 'en-US' /ms "Native ASPM" /qv 0x0 /cpwd YOUR-BIOS-ADMIN-PASSWORD /hb
.\SCEWIN_64.exe /i /lang 'en-US' /ms "Discrete Bluetooth Interface" /qv 0x0 /cpwd YOUR-BIOS-ADMIN-PASSWORD /hb
.\SCEWIN_64.exe /i /lang 'en-US' /ms "UnderVolt Protection" /qv 0x0 /hb
.\SCEWIN_64.exe /i /lang 'en-US' /ms "Password protection of Runtime Variables" /qv 0x0 /cpwd YOUR-BIOS-ADMIN-PASSWORD /hb

Another approach is to edit setup_script_file.txt manually by changing the asterisk location. And then:

.\SCEWIN_64.exe /i /s '.\setup_script_file_S0_enable.txt' /ds /r

Finally you have to reboot your machine.

In Windows I have C8 pkg C-State (Throttlestop utility) and 4.5 watts from the wall at idle (display went to sleep) in Proxmox (sorry I don't use Unraid but this forum is a godsend) as you see I have C10 (couldn't believe my eyes at first) and 5.5-6 watts from the wall with disks spinned down (added 2 2,5" HDDs: 250GB Samsung HM250HI and 4TB Seagate ST4000LM016 instead of Plextor SSD)

This guide was heavily inspired by another guide

https://winraid.level1techs.com/t/guide-enabling-hidden-bios-settings-on-gigabyte-z690-mainboards/94039

This is very interesting and thank you very much for the time to make the post. It is something I will definitely try when I get some free time, though, I wonder why this would be hidden in the first place. Do you have any idea if this affects the stability of the board? It's the only thing I can think of for why they have it hidden. — bleptoga, Jan 19 '24 at 23:41
Looking at this again. Why do I have to enable the administrator password in the BIOS? — bleptoga, Jan 20 '24 at 07:00
You are welcome. AMISCE utility cannot change protected runtime variables without enabling the administrator password, at least on modern Asrock motherboards, that's why. — voido, Jan 21 '24 at 15:05
No I don't have any stability issues. Asrock (and other manufacturers) hide BIOS settings for better compatibility in most cases. I think mgutt's answer is pretty good:
"That's typically ASRock On some Asrock boards you can only choose C6. It seems ASRock made an internal decision to limit C-States on all their boards to reduce driver issues / support requests maybe the customers than ask ASRock for support although it's because of a bad driver of a PCIe card of a different manufacturer" https://forums.unraid.net/topic/98070-reduce-power-consumption-with-powertop/page/19/#comment-1303201 — voido, Jan 21 '24 at 15:22

C-states not fuctioning on Ubuntu Server 22.04. Please help

2 Answers2