134

I am looking for a utility that will benchmark CPU performance under single and multi threaded instances. At present I have an old rig with a dual core CPU (E7500) at 3.6 Ghz and I am looking at replacing it with a quad core CPU (Q9400) at 3.2 Ghz. I want to see if I will notice a performance improvement with the extra 2 cores (albeit with a drop in core speed). I will clock the CPU's with the same FSB (400Mhz) and the cache size is the same per CPU (1.5MB) and for what its worth I have 4GB ram (with potential to upgrade to 6GB)

My son mainly uses the PC for playing TF2 (which I am trying to still get working under Linux) and I also use it for some video encoding (MP4 to DVD)

I am thinking that I could be better off with the quad core but any feedback would be appreciated.

fatso
  • 1,341

5 Answers5

168

Actually there is a a tool named sysbench.

You can install it with:

sudo apt-get install sysbench

To do CPU benchmarking you can use:

sysbench cpu run

This will run a single-threaded CPU benchmark. To use all cores, use:

sysbench --threads="$(nproc)" cpu run
Raja G
  • 105,627
  • 107
  • 263
  • 332
  • 11
    --cpu-max-prime=20000 is optional, the default being 10000. I suggest to keep the default and fiddle with --max-requests instead (which is the number of operations performed) – MestreLion Mar 22 '16 at 17:29
  • 25
    Also note that by default this is a single-threaded test. For testing N cores you can use --num-threads=N, and compare results using the per-request statistics output. – MestreLion Mar 22 '16 at 17:31
  • 4
    Is there a place to compare the output to get relative performance? – jjxtra Sep 13 '19 at 22:33
  • 1
    @jjxtra that's exactly the problem, all these benchmarks are close to completely useless, I could easily write my own little stress test without any problem but there is no reference (or you don't easily find it, which makes any benchmark trivial). – meow Nov 19 '19 at 12:50
  • 1
    I've just resorted to https://www.cpubenchmark.net/ to get ballpark figures :) – jjxtra Nov 19 '19 at 16:03
  • 3
    With newer version, the --test= option can be omitted, just use sysbench cpu.... Otherwise sysbench 1.0.18 prints: WARNING: the --test option is deprecated. You can pass a script name or path on the command line without any options. – ckujau Feb 16 '20 at 22:30
  • 3
    @MestreLion WARNING: --num-threads is deprecated, use --threads instead – Madacol Sep 03 '20 at 21:33
  • @Madacol: Yes, it seems sysbench changed its CLI interface since 2016 when I made the comment. --threads is preferred now, thanks for pointing it out – MestreLion Sep 04 '20 at 19:17
  • Keep in mind that sysbench isn't really comparable between systems: https://github.com/akopytov/sysbench/issues/469 – kiler129 Mar 18 '23 at 23:16
57

I was recently looking for a tool available in multiple distros (and in Termux), and while the packages mentioned in other answers are a common good choice, I read in this Linuxhint article that 7-zip has a built-in benchmarking tool! And 7zip can be found in nearly every distro's repository.

To run a single-thread benchmark:

7z b -mmt1

To run a multi-thread benchmark:

7z b

The test does some in-memory compression and decompression, so it exercises CPU, RAM and caches and i/o between them.

Results from my Pixel 2 phone:

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,8 CPUs LE)

LE CPU Freq: 1509 2234 2434 2447 2433 2406 2430 2425 2400

RAM size: 3657 MB, # CPU hardware threads: 8 RAM usage: 435 MB, # Benchmark threads: 1

                   Compressing  |                  Decompressing

Dict Speed Usage R/U Rating | Speed Usage R/U Rating KiB/s % MIPS MIPS | KiB/s % MIPS MIPS

22: 1666 99 1631 1621 | 30427 100 2608 2598 23: 1602 99 1644 1633 | 29815 100 2589 2581 24: 1517 99 1644 1632 | 29441 100 2595 2585

25: 1397 99 1607 1596 28748 100 2567 2559
Avr: 99 1632 1620 100 2590 2581

Tot: 99 2111 2100

The columns mean:

  • Dict: size of dataset, i.e. 22 means 2^22 bytes, i.e. 5MB.
  • Usage: column shows the percentage of time the processor is working, normalized for a one-thread load. (so a 22 thread system has a max of 2200%)
  • R/U: Rating normalized for 100% of CPU usage, i.e. the performance of one average CPU thread.
  • Tot actually shows the average of the compression and decompression ratings.

You can read more about this benchmark here. (Does anyone have a more authoritative or informative link?)

Zanna
  • 72,471
45

Alternatively, one can use stress-ng. It has a CPU stress test as one of the many stress tests built into the tool. The cpu stress test contains many different CPU stress methods covering integer, floating point, bit operations, mixed compute, prime computation, and a wide range of computations.

Install using:

sudo apt-get install stress-ng

To see the cpu related stress methods use:

stress-ng --cpu-method which

To benchmark, for example, matrix product for 60 seconds on 4 CPU threads, use:

stress-ng --cpu 4 --cpu-method matrixprod  --metrics-brief --perf -t 60
stress-ng: info:  [15876] dispatching hogs: 4 cpu
stress-ng: info:  [15876] successful run completed in 60.00s (1 min, 0.00 secs)
stress-ng: info:  [15876] stressor      bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [15876]                          (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [15876] cpu              71657     60.00    239.60      0.00      1194.25       299.07
stress-ng: info:  [15876] cpu:
stress-ng: info:  [15876]            885,244,279,148 CPU Cycles                    14.75 B/sec
stress-ng: info:  [15876]          1,289,303,858,968 Instructions                  21.49 B/sec (1.456 instr. per cycle)
stress-ng: info:  [15876]            201,499,961,692 Cache References               3.36 B/sec
stress-ng: info:  [15876]                    790,424 Cache Misses                  13.17 K/sec ( 0.00%)
stress-ng: info:  [15876]            157,689,508,544 Branch Instructions            2.63 B/sec
stress-ng: info:  [15876]              1,232,539,732 Branch Misses                 20.54 M/sec ( 0.78%)
stress-ng: info:  [15876]              5,755,605,036 Bus Cycles                    95.92 M/sec
stress-ng: info:  [15876]            817,296,440,876 Total Cycles                  13.62 B/sec
stress-ng: info:  [15876]                      8,532 Page Faults Minor            142.19 sec  
stress-ng: info:  [15876]                          0 Page Faults Major              0.00 sec  
stress-ng: info:  [15876]                        220 Context Switches               3.67 sec  
stress-ng: info:  [15876]                          0 CPU Migrations                 0.00 sec  
stress-ng: info:  [15876]                          0 Alignment Faults               0.00 sec  
yprez
  • 943
  • 4
    " Unable to locate package stress-ng!" – Ehsan M. Kermani Feb 05 '16 at 00:09
  • 1
    If you have an older release, stress-ng won't be available from the archive. However, they are packaged in ppa:colin-king/white – Colin Ian King Mar 03 '16 at 16:56
  • @EhsanM.Kermani : it's available in the repositories only from Ubuntu 15.14 onwards (and to 14.04 using the backports repository). See http://packages.ubuntu.com/search?searchon=names&suite=all&section=all&keywords=stress-ng – MestreLion Mar 22 '16 at 17:36
  • "--perf: Unknown option (364)". stress-ng version 0.07.11 – rustyx Feb 10 '17 at 21:23
  • I have newer versions that are built for older releases in my ppa: ppa:colin-king/white – Colin Ian King Feb 10 '17 at 23:15
  • http://kernel.ubuntu.com/~cking/stress-ng/ – Colin Ian King Sep 16 '17 at 10:48
  • 2
    From the stress-ng manpage: "stress-ng can also measure test throughput rates; this can be useful to observe performance changes across different operating system releases or types of hardware. However, it has never been intended to be used as a precise benchmark test suite, so do NOT use it in this manner." – Michael F Feb 23 '18 at 10:17
  • 6
    As the author of stress-ng, I'd better elaborate on this. stress-ng is good enough to get some comparative benchmarks results out of it, but it is not been thoroughly calibrated to say how much deviation there is on the each specific stressor. I therefore suggest running a stress-ng stressor several times and seeing how much variation there is on a specific stress test, and if it does not vary much, then it can be considered reliable enough for a benchmark for that specific use case. It all depends on now noisy/busy a system is, how well I/O performs, if it swaps, etc. – Colin Ian King Feb 23 '18 at 11:49
  • 5
    Comparing stress-ng and sysbench, one tells me that system A is faster (125%), the other tells me that system B is faster (140%), which one is correct? – W.M. Aug 08 '19 at 14:40
  • Wow. Running sudo stress-ng makes my dell server reboot. No bueno. – Leopd Apr 02 '21 at 21:33
  • Also rebooted my ubuntu hetzner server! So use with caution and not on production hardware – yvess Mar 27 '23 at 05:25
17

Phoronix Test Suite

This is an open source test suite licensed under the GPLv3:

Phoronix automates and standardizes the benchmarking of several real world use cases like compression, encryption and databases. Most tests benchmark open source software projects, but some also benchmark closed source software. They also host test results at: https://openbenchmarking.org which anyone can upload to, so you can compare test results with other different systems.

Since 18.04 there isn't a package on the official repos for it, so tested on Ubuntu 23.10 you just have install the dependencies:

sudo apt-get install php-cli php-xml

and then either:

  • download it from https://www.phoronix-test-suite.com/?k=downloads

    wget https://phoronix-test-suite.com/releases/phoronix-test-suite-10.8.4.tar.gz
    tar xfz phoronix-test-suite-10.8.4.tar.gz
    
  • git clone it:

    git clone https://github.com/phoronix-test-suite/phoronix-test-suite
    

and then in either case optionally add it to the PATH with:

export PATH="$PATH:$(pwd)/phoronix-test-suite"

PHP is needed because it is actually the main scripting language of the suite. These were different times altogether!

Until 18.04 there was a package for it, tested on Ubuntu 16.10:

sudo apt-get install phoronix-test-suite

Tested on version 10.8.4, we can list all tests with:

phoronix-test-suite list-all-tests

The first few lines are:

pts/3dmark                  3DMark Wild Life Extreme        Graphics  
pts/ai-benchmark            AI Benchmark Alpha              System    
pts/aircrack-ng             Aircrack-ng                     Processor

The last column categorizes the main thing being benchmarked, some possible values are:

  • Processor: CPU
  • Graphics: GPU
  • Memory: RAM
  • Network
  • Disk
  • System: likely mean a mixture of multiple above categories

Let's try a self explanatory processor bound one which basically benchmarks GCC:

phoronix-test-suite install pts/build-linux-kernel
phoronix-test-suite run pts/build-linux-kernel

First it asks some annoying configuration questions, first which kernel configuration we want to build, I select 1 for defconfig:

Timed Linux Kernel Compilation 6.1:
    pts/build-linux-kernel-1.15.0
    Processor Test Configuration
        1: defconfig    [Default Kernel Build]
        2: allmodconfig [This option is *much* more time consuming...]
        3: Test All Options
        ** Multiple items can be selected, delimit by a comma. **
        Build: 1

then:

Would you like to save these test results (Y/n):
Enter a name for the result file:
Enter a unique name to describe this test run / configuration:
New Description:

and then it proceeds to build the Linux kernel a bunch of times.

How to automate answering those questions was asked at: https://stackoverflow.com/questions/28581447/how-to-skip-phoronix-test-suite-inital-questions It is quite annoying really.

Running the test so many times is a bit excessive for a casual run, so let's limit it to a single run with FORCE_TIMES_TO_RUN=1:

FORCE_TIMES_TO_RUN=1 phoronix-test-suite run pts/build-linux-kernel

After dumping my system's configuration, at the end we see:

Timed Linux Kernel Compilation 6.1:
    pts/build-linux-kernel-1.15.0 [Build: defconfig]
    Test 1 of 1
    Estimated Trial Run Count:    1                     
    Estimated Time To Completion: 9 Minutes [04:37 UTC] 
        Running Pre-Test Script @ 04:29:14
        Started Run 1 @ 04:29:27
        Running Post-Test Script @ 04:32:23

Build: defconfig:
    173.449

Average: 173.449 Seconds
Samples: 1

so the test result was how long it took to build the kernel, which was 173.449 seconds on my system.

We can see how that compares with other systems at: https://openbenchmarking.org/test/pts/build-linux-kernel I'm running an AMD Ryzen 7 PRO 7840U on a Lenovo ThinkPad P14s, and for that CPU the public results were 129 +/- 9. So sadface, there seems to be something wrong with my system, as I'm considerably slower than those tests, maybe a performance mode issue? Setting to High Performance But at least this illustrates the awesome value of having public results available!

By tweaking "Analyze Test Configuration" we can also see:

Let's try another benchmark, how about stress-ng which was mentioned in another answer with configuration 1: CPU Stress

printf 1 | FORCE_TIMES_TO_RUN=1 phoronix-test-suite batch-run pts/stress-ng

and this time we see:

Stress-NG 0.16.04:
    pts/stress-ng-1.11.0 [Test: CPU Stress]
    Test 1 of 1
    Estimated Trial Run Count:    1                    
    Estimated Time To Completion: 1 Minute [12:15 UTC] 
        Started Run 1 @ 12:14:56

Test: CPU Stress:
    22191.17

Average: 22191.17 Bogo Ops/s
Samples: 1

where the test results is 22191.17 Bogo Ops/s, which is how stress-ng reports its output, and means just how many operations it did in a given amount of time. So we see that this is a different type of test, which rather than benchmarking the time to completion of a task rather set a timer and ran as many times as possible.

While it runs, I try to observe the final commands it is running with:

ciro     2117375  0.0  0.0   2780  1920 pts/11   S+   12:20   0:00 /bin/sh /home/ciro/down/phoronix-test-suite/phoronix-test-suite batch-run pts/stress-ng
ciro     2117388  0.0  0.0   2780  1792 pts/11   S+   12:20   0:00 sh -c  php /home/ciro/down/phoronix-test-suite/pts-core/phoronix-test-suite.php batch-run pts/stress-ng
ciro     2117673  0.0  0.0   2784  1792 pts/11   S+   12:20   0:00 /bin/sh ./stress-ng -t 30 --metrics-brief --cpu -1 --cpu-method all --no-rand-seed
ciro     2117674  0.0  0.0  39132  6528 pts/11   SL+  12:20   0:00 ./stress-ng -t 30 --metrics-brief --cpu -1 --cpu-method all --no-rand-seed
...

A bit more digging shows that the shell script is something from Phoronix: ~/.phoronix-test-suite/installed-tests/pts/stress-ng-1.11.0 which simply does:

#!/bin/sh
cd stress-ng-0.16.04
./stress-ng $@ > $LOG_FILE 2>&1
echo $? > ~/test-exit-status

where stress-ng is the final executable.

Test suites

Besides individual tests, Phoronix Test Suite also has the concept of test suites, which are just groups of tests. TODO how to list all test suites? I tried:

phoronix-test-suite list-available-suites

but it does not contain all suites. Notably, it is missing pts/cpu, which contains several CPU benchmarks. Note that this will be slow and download a bunch of test executables

Running a suite is similar to running a single test:

phoronix-test-suite install pts/cpu
phoronix-test-suite run pts/cpu

We can see which tests are contained in a suite with:

phoronix-test-suite info pts/cpu

which contains a list such as:

Contained Tests:
                    Rodinia      Test: OpenMP CFD Solver
                    Rodinia      Test: OpenMP LavaMD
                    NAMD
                    Stockfish
                    x264         Video Input: Bosphorus 1080p
                    x264         Video Input: Bosphorus 4K

so we quickly understand that is it picking certain configs of certain tests. TODO: is there a way to list the actual test identifiers, e.g. pts/rodinia and so on? One way is to look at: https://openbenchmarking.org/suite/pts/cpu

Another one of interest for CPU benchmarking which I found by randomly doing find is:

phoronix-test-suite info pts/cpu-massive

which contains a whopping 373 tests.

Upload a test result

First we create an account: https://openbenchmarking.org/register

TODO haven't managed to login yet. Confirmation email didn't arrive? I'll try again later some day.

File structure

Finally, let's have a look at the file that Phoronix keeps around:

ncdu ~/.phoronix-test-suite/

shows:

  167.1 MiB [###########################] /installed-tests
   44.1 MiB [#######                    ] /test-profiles
    2.9 MiB [                           ] /test-results
    1.1 MiB [                           ] /openbenchmarking.org
  636.0 KiB [                           ] /test-suites
   12.0 KiB [                           ]  core.pt2so
e   4.0 KiB [                           ] /modules-data
e   4.0 KiB [                           ] /modules
e   4.0 KiB [                           ] /download-cache
    4.0 KiB [                           ]  user-config.xml
    4.0 KiB [                           ]  graph-config.json
    4.0 KiB [                           ]  phoronix-test-suite-dependencies.log
    0.0   B [                           ]  phoronix-test-suite-benchmark.log

installed-tests contains source code and prebuilts, e.g. the Linuxk kernel build only has the source code txz:

--- /home/ciro/.phoronix-test-suite/installed-tests/pts/build-linux-kernel-1.15.0 -------------
                                          /..
  128.5 MiB [###########################]  linux-6.1.tar.xz

The build dir was either deleted, or lies outside of of ~/.phoronix-test-suite.

stress-ng has the packed source, the unpacked source, and the built executable:

--- ~/.phoronix-test-suite/installed-tests/pts/stress-ng-1.11.0 -
                                          /..
   34.8 MiB [###########################] /stress-ng-0.16.04
    3.7 MiB [##                         ]  stress-ng-0.16.04.tar.gz

Another interesting directory is ~/.phoronix-test-suite/test-results which stores run results under:

  • Enter a name for the result file: if you entered that interactively
  • a datetime such as 2023-12-23-0414 when running with batch-run
  • 3
    @downvoters please explain :-) – Ciro Santilli OurBigBook.com Oct 30 '17 at 17:41
  • 3
    Thanks. I drowned in test options, using build-suite on my first round with phoronix. – andersoyvind Jan 16 '18 at 08:53
  • 1
    it's not in debian 9, but you can still download the .deb from https://www.phoronix-test-suite.com/?k=downloads and run sudo apt install ./phoronix-test-suite(...).deb – hanshenrik Jul 18 '18 at 11:02
  • 4
    One thing that should be mentioned when describing the PTS is the size of its benchmarks. Running the mentioned pts/cpu benchmark downloads about 3GB of data and uses about 7GB of disk space (in the user's home directory). – stefanct Jul 23 '18 at 16:31
  • 6
    Agreed, if you are looking for a simple benchmark then you don't want phoronix-test-suite, it's massive, and keeps asking to download more and more dependencies to run tests. (I'm sure it's very thorough when you actually want this though) – Jamie Pate Sep 08 '18 at 01:11
  • This hasn't been available in the Ubuntu repository since 18.04 (see https://repology.org/project/phoronix-test-suite/versions for a complete rundown of availability). As @hanshenrik notes, however, it can be easily downloaded and installed from their website. – thisisrandy Dec 23 '23 at 01:07
11

If you don't have root permissions (e.g. managed web servers)

Measure single core:

-bash-4.2$ time cat </dev/urandom | head -c 1G | gzip >/dev/null

real 0m43.011s user 0m42.608s sys 0m8.819s

Measure all cores:

-bash-4.2$ time cat </dev/urandom | head -c 1G | pigz >/dev/null

real 0m13.870s user 0m45.988s sys 0m6.803s

If you already have pv but don't feel like installing special benchmarking software it's even more straightforward

a@a-virtual-machine:~$ pv </dev/urandom | gzip >/dev/null 
 149MiB 0:00:04 [37.5MiB/s]
a@a-virtual-machine:~$ pv </dev/urandom | pigz >/dev/null 
 348MiB 0:00:02 [ 173MiB/s]
Anton Duzenko
  • 283
  • 3
  • 9