31

Previously I was checking the memory usage on my GPU with the following command: 

nvidia-settings -q all | grep Memory

I am processing some scientific data on my GPU with numpy and theano. I was doing this with the gnome desktop running, and there was already 380 Mb of memory used on the device. So I stopped lightdm to free up a little more GPU memory; but, now the nvidia-settings tool no longer works.

It simply errors that there is no display

ERROR: Cannot open display ':0.0'.

Is there some other way of getting GPU memory without using this tool? Is there some way to use make it function in headless mode?

Kevin Bowen
  • 20,125
  • 57
  • 82
  • 84
Nathan
  • 873
  • 1
    Mitch (credit to) gives the answer here: http://askubuntu.com/questions/387594/how-to-measure-gpu-usage nvidia-smi – xxjjnn Dec 07 '13 at 19:51
  • 1
    possible duplicate of http://askubuntu.com/questions/387594/how-to-measure-gpu-usage? – Kevin Bowen May 19 '14 at 07:17

4 Answers4

43

For Nvidia GPUs:

nvidia-smi

For Intel GPUs:

intel_gpu_tools

For AMD GPUs:

aticonfig --odgc --odgt

For real time watching -- example:

watch nvidia-smi
Adel - MageBinary
  • 1,014
8

I think the better way is to install a utility nvtop

E.g. In the following picture almost 100% memory is used on all 3 GPUs and one gpu processor is used 100%. The other two GPU's are used 0%. command is

nvtop

enter image description here

drmaa
  • 363
4

If you want to check memory usage for every 1 second, then the watch command does the job.

watch -n 1 nvidia-smi
Rohan Shetty
  • 141
  • 2
0

Watch feature is actually built-in to nvidia-smi itself

Run

nvidia-smi -l 1

to monitor memory and other metrics at a 1s interval.

Michal Stolarczyk
  • 1