8

I am trying to install a number of python 3 modules (e.g. regex, cytoolsz, spacy) that require compilation, but they all fail with an error identical to the below (bottom). I have tried to check for the presence of "limits.h" using the the grep below. I have reinstalled gcc, g++ build-essentials, python3-dev etc, but to no avail.

x86_64-linux-gnu-gcc -xc -E -v /dev/null

Using built-in specs.
COLLECT_GCC=x86_64-linux-gnu-gcc
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 8.2.0-7ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 8.2.0 (Ubuntu 8.2.0-7ubuntu1) 
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'
 /usr/lib/gcc/x86_64-linux-gnu/8/cc1 -E -quiet -v -imultiarch x86_64-linux-gnu /dev/null -mtune=generic -march=x86-64 -fstack-protector-strong -Wformat -Wformat-security
ignoring nonexistent directory "/usr/local/include/x86_64-linux-gnu"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/8/include-fixed"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-linux-gnu/8/../../../../x86_64-linux-gnu/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/8/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
# 1 "/dev/null"
# 1 "<built-in>"
# 1 "<command-line>"
# 31 "<command-line>"
# 1 "/usr/include/stdc-predef.h" 1 3 4
# 32 "<command-line>" 2
# 1 "/dev/null"
COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/8/:/usr/lib/gcc/x86_64-linux-gnu/8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/8/:/usr/lib/gcc/x86_64-linux-gnu/
LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/8/:/usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/8/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/8/../../../:/lib/:/usr/lib/
COLLECT_GCC_OPTIONS='-E' '-v' '-mtune=generic' '-march=x86-64'

Check:

dpkg -S limits.h | grep linux
linux-headers-4.18.0-15: /usr/src/linux-headers-4.18.0-15/include/linux/dynamic_queue_limits.h
linux-libc-dev:amd64: /usr/include/linux/limits.h
linux-headers-4.19.0-041900rc8: /usr/src/linux-headers-4.19.0-041900rc8/include/uapi/linux/limits.h
linux-headers-4.18.0-14: /usr/src/linux-headers-4.18.0-14/include/linux/drbd_limits.h
linux-headers-4.18.0-15: /usr/src/linux-headers-4.18.0-15/arch/arm/include/asm/limits.h
linux-headers-4.18.0-14: /usr/src/linux-headers-4.18.0-14/include/uapi/linux/limits.h
linux-headers-4.18.0-14: /usr/src/linux-headers-4.18.0-14/include/linux/dynamic_queue_limits.h
libgcc-8-dev:amd64: /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed/limits.h
linux-headers-4.18.0-14: /usr/src/linux-headers-4.18.0-14/arch/arm/include/asm/limits.h
linux-headers-4.18.0-15: /usr/src/linux-headers-4.18.0-15/include/linux/drbd_limits.h
linux-headers-4.19.0-041900rc8: /usr/src/linux-headers-4.19.0-041900rc8/include/linux/drbd_limits.h
linux-headers-4.18.0-15: /usr/src/linux-headers-4.18.0-15/include/uapi/linux/limits.h
linux-headers-4.19.0-041900rc8: /usr/src/linux-headers-4.19.0-041900rc8/include/linux/dynamic_queue_limits.h
linux-headers-4.19.0-041900rc8: /usr/src/linux-headers-4.19.0-041900rc8/arch/arm/include/asm/limits.h
libgcc-8-dev:amd64: /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed/syslimits.h

Error:

    sudo pip3 install regex                   
The directory '/home/mac/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/mac/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting regex
  Downloading https://files.pythonhosted.org/packages/9a/6f/8c1479c781bbc94394f9c4e33ad4139068bcc6a1b018c5a5525471262b8a/regex-2019.02.18.tar.gz (643kB)
    100% |████████████████████████████████| 645kB 813kB/s 
Installing collected packages: regex
  Running setup.py install for regex ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-n16bk3y6/regex/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-z1rqj4ab-record/install-record.txt --single-version-externally-managed --compile:
    /home/mac/.local/lib/python3.6/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2019.02.18' to '2019.2.18'
      normalized_version,
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    copying regex_3/regex.py -> build/lib.linux-x86_64-3.6
    copying regex_3/_regex_core.py -> build/lib.linux-x86_64-3.6
    copying regex_3/test_regex.py -> build/lib.linux-x86_64-3.6
    running build_ext
    building '_regex' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/regex_3
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.6/regex_3/_regex.o
    In file included from /usr/include/python3.6m/Python.h:11,
                     from regex_3/_regex.c:48:
    /usr/include/limits.h:124:26: error: no include path in which to search for limits.h
     # include_next <limits.h>
                              ^
    In file included from regex_3/_regex.c:48:
    /usr/include/python3.6m/Python.h:14:2: error: #error "Something's broken.  UCHAR_MAX should be defined in limits.h."
     #error "Something's broken.  UCHAR_MAX should be defined in limits.h."
      ^~~~~
    /usr/include/python3.6m/Python.h:18:2: error: #error "Python's source code assumes C's unsigned char is an 8-bit type."
     #error "Python's source code assumes C's unsigned char is an 8-bit type."
      ^~~~~
    In file included from /usr/include/python3.6m/Python.h:25,
                     from regex_3/_regex.c:48:
    /usr/include/stdio.h:33:10: fatal error: stddef.h: No such file or directory
     #include <stddef.h>
              ^~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-n16bk3y6/regex/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-z1rqj4ab-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-n16bk3y6/regex/
mac
  • 250
  • 2
  • 3
  • 10
  • 1
    What was wrong with sudo apt install python3-regex? – N0rbert Feb 19 '19 at 20:24
  • Good question. I am trying to install ludwig in a virtual environment and regex is one of the required packages. – mac Feb 19 '19 at 23:07
  • 1
    usually you not need sudo for pip3. As well please try to install libc6-dev and linux-libc-dev and tell me if this solved your problem. – Videonauth Feb 21 '19 at 11:08
  • Can you add the output of x86_64-linux-gnu-gcc -xc -E -v /dev/null to your question? I'm curious to see the default include paths on your system. Also does it work with sudo -H CPATH=/usr/include python3 -m pip install regex? (Assuming libc6-dev is installed as described in Videonauth's answer) – Byte Commander Feb 25 '19 at 17:35
  • Thanks for your feedback. I have added the requested output. Adding the CPATH made no difference. – mac Feb 25 '19 at 21:10
  • Not what I expected, everything there looks ok to me. No clue then why it doesn't work :( – Byte Commander Feb 25 '19 at 23:07
  • If you booted with Kernel 4.19 Release Candidate 8 try booting with a stable kernel instead. Just a shot in the dark... – WinEunuuchs2Unix Feb 28 '19 at 00:54
  • Thanks. I tried booting with 4.18.0 and the result is the same. – mac Mar 01 '19 at 16:18

1 Answers1

12

First and foremost to address your question completely. The file limits.h in /usr/include/linux/ is provided by the linux-libc-dev package. So to have the header files in this location you simply have to install it by:

sudo apt install linux-libc-dev

to have header files provided in /usr/include/ which are occasionally used as well you need the libc6-dev package which provides them. Install it by:

sudo apt install libc6-dev

Furthermore it is bad practice to use pip3 with sudo as it tends to create root owned folders in your home directory. To fix this, I assume you already did a couple of installs that way simply take back ownership of /home/$USER/.cache/pip by:

sudo chown -Rv <your-username>:<your-username> /home/<your-username>/.cache/pip
Videonauth
  • 33,845
  • Tanks for your input. Both linux-libc-dev and libc6-dev are both already installed and I have also tried reinstalling both. – mac Feb 21 '19 at 16:06
  • The its not sure why it should not compile. Cannot see what else could be wrong there except checking the PATH variable if it contains the proper pathing. – Videonauth Feb 21 '19 at 16:51
  • echo $PATH -> /home/mac/.opam/system/bin:/home/mac/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin – mac Feb 21 '19 at 17:58
  • Looks like on my system. Slowly out of clues why this happens, did you try an sudo apt update && sudo apt dist-upgrade to make sure all packages are in a proper state and updated, should normally not be needed but it is worth a try. – Videonauth Feb 21 '19 at 19:19
  • Yeah, makes no difference. – mac Feb 21 '19 at 19:53