Error when pip installing files from requirements.txt

I cloned a repo from github for research purposes, and I'm trying to install some of the requirements from the requirements.txt file. The installation ran just fine on Windows, but I've been having errors when installing on Ubuntu 18.04

I'm installing it in a conda created virtual env. And I'm using Python 3.8. Below is the error traceback.

Collecting flask
  Using cached https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl
Collecting mysqlclient
  Using cached https://files.pythonhosted.org/packages/f8/9b/5db9a03e2088a87c26e3e4d4c7f7e8f4c2dbae610f9521cdfac15755a795/mysqlclient-1.4.5.tar.gz
Collecting sklearn
  Downloading https://files.pythonhosted.org/packages/1e/7a/dbb3be0ce9bd5c8b7e3d87328e79063f8b263b2b1bfa4774cb1147bfcd3f/sklearn-0.0.tar.gz
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/3a/8f/f9ee25c0ae608f86180c26a1e35fe7ea9d71b473ea7f54db20759ba2745e/numpy-1.17.3-cp38-cp38-manylinux1_x86_64.whl (20.5MB)
     |████████████████████████████████| 20.5MB 153kB/s 
Collecting scipy
  Downloading https://files.pythonhosted.org/packages/ee/5b/5afcd1c46f97b3c2ac3489dbc95d6ca28eacf8e3634e51f495da68d97f0f/scipy-1.3.1.tar.gz (23.6MB)
     |████████████████████████████████| 23.6MB 82kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... error
    ERROR: Command errored out with exit status 1:
     command: /home/agozie/miniconda3/envs/facial/bin/python /home/agozie/miniconda3/envs/facial/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpolyoz9vt
         cwd: /tmp/pip-install-sw1gf9a3/scipy
    Complete output (153 lines):
    lapack_opt_info:
    lapack_mkl_info:
    customize UnixCCompiler
      libraries mkl_rt not found in ['/home/agozie/miniconda3/envs/facial/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    openblas_lapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas not found in ['/home/agozie/miniconda3/envs/facial/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    openblas_clapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas,lapack not found in ['/home/agozie/miniconda3/envs/facial/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    atlas_3_10_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE

    atlas_3_10_info:
    customize UnixCCompiler
      libraries satlas,satlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE

    atlas_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE

    atlas_info:
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /home/agozie/miniconda3/envs/facial/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE

    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/home/agozie/miniconda3/envs/facial/lib', '/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE

    lapack_src_info:
      NOT AVAILABLE

      NOT AVAILABLE

    setup.py:386: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-l1gkp34t'), proceeding with generating Cython sources and expanding templates
      warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
    Running from scipy source directory.
    /tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:624: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      self.calc_info()
    /tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:624: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      self.calc_info()
    /tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:624: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      self.calc_info()
    Traceback (most recent call last):
      File "/home/agozie/miniconda3/envs/facial/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 257, in <module>
        main()
      File "/home/agozie/miniconda3/envs/facial/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 240, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/home/agozie/miniconda3/envs/facial/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 110, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 156, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 236, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 142, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 505, in <module>
        setup_package()
      File "setup.py", line 501, in setup_package
        setup(**metadata)
      File "/tmp/pip-build-env-ikhv85cu/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "setup.py", line 403, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/agozie/miniconda3/envs/facial/bin/python /home/agozie/miniconda3/envs/facial/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpolyoz9vt Check the logs for full command output.

And below is the requirements.txt file

mysqlclient
sklearn
numpy
scipy
pillow
dlib
face_recognition
flask_bootstrap
pytz
xlsxwriter
pandas
flask_mail
tensorflow
xlrd

I think it has something to do with the OS, but I can't be sure about that. If anyone can help, I'd be most grateful.

Edit: I tried installing scipy first using conda install scipy and I got the following output

Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: - 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abor\ 
Examining certifi:  16%|███▍                  | 5/32 [08:11<1:13:42, 163.79s/it]
Comparing s/ cs that have this dependency:   0%|          | 0/2 [00:00<?, ?it/s]
Finding shortest conflict path for certifi[version='>=2016.09']:   0%| | 0/4 [00Finding- hortest conflict path for ca-certificates:  25%|▎| 1/4 [08:43<26:09, 52Finding shortest conflict path for ca-certificates:  50%|▌| 2/4 [08:43<08:43, 26/ nding shortest conflict path for certifi:  50%|▌| 2/4 [09:05<08:43, 261.61s/itFinding shortest conflict path for certifi:  75%|▊| 3/4 [09:05<03:09, 189.68s/itFinding shortest confl| t path for certifi[version='>=2016.9.26']:  75%|▊| 3/4 [Finding shortest conflict path for certifi[version='>=2016.9.26']: 100%|█| 4/4 [Comparing specs that have this dependency:  50%|▌| 1/2 [25:37<25:37, 1537.78s/it]                       
Finding shortest conflict path for certifi[version='>=2016.09']:   0%| | 0/3 [00Finding shortest conf/ ct path for ca-certificates:  33%|▎| 1/3 [00:00<00:00, 32Finding shortest conflict path for certifi[version='>=2016.9.26']:  67%|▋| 2/3 [Examining python:  47%|███████████▎            | 15/32 [41:46<51:46, 182.75s// ]Comparing specs- hat have this dependency:   0%|          | 0/2 [00:00<?, ?it/s]
Finding shortes- conflict path for python[version='>=3.7,<3.8.0a0']:   0%| | 0/5Finding shortest conflict path for python=3.8:  20%|▏| 1/5 [00:00<00:00, 10.88itFinding shortest conflict path for python[version='>=3.6,<3.7.0a0']:  40%|▍| 2/5Finding shortest conflic| path for python[version='>=3.6,<3.7.0a0']:  60%|▌| 3/5Finding shortest conflict path for python[version='>=3.5,<3.6.0a0']:  60%|▌| 3/5Finding shortest conflic- path for python[version='>=2.7,<2.8.0a0']:  80%|▊| 4/5Finding shortest conflict path for python[version='>=2.7,<2.8.0a0']: 100%|█| 5/5Comparing specs that have this dependency:  50%|▌| 1/2 [05:30<05:30, 330.15s/it]Finding shortest conflict path for python[version='>=3.8,<3.9.0a0']:   0%| | 0/2failed                                                                                                                                                          UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - scipy -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

Your python: python=3.8

If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

The following specifications were found to be incompatible with each other:



Package certifi conflicts for:
scipy -> python[version='>=3.7,<3.8.0a0'] -> pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']
python=3.8 -> pip -> setuptools -> certifi[version='>=2016.09|>=2016.9.26']
Package setuptools conflicts for:
python=3.8 -> pip -> setuptools
scipy -> python[version='>=3.7,<3.8.0a0'] -> pip -> setuptools
Package pip conflicts for:
python=3.8 -> pip
scipy -> python[version='>=3.7,<3.8.0a0'] -> pip
Package ca-certificates conflicts for:
scipy -> python[version='>=3.7,<3.8.0a0'] -> ca-certificates
python=3.8 -> openssl[version='>=1.1.1d,<1.1.2a'] -> ca-certificates
Package wheel conflicts for:
scipy -> python[version='>=3.7,<3.8.0a0'] -> pip -> wheel
python=3.8 -> pip -> wheel

1 answer

  • answered 2019-11-08 14:03 Iguananaut

    It looks like you errored out installing SciPy, which it seems does not have a binary wheel on PyPI for your platform (which is surprising because there are many wheels for SciPy 1.3.1.

    However, since you're already using conda, you should start by installing as many dependencies through the appropriate conda packages using conda install. In particular complex packages like NumPy and SciPy have already been expertly packaged for conda, similarly for packages like tensorflow and pandas.

    Although not made obvious in the docs, conda-install can read a basic requirements.txt file. I tried passing the above requirements.txt like

    conda install --file requirements.txt
    

    but a few of the packages either were not in any of my default conda channels, or might have a different name (e.g. for "sklearn" the conda package is "scikit-learn"). Specifically:

    PackagesNotFoundError: The following packages are not available from current channels:
    
      - face_recognition
      - flask_bootstrap
      - flask_mail
      - sklearn
    

    Some of your dependencies might not have any conda package at all for them.

    I think the best thing to do is install as many dependencies through conda as possible, then use conda env export to create an environment.yml file.

    Then, when setting up a development environment, first install packages from the environment.yml, then run pip install -r requirements.txt. The latter should ignore any requirements already satisfied by conda, and just install additional packages that might not already be managed by conda, such as "flask-bootstrap".