Skip to content

Can't install plyvel with standalone toolchain (0.10.2) #775

Closed

Description

@dhalperi

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

No, I'm not sure this ever worked in a previous version. This is the first version of standalone I've tried.

The library does work with a custom interpreter.

Description

plyvel requires leveldb installed on the system. With brew install leveldb, I get the file at:

/usr/local/Cellar/leveldb@1.22/1.22/include/leveldb/db.h

When resolving plyvel, the below stack trace is thrown. It seems that plyvel can't find the C headers.

🔬 Minimal Reproduction

  1. Using macOS Monterey (12.5) on a 2017 Macbook Pro (x64)
  2. Install leveldb from Homebrew
  3. Use the following simplish repo:
  4. bazel build @pip_plyvel//:pkg - will throw the given error
  5. Comment out python_interpreter_target and rerun; will succeed.

Repo:
WORKSPACE:

workspace(name = "demo")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

##########################################################
## Python setup

http_archive(
    name = "rules_python",
    sha256 = "a3a6e99f497be089f81ec082882e40246bfd435f52f4e82f37e89449b04573f6",
    strip_prefix = "rules_python-0.10.2",
    url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.10.2.tar.gz",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python3_9",
    # Available versions are listed in @rules_python//python:versions.bzl.
    python_version = "3.9",
)

load("@python3_9//:defs.bzl", "interpreter")
load("@rules_python//python:pip.bzl", "pip_parse")

pip_parse(
    name = "pip",
    python_interpreter_target = interpreter,
    requirements_lock = "//:requirements.txt",
)

load("@pip//:requirements.bzl", "install_deps")

install_deps()

requirements.txt:

#
# This file is autogenerated by pip-compile with python 3.7
# To update, run:
#
#    pip-compile --allow-unsafe --generate-hashes --output-file=requirements.txt requirements.in
#
plyvel==1.4.0 \
    --hash=sha256:0741be98096db98929ba507c5d99be6b9e009e38fb1361e755e52e69b2a6a496 \
    --hash=sha256:4ea98bea04ebf0f44747bacdfafefc8827787106fbb787f0aedc46482b2dfd53 \
    --hash=sha256:61d6dc2edff07412be0bcfb20cd3a6cb0b2ac6e136e0d53ce91a2d9b10e89ebc \
    --hash=sha256:b4a4e25b75852a379744f14e743f17ae78513d0ba4f354be54199735487c71d7 \
    --hash=sha256:d50eb65421e40d788868943bbaa0e1b62e806a07323a21ce9a4b7bf6bea022ec \
    --hash=sha256:daef3f2076db41fa0a0fcbb3eabf1a36445739e101367356b7ad3c455b77b44d
    # via -r requirements.in

🔥 Exception or Error


ERROR: An error occurred during the fetch of repository 'pip_plyvel':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/pip_repository.bzl", line 445, column 13, in _whl_library_impl
		fail("whl_library %s failed: %s (%s)" % (rctx.attr.name, result.stdout, result.stderr))
Error in fail: whl_library pip_plyvel failed: Collecting plyvel==1.4.0
  Using cached plyvel-1.4.0.tar.gz (151 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: plyvel
  Building wheel for plyvel (setup.py): started
  Building wheel for plyvel (setup.py): finished with status 'error'
  Running setup.py clean for plyvel
Failed to build plyvel
 (  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.9-x86_64-3.9
      creating build/lib.macosx-10.9-x86_64-3.9/plyvel
      copying plyvel/_version.py -> build/lib.macosx-10.9-x86_64-3.9/plyvel
      copying plyvel/__init__.py -> build/lib.macosx-10.9-x86_64-3.9/plyvel
      running build_ext
      building 'plyvel._plyvel' extension
      creating build/temp.macosx-10.9-x86_64-3.9
      creating build/temp.macosx-10.9-x86_64-3.9/plyvel
      Compiling with an SDK that doesn't seem to exist: /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
      Please check your Xcode installation
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -mmacosx-version-min=10.9 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -fPIC -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include/ncursesw -I/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmphaz3a_ee/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/include/python3.9 -c plyvel/_plyvel.cpp -o build/temp.macosx-10.9-x86_64-3.9/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++
      plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found
      #include "leveldb/db.h"
               ^~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for plyvel
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 81, in 
    main()
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 57, in main
    subprocess.run(pip_args, check=True, env=env)
  File "/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_dan/b3405f8b4e31d4c991d0d779169ce930/external/python3_9_x86_64-apple-darwin/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/var/folders/zy/32xth96x23vb7bddgpscl7g40000gn/T/tmpf70uqj3w']' returned non-zero exit status 1.
)

🌍 Your Environment

Operating System:

macOS Monterey, x86-64 processor

Output of bazel version:

Bazelisk version: development
Build label: 5.2.0
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 7 16:11:03 2022 (1654618263)
Build timestamp: 1654618263
Build timestamp as int: 1654618263

Rules_python version:

0.10.2

Anything else relevant?

Is there a way to supply C headers in-project?

Activity

  1. dhalperi commented on Aug 2, 2022

    @dhalperi
    ContributorAuthor

    It seems to me that /usr/local/include is not on the compile path. CC: @hrfuller who may know the most about this?

  2. mvgijssel commented on Aug 29, 2022

    @mvgijssel

    Running into a similar problem trying to install gevent using pip_parse on macOS 12.5.1

          clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -mmacosx-version-min=11.0 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fPIC -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/ncursesw -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DLIBEV_EMBED=1 -DEV_CLEANUP_ENABLE=0 -DEV_EMBED_ENABLE=0 -DEV_PERIODIC_ENABLE=0 -DEV_USE_REALTIME=1 -DEV_USE_MONOTONIC=1 -DEV_USE_FLOOR=1 -I/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/include/python3.9 -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent/libev -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent -I/install/include/python3.9 -c build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.c -o build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.o
          clang -bundle -undefined dynamic_lookup -arch arm64 -mmacosx-version-min=11.0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -L/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/lib -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libev._corecffi.o -L/install/lib -o build/lib.macosx-11.0-arm64-cpython-39/gevent/libev/_corecffi.abi3.so
          ld: warning: directory not found for option '-L/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/lib'
          ld: warning: directory not found for option '-L/install/lib'
          building 'gevent.libuv._corecffi' extension
          creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv
          creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv/src
          creating build/temp.macosx-11.0-arm64-cpython-39/deps/libuv/src/unix
          clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -mmacosx-version-min=11.0 -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-undef-prefix -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk -fPIC -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/ncursesw -I/var/folders/c8/z52g4gc5679gygt8z0dx0jqm0000gn/T/tmps5wb7dre/tools/deps/include/uuid -Werror=unguarded-availability-new -g0 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -DLIBUV_EMBED=1 -D_LARGEFILE_SOURCE=1 -D_FILE_OFFSET_BITS=64 -D_DARWIN_USE_64_BIT_INODE=1 -D_DARWIN_UNLIMITED_SELECT=1 -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libuv/include -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/deps/libuv/src -I/private/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/pip-wheel-qrxquzu4/gevent_3c0ddb0a3bc74ed18e829b5f418459e4/src/gevent -I/install/include/python3.9 -c build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.c -o build/temp.macosx-11.0-arm64-cpython-39/build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.o -Wno-unreachable-code -Wno-deprecated-declarations -Wno-incompatible-sysroot -Wno-tautological-compare -Wno-implicit-function-declaration -Wno-unused-value -Wno-macro-redefined
          build/temp.macosx-11.0-arm64-cpython-39/gevent.libuv._corecffi.c:50:14: fatal error: 'pyconfig.h' file not found
          #    include <pyconfig.h>
                       ^~~~~~~~~~~~
          1 error generated.
          error: command '/usr/bin/clang' failed with exit code 1
          [end of output]
    
      note: This error originates from a subprocess, and is likely not a problem with pip.
      ERROR: Failed building wheel for gevent
    ERROR: Failed to build one or more wheels
    Traceback (most recent call last):
      File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 197, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 83, in <module>
        main()
      File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/rules_python/python/pip_install/extract_wheels/extract_single_wheel.py", line 59, in main
        subprocess.run(pip_args, check=True, env=env)
      File "/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/lib/python3.9/subprocess.py", line 528, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/private/var/tmp/_bazel_maarten/8c4bfcd606c4d7b95cf418885251b08c/external/python3_9_aarch64-apple-darwin/bin/python3', '-m', 'pip', '--isolated', 'wheel', '--no-deps', '-r', '/var/folders/86/9v04xnt92zgc3djqqdy4c7_00000gn/T/tmpt5hi3lcn']' returned non-zero exit status 1.
    )
  3. mvgijssel commented on Aug 29, 2022

    @mvgijssel

    Just came across #809 which might help for me, validating if this is the case and will post back here.

  4. mvgijssel commented on Aug 29, 2022

    @mvgijssel

    Adding the most recent rules_python solved the issue for me 🎉

    http_archive(
        name = "rules_python",
        sha256 = "b593d13bb43c94ce94b483c2858e53a9b811f6f10e1e0eedc61073bd90e58d9c",
        strip_prefix = "rules_python-0.12.0",
        url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.12.0.tar.gz",
    )
  5. github-actions commented on Feb 25, 2023

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
    Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

  6. dhalperi commented on Feb 26, 2023

    @dhalperi
    ContributorAuthor

    activity

  7. chrislovecnm commented on Aug 7, 2023

    @chrislovecnm
    Contributor

    @dhalperi can you test with the latest version of rules_python?

  8. github-actions commented on Feb 3, 2024

    @github-actions
    Contributor

    This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
    Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

  9. github-actions commented on Mar 4, 2024

    @github-actions
    Contributor

    This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Can Close?Will close in 30 days if there is no new activity

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions