Repository navigation
Closed
Labels
Can Close?Will close in 30 days if there is no new activityWill close in 30 days if there is no new activity
Description
🐞 bug report
Affected Rule
The issue is caused by the rule: pip_installIs this a regression?
Yes, the previous version in which this bug was not present was: Not a regressionDescription
pip_install fails for only one requirement `tree-sitter` from `requirements.txt`🔥 Exception or Error
Failed to build tree-sitter
( ERROR: Command errored out with exit status 1:
command: /home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/python_bin /home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/pypi__pip/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpxxrp34vx
cwd: /tmp/pip-wheel-hw8ksvfm/tree-sitter
Complete output (20 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/tree_sitter
copying tree_sitter/__init__.py -> build/lib.linux-x86_64-3.7/tree_sitter
running build_ext
building 'tree_sitter.binding' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/tree_sitter
creating build/temp.linux-x86_64-3.7/tree_sitter/core
creating build/temp.linux-x86_64-3.7/tree_sitter/core/lib
creating build/temp.linux-x86_64-3.7/tree_sitter/core/lib/src
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -g0 -fPIC -Itree_sitter/core/lib/include -Itree_sitter/core/lib/src -I/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/bazel_install/lib/python3.7/config-3.7m-x86_64-linux-gnu/Include -I/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter -c tree_sitter/binding.c -o build/temp.linux-x86_64-3.7/tree_sitter/binding.o -std=c99 -Wno-unused-variable
tree_sitter/binding.c:2:10: fatal error: Python.h: No such file or directory
#include "Python.h"
^~~~~~~~~~
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for tree-sitter
ERROR: Failed to build one or more wheels
Traceback (most recent call last):
File "/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/bazel_install/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/bazel_install/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/rules_python/python/pip_install/extract_wheels/__main__.py", line 5, in
main()
File "/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/rules_python/python/pip_install/extract_wheels/__init__.py", line 87, in main
subprocess.run(pip_args, check=True)
File "/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/bazel_install/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/python_bin', '-m', 'pip', 'wheel', '-r', '/home/user/natural-cubert/requirements.txt']' returned non-zero exit status 1.
)
🌍 Your Environment
Operating System:
Ubuntu
Output of bazel version:
Build label: 4.0.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Jan 21 07:33:24 2021 (1611214404)
Build timestamp: 1611214404
Rules_python version:
0.1.0
Anything else relevant?
The bazel setup used to build and run, not sure what changed.
Metadata
Metadata
Assignees
Labels
Can Close?Will close in 30 days if there is no new activityWill close in 30 days if there is no new activity
Activity
hrfuller commentedon May 8, 2022
I believe the issue lies with the hermetic interpreters which are now used, apparently by default. The installed interpreters libs don't seem have the correct header files present.
hrfuller commentedon May 8, 2022
@davians12 I can't repro this exact issue because I develop on macos. Can you put together a repro of the WORKSPACE which create this issue. In particular are you using a custom python interpreter toolchain target or the built in hermetic toolchains?
thundergolfer commentedon May 10, 2022
@hrfuller Yep this is totally expected with the python-build-standalone intepreter.
It's failed to find a wheel for
tree-sitterso fell back to compiling it. It appearstree-sitterdoesn't distribute a Linux wheel. https://pypi.org/project/tree-sitter/#fileshrfuller commentedon May 11, 2022
So this issue is basically the linux equivalent of #646 but in the integration test I added to #679 I can't repro this failure with linux. To me that implies that the interpreter setup here is slightly different. In particular this include path
-I/home/user/.cache/bazel/_bazel_user/1ced21babfe30b5e39b8b34411e58735/external/python_interpreter/bazel_install/lib/python3.7/config-3.7m-x86_64-linux-gnu/Includedoesn't look like it comes from the standalone interpreters because the external repo name in the path doesn't have the standard form that rules python creates e.g something like `python39_x86_64-apple-darwin. Thats why I wanted to see a repro.aaliddell commentedon May 16, 2022
I am seeing the same thing, using a custom Python toolchain and the
hiredisandpython-prctlpackages. The build works with 0.7.0 and fails with 0.8.0, so I have bisected the issue back to fe30f15 where a set of dependencies were updated. It appears that new combination of packages has changed how the include paths are being found when creating the gcc compile args, resulting in the-Iflags pointing to the wrong locations.I've tried looking through the issues on those packages for relevant changes, but not found anything obvious yet. The simple test would be to update these packages one-by-one or just bulk upgrade all of these to their latest version to see if there was a bug that's been resolved.
This works (commit prior to fe30f15):
This fails:
@davians12: Do you see the same behaviour when you use the above rules_python versions?
I'll try to extract a repro repo too.
aaliddell commentedon May 16, 2022
So, this has been a rabbit hole into Python builds. The repo I have that reproduces this is similar in setup to the demo code you have here in this repo, where Python is pulled and built in patch commands. Given the
.../bazel_install/...path segment shown in the top comment, I would guess @davians12 has the same setup, as this was a common-ish setup prior to the hermetic Python introduction recently. This has always been a bit of a hack and it seems we are finally seeing a consequence of that.The package update that breaks the build is setuptools between v60.1.1 and v60.2.0. Looking at the diff, there are changes to the way Python build config files are detected, which appears to have a knock-on effect in how the include paths are generated in build_ext when building the wheel.
The section of interest appears to be in how distutils checks if a Python executable is part of a 'non-installed' build and changes behaviour if so. The critical piece here is the final Python binary is symlinked back into the source tree, which makes distutils think we are in an uninstalled Python build due to the presence of the source files alongside the binary.
So, as far as I can tell, this is not a bug in rules_python nor any of the upstream Python packages, but rather a latent problem in mangling Python build directories that hasn't mattered until an unrelated cleanup in distutils exposed it.
The fix for users with this build setup is to ensure that the final binary/symlink that is given to your toolchain/pip_install/pip_parse is not positioned alongside the source files, to prevent distutils getting confused. I have done this by separating the source and build dirs in the
@pythonrepo. Alternatively, switch to the new hermetic toolchains.groodt commentedon May 17, 2022
Coincidentally running into the same issue. The setuptools upgrade was introduced here: #661
groodt commentedon May 17, 2022
I'm currently winding back
setuptoolsversions to find out when it breaks in my environment. I'm not sure how necessary #661 was. I think we need to be more careful withsetuptoolsupgrades and stay on old versions as long as possible.aaliddell commentedon May 17, 2022
Are you running with the new built-in toolchains or a custom Python toolchain?
Rolling back setuptools may be a short-term solution, but that only hides the issue and we should be able to upgrade it freely in the future (particularly as new Python releases come out that could start to have issues with a very old setuptools version). In this particular case for this issue: there's nothing that rules_python can fix (besides maybe updating the demo code I linked), as it's my local custom toolchain layout that was breaking distutils. It just so happens that a fair number of us have adopted and adapted that original demo code, so are seeing similar bugs.
groodt commentedon Jun 12, 2022
My issue is now resolved after upgrading
rules_pythonto the latest and upgrading the failing dependency to awheel, rather than relying on thesdist. Relying on thesdistwas fragile tosetuptoolsanddistutilschanges as you mention.I no longer believe I was seeing the same issue, so I think we still need to see a repro from the original poster to be certain (although I do think @aaliddell has the best explanation so far).
hrfuller commentedon Jun 13, 2022
Agreed the @aaliddell bisect makes sense to me.
groodt commentedon Jul 3, 2022
@davians12 Can you confirm whether @aaliddell fix solves your issue?
aaliddell commentedon Jul 3, 2022
A useful closure here could be to update the example python build toolchain here to fix the symlink being inside the source tree.
garymm commentedon Aug 29, 2022
For me this was fixed on MacOS by version 0.12.0. From the release notes, I'm guessing fixed by #809
github-actions commentedon Oct 10, 2022
This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"