You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
pip_install_dependencies macro from @rules_python//python/pip_install/repositories.bzl
Description
In our CI, we frequently see download errors caused by failing to download dependencies from rules_python. For example, the following bazel error:
Error in download_and_extract: java.io.IOException: Error downloading [https://files.pythonhosted.org/packages/76/0a/b6c5f311e32aeb3b406e03c079ade51e905ea630fc19d1262a46249c1c86/click-8.0.1-py3-none-any.whl] to /tmp/bazel/c75bfb83ea925bcd720c63452511dc13/external/pypi__click/temp13170534750999975287/click-8.0.1-py3-none-any.whl.zip: connect timed out
Our solution is to fork the @rules_python//python/pip_install/repositories.bzl file in our repo, and add additional mirror URL's in case files.pythonhosted.org is failing. This works great, but it is a bit painful to maintain, since we have to update it to keep it in sync whenever updating rules_python.
Describe the solution you'd like
Add additional mirrors for external dependencies to make the downloads more reliable.
Another idea would be to add the ability to specify our own mirrors to the call to pip_parse without forking the repositories.bzl file.
@kgreenek , is it possible to use --distdir as a workaround in your situation? I'm using this to eliminate network requests for dependencies in my projects including those listed in @rules_python//python/pip_install/repositories.bzl
http_archive (used by pip_install_dependencies) can be instructed to search locally before making a network call. (details: bazelbuild/bazel#9265 (comment))
I realize this isn't exactly what you're asking for, but it might offer some improvement over the current situation.
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
Activity
driftregion commentedon Sep 5, 2022
@kgreenek , is it possible to use
--distdiras a workaround in your situation? I'm using this to eliminate network requests for dependencies in my projects including those listed in@rules_python//python/pip_install/repositories.bzlhttps://docs.bazel.build/versions/main/external.html#offline-builds
http_archive(used bypip_install_dependencies) can be instructed to search locally before making a network call. (details: bazelbuild/bazel#9265 (comment))I realize this isn't exactly what you're asking for, but it might offer some improvement over the current situation.
kgreenek commentedon Sep 6, 2022
That looks like a helpful suggestion! I will look into using this.
github-actions commentedon Mar 5, 2023
This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!
github-actions commentedon Apr 5, 2023
This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"