Repository navigation
fix(pypi): respect uv.lock dependency platform markers - #4286
Conversation
|
Hi @aignas, could you take a look when you have time? This fixes uv.lock dependency-edge markers being ignored during platform selection. It includes a confirmed uvloop reproduction and regression tests, and the required CI checks pass. Thanks! |
| logger = logger, | ||
| ) | ||
|
|
||
| def _uv_lock_package_platforms(packages, all_platforms, platforms, extras_map): |
There was a problem hiding this comment.
the code in this function is heavily nested and I could not fully understand everything by just reading and I have reasonably high familiarity with the code base. Would it be possible to refactor it to somehow make it easier to read?
If I understand the general idea, we have:
uv.lockfile has aselfpackage entry that will be named based on the mainpyproject.tomlfile name.- We could look at the deps in there and calculate the markers that we need to add based on the markers for the deps for this
selfpackage. They should be basically "ANDed" with thetarget_platforms. - If the packages are not included, we need to discard them, which might happen automatically with the existing code?
|
Thank you for the fix, I love that this is surgical and is affecting only very few files, however, I would like the added function to be a little bit more easily readable before accepting the contribution. Let me know what you think. |
aignas
left a comment
There was a problem hiding this comment.
Note
This review was generated by an AI coding agent (Claude Code) at the request of @aignas. Please provide feedback if any of the comments are nonsensical or wrong, and feel free to ignore the suggestions that do not make sense.
Thanks for the fix! I tried to simplify the implementation locally while keeping the same approach of walking the dependency edges from the workspace roots, and I think we can achieve the same with less code by reusing the existing behaviour:
- No
(package, extra)states.uv_lock_extras_mapalready adds everyoptional-dependencieskey of a package to its requirement line, so the packages are installed with all of their extras on all of their platforms. Following all of theoptional-dependenciesof a reachable package is consistent with that and the walk only needs to track the packages. The reachability helper goes from ~80 to ~58 lines (including the docstring) andextras_mapis not needed. - Possible regression: the current walk only follows
dependenciesandoptional-dependencies, so the packages that are only used by thedev-dependencies(dependency groups) of the workspace roots get nosrcsand are dropped from the hub. The simplified version followsdev-dependenciestoo, and the suggested test covers it. - Possible dangling deps: with the current code a package reachable only through an extra that is not requested is dropped, but its parent is still installed with that extra, so the parent wheel would depend on a package that does not exist.
test_uv_lock_requested_extrasencodes that behaviour, so I suggested updated expectations. - The trade-off is that a package reachable only through an extra that is requested on some of the platforms now gets all of the platforms of its parent, which matches how the parent is installed.
With the suggestions applied, bazel test --config=fast-tests //tests/pypi/parse_requirements:all //tests/uv/lock:all passes locally (94 tests).
A possible follow-up would be to move the walk next to uv_lock_extras_map in python/uv/private/uv_lock_to_requirements.bzl so that uv_lock_to_requirements can use the same platform information.
Use the (pypi) subsystem prefix, focus the news entry on user-visible behavior, and link bazel-contrib#4287.
|
Thanks for the review! The walk now tracks packages and follows all recorded optional extras, consistent with the existing importer behavior. It also includes development dependency groups while respecting platform markers. The nested-extra fixture now uses a list. Regression checks reproduced both issues and pass with the revised traversal. All 94 parser and uv-lock tests and lint checks pass locally; the real uvloop reproduction still selects only the Linux wheel. |
|
AI disclosure: Codex assisted with the implementation, tests, and PR text. I direct the work in chat and authorize it to post responses on my behalf. Responses may be drafted and posted by Codex at my request; it does not automatically reply to review comments. A background agent monitors CI. |
|
Hey @aignas, any chance to get this cherry-picked into 2.5.0? Thanks |
|
/backport |
|
Updated release tracking issue #4388 (Release 2.5.0) with this PR added to the backports checklist. |
Fixes #4287. `pip.parse` can select Linux-only dependencies on macOS because it ignores platform conditions on dependency edges in `uv.lock`. For example, `uvloop; sys_platform == 'linux'` incorrectly selects a source archive on macOS. This change follows dependencies from the workspace roots and keeps each package only on the platforms where it is reachable. It preserves support for optional extras and development dependency groups. Validation: - All 94 parser and uv-lock tests pass, including regression coverage for platform filtering, extras, and development groups. - The real uvloop reproduction now selects only the Linux wheel. - Buildifier and `git diff --check` pass. Includes updated API documentation and a release note. --------- Co-authored-by: Richard Levasseur <richardlev@gmail.com> (cherry picked from commit 8a3b77c) Work towards #4388
Updates CHANGELOG.md and removes news files for backports: - bazel-contrib#4286 Work towards bazel-contrib#4388 Release-Tracking-Issue: bazel-contrib#4388 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Fixes #4287.
pip.parsecan select Linux-only dependencies on macOS because it ignores platform conditions on dependency edges inuv.lock. For example,uvloop; sys_platform == 'linux'incorrectly selects a source archive on macOS.This change follows dependencies from the workspace roots and keeps each package only on the platforms where it is reachable. It preserves support for optional extras and development dependency groups.
Validation:
git diff --checkpass.Includes updated API documentation and a release note.