Skip to content

feat(py_info): support TreeArtifact venv entries via link_to_subpaths - #6

Open
jvolkman wants to merge 8 commits into
mainfrom
refactor/action-install-prep
Open

jvolkman wants to merge 8 commits into
mainfrom
refactor/action-install-prep

Conversation

@jvolkman

@jvolkman jvolkman commented Nov 1, 2026 •

Copy link
Copy Markdown
Owner

Targets that install a Python package into a directory artifact
(TreeArtifact) during the build phase cannot populate individual File
objects in VenvSymlinkEntry.files when constructing virtual environment
symlinks. Without sub-path metadata, overlapping package directories
across multiple dependencies cannot be merged at file granularity.

Add an experimental VenvSymlinkEntry.link_to_subpaths field (and
builder support) so directory-backed entries can list their relative
file paths for conflict merging, and teach virtual environment link
generation to produce and merge TreeArtifact-backed entries.
Single-file entries whose files holds an ancestor directory of
link_to_path are now linked during conflict merging rather than
skipped.

Also prepare internal rule and tool helpers for build-action wheel
installation with no change in default behavior:

  • Support multi-file bootstrap interpreter programs by staging sibling
    Python modules into a private directory with bytecode writing
    disabled.
  • Extract pure in-memory helpers for wheel RECORD and console-script
    shebang rewriting so build-time installers can reuse them directly.

Targets that install a Python package into a directory artifact
(TreeArtifact) during the build phase cannot populate individual File
objects in VenvSymlinkEntry.files when constructing virtual environment
symlinks. Without sub-path metadata, overlapping package directories
across multiple dependencies cannot be merged at file granularity.

Add an experimental VenvSymlinkEntry.link_to_subpaths field (and
builder support) so directory-backed entries can list their relative
file paths for conflict merging, and teach virtual environment link
generation to produce and merge TreeArtifact-backed entries.
Single-file entries whose files holds an ancestor directory of
link_to_path are now linked during conflict merging rather than
skipped.

Also prepare internal rule and tool helpers for build-action wheel
installation with no change in default behavior:
* Support multi-file bootstrap interpreter programs by staging sibling
  Python modules into a private directory with bytecode writing
  disabled.
* Extract pure in-memory helpers for wheel RECORD and console-script
  shebang rewriting so build-time installers can reuse them directly.
@jvolkman jvolkman changed the title feat(py_info): support TreeArtifact venv entries via link_to_paths feat(py_info): support TreeArtifact venv entries via link_to_subpaths Nov 1, 2026
@jvolkman
jvolkman force-pushed the refactor/action-install-prep branch from 1ba2b25 to a56064b Compare November 1, 2026 16:10
Drop the py_interpreter_program staging changes so it stays a thin
bootstrap shim, normalize get_venv_symlinks inputs into typed candidate
structs before grouping, clarify VenvSymlinkEntry and merge comments,
fix MyST directive nesting, and rename the changelog fragment to 4396.
Simplify the `files` expression on shared-library `VenvSymlinkEntry`
records in `get_venv_symlinks`, document `tree_venv_lib` and
`_assert_tree_artifact_equivalence`, and add
`tree_shared_lib_loading_test` to verify loading compiled C extensions
and transitive shared libraries from a `TreeArtifact`-backed venv at
runtime.
Switch `tree_venv_merge_test` and `tree_shared_lib_loading_test` from
`unittest.TestCase` + `py_reconfig_test` to `pytest_test` with plain
`pytest` test functions and assertions.
An entry that is linked as a single unit (a directory File, or a path
inside a directory artifact without link_to_subpaths) can't share its
venv path with files from other entries. Such overlaps used to surface
only when creating the runfiles tree, or produced an incorrect venv.

Detect the overlap while merging and fail with the entries involved.
Clarify in the VenvSymlinkEntry docs that link_to_subpaths is optional
and what leaving it empty means.
The check for unmergeable venv entries walked every parent directory
of every merged path, which roughly doubled the cost of merging
overlapping entries. Files usually share directories, so remember the
directories that were already walked and stop there.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant