Repository navigation
feat(toolchains): allow selecting debug python-build-standalone runtimes - #4502
vadikmironov wants to merge 2 commits into
Conversation
Add a `debug` value to the `py_pbs_distribution` flag. Debug (`Py_DEBUG`) archives from runtime manifests get a `-debug` platform key, like `-full`, so a build flag can select them. Unlike the other values, `debug` has no fallback to `install_only`: only the versions and platforms that have a debug archive get a debug toolchain. The debug runtime test now uses the flag and a standard platform key in place of a custom platform. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
I prepared this with Claude Code. I reviewed it myself, and I'll answer review comments myself. |
|
As I mentioned in #4471, |
|
I was poking around the PBS format and the "debug" aspect of a distribution is separate from the "archive flavor" (distribution). The basic naming scheme of them is So I think it makes more sense to expose selection of this as a separate flag and not fold it under the distribution flag |
Oh good point. Yeah, we don't want |
rickeylev
left a comment
There was a problem hiding this comment.
Trade-offs across the three flag options under discussion:
- --compilation_mode=dbg driving
auto: Beyond missing built-in debug archives, Py_DEBUG changes the C ABI (cp314d) and breaks prebuilt release wheels when -c dbg is used to debug C/C++ targets, whereas install_only already keeps debug symbols without changing the ABI. - py_pbs_distribution=debug (current PR): Limits PLATFORMS growth since upstream PBS only ships debug-full, but folds an ABI-changing build option into an archive-packaging flag with mixed fallback rules (archive flavors fall back to install_only, whereas
debugcannot). - Separate flag (such as py_debug=yes|no, like py_freethreaded): Keeps ABI-changing build options orthogonal to archive flavors with uniform no-fallback semantics, at the cost of another public flag and doubling the PLATFORMS cross-product.
This is an agent-generated comment reviewed and approved by the user.
| keys = [platform] | ||
| if entry.archive_flavor in PBS_DISTRIBUTION_SUFFIXES: | ||
| keys.append(platform + PBS_DISTRIBUTION_SUFFIXES[entry.archive_flavor]) | ||
| flavor_parts = entry.build_flavor.replace("-", "+").split("+") | ||
| if "debug" in flavor_parts: | ||
| keys.append(platform + PBS_DISTRIBUTION_SUFFIXES["debug"]) | ||
| return keys |
There was a problem hiding this comment.
Debug builds here still emit plain and -full platform keys, assigning an ABI-incompatible Py_DEBUG archive when non-debug full entries are absent. Windows debug entries in SHA256SUMS also register a broken -debug toolchain expecting python.exe instead of python_d.exe.
Suggested fix: Extract _is_debug_build(entry) and return early for debug builds, skipping Windows:
def _is_debug_build(entry):
return "debug" in entry.build_flavor.replace("-", "+").split("+")
def manifest_entry_platform_keys(entry, platform):
"""...
Returns:
A list of platform keys: the debug key for a non-Windows debug build, or
the plain key plus the archive-flavor key (if any) for non-debug builds.
"""
if _is_debug_build(entry):
if entry.os == "windows":
return []
return [platform + PBS_DISTRIBUTION_SUFFIXES["debug"]]
keys = [platform]
if entry.archive_flavor in PBS_DISTRIBUTION_SUFFIXES:
keys.append(platform + PBS_DISTRIBUTION_SUFFIXES[entry.archive_flavor])
return keysWhere: Reuse _is_debug_build(entry) in manifest_entry_sort_key at python/private/pbs_manifest.bzl:63-64 (debug_rank = 1 if _is_debug_build(entry) else 0), and cover debug-only and Windows debug manifest entries in tests/python_bzlmod_ext/runtime_manifests_tests.bzl.
This is an agent-generated comment reviewed and approved by the user.
| for distribution in [ | ||
| "install_only", | ||
| "install_only_stripped", | ||
| "full", | ||
| "debug", | ||
| ]: |
There was a problem hiding this comment.
This distribution list is unsorted and missing # keep-sorted markers.
Suggested fix:
for distribution in [
# keep-sorted start
"debug",
"full",
"install_only",
"install_only_stripped",
# keep-sorted end
]:Where: Also add # keep-sorted start/end markers inside PBS_DISTRIBUTION_SUFFIXES at python/private/pbs_manifest.bzl:14-18.
This is an agent-generated comment reviewed and approved by the user.
|
Thanks, @rickeylev. Sure, I'll do |
Follow-up to #4471 this change adds a flag to choose a debug runtime.
python-build-standalone (PBS) publishes debug (
Py_DEBUG) builds ofthe runtime, which can be helpful for anyone developing/debugging C
extensions. Since #4471 they work on Linux and macOS, but selecting one
per build takes a
python.single_version_platform_overridewith acustom platform name and custom
target_settings, for each version andplatform.
To fix, add a
debugvalue to--@rules_python//python/config_settings:py_pbs_distribution. Debugarchives from runtime manifests get a
-debugplatform key, the sameway
fullarchives get-full, so the existing platform, toolchainand alias code handles them. The built-in manifest has no debug
archives; a release's
SHA256SUMSadded withpython.override(add_runtime_manifest_urls = ...)makes themavailable.
Similar to
freethreaded,debugdoes not fall back toinstall_onlywhen a version or platform has no debug archive. Windows debug archives
are not supported yet.