Skip to content

feat(toolchains): allow selecting debug python-build-standalone runtimes - #4502

Open
vadikmironov wants to merge 2 commits into
bazel-contrib:mainfrom
vadikmironov:feat/select-debug-runtime
Open

vadikmironov wants to merge 2 commits into
bazel-contrib:mainfrom
vadikmironov:feat/select-debug-runtime

Conversation

@vadikmironov

@vadikmironov vadikmironov commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #4471 this change adds a flag to choose a debug runtime.

python-build-standalone (PBS) publishes debug (Py_DEBUG) builds of
the runtime, which can be helpful for anyone developing/debugging C
extensions. Since #4471 they work on Linux and macOS, but selecting one
per build takes a python.single_version_platform_override with a
custom platform name and custom target_settings, for each version and
platform.

To fix, add a debug value to
--@rules_python//python/config_settings:py_pbs_distribution. Debug
archives from runtime manifests get a -debug platform key, the same
way full archives get -full, so the existing platform, toolchain
and alias code handles them. The built-in manifest has no debug
archives; a release's SHA256SUMS added with
python.override(add_runtime_manifest_urls = ...) makes them
available.

Similar to freethreaded, debug does not fall back to install_only
when a version or platform has no debug archive. Windows debug archives
are not supported yet.

Add a `debug` value to the `py_pbs_distribution` flag. Debug
(`Py_DEBUG`) archives from runtime manifests get a `-debug` platform
key, like `-full`, so a build flag can select them. Unlike the other
values, `debug` has no fallback to `install_only`: only the versions
and platforms that have a debug archive get a debug toolchain.

The debug runtime test now uses the flag and a standard platform key
in place of a custom platform.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@vadikmironov

Copy link
Copy Markdown
Contributor Author

I prepared this with Claude Code. I reviewed it myself, and I'll answer review comments myself.

@vadikmironov

Copy link
Copy Markdown
Contributor Author

As I mentioned in #4471, --compilation_mode=dbg felt as a natural choice for driving the choice of debug toolchain, but in the end I left it out of this PR. Making auto resolve to debug under -c dbg is a few lines in the flag rule, but it needs either the fallback from debug or debug archives in the built-in manifest. No fallback choice is logical and matches py_freethreaded=yes, thus leaving only one path. Not sure if reviewers would agree if it is the best path forward and would welcome any comments. It also would change the interpreter for everyone who already builds with -c dbg, but this is sort of rather niche case in my view.

@rickeylev

Copy link
Copy Markdown
Collaborator

I was poking around the PBS format and the "debug" aspect of a distribution is separate from the "archive flavor" (distribution). The basic naming scheme of them is {pythonVersionAndPlatform}-{buildOptions}-{archiveFlavor}, where buildOptions is the options separated by pluses, e.g. [freethreaded, pgo, static] or [debug]. And archiveFlavor is one of full, install_only, install_only_stripped

So I think it makes more sense to expose selection of this as a separate flag and not fold it under the distribution flag

@rickeylev

Copy link
Copy Markdown
Collaborator

Making auto resolve to debug under -c dbg is a few lines in the flag rule, but it needs either the fallback from debug or debug archives in the built-in manifest.

Oh good point. Yeah, we don't want -c dbg to fail to find an runtime by default and break the whole build.

@rickeylev rickeylev added the do not merge Tag that prevents merging label Nov 14, 2026

@rickeylev rickeylev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trade-offs across the three flag options under discussion:

  • --compilation_mode=dbg driving auto: Beyond missing built-in debug archives, Py_DEBUG changes the C ABI (cp314d) and breaks prebuilt release wheels when -c dbg is used to debug C/C++ targets, whereas install_only already keeps debug symbols without changing the ABI.
  • py_pbs_distribution=debug (current PR): Limits PLATFORMS growth since upstream PBS only ships debug-full, but folds an ABI-changing build option into an archive-packaging flag with mixed fallback rules (archive flavors fall back to install_only, whereas debug cannot).
  • Separate flag (such as py_debug=yes|no, like py_freethreaded): Keeps ABI-changing build options orthogonal to archive flavors with uniform no-fallback semantics, at the cost of another public flag and doubling the PLATFORMS cross-product.

This is an agent-generated comment reviewed and approved by the user.

Comment on lines 46 to 52
keys = [platform]
if entry.archive_flavor in PBS_DISTRIBUTION_SUFFIXES:
keys.append(platform + PBS_DISTRIBUTION_SUFFIXES[entry.archive_flavor])
flavor_parts = entry.build_flavor.replace("-", "+").split("+")
if "debug" in flavor_parts:
keys.append(platform + PBS_DISTRIBUTION_SUFFIXES["debug"])
return keys

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug builds here still emit plain and -full platform keys, assigning an ABI-incompatible Py_DEBUG archive when non-debug full entries are absent. Windows debug entries in SHA256SUMS also register a broken -debug toolchain expecting python.exe instead of python_d.exe.

Suggested fix: Extract _is_debug_build(entry) and return early for debug builds, skipping Windows:

def _is_debug_build(entry):
    return "debug" in entry.build_flavor.replace("-", "+").split("+")

def manifest_entry_platform_keys(entry, platform):
    """...
    Returns:
      A list of platform keys: the debug key for a non-Windows debug build, or
      the plain key plus the archive-flavor key (if any) for non-debug builds.
    """
    if _is_debug_build(entry):
        if entry.os == "windows":
            return []
        return [platform + PBS_DISTRIBUTION_SUFFIXES["debug"]]
    keys = [platform]
    if entry.archive_flavor in PBS_DISTRIBUTION_SUFFIXES:
        keys.append(platform + PBS_DISTRIBUTION_SUFFIXES[entry.archive_flavor])
    return keys

Where: Reuse _is_debug_build(entry) in manifest_entry_sort_key at python/private/pbs_manifest.bzl:63-64 (debug_rank = 1 if _is_debug_build(entry) else 0), and cover debug-only and Windows debug manifest entries in tests/python_bzlmod_ext/runtime_manifests_tests.bzl.

This is an agent-generated comment reviewed and approved by the user.

Comment on lines +193 to +198
for distribution in [
"install_only",
"install_only_stripped",
"full",
"debug",
]:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This distribution list is unsorted and missing # keep-sorted markers.

Suggested fix:

    for distribution in [
        # keep-sorted start
        "debug",
        "full",
        "install_only",
        "install_only_stripped",
        # keep-sorted end
    ]:

Where: Also add # keep-sorted start/end markers inside PBS_DISTRIBUTION_SUFFIXES at python/private/pbs_manifest.bzl:14-18.

This is an agent-generated comment reviewed and approved by the user.

@vadikmironov

Copy link
Copy Markdown
Contributor Author

Thanks, @rickeylev. Sure, I'll do debug as a build option like freethreaded then and not an archive flavor (separate flag, --@rules_python//python/config_settings:py_debug=yes|no (default no)), with the same no-fallback behaviour as py_freethreaded.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Tag that prevents merging

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants