Skip to content

Allow models to customize tensor parallel plans - #15197

Open
yzhautouskay wants to merge 1 commit into
huggingface:mainfrom
yzhautouskay:yzhautouskay/tp-plan-resolution
Open

yzhautouskay wants to merge 1 commit into
huggingface:mainfrom
yzhautouskay:yzhautouskay/tp-plan-resolution

Conversation

@yzhautouskay

@yzhautouskay yzhautouskay commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

Route TP shard resolution and hook installation through _get_tp_plan(). It returns the existing class-level plan by default, preserving current behavior.

This enables configuration-dependent plans without modifying the class attribute. Cosmos 3 needs this because SiLU uses gate_proj layers while ReLU2 does not – differs between model flavors. Extracted from #15137, needed here https://github.com/huggingface/diffusers/pull/15137/changes#diff-2e4d21f189d670c5f6d027711540b335373c8ab2eaff5bf10471ec8dc2e945b5R405, as requested in review

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
    • Did you read the Coding with AI agents guide?
    • Did you run the self-review skill on the diff?
    • Did you share the final self-review notes in the PR description or a comment?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc? (important for complex PRs)
  • Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?
  • Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Who can review?

@sayakpaul

Core library:

Integrations:

HF projects:

-->

@github-actions github-actions Bot added models size/S PR with diff < 50 LOC labels Nov 6, 2026
@sayakpaul
sayakpaul requested a review from DN6 November 6, 2026 14:14

@sayakpaul sayakpaul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Could you also preserve the contribution history?

@HuggingFaceDocBuilderDev

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yzhautouskay
yzhautouskay force-pushed the yzhautouskay/tp-plan-resolution branch from cb1a570 to feb7dfd Compare November 6, 2026 14:51
@github-actions github-actions Bot added the tests label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Hi @yzhautouskay, thanks for the PR! It does not appear to link an issue it fixes.

  • If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked.
  • If you are the author of the model/pipeline this PR relates to, or part of the team we work with on it, tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box in the PR description (if you removed the template, paste that line back, checked). This reminder then no longer applies.
  • Otherwise there is nothing you need to do: a maintainer will look at the PR and add the no-issue-needed label to silence this reminder if it should stay open without an issue.

See the contribution guide for more details.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an existing issue, the author box is ticked, or the no-issue-needed label is added, you can ignore this message — it stays here as a comment, but it no longer applies.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models no-issue-needed for PRs that do not require link to an issue size/S PR with diff < 50 LOC tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants