Skip to content

Fix set_ip_adapter_scale with a single scale and multiple IP-Adapters - #15217

Open
sohailshk wants to merge 1 commit into
huggingface:mainfrom
sohailshk:fix/ip-adapter-scale-broadcast
Open

sohailshk wants to merge 1 commit into
huggingface:mainfrom
sohailshk:fix/ip-adapter-scale-broadcast

Conversation

@sohailshk

@sohailshk sohailshk commented Nov 9, 2026 •

Copy link
Copy Markdown

What does this PR do?

Fixes set_ip_adapter_scale() raising a ValueError when it is given a single scale while more than one IP-Adapter is loaded.

Problem

In IPAdapterMixin.set_ip_adapter_scale and ModularIPAdapterMixin.set_ip_adapter_scale, a single scale is wrapped into a one-element list and is meant to be broadcast across all loaded IP-Adapters. But the length-mismatch check ran first:

if len(scale_configs) != len(attn_processor.scale):
    raise ValueError(...)
elif len(scale_configs) == 1:
    scale_configs = scale_configs * len(attn_processor.scale)

With 2 adapters loaded, len(scale_configs) == 1 and len(attn_processor.scale) == 2, so the first branch always raised and the elif broadcast branch was unreachable for any case where it was needed (it could only trigger when a single adapter was loaded, where it is a no-op).

pipe.load_ip_adapter(..., weight_name=["adapter_a.bin", "adapter_b.bin"])
pipe.set_ip_adapter_scale(0.5)
# ValueError: Cannot assign 1 scale_configs to 2 IP-Adapter.

Passing a list ([0.5, 0.5]) works, so only the single-scale shorthand was broken.

Fix

Swap the order of the two branches so the single-scale broadcast is handled first, and the ValueError is only raised for a real mismatch (e.g. 2 scales for 3 adapters). Applied in both places the logic is duplicated in src/diffusers/loaders/ip_adapter.py (IPAdapterMixin and ModularIPAdapterMixin).

Behavior is unchanged for lists with a matching length, and for genuine mismatches the same error is still raised.

Tests

  • tests/pipelines/testing_utils/ip_adapter.py: new test_ip_adapter_single_scale_multi_adapter in the shared IP-Adapter tester mixin. It loads two adapters, calls set_ip_adapter_scale(0.5), and asserts every IP-Adapter attention processor ends up with scale == [0.5, 0.5]. Every pipeline using the mixin picks it up.
  • tests/modular_pipelines/stable_diffusion_xl/test_modular_pipeline_stable_diffusion_xl.py: the same check for the modular SDXL pipeline.

Before submitting

  • Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR?
  • Did you read the contributor guideline?
  • Did you read our philosophy doc?
  • Was this discussed/approved via a GitHub issue or GitHub Discussions?
  • Did you make sure to update the documentation with your changes? (No docs change needed: this is a bug fix, the intended behavior is unchanged.)
  • Did you write any new necessary tests?
  • Are you the author (or part of the team we work with) of the model/pipeline this PR relates to?

Who can review?

@yiyixuxu @sayakpaul

@github-actions github-actions Bot added tests ip-adapter size/S PR with diff < 50 LOC labels Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Hi @sohailshk, thanks for the PR! It does not appear to link an issue it fixes.

  • If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked.
  • If you are the author of the model/pipeline this PR relates to, or part of the team we work with on it, tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box in the PR description (if you removed the template, paste that line back, checked). This reminder then no longer applies.
  • Otherwise there is nothing you need to do: a maintainer will look at the PR and add the no-issue-needed label to silence this reminder if it should stay open without an issue.

See the contribution guide for more details.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an existing issue, the author box is ticked, or the no-issue-needed label is added, you can ignore this message — it stays here as a comment, but it no longer applies.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ip-adapter size/S PR with diff < 50 LOC tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant