Skip to content

[MiniMax-H3, Pika H3 Flex] Resample reference soundtracks without torchaudio - #15199

Open
christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_audio_resample_without_torchaudio
Open

christopher5106 wants to merge 1 commit into
huggingface:mainfrom
scenario-labs:fix_audio_resample_without_torchaudio

Conversation

@christopher5106

@christopher5106 christopher5106 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

The ref2va setup steps of MiniMax-H3 (MiniMaxH3Ref2VASetupStep) and Pika H3 Flex (PikaH3FlexRef2VASetupStep) resample a reference soundtrack onto the audio VAE's sample rate with torchaudio.transforms.Resample. torchaudio is not a diffusers dependency, so a reference soundtrack at any other rate than the VAE's 32 kHz (the 44.1 / 48 kHz audio track of most videos) fails:

ImportError: Resampling a MiniMax-H3 reference soundtrack from 48000 Hz to 32000 Hz needs `torchaudio`. Pass a waveform already at the audio VAE's sample rate to do without it.

Installing torchaudio doesn't help everywhere: it has no release for recent torch versions (the last on PyPI is 2.11.1).

This PR adds diffusers.utils.torch_utils.resample_audio, a plain-PyTorch port of torchaudio's default band-limited resampler (Hann-windowed sinc, lowpass_filter_width=6, rolloff=0.99), and uses it in both steps. Its output matches torchaudio.transforms.Resample to float32 rounding: identical at 48 → 32 kHz, within 4e-5 at 44.1 kHz. torchaudio's own transforms.Resample and functional.resample differ from each other by about as much.

echo still imports torchaudio, because it also needs MelSpectrogram, so this PR doesn't change it.

Tests

  • tests/others/test_utils.py::TestResampleAudio: parity with torchaudio.transforms.Resample for 48→32, 44.1→32, 16→48 and 22.05→44.1 kHz (skipped without torchaudio), a 1 kHz tone keeping its frequency through 48 → 32 kHz, and checks on shape, dtype, the same-rate identity and the integer-input error.
  • tests/modular_pipelines/minimax_h3: _normalize_audio_condition resamples a 48 kHz mono reference with torchaudio blocked in sys.modules.

The full minimax_h3 and pika_h3_flex suites pass on CPU, except 4 Pika H3 Flex ref2va tests (test_inference_is_not_nan, test_save_from_pretrained, and the two auto_cpu_offload ones), which fail the same way on main without this change.

Before submitting

…chaudio

The ref2va setup steps of MiniMax-H3 and Pika H3 Flex resample a reference
soundtrack onto the audio VAE's sample rate with `torchaudio.transforms.Resample`,
an optional dependency that is not installed with diffusers. Any soundtrack at
another rate than the VAE's (e.g. the 44.1 / 48 kHz track of most videos) fails
with an ImportError unless torchaudio is installed, and torchaudio has no release
for recent torch versions.

Add `diffusers.utils.torch_utils.resample_audio`, a plain-PyTorch port of
torchaudio's default windowed-sinc resampler (`sinc_interp_hann`,
`lowpass_filter_width=6`, `rolloff=0.99`), and use it in both steps. It matches
`torchaudio.transforms.Resample` to float32 rounding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Hi @christopher5106, thanks for the PR! It does not appear to link an issue it fixes.

  • If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked.
  • If you are the author of the model/pipeline this PR relates to, or part of the team we work with on it, tick the Are you the author (or part of the team we work with) of the model/pipeline this PR relates to? box in the PR description (if you removed the template, paste that line back, checked). This reminder then no longer applies.
  • Otherwise there is nothing you need to do: a maintainer will look at the PR and add the no-issue-needed label to silence this reminder if it should stay open without an issue.

See the contribution guide for more details.

Please note that PRs without a linked issue are likely to be automatically closed 10 days after this notice.

Once the PR links an existing issue, the author box is ticked, or the no-issue-needed label is added, you can ignore this message — it stays here as a comment, but it no longer applies.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant