[tests] fix SVD tests

[sayakpaul]

Fixes https://github.com/huggingface/diffusers/actions/runs/33510236955/job/99864282042

It never showed up because we didn't do testing of the corresponding features in the previous testing mixins. I didn't want to touch the src during my refactors of the test suites for a better reviewing experience.

[sayakpaul]

/diffusers-bot pytest tests/pipelines/stable_video_diffusion/

[github-actions]

✅ pytest tests/pipelines/stable_video_diffusion/ passed on GPU — view logs.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[dg845]

I think this comment is confusing because the change addresses the needs_upcasting=False case, not the needs_upcasting=True case (where both the image and vae would be in FP32 and we wouldn't need to cast). So we should rewrite it or remove it.

(needs_upcasting only checks whether the VAE dtype is FP16, which seems like a bug. The tests in https://github.com/huggingface/diffusers/actions/runs/33510236955/job/99864282042 fail when the input is BF16.)

[sayakpaul]

I think this comment is confusing because the change addresses the needs_upcasting=False case, not the needs_upcasting=True case (where both the image and vae would be in FP32 and we wouldn't need to cast). So we should rewrite it or remove it.

But the current change would still remain agnostic to it because we're casting to the dtype of the VAE no?

[sayakpaul]

So maybe a better comment is:

# The image comes out of `video_processor.preprocess()` in float32, so it has to follow the VAE dtype.
# `needs_upcasting=False` in `__call__` has already moved a float16 VAE to float32 by this point. But regardless of that, this change should be harmless.

[sayakpaul]

(needs_upcasting only checks whether the VAE dtype is FP16, which seems like a bug.

I don't think it's a bug, actually. The underlying VAE from the pretrained SVD checkpoint is very sensitive to FP16, so it has historical reasons. So you might remember: https://huggingface.co/madebyollin/sdxl-vae-fp16-fix

[dg845]

Yeah, I think the current change still makes sense regardless. I would suggest something like

# The image comes out of `video_processor.preprocess()` in float32. When `needs_upcasting=False`, the VAE may not
# be in FP32 (for example, it could be in BF16) so we need to cast in this case.

[dg845]

Thanks! Left one comment.