Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ai/references/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Follow the style introduced in [#14113](https://github.com/huggingface/diffusers
- `HunyuanDiTAttentionPool` (`src/diffusers/models/embeddings.py`) shows the same failure without an MHA module: a plain `nn.Module` that hands its `q_proj` / `k_proj` / `v_proj` / `c_proj` weights to `torch.nn.functional.multi_head_attention_forward`, so all four projections stay offloaded rather than just one. `HunyuanDiT2DModel` opts out of group offloading entirely with `_supports_group_offloading = False`.
- Before adding a skip or an exclusion, confirm the failure still reproduces — several existing skips are stale, having outlived the upstream cause.
- **A migration that surfaces a `src/` gap marks the test `xfail`, it does not patch the pipeline.** Give the marker a module-level name and a `reason` naming the exact gap (`PNDM_*` in `tests/pipelines/pndm/test_pndm.py` is the worked example), and prefer `strict=True` so the marker reports XPASS — and gets deleted — the day the pipeline is fixed. Use `strict=False` only when one mark covers a group whose members do not all fail. Marking a whole test class keeps the mixin's own marks (`@is_memory`, `@require_accelerator`) intact; overriding individual inherited tests drops the decorators they were declared with, so re-declare those too.
- **`from_pipe` tests** (a pipeline that is a variant of an existing one — PAG, AnimateDiff, ...) compose the shared `FromPipeTesterMixin` (`tests/pipelines/testing_utils/from_pipe.py`, exported from `..testing_utils`) in their own test class. It derives the original pipeline from `pipeline_class.__name__`; set `original_pipeline_repo` on the test class to pull it from a repo other than the default for that class. The unittest-era `PipelineFromPipeTesterMixin` in `tests/pipelines/test_pipelines_common.py` is what it replaces.
- **`from_pipe` tests** (a pipeline that is a variant of an existing one — PAG, AnimateDiff, ...) compose the shared `FromPipeTesterMixin` (`tests/pipelines/testing_utils/from_pipe.py`, exported from `..testing_utils`) in their own test class. It derives the original pipeline from `pipeline_class.__name__`; set `original_pipeline_repo` on the test class to pull it from a repo other than the default for that class.
- **A hardware gap is a conditional skip, not an xfail.** When a test fails only because the runner's cuDNN build has no kernel for an op — `RuntimeError: GET was unable to find an engine to execute this computation`, as Sana's depthwise `Conv2d` hits in bfloat16 — wrap the call in `skip_if_no_cudnn_engine()` (`tests/testing_utils.py`). It skips on that error and re-raises every other `RuntimeError`, so the test still runs wherever the kernel exists.
- **PAG pipelines** also compose `PAGPipelineTesterMixin` (`tests/pipelines/pag/testing_utils.py`) in place of `PipelineTesterMixin`: it adds `test_pag_disable_enable` and `test_pag_inference` on top, driven by `base_pipeline_class` and the `pag_*` knobs on the test class. Keep `test_pag_applied_layers` per pipeline — which layers PAG resolves to is model-specific.
- **`encode_prompt` reading a component that isn't a text encoder or tokenizer?** `test_encode_prompt_works_in_isolation` rebuilds the pipeline with only the components whose names contain `text` or `tokenizer`. When `encode_prompt` also needs another one — a `processor` used for chat templating, say — list it in `text_stack_component_names` on the config class rather than re-implementing the test.
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/nightly_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,6 @@ jobs:
pytest -n 1 --max-worker-restart=0 --dist=loadfile \
--make-reports=tests_torch_minimum_version_cuda \
tests/models/test_modeling_common.py \
tests/pipelines/test_pipelines_common.py \

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer exists.

tests/pipelines/test_pipeline_utils.py \
tests/pipelines/test_pipelines.py \
tests/pipelines/test_pipelines_auto.py \
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/pr_tests_gpu.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ on:
- "src/diffusers/loaders/lora_base.py"
- "src/diffusers/loaders/lora_pipeline.py"
- "src/diffusers/loaders/peft.py"
- "tests/pipelines/test_pipelines_common.py"
- "tests/pipelines/testing_utils/**.py"
- "tests/models/test_modeling_common.py"
- "tests/models/testing_utils/**.py"
Comment on lines -17 to +19

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To have parity in our CI when test changes happen.

- "examples/**/*.py"
workflow_dispatch:

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/release_tests_fast.yml
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,6 @@ jobs:
pytest -n 1 --max-worker-restart=0 --dist=loadfile \
--make-reports=tests_torch_minimum_cuda \
tests/models/test_modeling_common.py \
tests/pipelines/test_pipelines_common.py \

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer exist.

tests/pipelines/test_pipeline_utils.py \
tests/pipelines/test_pipelines.py \
tests/pipelines/test_pipelines_auto.py \
Expand Down
2 changes: 1 addition & 1 deletion tests/models/transformers/test_models_transformer_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@


# TODO: This standalone function maintains backward compatibility with pipeline tests
# (tests/pipelines/test_pipelines_common.py) and will be refactored.
# (tests/pipelines/testing_utils/ip_adapter.py) and will be refactored.
def create_flux_ip_adapter_state_dict(model) -> dict[str, dict[str, Any]]:
"""Create a dummy IP Adapter state dict for Flux transformer testing."""
ip_cross_attn_state_dict = {}
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/chroma/test_pipeline_chroma.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
from diffusers import AutoencoderKL, ChromaPipeline, ChromaTransformer2DModel, FlowMatchEulerDiscreteScheduler

from ...testing_utils import assert_tensors_close, torch_device
from ..flux.testing_utils import FluxIPAdapterTesterMixin
from ..testing_utils import (
BasePipelineTesterConfig,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
check_qkv_fused_layers_exist,
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/chroma/test_pipeline_chroma_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
from diffusers import AutoencoderKL, ChromaImg2ImgPipeline, ChromaTransformer2DModel, FlowMatchEulerDiscreteScheduler

from ...testing_utils import assert_tensors_close, floats_tensor, torch_device
from ..flux.testing_utils import FluxIPAdapterTesterMixin
from ..testing_utils import (
BasePipelineTesterConfig,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
check_qkv_fused_layers_exist,
Expand Down
8 changes: 6 additions & 2 deletions tests/pipelines/controlnet_flux/test_controlnet_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,12 @@
require_big_accelerator,
torch_device,
)
from ..flux.testing_utils import FluxIPAdapterTesterMixin
from ..testing_utils import BasePipelineTesterConfig, MemoryTesterMixin, PipelineTesterMixin
from ..testing_utils import (
BasePipelineTesterConfig,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
)


enable_full_determinism()
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_inpainting.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/deepfloyd_if/test_if_superresolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,11 @@
slow,
torch_device,
)
from ..test_pipelines_common import assert_mean_pixel_difference
from ..testing_utils import (
BasePipelineTesterConfig,
PipelineOffloadTesterMixin,
PipelineTesterMixin,
assert_mean_pixel_difference,
)


Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/flux/test_pipeline_flux.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
BasePipelineTesterConfig,
FasterCacheTesterMixin,
FirstBlockCacheTesterMixin,
FluxIPAdapterTesterMixin,
LoraMemoryTesterMixin,
LoraTesterMixin,
MagCacheTesterMixin,
Expand All @@ -39,7 +40,6 @@
PyramidAttentionBroadcastTesterMixin,
TaylorSeerCacheTesterMixin,
)
from .testing_utils import FluxIPAdapterTesterMixin


if is_peft_available():
Expand Down
8 changes: 6 additions & 2 deletions tests/pipelines/flux/test_pipeline_flux_img2img.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@
from diffusers import AutoencoderKL, FlowMatchEulerDiscreteScheduler, FluxImg2ImgPipeline, FluxTransformer2DModel

from ...testing_utils import floats_tensor, torch_device
from ..testing_utils import BasePipelineTesterConfig, MemoryTesterMixin, PipelineTesterMixin
from .testing_utils import FluxIPAdapterTesterMixin
from ..testing_utils import (
BasePipelineTesterConfig,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
)


class FluxImg2ImgPipelineTesterConfig(BasePipelineTesterConfig):
Expand Down
8 changes: 6 additions & 2 deletions tests/pipelines/flux/test_pipeline_flux_inpaint.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,12 @@
from diffusers import AutoencoderKL, FlowMatchEulerDiscreteScheduler, FluxInpaintPipeline, FluxTransformer2DModel

from ...testing_utils import floats_tensor, torch_device
from ..testing_utils import BasePipelineTesterConfig, MemoryTesterMixin, PipelineTesterMixin
from .testing_utils import FluxIPAdapterTesterMixin
from ..testing_utils import (
BasePipelineTesterConfig,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
)


class FluxInpaintPipelineTesterConfig(BasePipelineTesterConfig):
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/flux/test_pipeline_flux_kontext.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@
from ..testing_utils import (
BasePipelineTesterConfig,
FasterCacheTesterMixin,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
PyramidAttentionBroadcastTesterMixin,
)
from .testing_utils import FluxIPAdapterTesterMixin


class FluxKontextPipelineTesterConfig(BasePipelineTesterConfig):
Expand Down
2 changes: 1 addition & 1 deletion tests/pipelines/flux/test_pipeline_flux_kontext_inpaint.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
from ..testing_utils import (
BasePipelineTesterConfig,
FasterCacheTesterMixin,
FluxIPAdapterTesterMixin,
MemoryTesterMixin,
PipelineTesterMixin,
PyramidAttentionBroadcastTesterMixin,
)
from .testing_utils import FluxIPAdapterTesterMixin


class FluxKontextInpaintPipelineTesterConfig(BasePipelineTesterConfig):
Expand Down
162 changes: 0 additions & 162 deletions tests/pipelines/flux/testing_utils.py

This file was deleted.

Loading
Loading