From 754c619ce0da1184bcc86879801cb781e421de96 Mon Sep 17 00:00:00 2001 From: Zeinab Dehghani Date: Wed, 19 Aug 2026 15:20:06 +0100 Subject: [PATCH] Revise Pix2Pix documentation for clarity and scope Signed-off-by: Zeinab Dehghani --- .../image-generation/pix2pix-models.md | 71 ++++++++++++++----- 1 file changed, 53 insertions(+), 18 deletions(-) diff --git a/docs/explainers/image-generation/pix2pix-models.md b/docs/explainers/image-generation/pix2pix-models.md index 69c26407..c9142bca 100644 --- a/docs/explainers/image-generation/pix2pix-models.md +++ b/docs/explainers/image-generation/pix2pix-models.md @@ -1,34 +1,69 @@ --- -title: Pix2Pix Models in XWhy Image Generation -description: Planned examples showing how Pix2Pix-style image-to-image models may be explained within the broader XWhy image-generation documentation. +title: Image Generation and Image Editing +description: An overview of model-agnostic explanation approaches for image-generation and image-editing models in XWhy. --- -# Pix2Pix model examples +# Image Generation and Image Editing -!!! danger "Experimental interface — under construction" - XWhy currently exports `Pix2PixExplainer`, but its `explain()` method raises `NotImplementedError`. The examples described here are a documentation roadmap, not an executable workflow. +Generative image models either create new images or transform existing ones. Within XWhy, these tasks are organised into two related categories: + +- **Image generation**, where a model creates an image from a conditioning input such as text, noise, or another representation. +- **Image editing**, where a source image is modified according to an instruction, mask, or target condition. + +Unlike image classification, these models do not return a single class score. Their explanations must therefore consider the relationship between the input condition, the source image, and the generated output. + +## Model-agnostic explanation workflow -Pix2Pix-style models are one example of conditional image-to-image generation. They belong under **Image Generation → Image Editing** because the model transforms a source image into a target image rather than returning a classification score. +A local explanation workflow for image-generation and image-editing models may involve: -## Planned first example +1. generating a reference output using the original input; +2. perturbing the source image, conditioning input, or editing instruction; +3. generating outputs for the perturbed inputs; +4. comparing each perturbed output with the reference output; +5. measuring differences using pixel-space, perceptual, or semantic distances; +6. fitting a local interpretable surrogate model; +7. producing feature, region, or word-level attributions. -A future worked example should document: +This approach can be applied without requiring access to the internal architecture or gradients of the generative model. -1. the source image and target transformation task; -2. the Pix2Pix model, weights, and preprocessing; +## Image-editing example: Pix2Pix + +Pix2Pix-style models are an example of conditional image-to-image generation. They transform a source image into a target image rather than returning a classification score. + +!!! danger "Experimental interface — under construction" + XWhy currently exports `Pix2PixExplainer`, but its `explain()` method raises `NotImplementedError`. The interface is therefore experimental and should not yet be treated as an executable explanation workflow. + +A future Pix2Pix example should document: + +1. the source image and target transformation; +2. the model, weights, and preprocessing steps; 3. the source-image regions selected for perturbation; 4. the output-distance or perceptual-similarity measure; 5. the local surrogate configuration; -6. the generated attribution map; +6. the resulting attribution map; 7. fidelity, stability, and runtime evidence; -8. limitations of interpreting generative outputs. +8. limitations when interpreting generative outputs. + +## Planned examples + +Future documentation may include: + +- text-to-image generation; +- instruction-based image editing; +- mask-based image editing; +- Pix2Pix-style image translation; +- perturbation of textual conditioning inputs; +- comparison of pixel, perceptual, and semantic distances; +- detection of unintended changes outside the requested edit region. + +## Evaluation considerations -## Planned comparison cases +Explanations for generative models should be assessed across multiple dimensions, including: -- removing or masking source-image regions; -- perturbing the conditioning input; -- comparing pixel-space and perceptual distances; -- checking attribution stability across generation seeds; -- detecting unintended changes outside the target edit region. +- **fidelity** to the behaviour of the generative model; +- **stability** under small or semantically irrelevant input changes; +- **consistency** across repeated generations; +- **runtime and computational cost**; +- **localisation** of intended and unintended visual changes. [View the current `Pix2PixExplainer` API interface](../../reference/xwhy/explainers/pix2pix.md)