diff --git a/website/blog/2026-07-04-image-generation.md b/website/blog/2026-07-04-image-generation.md new file mode 100644 index 0000000..782d951 --- /dev/null +++ b/website/blog/2026-07-04-image-generation.md @@ -0,0 +1,763 @@ +--- +title: "From Idea to Image: A Complete Guide to SDXL Image Generation" +date: 2026-06-28 +description: "Master Stable Diffusion XL image generation — from quick drafts to production-ready visuals. A guide for developers, designers, and creative teams." +draft: true +--- + +# From Idea to Image: A Complete Guide to SDXL Image Generation + +## Introduction: Why SDXL Matters + +Imagine being able to generate stunning, blog-ready illustrations in under 30 minutes. No stock photo subscriptions. No hiring illustrators. Just a clear idea and the right tools. + +**Stable Diffusion XL (SDXL)** makes this possible. It's a state-of-the-art AI image generation model that understands context, follows detailed prompts, and produces artwork quality that rivals professional designers. Unlike earlier diffusion models, SDXL excels at text rendering, human anatomy, hands (historically a nightmare for AI), and precise scene composition. + +This tool is for **anyone** who needs images: +- **Developers** building content platforms or automation workflows +- **Content creators** who want original illustrations for blog posts, social media, or newsletters +- **Designers** exploring AI as a creative partner, not a replacement +- **Storytellers** bringing characters and worlds to life without leaving the terminal +- **Teams** that need consistency across dozens of images without manual tweaking + +Here's what you'll learn: **We'll go from a 10-minute draft (quick and rough) to a 28-minute polished production image** — the same image, progressively refined. You'll understand why each step matters, what settings control quality, and when speed matters more than perfection. + +--- + +## Section 1: The Basics + +### Installation & Setup + +To get started: + +```bash +# 1. Clone the repository (or navigate to image-generation/) +git clone https://github.com/your-org/image-generation.git +cd image-generation + +# 2. Install dependencies +pip install -r requirements.txt + +# 3. Verify installation +python -m image_generation.cli.generate --help +``` + +### Three Core Concepts (Explained Simply) + +#### **1. Prompts & Negative Prompts** + +Your **prompt** is an instruction to the AI: *"Create a glowing tropical garden with magenta flowers, teal pools, warm sunset, no text."* + +Think of it like giving a director notes for a film scene. The more specific you are, the better the result. But there's a catch: the AI sometimes misinterprets instructions or adds unwanted elements (blurry edges, duplicate hands, watermarks). + +That's where **negative prompts** come in. They're anti-instructions: *"Don't create: blurry, low quality, text, watermark, deformed hands, duplicate bodies."* + +**Why negatives matter:** +- **Without them:** The AI might add visual noise you didn't ask for. +- **With them:** You explicitly rule out common failure modes. + +A good negative prompt is usually a short list of things you *never* want: +``` +blurry, bad quality, worst quality, low resolution, text, watermark, +deformed, ugly, duplicate, morbid +``` + +Start with this list and customize based on what you see in outputs. + +#### **2. Steps & Guidance: The Quality vs. Speed Trade-off** + +Imagine drawing a portrait: with 5 quick sketches, you get a rough outline. With 50 passes, you refine every detail. SDXL works the same way. + +**Steps** = how many refinement passes the AI makes (default: 22). +- **Fewer steps (10–15):** Fast drafts, less detail (~10 min on CPU) +- **More steps (35+):** Polished, detailed images, slower (~19 min on CPU) + +**Guidance** = how strictly the AI follows your prompt (default: 6.5). +- **Low guidance (3–5):** More creative freedom, looser interpretation, dreamier +- **Medium guidance (6–8):** Balanced; follows your prompt closely +- **High guidance (9+):** Strict adherence; may look "overcooked" + +**The trade-off:** More steps = better quality, slower generation. Higher guidance = more predictable, but sometimes less natural-looking. + +#### **3. Presets: Speed Buttons for Common Scenarios** + +Tired of remembering parameters? Presets bundle everything into one word: + +| Preset | Steps | Refine? | CPU Time | Best For | +|--------|-------|---------|----------|----------| +| `quick-draft` | 15 | No | ~10 min | Fast experiments, iteration | +| `standard` | 22 | No | ~14 min | Default, balanced quality | +| `high-quality` | 35 | No | ~19 min | Detailed, final images | +| `production` | 35 | Yes | ~28 min | Blog headers, print-ready | + +So instead of remembering `--steps 35 --refine --refiner-steps 12`, you just use `--preset production`. + +### Your First Generation (The Simple Way) + +Ready to generate your first image? Here's the simplest possible command: + +```bash +cd image-generation + +# Dry run: see what parameters you get (no generation, instant) +python simple_config.py \ + --prompt "A peaceful mountain lake at sunrise, morning mist, no text" \ + --preset standard --dry-run + +# Actual generation (takes ~14 min on CPU) +python simple_config.py \ + --prompt "A peaceful mountain lake at sunrise, morning mist, no text" \ + --preset standard \ + --output outputs/my-first-image.png \ + --cpu +``` + +That's it. You're generating an image. + +**What just happened:** +1. You described a scene in plain English +2. The tool translated your preset (`standard`) into concrete parameters +3. SDXL's base model ran 22 refinement steps to generate the image +4. The result was saved as a PNG with metadata (parameters embedded) + +--- + +## Section 2: Five Progression Examples + +Each example shows a **journey from draft to production** — the same image, refined three times. + +### Example 1: Blog Hero Image (1200×632, Watercolor Style) + +**Goal:** Create a striking blog header for an article about "Developer Well-being." The image should be warm, inviting, and visually interesting without overwhelming the text. + +#### STEP 1: Simple Draft (Quick Test) +**Time: ~10 minutes | Quality: Raw concept | Use: Confirm the idea works** + +```bash +python simple_config.py \ + --prompt "A developer meditating in a sunny office" \ + --preset quick-draft \ + --style watercolor \ + --size blog-hero \ + --seed 42 \ + --output outputs/hero-v1.png \ + --cpu +``` + +**What changed:** +- `--preset quick-draft`: Only 15 steps (fast) +- `--style watercolor`: Applies a watercolor LoRA for soft, painterly look +- `--size blog-hero`: Sets dimensions to 1200×632 (perfect for blog headers) +- `--seed 42`: Reproducible (same seed = same image layout) + +**What you'd see:** A colorful scene of someone at a desk, but slightly abstract, soft edges, painterly. Not polished, but you can immediately see if the composition works. + +**Problem we're solving:** "Does this concept work visually?" + +--- + +#### STEP 2: Better (Refined Prompt) +**Time: ~14 minutes | Quality: Good | Use: Getting close to final** + +```bash +python simple_config.py \ + --prompt "A developer in a cozy office with warm sunlight, working peacefully at a glass desk, plants nearby, soft watercolor, no text, no people outside window" \ + --preset standard \ + --style watercolor \ + --size blog-hero \ + --modifier crisper \ + --seed 42 \ + --output outputs/hero-v2.png \ + --cpu +``` + +**What changed:** +- Richer prompt with more details (glass desk, plants, warm light) +- Added negative details ("no people outside window") to avoid distractions +- `--preset standard`: Now 22 steps (better detail) +- `--modifier crisper`: Guidance 7.5 (stricter adherence to prompt) + +**What you'd see:** Same composition, but sharper edges, more intentional details. The plants are clearer, the desk reflects light, the overall mood is more controlled. + +**Problem we're solving:** "How can we make the composition tighter and more intentional?" + +--- + +#### STEP 3: Beautiful (Production Polish) +**Time: ~28 minutes | Quality: Production-ready | Use: Final blog header** + +```bash +python simple_config.py \ + --prompt "A developer in a modern glass-top desk by a sunny window, surrounded by plants and a hot cup of tea, morning light pouring in, warm watercolor, peaceful, serene, no text, no people visible outside" \ + --negative-prompt "blurry, bad quality, worst quality, low resolution, text, watermark, deformed, ugly, duplicate, morbid, oversaturated, cluttered, busy, too many objects" \ + --preset production \ + --style watercolor \ + --size blog-hero \ + --modifier crisper \ + --seed 42 \ + --output outputs/hero-final.png \ + --cpu +``` + +**What changed:** +- `--preset production`: 35 base steps + refiner (two-stage polish) +- Full negative prompt (more comprehensive) +- Even richer positive prompt (tea cup adds a warm detail) +- Refiner step sharpens edges and enhances fine details + +**What you'd see:** A polished, publication-ready image. Sharp details, rich colors, intentional composition. The refiner pass has sharpened the desk edge, made the plants crisper, and enhanced the light effects. + +**Production command (all-in-one):** +```bash +python simple_config.py \ + --prompt "A developer in a modern glass-top desk by a sunny window, surrounded by plants and a hot cup of tea, morning light pouring in, warm watercolor, peaceful, serene, no text, no people visible outside" \ + --negative-prompt "blurry, bad quality, worst quality, low resolution, text, watermark, deformed, ugly, duplicate, morbid, oversaturated, cluttered, busy, too many objects" \ + --preset production \ + --style watercolor \ + --size blog-hero \ + --modifier crisper \ + --seed 52 \ + --output outputs/blog-hero-final.png \ + --cpu +``` + +--- + +### Example 2: Character Portrait (768×1024, Portrait Orientation) + +**Goal:** Create a character illustration for a fantasy story — a wise elder with kind eyes and weathered features. + +#### STEP 1: Simple Draft +```bash +python simple_config.py \ + --prompt "A wise elder with kind eyes" \ + --preset quick-draft \ + --size portrait \ + --seed 100 \ + --output outputs/character-v1.png \ + --cpu +``` + +**What you'd see:** A simple, abstract portrait. The character is vaguely recognizable but not detailed. + +--- + +#### STEP 2: Better +```bash +python simple_config.py \ + --prompt "An elderly wizard with kind blue eyes, weathered face, long white beard, wise expression, fantasy art style, detailed face, no hat" \ + --preset standard \ + --size portrait \ + --modifier more-detailed \ + --seed 100 \ + --output outputs/character-v2.png \ + --cpu +``` + +**What changed:** +- Specific features (blue eyes, white beard, wizard) +- `--modifier more-detailed`: Adds 10 steps + auto-enables refiner for extra polish +- Details matter for faces: mention eyes, expression, facial hair + +**What you'd see:** A recognizable character with personality. The beard texture is visible, eyes are expressive. + +--- + +#### STEP 3: Beautiful +```bash +python simple_config.py \ + --prompt "A wise elderly wizard with kind, penetrating blue eyes, deeply weathered face with laugh lines, long flowing white beard, serene expression, ornate robes, fantasy art, highly detailed, perfect face anatomy, no hat, no hat covering face" \ + --negative-prompt "blurry, low quality, text, deformed face, asymmetrical face, bad anatomy, too young, cartoonish, watermark, oversaturated" \ + --preset production \ + --modifier more-detailed \ + --modifier crisper \ + --seed 100 \ + --output outputs/character-final.png \ + --cpu +``` + +**What changed:** +- Hyper-specific (laugh lines, serene expression, ornate robes) +- "Perfect face anatomy" (prevents SDXL's occasional hand/face issues) +- Stacked modifiers: `more-detailed` + `crisper` for maximum sharpness + steps +- Detailed negative prompt focuses on face quality + +**What you'd see:** A production-ready character. Eyes have depth, beard has texture, robes are ornate, face is anatomically sound. + +--- + +### Example 3: Product Mockup (1024×1024, Photorealistic) + +**Goal:** Generate a product image for a tech company — sleek headphones on a minimalist white desk with subtle lighting. + +#### STEP 1: Simple Draft +```bash +python simple_config.py \ + --prompt "Sleek headphones on a white desk" \ + --preset quick-draft \ + --modifier photorealistic \ + --size square \ + --seed 200 \ + --output outputs/product-v1.png \ + --cpu +``` + +**What you'd see:** Basic headphones shape, generic desk. Useful for confirming the product concept works. + +--- + +#### STEP 2: Better +```bash +python simple_config.py \ + --prompt "Premium wireless headphones with matte black finish and gold accents, resting on a minimalist white marble desk, soft studio lighting from the left, no cable, no text, no branding visible" \ + --preset standard \ + --modifier photorealistic \ + --modifier crisper \ + --size square \ + --seed 200 \ + --output outputs/product-v2.png \ + --cpu +``` + +**What changed:** +- Specific materials (matte black, gold accents, marble) +- Lighting direction ("from the left") gives the image depth +- Negatives prevent mistakes (no cable, no branding) +- `photorealistic` modifier bumps guidance to 9.0 + +**What you'd see:** A recognizable, studio-quality product photo. The headphones have sheen, the marble has subtle reflection, lighting creates depth. + +--- + +#### STEP 3: Beautiful +```bash +python simple_config.py \ + --prompt "Premium wireless headphones, matte black with brushed gold accents, detailed ear cups, resting on pristine white marble desktop, soft diffused studio lighting from upper left, shadows cast gently on marble, shallow depth of field background, product photography, highly detailed, sharp focus on product" \ + --negative-prompt "blurry, low quality, text, watermark, cables, branding, fingerprints, dust, distorted product, asymmetrical, oversaturated, too dark, cluttered background" \ + --preset production \ + --modifier photorealistic \ + --modifier crisper \ + --size square \ + --seed 200 \ + --output outputs/product-final.png \ + --cpu +``` + +**What changed:** +- Professional photography language (shallow depth of field, product photography, sharp focus) +- Specific lighting description (diffused, upper left, gentle shadows) +- Material details (brushed gold, pristine marble) +- Production preset ensures maximum polish + +**What you'd see:** Magazine-quality product photography. The headphones are sharp, the desk material is pristine, lighting is professional, depth of field separates product from background. + +--- + +### Example 4: Abstract/Artistic (1024×1024, Experimental) + +**Goal:** Generate abstract art for a technology blog — something that evokes "data flow" without being literal. + +#### STEP 1: Simple Draft +```bash +python simple_config.py \ + --prompt "Abstract data visualization" \ + --preset quick-draft \ + --modifier dreamier \ + --size square \ + --seed 300 \ + --output outputs/abstract-v1.png \ + --cpu +``` + +**What you'd see:** Vague color patterns. Establishes mood but not refined. + +--- + +#### STEP 2: Better +```bash +python simple_config.py \ + --prompt "Abstract flowing data, bioluminescent colors, glowing lines intersecting, deep blues and purples, energy radiating outward, digital art, modern, no text" \ + --preset standard \ + --modifier artistic \ + --modifier softer \ + --size square \ + --seed 300 \ + --output outputs/abstract-v2.png \ + --cpu +``` + +**What changed:** +- Poetic language (bioluminescent, radiating) +- Specific colors (blues, purples) +- `artistic` modifier + `softer` guidance creates more freedom +- "Digital art" sets the style + +**What you'd see:** Flowing, colorful abstract patterns. Less dreamlike, more intentional. Energy is visible. + +--- + +#### STEP 3: Beautiful +```bash +python simple_config.py \ + --prompt "Abstract digital art of flowing data and information, bioluminescent energy lines glowing in deep blues, purples, and teals, complex geometric patterns intersecting and branching, sense of motion and growth, futuristic, no text, ethereal and energetic atmosphere" \ + --negative-prompt "blurry, low quality, text, watermark, faces, people, animals, recognizable objects, too literal, oversaturated, chaotic, muddy colors" \ + --preset production \ + --modifier artistic \ + --modifier softer \ + --size square \ + --seed 300 \ + --output outputs/abstract-final.png \ + --cpu +``` + +**What changed:** +- Layered poetic details (branching, sense of motion) +- Production preset with artistic + softer for maximum expression +- Explicit negatives prevent literal interpretations (no faces, no animals) + +**What you'd see:** A stunning, cohesive abstract image. Colors blend smoothly, patterns feel intentional and complex, energy flows. Gallery-quality. + +--- + +### Example 5: Landscape/Environment (1280×720, Wide/Cinematic) + +**Goal:** Generate a cinematic landscape for a travel blog — a misty mountain valley at dawn. + +#### STEP 1: Simple Draft +```bash +python simple_config.py \ + --prompt "Mountain valley at sunrise" \ + --preset quick-draft \ + --size wide \ + --seed 400 \ + --output outputs/landscape-v1.png \ + --cpu +``` + +**What you'd see:** Recognizable landscape, but soft and impressionistic. Establishes composition. + +--- + +#### STEP 2: Better +```bash +python simple_config.py \ + --prompt "Misty mountain valley at dawn, layers of blue mountains receding into distance, golden morning light breaking through mist, coniferous forest in foreground, soft atmospheric perspective, serene, no text, no people" \ + --preset standard \ + --modifier crisper \ + --size wide \ + --seed 400 \ + --output outputs/landscape-v2.png \ + --cpu +``` + +**What changed:** +- Atmospheric depth language (layers, receding, atmospheric perspective) +- Specific elements (coniferous forest, golden light) +- `crisper` modifier for sharpness on distant details + +**What you'd see:** A cohesive landscape with clear depth. The foreground is detailed, mountains recede into misty distance, lighting creates mood. + +--- + +#### STEP 3: Beautiful +```bash +python simple_config.py \ + --prompt "Cinematic landscape: misty mountain valley at dawn, multiple layers of blue-purple mountains receding into soft mist, golden sunlight breaking through clouds and illuminating a coniferous forest in the foreground, perfect atmospheric perspective, volumetric light rays, calm peaceful mood, no text, no people, highly detailed, photorealistic landscape painting" \ + --negative-prompt "blurry, low quality, text, watermark, people, animals, buildings, roads, too much detail, oversaturated, too dark, muddy colors, flat composition" \ + --preset production \ + --modifier crisper \ + --modifier more-detailed \ + --size wide \ + --seed 400 \ + --output outputs/landscape-final.png \ + --cpu +``` + +**What changed:** +- Professional photography language (cinematic, volumetric light rays) +- "Photorealistic landscape painting" sets high bar for detail +- Production preset + stacked modifiers for maximum quality +- Detailed negatives (no buildings, roads, etc.) + +**What you'd see:** A breathtaking, publication-quality landscape. Atmospheric effects are present (volumetric light), depth is profound, colors are rich, details are sharp. Ready for print or web. + +--- + +## Section 3: Parameter Deep Dive + +You've seen these in action; now let's understand them deeply. + +### The Four Core Parameters + +#### **`--steps`** (How many refinement passes?) +- **Default:** 22 +- **Range:** 1 to infinity (but 50+ has diminishing returns) +- **What it controls:** Number of denoising iterations the base model runs +- **More steps = higher quality, more detail, slower, more GPU memory** +- **Analogy:** Like brush strokes in painting. More strokes = more refined, but 100 strokes don't make a painting 5× better than 20. + +**When to adjust:** +- `10–15`: Quick experiments, iteration loops (fast) +- `22` (default): Balanced quality and speed +- `30–40`: Detailed, final images +- `50+`: Diminishing returns; use refiner instead for polish + +--- + +#### **`--guidance`** (How strictly follow the prompt?) +- **Default:** 6.5 +- **Range:** 0 to 20+ +- **What it controls:** Classifier-free guidance scale (prompt adherence) +- **0 = creative chaos; 6.5 = balanced; 10+ = very strict** + +**Analogy:** A director giving notes. Guidance 0 = actor ignores all notes and improvises wildly. Guidance 9 = actor follows every note rigidly, sometimes awkwardly. + +**When to adjust:** +- `3–4` (`dreamier`): More creative, looser interpretation, experimental +- `5–6`: Balanced +- `7–8` (`crisper`/`sharper`): Strict adherence, predictable +- `9+` (`photorealistic`): Lock to prompt, may look stiff + +--- + +#### **`--refine`** (Enable two-stage polish?) +- **Default:** Off +- **Type:** Boolean flag (`--refine` to enable) +- **What it does:** Adds a refinement stage after base model completes + +**How it works:** +1. Base model runs for 80% of denoising steps, outputs raw latents (unfinished work) +2. Base unloads from GPU (frees memory) +3. Refiner model loads and completes final 20% (adds polish) + +**Results:** Sharper edges, better hand anatomy, more polished overall appearance. + +**Cost:** Slower (two model loads) = ~1.5–2× total time; uses more VRAM. + +**When to use:** +- ✅ You have enough GPU VRAM (refiner is memory-intensive) +- ✅ Final output matters (blog heroes, covers, portfolio pieces) +- ✅ You're not iterating quickly +- ❌ You're prototyping or hit OOM errors +- ❌ Speed is your priority + +--- + +#### **`--refiner-steps`** (How many refinement passes in stage 2?) +- **Default:** 10 +- **Range:** 1 to 50+ +- **What it controls:** Denoising iterations for the refiner model (only used when `--refine` enabled) +- **Independent from `--steps`** (can have 35 base steps + 12 refiner steps) + +**When to adjust:** +- `5–8`: Quick refinement pass +- `10` (default): Good balance +- `15+`: Intensive polish, slower + +--- + +### Parameter Interaction Table + +This table shows common scenarios and what parameters work best together: + +| Goal | Steps | Guidance | Refine | Refiner Steps | Time | Best For | +|------|-------|----------|--------|---------------|------|----------| +| **Fast draft** | 15 | 5.0 | No | — | ~10 min | Idea validation | +| **Balanced (default)** | 22 | 6.5 | No | — | ~14 min | General use | +| **High quality** | 35 | 7.0 | No | — | ~19 min | Final images without polish | +| **Production polish** | 35 | 6.5 | Yes | 12 | ~28 min | Blog headers, print-ready | +| **Memory constrained** | 18 | 6.5 | No | — | ~12 min | Reduce OOM risk | +| **Strict adherence** | 30 | 9.0 | No | — | ~16 min | Photorealistic, precise | +| **Creative/dreamy** | 22 | 4.0 | No | — | ~14 min | Experimental, artistic | +| **Ultra-detailed** | 45 | 7.0 | Yes | 15 | ~35 min | Showcase piece | + +--- + +## Section 4: Pro Tips for Perfect Images + +### Negative Prompt Secrets + +A good negative prompt prevents common failures. Start with this baseline: + +``` +blurry, bad quality, worst quality, low resolution, text, watermark, +signature, deformed, ugly, duplicate, morbid +``` + +**Then customize based on your scenario:** + +**For portraits:** +``` +blurry, low quality, text, deformed face, asymmetrical face, bad anatomy, +twisted features, too many fingers, too many eyes, duplicate eyes, duplicate head +``` + +**For products:** +``` +blurry, low quality, text, watermark, cables, branding, fingerprints, +dust, distorted product, asymmetrical, oversaturated, too dark, cluttered +``` + +**For landscapes:** +``` +blurry, low quality, text, watermark, people, animals, buildings, +roads, too much detail, oversaturated, too dark, muddy colors, flat +``` + +**Golden rules for negatives:** +- Keep them concise (15–20 words maximum) +- Focus on things you *never* want +- Use for common failure modes (blurry, deformed, text) +- Test and refine based on outputs + +--- + +### Seed Reproducibility + +A **seed** locks the random number generator, making generations reproducible: + +```bash +python simple_config.py \ + --prompt "A peaceful lake at sunset" \ + --seed 42 \ + --output outputs/lake-seed-42.png \ + --cpu +``` + +Run it twice with `--seed 42`: you get identical images (same layout, colors, composition). + +**Why it matters:** +- **Iteration:** Generate once with seed, tweak prompt/params, regenerate with new seed, compare +- **Consistency:** Same seed across images ensures visual consistency (e.g., character appearance in a series) +- **Reproducibility:** Document your seed in notes for future reference + +**Tip:** Use meaningful seed numbers (e.g., 52 for a 5/2 birthday, 2024 for the year). + +--- + +### LoRA Selection & Custom Styling + +LoRAs are lightweight model adapters that inject a specific style. Examples: + +```bash +# List available LoRAs +python simple_config.py lora list + +# Use a watercolor LoRA (pre-configured) +python simple_config.py \ + --prompt "A garden in watercolor style" \ + --lora aether-watercolor \ + --output outputs/watercolor-garden.png \ + --cpu + +# Add a custom LoRA +python simple_config.py lora add "my-custom-style" \ + --id "huggingface-user/my-sdxl-lora" \ + --weight 0.7 \ + --models sdxl \ + --triggers "my custom style" \ + --description "My personal artistic style" + +# Use it +python simple_config.py \ + --prompt "A landscape in my custom style" \ + --lora my-custom-style \ + --output outputs/custom-landscape.png \ + --cpu +``` + +**Pro tip:** LoRA weight 0.6–0.8 is usually optimal. Higher (0.9+) oversaturates the style. + +--- + +### When to Use `--cpu` vs GPU + +**`--cpu` (slow, universal):** +- No GPU required +- ~10–28 minutes per image +- Useful for: Servers without GPU, CI/CD pipelines, laptops +- Use: When speed isn't critical + +**GPU mode (fast, requires setup):** +- NVIDIA (CUDA): ~2–8 minutes per image +- Apple (MPS): ~1–5 minutes per image +- AMD (ROCm): ~3–10 minutes per image +- Use: When speed matters, you have a GPU available + +**Rule of thumb:** +- If generating once or twice: `--cpu` is fine +- If batch processing (10+ images): use GPU + +--- + +### Troubleshooting OOM (Out of Memory) Errors + +If you get an OOM error, the model doesn't fit in your GPU memory: + +**Quick fixes (try in order):** +1. Add `--cpu` to use CPU instead +2. Reduce dimensions: `--width 768 --height 768` instead of 1024×1024 +3. Reduce steps: `--preset quick-draft` instead of `production` +4. Disable refiner: Remove `--refine` flag +5. Disable LoRA: Remove `--lora` flag + +**Example fallback:** +```bash +# If this fails with OOM: +python simple_config.py --prompt "..." --preset production --size wide + +# Try this instead: +python simple_config.py --prompt "..." --preset high-quality --size square --cpu +``` + +--- + +## Section 5: Next Steps & Resources + +### Links & Documentation + +- **README.md:** Full setup, GPU configuration, batch processing guides +- **prompts/examples.md:** 50+ pre-written prompts for different styles +- **simple_config.py:** Parameter translation and presets +- **generate.py:** Implementation details (for advanced users) + +### Advanced Workflows + +**Batch processing** (generate 10+ images at once): +```bash +python -m image_generation.cli.batch --batch-file batch-06192026-2.json --output outputs/batch/ +``` + +**CLIP preflight validation** (check prompt length before generating): +```bash +python -m image_generation.cli.check_clip --batch-file batch.json +``` + +**Custom presets** (create your own preset combinations): +Edit `simple_config.py` to add a new preset, or use profiles: +```bash +python simple_config.py --profile my-blog-series --prompt "..." +``` + +### Contributing & Community + +- **Found a bug?** Open an issue on GitHub +- **Want to add a LoRA?** Submit a PR to `loras.json` +- **Built something cool?** Share in discussions! + +### Final Thoughts + +SDXL is powerful, but it's a tool — not magic. The best images come from: + +1. **Clear prompts:** Specific, descriptive language (not vague wishes) +2. **Intentional parameters:** Understanding why you're changing steps/guidance +3. **Iteration:** Generating multiple times, comparing, refining +4. **Experimentation:** Testing new combinations, pushing boundaries + +You're now equipped to go from idea to image in 28 minutes — or 10 if you're in a hurry. The journey from draft to production is shorter than you think. + +**Happy generating! 🎨** + +--- + +*For questions, see the FAQ in README.md or open an issue on GitHub.* diff --git a/website/blog/2026-07-05-containerized-model-generation.md b/website/blog/2026-07-05-containerized-model-generation.md new file mode 100644 index 0000000..1faa1d1 --- /dev/null +++ b/website/blog/2026-07-05-containerized-model-generation.md @@ -0,0 +1,564 @@ +--- +title: "From Custom Local Setup to Elastic Cloud: Containerizing SDXL for Azure Container Apps" +date: 2026-07-05 +description: "I needed my exact model configs, not a vendor's API. Here's how I went from a local Python CLI to a production Flask API deployed on Azure—no lock-in, full control." +tags: ["AI", "DevOps", "Azure", "Docker", "SDXL", "Cloud Deployment", "Production"] +draft: true +--- + +## The Problem: Vendor APIs Weren't Mine + +I've been generating blog illustrations with Stable Diffusion XL for a few months now. My local setup works perfectly—custom prompts, specific seeds for reproducibility, precise step counts and guidance scales. I own the entire workflow. + +Then came the pressure to "scale." What if I needed 100 images one week and five the next? Running generation on my MacBook Pro works great when I'm home, but what about when I'm traveling? And the electricity cost of a local GPU running 24/7 adds up. + +The obvious answer: vendor APIs. Stability AI, Replicate, Amazon Bedrock—they all offer SDXL inference. But here's what I found myself asking: + +- Can I use *my exact* inference settings? (Often locked to presets.) +- Can I run this on my own infrastructure if I want to? (Usually locked to their cloud.) +- Will they keep offering SDXL at the same price? (Who knows.) +- Can I add custom LoRAs or refine the pipeline? (Usually no.) + +I realized I didn't actually need a vendor API. I needed **my code, in a container, running in elastic cloud infrastructure**. That's cheaper, more flexible, and surprisingly easy. + +This is my journey from local Python CLI to production Flask API on Azure Container Apps—and what I learned about containerization, cloud deployment, and cost control along the way. + +## Part 1: Understanding What I Already Had + +My local image generation pipeline is a Python CLI (`generate.py`) that: +- Loads SDXL base and optional refiner models from Hugging Face +- Takes text prompts and generates 1024×1024 PNG images +- Supports batch processing from JSON configs +- Auto-detects GPU (NVIDIA, Apple Silicon, CPU fallback) +- Handles out-of-memory gracefully by retrying with fewer steps + +**Example local usage:** +```bash +python generate.py --prompt "tropical sunset, oil painting" --steps 40 --seed 42 +``` + +Or batch mode: +```bash +python generate.py --batch-file batches/blog-images.json +``` + +Where `batches/blog-images.json` is: +```json +[ + { + "prompt": "underwater kingdom with coral castles, fantasy art", + "output": "outputs/underwater.png", + "seed": 42 + }, + { + "prompt": "floating islands in clouds, magical realism", + "output": "outputs/islands.png", + "seed": 43 + } +] +``` + +This works perfectly on my local machine. But it's not: +- **Accessible remotely** — I need to SSH in or run it locally +- **Elastic** — one request at a time, one machine, can't scale +- **Pay-as-you-go** — GPU costs whether I'm using it or not +- **Integrated** — hard to call from other apps or automate + +## Part 2: The Bridge—Flask + Docker + +The key insight: I don't need to rewrite my CLI. I just need to: +1. Wrap it in a web server (Flask) +2. Put the web server in a container (Docker) +3. Deploy the container to managed infrastructure (Azure Container Apps) + +**Step 1: Create a Flask wrapper** (`app.py`) + +```python +from flask import Flask, request, jsonify +from src.image_generation.generate import generate_with_retry +from types import SimpleNamespace + +app = Flask(__name__) + +@app.route("/generate", methods=["POST"]) +def generate_endpoint(): + config = request.get_json() + + results = [] + for prompt_obj in config.get("prompts", []): + args = SimpleNamespace( + prompt=prompt_obj["prompt"], + output=prompt_obj.get("output", "outputs/image.png"), + seed=prompt_obj.get("seed"), + steps=config.get("steps", 40), + guidance=config.get("guidance", 7.5), + width=config.get("width", 1024), + height=config.get("height", 1024), + refine=config.get("refine", False), + cpu=False + ) + + try: + output_path = generate_with_retry(args, max_retries=2) + results.append({ + "prompt": prompt_obj["prompt"], + "output": output_path, + "status": "ok" + }) + except Exception as e: + results.append({ + "prompt": prompt_obj["prompt"], + "error": str(e), + "status": "error" + }) + + return jsonify({ + "status": "success", + "results": results, + "timestamp": datetime.utcnow().isoformat() + }) + +if __name__ == "__main__": + app.run(host="0.0.0.0", port=8000) +``` + +That's it. Same CLI code, now exposed as `/generate POST` endpoint. + +**Test locally:** +```bash +python app.py +``` + +Then: +```bash +curl -X POST http://localhost:8000/generate \ + -H "Content-Type: application/json" \ + -d '{ + "prompts": [ + {"prompt": "tropical sunset", "seed": 42} + ], + "steps": 40, + "guidance": 7.5 + }' +``` + +Response: +```json +{ + "status": "success", + "results": [ + { + "prompt": "tropical sunset", + "output": "outputs/image_20260703_092314.png", + "status": "ok" + } + ], + "timestamp": "2026-07-03T09:23:14Z" +} +``` + +**Step 2: Containerize with Docker** + +```dockerfile +FROM nvidia/cuda:12.1-runtime-ubuntu22.04 + +ENV PYTHONUNBUFFERED=1 + +WORKDIR /app + +# Install dependencies +COPY requirements.txt . +RUN pip install -r requirements.txt + +# Copy code +COPY app.py . +COPY src/ ./src/ + +EXPOSE 8000 + +CMD ["python3", "app.py"] +``` + +Build it: +```bash +docker build -t sdxl-api:latest . +``` + +Run it: +```bash +docker run --gpus all -p 8000:8000 \ + --mount type=bind,source=$(pwd)/outputs,target=/app/outputs \ + sdxl-api:latest +``` + +Same API, now in a container. I can run it on any machine with Docker and a GPU. + +## Part 3: From Container to Cloud + +I chose Azure Container Apps because: +- **Managed Kubernetes** — no cluster management +- **GPU support** — up to 1 GPU vCPU per container, at $0.30/hour +- **Elastic scaling** — scales from 0 to N replicas, pay only for active time +- **HTTPS by default** — ingress configured automatically +- **Easy to define** — Bicep templates (IaC in Azure's language) + +### Infrastructure as Code (Bicep) + +Instead of clicking through the Azure portal, I defined everything in `infra/main.bicep`: + +```bicep +param location string = resourceGroup().location +param containerRegistryName string = 'sdxlregistry' +param containerAppName string = 'sdxl-generation-api' +param environmentName string = 'sdxl-env' + +// Create Container Registry (for hosting my Docker images) +module acr 'resources/acr.bicep' = { + name: 'acr-deployment' + params: { + location: location + containerRegistryName: containerRegistryName + } +} + +// Create Container Apps Environment +module caEnvironment 'resources/aca-env.bicep' = { + name: 'aca-env-deployment' + params: { + location: location + environmentName: environmentName + } +} + +// Create Container App (the actual service) +module containerApp 'resources/aca.bicep' = { + name: 'container-app-deployment' + params: { + location: location + containerAppName: containerAppName + containerAppsEnvironmentId: caEnvironment.outputs.environmentId + imageName: '${acr.outputs.loginServer}/sdxl-api:latest' + } +} + +output containerAppUrl string = containerApp.outputs.fqdn +``` + +The container app module specifies: +```bicep +scale: { + minReplicas: 0 # Scale to zero when idle (no cost) + maxReplicas: 1 # Max 1 replica (can handle 1 request at a time) +} + +resources: { + cpu: json('4') # 4 vCPU + memory: '16Gi' # 16 GB RAM for model loading +} +``` + +Key decision: **minReplicas: 0** means the container stops entirely when idle. I pay nothing when I'm not generating. First request takes ~10-30 seconds to start (cold start), but that's for the container itself—the models then need to be downloaded from Hugging Face (5-10 minutes total for first inference). Subsequent requests on the same container are fast. + +### Deployment with Azure Developer CLI (azd) + +Azure's `azd` tool orchestrates the entire deployment: + +```bash +# Authenticate +azd auth login + +# Initialize (creates resource group, configures subscription) +azd init + +# Deploy +azd up +``` + +What happens: +1. Docker image is built +2. Image is pushed to Azure Container Registry +3. Bicep template is deployed +4. Container App is created and started +5. Public HTTPS URL is generated +6. Health check is configured + +Output: +``` +✓ Building image 'sdxl-api:latest' +✓ Pushing image to 'sdxlregistry.azurecr.io' +✓ Deploying infrastructure +✓ Service deployed to https://sdxl-generation-api.victoriousforest-12ab.eastus.azurecontainerapps.io +``` + +That URL is live and accepts requests immediately. + +## Part 4: Using the Cloud Service + +Now I can call my API from anywhere: + +```bash +ENDPOINT="https://sdxl-generation-api.victoriousforest-12ab.eastus.azurecontainerapps.io" + +curl -X POST $ENDPOINT/generate \ + -H "Content-Type: application/json" \ + -d '{ + "prompts": [ + {"prompt": "a tropical sunset with palm trees, oil painting style", "seed": 42}, + {"prompt": "underwater coral reef, fantasy art", "seed": 43} + ], + "steps": 40, + "guidance": 7.5, + "width": 1024, + "height": 1024 + }' +``` + +The API queues both requests, generates both images (or one at a time if minReplicas: 0 means they're sequential), and returns results in ~60-90 seconds. + +If I suddenly needed to handle 10 concurrent requests, I'd edit `infra/resources/aca.bicep`: + +```bicep +maxReplicas: 10 # now can handle up to 10 parallel generations +``` + +Then: +```bash +azd deploy +``` + +Azure auto-scales to 10 replicas (10 containers × $0.30/hr each). If demand drops, it scales back down. **I only pay for what I use.** + +## Part 5: Cost Reality + +Here's what I'm actually paying: + +| Component | Cost | Notes | +|-----------|------|-------| +| GPU compute (active) | ~$0.30–0.50/hr | Only when generating; varies by region | +| Container Registry | ~$5–10/month | Depends on image size and updates | +| Model cache | Needs persistent storage | Models re-download after scale-to-zero (~$5/mo for Azure Files) | + +Without persistent model storage, **each scale-to-zero cycle means re-downloading SDXL (~7GB, 5-10 min latency)**. For budget-conscious deployments, I can: +- Pre-build the Docker image with models included (heavier image, eliminates runtime downloads) +- OR mount Azure Files for persistent `/root/.cache/huggingface` (adds cost but keeps cold start fast) + +**Example usage: 10 batches/month, 5 min each (realistic for blog illustrations)** +- Total GPU time: 50 min = 0.83 hours +- Compute cost: 0.83 × $0.30 = ~$0.25/month +- Plus ACR storage: ~$5/month +- Plus persistent model cache (if using Azure Files): ~$5/month +- **Total: ~$10/month for reliable, fast generation** + +By contrast: +- **Local GPU:** $3000–5000 hardware ÷ 36 months = $83–139/month amortized + electricity + maintenance +- **Vendor API (Replicate):** $0.025 per image = $0.25–2.50/month for 10–100 images, but limited customization + +**If I were running this 24/7**, Azure would cost ~$216–360/month (1 replica × $0.30–0.50/hr × 730 hours), plus persistent storage. But I'm not—I generate when I need images. The benefit: zero cost when idle. + +## Part 6: Portability vs Lock-In + +I often hear "containers are portable, zero lock-in." That's *partially* true here: + +**Portable:** +- The Docker container runs on any cloud (AWS, GCP, DigitalOcean) +- The Python code is framework-agnostic + +**Azure-specific:** +- The Bicep templates are Azure's Infrastructure as Code language +- Porting to AWS requires writing CloudFormation or Terraform (~2–4 hours of work) +- You'd need to replicate: Container Registry, Managed Environment, Container App, persistent storage + +**Practical take:** This architecture *can* move clouds, but it requires effort. The value isn't "zero lock-in"—it's **not being locked into a specific model vendor's API** (Stability AI, Replicate). I can swap SDXL for Llama, Mistral, or a custom fine-tune without paying a vendor premium. + +**Full control:** I can: +- Add custom LoRAs to the model +- Adjust step counts and guidance globally +- Implement persistent model caching to avoid re-downloads +- Add authentication if I expose this to teammates +- Switch models entirely (e.g., Stable Diffusion 3, custom fine-tunes) +- Monitor logs and performance +- Run batch inference efficiently + +**Deterministic:** Same seeds produce same images. Batch configuration is versioned. I'm not at the mercy of a vendor changing their API. + +**Scalable:** From 0 to N replicas with one parameter change. Batch inference becomes trivial—instead of running 100 prompts one at a time locally, I can send them to my cloud API and let Azure handle concurrency. + +## Part 7: What I Learned + +1. **Containerization isn't just for production.** It's for portability and reproducibility. Even local development benefits. + +2. **Managed services are worth it.** Azure Container Apps handles scaling, networking, and SSL. I focus on my code. + +3. **Infrastructure as Code (Bicep) scales better than ClickOps.** I can redeploy in seconds, version control everything, and reproduce environments exactly. + +4. **Cold start vs model download are different problems.** Container startup is ~10–30s. Model download is 5–10 minutes. For batch work, I tolerate this. For real-time APIs, I'd use persistent storage or pre-download models. + +5. **Model persistence is essential at scale.** With scale-to-zero, containers are destroyed when idle. Models must be either: + - Pre-built into the Docker image (heavier, but always-fast) + - Mounted from persistent storage (adds cost, but smaller image) + +6. **Monitoring is critical.** I set up health checks (`GET /health`) so Azure can detect failures and restart. Logs go to stdout for Container Apps Log Analytics. + +7. **Scaling is a cost decision.** Each replica consumes full GPU resources. With 5 replicas, I'm budgeting $150–250/mo, not $10/mo. + +## Part 8: Real-World Complications I Encountered + +When I first deployed this, I hit a few snags that taught me a lot. + +**OOM on first request:** The model (~7GB) plus working memory can exceed available GPU RAM on startup. I solved this by: +- Implementing automatic retry logic that halves inference steps on OOM (40 → 20 → 10 steps) +- Pre-building the Docker image with models downloaded (heavier image, but eliminates runtime downloads) +- Allocating sufficient GPU memory (4 vCPU + 16GB) + +The retry is elegant: if I request 40 steps and hit OOM, the app auto-retries with 20 steps, then 10. The user still gets a result; it's just lower quality. + +**Cold start + model download latency:** First request after scaling to zero takes: +- ~10–30 seconds for container startup +- ~5–10 minutes for initial model download (if not pre-cached) +- **Total: 5–10 minutes before generating starts** + +For batch work, this is acceptable. For real-time UIs, it's too slow. I'm implementing persistent Azure Files mounts to cache models between scale-to-zero cycles. + +**Image persistence:** Generated images live in `/app/outputs/` inside the container. If the container restarts, they're gone. I solve this by: +- Mounting generated images to Azure Blob Storage (future enhancement) +- Or keeping them small (~5MB per 1024×1024 PNG) and accepting that they're ephemeral + +**Scaling beyond 1 replica:** If I need 5 replicas, each one is a separate container with its own Python interpreter and model cache. That's 5 × 16GB = 80GB memory. Azure enforces this automatically, so scaling is a conscious cost decision. + +**API versioning:** Changing inference parameters (adding new endpoints, changing response format) requires container rebuilds. I version my API in the URL path (`/v1/generate`, `/v2/generate`) to avoid breaking changes. + +## Part 9: Beyond SDXL—Generalizing the Approach + +This isn't just for SDXL. The architecture works for any model type: + +- **LLMs:** Wrap Llama 2, Mistral, or custom fine-tunes in Flask +- **Audio models:** Voice cloning, music generation +- **Computer vision:** Object detection, segmentation, classification +- **Retrieval:** Embed documents and expose vector search +- **Multimodal:** Image → text, text → speech pipelines + +The pattern is always: +1. Local Python code that works +2. Flask wrapper for HTTP interface +3. Docker for reproducibility +4. Bicep for infrastructure +5. `azd up` to deploy + +The only changes are: +- Adjust container resources (CPU, memory) based on model size +- Modify maxReplicas based on expected concurrency +- Add authentication if sharing with teammates +- Update requirements.txt with model-specific dependencies + +## Part 10: Should You Build This? + +This approach is ideal if: +- You have **custom model configurations** (specific seeds, steps, guidance values) +- You need **elastic scaling** (bursty usage patterns) +- You want **full control** over the inference pipeline +- You're cost-conscious and need to **pay only for what you use** +- You dislike **vendor lock-in** + +It's probably overkill if: +- You're calling a model **once per month** +- You don't care about **reproducibility** (seeds, exact parameters) +- You're okay with **vendor APIs** (simpler integration) +- You need **sub-second latency** (cold starts are slow) +- You need **GPUs 24/7** (local machine is cheaper than always-on cloud) + +For everything in between, this pattern is the sweet spot: production-ready, cost-efficient, and fully under your control. + +## Part 11: Monitoring and Observability in Production + +I quickly realized that deploying to Azure is only half the battle. Knowing what's happening in production is the other half. + +I set up: +- **Health checks:** `/health` endpoint that `curl`s back to verify the container is alive +- **Structured logging:** Every generation request logs `{"timestamp": "...", "prompt": "...", "duration_ms": ..., "status": "ok"|"error"}` +- **Alerts:** Azure Monitor can alert on failed requests, slow responses, or container restarts +- **Log aggregation:** Container Apps pipes all stdout/stderr to Log Analytics Workspace (queryable with KQL) + +Example query to find slow generations: +```kusto +ContainerAppConsoleLogs +| where Log contains "Generated" +| extend duration = extract(@"duration_ms.(\d+)", 1, Log) +| where duration > 120000 +| project TimeGenerated, duration +``` + +This simple observability caught two issues: +1. **GPU driver was outdated** — inference took 3× longer than expected +2. **Model cache wasn't persistent** — first generation on each new container was 2–3x slower + +Both were fixable with infrastructure changes. + +## Part 12: The Economics in Detail + +Let me break down the actual monthly cost more precisely, because it surprised me. + +**Scenario: 500 image generations per month, 40 steps each** + +Azure Container Apps pricing (US East): +- GPU vCPU: $0.30/hour +- Memory: included in compute cost +- Data transfer: free within Azure (egress is $0.087/GB, but my PNGs stay in Blob Storage) + +Inference time per image: ~1.5 minutes (model already loaded) to ~5 minutes (cold start with model load) + +Conservative estimate: average 3 min per image +- 500 images × 3 min = 1500 min = 25 hours/month active +- 25 hours × $0.30/hr = **$7.50/month compute** + +Container Registry storage: ~$0.10/day = **$3/month** + +**Total: ~$10.50/month** + +Compare to alternatives: +- **Replicate API:** $0.025 per image (no setup) = $12.50/month (vs. my $10.50) +- **Local GPU:** $5k GPU ÷ 36 months = $139/month + $20/month electricity = **$159/month** +- **Amazon SageMaker:** $0.50/hour ml.g4dn.xlarge + setup overhead = **$50–100/month** + +Azure is cheapest if you have bursty usage. Local GPU wins if you're generating 8+ hours/day. Replicate wins on simplicity, not cost. + +## Part 13: What's Next + +Future improvements I'm considering: + +1. **Persistent model cache:** Store models in Azure Blob Storage so containers don't re-download (~10 min saved per cold start) +2. **API authentication:** Add JWT tokens so only I (and authorized teammates) can call `/generate` +3. **Webhooks:** Make the API async—submit a batch, get a callback when done +4. **Cost optimization:** Use spot instances (Azure Spot VMs) for non-critical batches (70% cheaper, can be preempted) +5. **Multi-model support:** Host SDXL, Stable Diffusion 2, and a custom fine-tune on the same endpoint +6. **Batch job scheduling:** Integrate with Azure Batch or Logic Apps to trigger generation on a schedule + +Each of these is a ~2-hour implementation change because the base is solid. + +## The Full Setup + +If you want to replicate this: + +1. **Copy my local generation code** to a new Git repo +2. **Wrap it in Flask** (expose `/generate` endpoint) +3. **Create a Dockerfile** using `nvidia/cuda:12.1-runtime-ubuntu22.04` as base +4. **Write Bicep templates** for Container Registry and Container Apps +5. **Use `azd up`** to deploy everything + +It's ~200 lines of Flask, ~100 lines of Bicep, and a Dockerfile. Total time: 2–3 hours if you're new to this; 30 min if you've done it before. + +## Closing: The Trade-off + +Vendor APIs are faster to integrate. My approach requires more infrastructure knowledge. But the payoff—full control, elastic scaling, cost efficiency, no lock-in—is worth it. + +For my use case (generating 100–1000 images/month on demand), this costs ~$10/month and gives me complete ownership over my inference pipeline. + +If you're generating images regularly with custom configs, this might be your answer too. + +--- + +**Next steps:** +- Check out the [image-generation-ai repo](https://github.com/dfberry/image-generation-ai) for a complete, ready-to-deploy scaffold +- Read Azure's [Container Apps docs](https://aka.ms/aca-docs) for deeper customization +- Explore [Hugging Face Diffusers](https://huggingface.co/docs/diffusers) for model alternatives + +Let me know if you go this route or find a better way. I'm always learning. diff --git a/website/blog/2026-07-05-local-vs-cloud-image-generation.md b/website/blog/2026-07-05-local-vs-cloud-image-generation.md new file mode 100644 index 0000000..fefcf7f --- /dev/null +++ b/website/blog/2026-07-05-local-vs-cloud-image-generation.md @@ -0,0 +1,105 @@ +--- +title: "Three Ways to Run Models: API, Rented Hardware, or Your Own Machine" +date: 2026-07-05 +author: Niobe +description: "I've been thinking about the choices for running models—image generation, LLMs, speech, all of it. Here's how I think about the three paths and when each makes sense." +draft: true +--- + +# Three Ways to Run Models: API, Rented Hardware, or Your Own Machine + +I keep running into the same question from different angles: Should I use an API? Rent a GPU? Buy my own hardware? + +The question sounds simple until you realize there's no universal answer. It depends entirely on what you're actually doing—how often, at what scale, and what you care about. + +So I stopped and pulled it apart. Here's what I found. + +--- + +## The Three Real Paths + +Whether you're working with image generation (DALL-E, Stable Diffusion), language models (Claude, Llama), voice synthesis, or anything else, you have three genuine options. + +### Path 1: Cloud APIs + +This is the straightforward path. You use DALL-E, ChatGPT, Midjourney, or any API-based service. You authenticate, make a request, and get a result back in seconds. Setup takes minutes—literally just signing up and getting an API key. + +The appeal is obvious: zero infrastructure to manage, and the latest models automatically. You don't think about hardware or drivers or disk space. Someone else handles all of that. + +The cost structure is clean: you pay for what you use. A few cents per request or per image. If you're experimenting or building something small, this feels like almost nothing. + +But here's the thing: cost scales linearly with volume. Use more, pay more. There's no pricing ceiling. And when the provider updates a model, you get the update whether you want it or not. If something breaks or changes, you adapt or you're stuck. + +### Path 2: Rented Hardware + +This is the middle path. You rent GPU time or a container from someone like Runpod, Modal, AWS, or Azure. You have full control over which model you run and which version you use. You don't have to buy hardware. + +Setup takes a bit longer—maybe 10 or 20 minutes to authenticate and configure a container. But once that's done, you can run whatever model you want on whatever schedule you want. If a new model comes out and you're skeptical, you don't have to adopt it. You stay on what works. + +The cost is usually cheaper than cloud APIs if you're doing any serious volume. You pay for compute time. At moderate scale, this often hits a sweet spot: cheaper than APIs, but without the upfront hardware investment. + +The catch is that you need some technical comfort with containers and cloud infrastructure. It's not as hands-off as an API. You're managing something, even if it's someone else's server. + +### Path 3: Your Own Hardware + +This is the longest-term path. You buy a GPU—probably a few hundred to a couple thousand dollars depending on what you want—and run models locally on your machine. After that initial investment, the cost per use is just electricity. Pennies. + +You have complete control. You choose which models to run, which versions to keep, when to upgrade. Nothing leaves your machine. If you care about privacy, this is the only path where you truly have it. + +But inference is slower. Depending on your hardware, you might be waiting minutes instead of seconds. You manage the infrastructure: drivers, updates, troubleshooting when things break. You're responsible for keeping current with model versions. + +And there's the upfront cost, which matters. If you're only running models occasionally, paying for a GPU doesn't make sense. But if you're planning to use this for years and doing it at any real volume, it pays for itself. + +--- + +## What Actually Matters + +I think the comparison gets clearer when you stop looking at each in isolation and start thinking about what you're optimizing for. + +**Privacy** is one thing. Cloud APIs log your requests. Rented hardware keeps things in a container that's still on someone else's server. Only your own hardware keeps everything completely local. If you're working with confidential data or you simply prefer not to send things to external servers, there's only one real path. + +**Control** is another. Cloud APIs force you to adapt to whatever the provider decides. If they change a model's behavior, you get the change. Rented hardware and your own machine both give you control over which versions you run and when you upgrade. That matters if consistency is important or if you've built workflows around specific model behavior. + +**Speed** is obvious but worth stating clearly: cloud is fastest (seconds), rented hardware is middle ground, your own machine is slowest. But slowness is relative. If your model takes a few minutes to run, that might be completely acceptable depending on the context. + +**Cost** depends on volume. At low volume, cloud is cheapest because there's no upfront investment. As volume increases, rented hardware becomes more economical. At high volume over several years, your own hardware probably wins. + +But here's what I actually think matters most: **understanding the direction each path pulls you**. Cloud APIs pull you toward convenience and hands-off simplicity. Rented hardware pulls you toward balance. Your own hardware pulls you toward independence and control. + +--- + +## Examples Across Different Model Types + +These three paths apply to everything. The trade-offs don't change just because you're switching from images to language models. + +If you're generating images, you might use DALL-E (cloud API), run Flux on Runpod (rented hardware), or install Stable Diffusion locally via ollama (your machine). + +If you're working with language models, you might use ChatGPT's API (cloud), run Llama on Together AI or Replicate (rented compute), or use ollama running Mistral or Llama locally (your machine). + +Speech and voice synthesis work the same way: cloud APIs like Eleven Labs, containers running open-source TTS, or local Whisper running on your hardware. + +The specific tools change. The trade-offs stay identical. + +--- + +## Thinking Through Your Actual Situation + +Here's how I'd think through the decision. + +**First, how often are you actually running this?** If you're using it a few times a week and experimenting, a cloud API is probably the right move. If you're running it hundreds of times a month, the economics shift dramatically. If you're running it thousands of times a month, your own hardware starts looking inevitable. + +**Second, what's your priority?** If you need instant results and don't want to manage infrastructure, cloud is the answer. If you want model control without hardware investment, rented compute is worth exploring. If you're planning for the long term and privacy or control matters, your own hardware makes sense. + +**Third, what constraints actually matter to you?** Speed might not matter if you're batch processing. Privacy might not matter if your work is public. Cost might not be the deciding factor if it's a side project and convenience is worth it. + +**Fourth, do you already have suitable hardware?** If you've got a GPU sitting there, the economics of your own hardware immediately become more attractive. If you don't, factor in the upfront cost. + +--- + +## The Honest Answer + +There's no objectively "best" option here. I keep coming back to that. + +Cloud APIs are genuinely the right choice for a lot of situations. I use them regularly when I want simplicity or when I'm exploring something new. Rented hardware makes sense if you're doing enough volume that the setup effort pays off. Your own hardware is right if you're building something you plan to maintain for years. + +I'm genuinely interested in how people are actually handling this trade-off. Where does the friction appear? What's the decision point where you switched from one path to another? I'm learning as I go here, same as everyone else. diff --git a/website/blog/2026-07-05-model-deployment-options.md b/website/blog/2026-07-05-model-deployment-options.md new file mode 100644 index 0000000..e7e0ee5 --- /dev/null +++ b/website/blog/2026-07-05-model-deployment-options.md @@ -0,0 +1,73 @@ +--- +title: "Three Ways to Run Models: Pick the One That Fits" +date: 2026-07-05 +author: Niobe +description: "A straightforward look at cloud APIs, rented hardware, and your own machine—and when each one actually makes sense." +draft: true +--- + +# Three Ways to Run Models: Pick the One That Fits + +I keep running into the same question from different angles: Should I use an API? Rent a GPU? Buy my own hardware? + +The question sounds simple until you realize there's no universal answer. It depends entirely on what you're actually doing—how often, at what scale, and what you care about. + +So I stopped and pulled it apart. Here's what I found. + +## The Three Real Paths + +Whether you're working with image generation, language models, speech synthesis, or anything else, you have three genuine options. + +**Cloud APIs** are the straightforward path. You use ChatGPT, DALL-E, Midjourney, or any API-based service. Setup takes minutes—just sign up and get an API key. The appeal is obvious: zero infrastructure to manage, and the latest models automatically. You pay for what you use. A few cents per request. If you're experimenting or building something small, this feels like almost nothing. + +The catch: cost scales linearly with volume. Use more, pay more. There's no pricing ceiling. And when the provider updates a model, you get the update whether you want it or not. + +**Rented hardware** is the middle path. You rent GPU time from services like Runpod, Modal, AWS, or Azure. Setup takes longer—maybe 15 minutes to configure a container—but once that's done, you have full control over which model you run and which version you use. You don't have to adopt every update. You stay on what works. + +The cost is usually cheaper than cloud APIs at moderate volume. The catch is that you need some technical comfort with containers and cloud infrastructure. It's not as hands-off as an API. + +**Your own hardware** is the longest-term path. You buy a GPU—a few hundred to a couple thousand dollars—and run models locally on your machine. After that initial investment, the cost per use is just electricity. You have complete control. Nothing leaves your machine. If privacy matters, this is the only path where you truly have it. + +But inference is slower—you might be waiting minutes instead of seconds. You manage the infrastructure: drivers, updates, troubleshooting. And there's that upfront cost, which matters if you're only running models occasionally. But if you're planning to use this for years and doing it at any real volume, it pays for itself. + +## What Actually Matters + +**Privacy** is one thing. Cloud APIs log your requests. Rented hardware keeps things in a container on someone else's server. Only your own hardware keeps everything completely local. If you're working with confidential data or you simply prefer not to send things to external servers, there's only one real path. + +**Control** is another. Cloud APIs force you to adapt to whatever the provider decides. Rented hardware and your own machine both give you control over which versions you run and when you upgrade. That matters if consistency is important or if you've built workflows around specific model behavior. + +**Speed** is obvious: cloud is fastest (seconds), rented hardware is middle ground, your own machine is slowest. But slowness is relative. If your model takes a few minutes to run, that might be completely acceptable depending on the context. + +**Cost** depends on volume. At low volume, cloud is cheapest because there's no upfront investment. As volume increases, rented hardware becomes more economical. At high volume over several years, your own hardware probably wins. + +But here's what actually matters most: understanding the direction each path pulls you. Cloud APIs pull you toward convenience and hands-off simplicity. Rented hardware pulls you toward balance. Your own hardware pulls you toward independence and control. + +## When to Pick Each One + +**Use cloud APIs if:** You're experimenting. You need instant results. You don't want to manage infrastructure. You're okay with using whatever model the provider currently recommends. You're running this a few times a week at most. + +**Use rented hardware if:** You need model control without a large hardware investment. You're running models hundreds of times a month. You want to stick with a specific version or model. You have the technical comfort to configure containers. You want something cheaper than APIs but don't want to buy a GPU. + +**Use your own hardware if:** You care about privacy. You're running this thousands of times a month. You're building something you plan to maintain for years. You already own suitable hardware. You want complete independence from a provider's decisions. + +## The Framework Applies Everywhere + +These three paths apply to everything. The trade-offs don't change just because you're switching from images to language models. + +If you're generating images, you might use DALL-E (cloud API), run Flux on Runpod (rented hardware), or install Stable Diffusion locally via ollama (your machine). + +If you're working with language models, you might use ChatGPT's API (cloud), run Llama on Together AI (rented compute), or use ollama running Llama locally (your machine). + +Speech and voice synthesis work the same way: cloud APIs, containers running open-source TTS, or local Whisper on your hardware. + +The specific tools change. The trade-offs stay identical. + +## What Changed My Thinking + +I spent a lot of time looking at this as a pure economics problem—break-even points, amortization, per-use costs. That's useful, but it misses something important. + +The real question isn't which is cheapest. It's which one aligns with how you actually work and what you actually care about. + +Some people will always prefer cloud APIs because infrastructure stress isn't worth the savings. Some people need local hardware for privacy and are willing to manage it. Most people probably sit somewhere in the middle. + +I'm genuinely interested in how people are actually handling this trade-off. Where does the friction appear? What's the decision point where you switched from one path to another? I'm learning as I go here, same as everyone else. diff --git a/website/blog/2026-07-05-sdxl-azure-container-apps.md b/website/blog/2026-07-05-sdxl-azure-container-apps.md new file mode 100644 index 0000000..29186a9 --- /dev/null +++ b/website/blog/2026-07-05-sdxl-azure-container-apps.md @@ -0,0 +1,195 @@ +--- +title: "Running Custom Image Generation at Scale: SDXL on Azure Container Apps" +date: 2026-07-05 +author: Niobe +description: "I containerized my local SDXL setup and deployed it to Azure Container Apps. No vendor lock-in, complete control, and you only pay when it runs." +draft: true +--- + +# Running Custom Image Generation at Scale: SDXL on Azure Container Apps + +I've been doing image generation locally for a while—SDXL with custom LoRAs, specific presets, and precise controls over inference steps and seeds. Everything is reproducible. Everything is mine. + +Then I hit a real constraint: I needed to scale beyond what my local GPU could handle, but cloud APIs (DALL-E, Midjourney) wouldn't let me keep that control. They're black boxes. LoRAs don't exist. Seed control doesn't exist. Preset configurations don't exist. + +So I did what I should have done from the start: I containerized my setup and ran it on Azure Container Apps. Same configs, same results, same complete control—but now I can generate hundreds of images without buying more hardware. + +This is what actually makes sense for this specific problem. + +--- + +## The Setup I Built + +My local workflow is a Python CLI with SDXL: + +```yaml +images: + - prompt: "A glowing tropical garden with deep magenta flowers..." + seed: 42 + output: "outputs/quickstart/garden.png" + width: 1200 + height: 632 + preset: quick-draft + style: watercolor + lora: + - name: aether-watercolor + weight: 0.8 + modifiers: + - more-detailed + - crisper +``` + +It runs locally with `python -m image_generation.cli.generate --batch-file-yaml batch.yaml`. I control everything. + +None of that exists on a cloud API. + +So instead of abandoning my setup, I containerized it: + +1. **Dockerfile** — Package my SDXL code + dependencies (torch, diffusers, LoRA weights) +2. **Flask wrapper** — Add a thin HTTP layer so I can submit batches remotely +3. **Azure Container Apps** — Deploy the container to scale to zero when idle + +The batch configs don't change. The prompts don't change. The results are identical. I just call it remotely now. + +--- + +## Why This Matters: The Missing Middle Ground + +When I first researched this, the framing was always: + +- Cloud APIs: instant, hands-off, no control, pay per use +- Rent a GPU: middle ground, but still vendor's environment +- Your own hardware: total control, but finite capacity and high electricity + +None of those fit what I actually needed: **total control + elastic scaling + pay only when running**. + +Azure Container Apps gave me that. Particularly because of one thing: **minimum scale of zero**. The container shuts down completely when not in use. You don't pay for idle time. + +If I have 10 requests a month, I pay $0.50. If I have 10,000 requests a month, I pay maybe $50–100. There's no "always on" baseline cost. There's no infrastructure I'm not using. + +--- + +## How It Works + +**Step 1: Wrap the CLI in a web endpoint** + +Flask is 30 lines: + +```python +from flask import Flask, request +from image_generation.cli.generate import generate_from_batch + +app = Flask(__name__) + +@app.route("/generate", methods=["POST"]) +def generate(): + batch = request.json + images = generate_from_batch(batch) + return {"status": "done", "images": images} + +if __name__ == "__main__": + app.run(host="0.0.0.0", port=8000) +``` + +**Step 2: Containerize** + +```dockerfile +FROM nvidia/cuda:12.1-runtime-ubuntu22.04 +WORKDIR /app +COPY requirements.txt . +RUN pip install -r requirements.txt flask +COPY image-generation/ . +COPY app.py . +EXPOSE 8000 +CMD ["python", "app.py"] +``` + +**Step 3: Deploy with `azd`** + +This is where modern Azure Developer CLI makes it simple. You use `azd`—open-source, no vendor lock-in, all your infrastructure is code: + +```bash +azd auth login +azd init +azd up +``` + +The `./infra` directory contains Bicep templates (infrastructure-as-code, human-readable, version-controlled). You get: + +- Container Registry (to store your image) +- Container Apps environment +- Your container running with GPU +- Auto-scaling from zero to N replicas +- No manual clicking in the Azure portal + +**Step 4: Call it** + +```bash +curl -X POST http:///generate \ + -H "Content-Type: application/json" \ + -d @batch_quickstart.yaml +``` + +--- + +## Cost Reality + +**Azure Container Apps pricing:** +- Compute: pay per vCPU-second (GPU = ~$0.30/hour) +- Memory: separate line item +- No charge if scaled to zero + +**My typical workflow:** +- Submit a batch of 50 images (takes ~20 minutes) +- Container runs: $0.10 +- Container idles the rest of the month: $0.00 + +**If I ran 500 batches a month:** +- 500 × $0.10 = $50/month +- Plus: Docker image storage ($5/month), container environment (~$10/month) +- Total: ~$65/month + +**Versus:** +- Cloud API per image: 10¢–50¢ each = $50–250/month for 500 images +- Local GPU: $600 hardware + $30/month electricity + +**The tradeoff:** I lose raw speed (images take 30 seconds instead of 3 seconds with cloud APIs) but keep total control and hit a cost sweet spot if I have any reasonable volume. + +--- + +## What Doesn't Lock You In + +This matters to me because I wanted to avoid vendor lock-in: + +- **Bicep templates** are readable and portable (unlike Azure-specific configuration formats) +- **Container image is vendor-agnostic** (run it on AWS ECS, GCP Cloud Run, or your own Kubernetes cluster tomorrow) +- **Python code stays mine** (not wrapped in framework magic) +- **Infrastructure is version-controlled** (not hidden in portal settings) + +If I wake up in a year and hate Azure's pricing, I copy the image to another provider and update the Bicep templates. My actual image generation code doesn't change. + +--- + +## The Decision Framework + +Cloud APIs are right if you want instant results and don't care about model control. They're genuinely good for exploration. + +Rented GPU time makes sense if you want some flexibility without infrastructure management. + +**This approach—containerized local code on serverless compute—is right if:** + +- You have custom model configurations (LoRAs, presets, specific prompts) that wouldn't survive an API migration +- You care about consistency and reproducibility +- You run batches, not single on-demand requests +- You want to avoid vendor lock-in +- You have some volume (more than a few images per month) to justify the setup time + +--- + +## What Changed My Thinking + +I spent months assuming I had to choose: either abandon my custom setup and use an API, or stay local and stay limited. + +I didn't realize there was a legitimate third path: keep the setup, containerize it, and run it elastically. It took pulling the code apart and asking "what if I just... made this callable from HTTP?" to unlock it. + +I'm curious whether this resonates with how other people are handling bespoke model work. Are you running custom fine-tuned models? Do you have reproducibility constraints that APIs can't meet? How do you scale past your local hardware? diff --git a/website/blog/2026-07-07-surprises-self-hosting-image-model.md b/website/blog/2026-07-07-surprises-self-hosting-image-model.md new file mode 100644 index 0000000..35d0a06 --- /dev/null +++ b/website/blog/2026-07-07-surprises-self-hosting-image-model.md @@ -0,0 +1,376 @@ +--- +title: "Self-Hosting SDXL on Azure Container Apps: What the Vendor API Was Hiding" +date: 2026-07-07 +slug: /2026-07-07-surprises-self-hosting-image-model +description: "Self-hosting SDXL on Azure Container Apps means taking responsibility for the model runtime, storage lifecycle, readiness state, and deployment behavior." +tags: ["AI", "Azure", "Docker", "SDXL", "Azure Container Apps", "Cloud Deployment", "Architecture", "Lessons Learned"] +draft: true +keywords: + - self-hosting sdxl + - sdxl azure container apps + - stable diffusion xl deployment + - azure files model cache + - model readiness state + - azd postdeploy hook + - flask azure container apps + - diffusers cpu optimization + - self-hosted image generation + - vendor api lock-in + +--- + +This project started as hands-on image generation, not an abstract model experiment. I was working with SDXL (Stable Diffusion XL), an open text-to-image generation model, and the path felt natural: first run it locally on my own machine, then package the same work in a local container with Docker, then deploy that container to Azure Container Apps. + +The local and local container stages were mostly smooth. They gave me enough confidence that moving from laptop to container to cloud would be more plumbing than discovery. The surprises showed up when self-hosting moved to Azure, where model files, process readiness, storage, and deployment behavior all became part of the system. + +## What the Model Produces + +Before the surprises, here is what the self-hosted SDXL pipeline actually generates once it is loaded and running. These are direct outputs from the same code, unretouched. + +![Self-hosted SDXL output: a warm, sunlit coffee shop interior with bookshelves, wooden tables, and afternoon light through tall windows](./media/2026-07-07-surprises-self-hosting-image-model/example-output-coffee-shop.png) + +This is the payoff I was working toward. The rest of the post is about everything that stood between the container starting and these images coming out. + +## The False Assumption + +Calling a vendor API makes image generation look like one operation: send a request, get an image back. Self-hosting a generative model turns that single operation into a system I have to own. + +I wanted control over inference settings, no lock-in to a hosted image API, and a cost model I could reason about. At the top level, the choice sounded simple: stop calling the vendor service and run the model myself. + +That choice moved hidden responsibilities into my application boundary. I inherited the model runtime, storage lifecycle, readiness state, and deployment behavior. I also inherited the difference between persisted model assets and a process that has loaded the model and can generate an image. + +The container was only the packaging format. Self-hosting meant owning everything the model needed after the container started. + +## The Architecture I Expected + +The system shape looked straightforward before the edge cases showed up. Local code would become a web API, the web API would run in a container, and the deployed container would use external storage for the model cache. + +The first version looked like this: + +![Expected architecture: local SDXL code wrapped in Flask, containerized, deployed to Azure Container Apps with cached model weights on an Azure Files share](./media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.png) + +The concrete version ran my Flask-wrapped SDXL (Stable Diffusion XL, an open image-generation model) code in Docker on Azure Container Apps (ACA, the managed container hosting service used here), with an Azure Files share (a network-attached mount) for cached model weights. An `azd` (Azure Developer CLI) postdeploy hook—an automation step after deploy—pulled the model. + +The real deployment runs on CPU in Azure Container Apps: **4 vCPU, 16Gi memory, `device=cpu`, and 136Gi ephemeral storage**, the container's local scratch disk, wiped on restart. That 4 vCPU / 16Gi shape requires an Azure Container Apps Dedicated (D4) workload profile; the Consumption plan caps at 4 vCPU / 8Gi. The mounted Azure Files share holds the model cache, because the SDXL assets are too large to treat as incidental container filesystem state. + +That architecture was directionally right, but it left out most of the operational work. + +### What Was Actually Happening + +The first clean picture hid two separate truths: the container could be up without running Flask, and model files could exist without the process being ready. + +![What actually happened: after deploy the placeholder static server command was preserved, so GET / returned 200 but /health and model routes returned 404, and /model/status reported not_started](./media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.png) + +## Surprise #1: Persistent Storage Does Not Mean Warm Application State + +The first lifecycle mistake was treating stored model assets as if they were the same thing as a ready application. Persistent storage can keep files between revisions, ACA's immutable deployment versions. It cannot keep a new container process warm. + +![Surprise 1: model files persist on the Azure Files share, but a new revision starts a cold process so /model/status reports not_started](./media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.png) + +After the first cold download succeeded, I expected the next revision to be warm. The model files were on the Azure Files share, the share was mounted, and the path existed. + +Then the app reported readiness state held in the process's memory, through the `/model/status` `state` field: + +```json +{ + "state": "not_started" +} +``` + +That looked wrong until I separated two states I had been mentally combining: + +- model assets persisted on disk +- model loaded and ready in this process + +Those are not the same lifecycle. The Azure Files share can be warm while the container process is cold. A new revision starts a new process. That process can see the cached files, but it still has to initialize the SDXL pipeline in memory. + +The specific `not_started` state did not mean "the share is empty." It meant "this process has not begun loading the model." The useful ready state was `ready`, also reported by `/model/status`. + +That distinction changed how I read status. A cold path downloads model assets and then loads the model. A warm path skips the download but still loads the model from the mounted cache. In my deployment, warm load from the cached share was about **48 seconds**. Cold download took minutes. + +Those are different costs, and they happen for different reasons. The deployment gate needed to care about readiness, not just file presence. Checking whether a directory exists is not enough. Checking whether the model cache is populated is not enough. The application has to report that this process is ready to serve generation requests. + +Persistent storage keeps assets. It does not keep application memory warm. For model-serving systems, readiness is process state. + +I built a small web console over these endpoints so I could see that distinction directly. Each endpoint the deployment depends on has its own section, and `/model/status` reports the process state in plain language: here it shows **READY - Model weights cached on disk. /generate will load from cache**, which is exactly the difference between "the files are on the share" and "this process can answer a request." + +![SDXL API Console web UI with one section per endpoint: GET /api, GET /health showing HEALTHY on device cpu, GET /model/status showing READY with model weights cached on disk, POST /model/pull, and a POST /generate form with prompt, steps, guidance, size, and Force CPU options](./media/2026-07-07-surprises-self-hosting-image-model/web-ui-api-console.png) + +![Surprise 1 fix: a deployment gate checks readiness while the cached share warm-loads the model in about 48 seconds before serving](./media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.png) + +## Surprise #2: Model Files Are Not Just Files + +The next storage mistake was treating model acquisition as a small deployment detail. For a generative model, the weights are a deployable asset with their own lifecycle. + +A model that is about 7GB in FP16, or roughly 14GB loaded as FP32 on CPU the way this deployment runs it, is not a small deployment detail. It is its own deployment phase. + +![Surprise 2: the empty model share plus a download that assumed POSIX flock broke on Azure Files SMB, so weights never appeared](./media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.png) + +The model weights do not appear during `azd up`. Infrastructure provisioning creates the place where the model can live, but it does not populate that place with model assets. + +I made model acquisition part of postdeploy. The deployment automation step that runs after deploy, the `azd` postdeploy hook, calls the app's model-download endpoint, `POST /model/pull`: + +```bash +curl --fail -X POST "$APP_URL/model/pull" +``` + +Then it blocks until the app reports through the model-status endpoint, `/model/status`, that the model is ready: + +```bash +curl --fail "$APP_URL/model/status" +``` + +A useful response includes the process readiness state, target device, and model path: + +```json +{ + "state": "ready", + "device": "cpu", + "model_path": "/models/stable-diffusion-xl-base-1.0" +} +``` + +That was the right shape, but the storage layer had its own constraints. The mounted network-attached file storage is an Azure Files share, which uses Server Message Block (SMB, the network file-sharing protocol Azure Files uses). SMB does not support POSIX `flock`, the file-locking call a local Linux filesystem supports. The first version of the download logic assumed file locking would behave like local disk, and that assumption broke on the mounted share. + +That kind of bug feels like an operating system problem until you remember that self-hosting makes the filesystem part of the application architecture. I had to rework the download logic so the app did not depend on unsupported locking behavior on the mounted share. + +Once that was fixed, the cold download was faster than I expected: about **2 minutes 23 seconds** over the Azure backbone. The newer Hugging Face transfer path helped here; `hf_xet`, Hugging Face's newer fast download transport, replaced the deprecated `hf_transfer`, and the transfer itself was not the bottleneck I feared. + +![Surprise 2 fix: a postdeploy hook posts to /model/pull, downloads without flock over hf_xet, and polls /model/status until ready](./media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.png) + +The useful takeaway was not simply that downloads can be fast. The model is a deployable asset with its own lifecycle. With a vendor API, the weights are someone else's problem. With self-hosting, model acquisition needs ordering, retries, logs, status, and a failure mode that stops the release instead of hiding the problem until the first image request. + +## Surprise #3: "CPU Offload" Doesn't Work on a CPU + +The runtime mistake was trusting a helper name before checking the hardware contract behind it. I expected memory to be an issue, and it was. The memory-saving helper I reached for had a name that sounded perfect for CPU hosting but failed because the container was actually running on CPU. + +![Surprise 3: on a pure-CPU container, enable_model_cpu_offload expects an accelerator and errors with requires accelerator but not found](./media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.png) + +In `diffusers`, Hugging Face's Python library for running diffusion image models, the helper is `enable_model_cpu_offload()`: + +```python +pipe.enable_model_cpu_offload() +``` + +The name sounds like exactly what a CPU deployment wants. I read it as: use CPU memory carefully, offload model pieces as needed, survive inside the container limits. + +That is not what it means. On a pure-CPU container, it raises the kind of error that makes the naming clear: + +```text +requires accelerator, but not found +``` + +`enable_model_cpu_offload()` means "offload *to* CPU *from* an accelerator." It is for a system that has an accelerator and wants to move parts of the model back to CPU memory. It is not a CPU execution mode. + +The fix was explicit CPU-safe initialization: + +```python +pipe = StableDiffusionXLPipeline.from_pretrained( + model_path, + torch_dtype=torch.float32, + use_safetensors=True, +) + +pipe.to("cpu") +pipe.enable_attention_slicing() +pipe.vae.enable_slicing() +pipe.vae.enable_tiling() +``` + +The idiomatic `diffusers` pipeline-level equivalents are `pipe.enable_vae_slicing()` and `pipe.enable_vae_tiling()`; both forms are equivalent. + +The memory-saving calls here are literal: attention slicing computes attention in smaller chunks, and VAE (the variational autoencoder stage that decodes latents into the final image) slicing and tiling decode the image in pieces. + +Model libraries encode hardware assumptions. Sometimes those assumptions are obvious. Sometimes they are hidden inside method names that sound like they were written for your exact scenario. + +![Surprise 3 fix: move the pipeline to CPU with pipe.to(cpu) plus attention slicing and VAE slicing and tiling to fit memory](./media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.png) + +For this deployment, "CPU offload" means "offload to CPU from somewhere else." It does not mean "run on CPU." The app is not just my Flask routes; it is also the model runtime, tensor dtype, memory behavior, and hardware profile lining up correctly. + +## Surprise #4: The Container Was Running, But Not My App + +The startup mistake was using a running container as proof that my application was running. A container can be healthy enough to accept traffic while the wrong process is listening. + +![Surprise 4: azd deploy preserves the placeholder command, leaving the static server running so GET / returns 200 and app routes return 404](./media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.png) + +The container app was up, the revision existed, the endpoint responded, and `GET /` returned `200`. Every real route still returned a `404 Not Found` from Python's static file server, with this line in the HTML body: + +```text +Message: File not found. +``` + +At first, that looked like my Flask routing was broken. Maybe the app was not binding correctly. Maybe the container port was wrong. Maybe the health route was missing. Maybe the image was stale. + +The error page pointed to the real problem. `Message: File not found.` is not Flask's default response; it is Python's `SimpleHTTPRequestHandler`, the built-in static file server returning its HTML error page. My container was running, but my Flask app was not. + +On a fresh environment, `azd` provisions the Azure Container App before the real application image exists. To make the infrastructure deployment succeed, it uses a temporary placeholder web server, `python3 -m http.server 8000`: + +```text +python3 -m http.server 8000 +``` + +That is reasonable during provisioning. The surprise came later, when `azd deploy` swapped in my real Flask image and preserved the placeholder command. The image changed, but the runtime command did not. + +So my real container image started successfully and then ran Python's static file server instead of my Flask app. That is why `/` returned `200`, and why `/health`, `/model/status`, and `/model/pull` returned `404 Not Found` responses whose HTML body said `Message: File not found.` Those routes only exist in Flask, and Flask was never running. + +I stopped treating "container is up" as proof that the application is running. I added a self-heal step in the postdeploy hook that resets the command explicitly: + +```bash +az containerapp update \ + --name "$CONTAINER_APP_NAME" \ + --resource-group "$RESOURCE_GROUP" \ + --command "python3" "app.py" +``` + +Then the hook waits for the actual application route before it does any model work: + +```bash +curl --fail "$APP_URL/health" +``` + +![Surprise 4 fix: reset the container command to python3 app.py so Flask starts, /health responds, and model work continues](./media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.png) + +Only after `/health` responds from Flask does the deployment continue. Calling `/model/pull` before proving Flask is running is just sending a request to whatever process happens to be listening. + +I now treat the command, the image, and the health endpoint as three separate facts. The deployment is not ready until all three are true. + +The self-heal works, but it treats a symptom. The root cause is that the container `command` was set in the Bicep template, and `azd deploy` swaps only `containers[0].image`, so the placeholder command survives and overrides the image's own start command. The cleaner pattern is to put `CMD ["python3", "app.py"]` in the Dockerfile, remove `command` and `args` from the Bicep entirely so the image command is used, and gate readiness with an ACA startup probe on `/health` instead of a manual wait loop. I kept the self-heal hook because it is what is working in this deployment, but if I were starting clean I would remove the Bicep command override and let the image plus a startup probe do this job. + +## Surprise #5: Tooling Silence Is Also a Failure Mode + +The verification mistake was trusting quiet tooling. Some failures throw obvious errors. Others look like nothing happened. + +![Surprise 5: silent tooling failures - invisible hook stdout, invalid azure.yaml keys defaulting the image, and a circular Bicep dependency](./media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.png) + +One problem was visibility. The `azd` postdeploy hook was running, but when its output was piped or non-interactive, stdout was invisible. Nothing in the terminal made it obvious what the hook was doing, so I verified through the platform logs: + +```bash +az containerapp logs show \ + --name "$CONTAINER_APP_NAME" \ + --resource-group "$RESOURCE_GROUP" \ + --follow +``` + +Those logs became the source of truth. + +Another problem was configuration shape. I had invalid keys in `azure.yaml` during one iteration. A top-level `dockerfile:` or `port:` looks plausible if you are moving fast, but `azd` did not fail the way I wanted. It ignored the invalid shape and fell back to default behavior. + +The Dockerfile must be configured through the supported `docker:` block in `azure.yaml`: + +```yaml +services: + image-generation: + project: . + language: docker + host: containerapp + docker: + path: ./Dockerfile.cpu + context: . +``` + +That small indentation decision changed what image was built. + +The Dockerfile also had to default to the Flask server as its entrypoint. If the platform command was absent or wrong, the image still needed to know how to run the app: + +```dockerfile +CMD ["python3", "app.py"] +``` + +Without that default, ACA could end up in `ContainerBackOff` or `ActivationFailed`, ACA states for a container that cannot start or stay up, depending on which part of startup failed. + +There was also an infrastructure bug: a circular dependency in the Bicep, Azure's infrastructure-as-code language, for the container app failed template validation until I broke the cycle. That was not an SDXL issue. The model deployment made the infrastructure graph more complicated, and the graph had to be correct before the app could even try to start. + +![Final architecture: verify logs as source of truth, use a supported docker block, rely on the Dockerfile CMD default, and break the Bicep cycle for an observable deploy](./media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.png) + +Automation needs observable verification. In this setup, a successful command did not prove the deployment was correct, a running container did not prove Flask was running, a mounted share did not prove the model was ready, and a quiet hook did not prove the hook was idle. + +## What the Final Architecture Became + +The final shape is the concrete version of the earlier diagram. Each piece now has an explicit responsibility, and deployment gates on the application being ready, not just the infrastructure existing. + +The container image defaults to Flask: + +```dockerfile +CMD ["python3", "app.py"] +``` + +The runtime behavior is explicitly CPU: + +```text +device=cpu +vCPU=4 +memory=16Gi +ephemeral storage=136Gi +``` + +The model pipeline uses CPU-safe initialization: + +```python +pipe.to("cpu") +pipe.enable_attention_slicing() +pipe.vae.enable_slicing() +pipe.vae.enable_tiling() +``` + +Here too, the pipeline-level `pipe.enable_vae_slicing()` and `pipe.enable_vae_tiling()` calls are the idiomatic `diffusers` form. + +The Azure Developer CLI configuration points at the CPU Dockerfile through the supported shape: + +```yaml +services: + image-generation: + language: docker + host: containerapp + docker: + path: ./Dockerfile.cpu + context: . +``` + +The postdeploy hook does four jobs, in order: + +1. Reset the container command to the Flask app. +2. Wait for `/health` so I know Flask is actually running. +3. POST `/model/pull` so model acquisition is part of deployment. +4. Poll `/model/status` until `state` is `ready`, with a configurable timeout and fail-fast behavior. + +In shell form, the core idea is simple: + +```bash +az containerapp update \ + --name "$CONTAINER_APP_NAME" \ + --resource-group "$RESOURCE_GROUP" \ + --command "python3" "app.py" + +curl --fail "$APP_URL/health" +curl --fail -X POST "$APP_URL/model/pull" + +until curl --fail "$APP_URL/model/status" | grep '"state":"ready"'; do + sleep 10 +done +``` + +The real script has more defensive handling, because production scripts should fail clearly. But that is the architecture. + +The app owns readiness. The hook gates deployment on readiness. Logs validate reality. I did not end up with just a container that runs SDXL; I ended up with a deployment lifecycle for a self-hosted generative model. + +## The Decision Framework I Actually Trust Now + +I still like the decision to self-host for this project. The tradeoff is just clearer now. + +Self-hosting buys control over inference settings, portability, model loading strategy, and deployment lifecycle. It also moves hidden responsibilities into your application boundary: runtime assumptions, model storage, download orchestration, readiness, deployment verification, logs, sizing, and the difference between "files exist" and "the model can answer this request." + +A vendor API charges for convenience, but the convenience is real. It is not just inference. It is the operational surface area you do not have to build. + +For a prototype, that surface area may not be worth it. For a workflow where settings, portability, and control matter, it can be. + +The question I trust now is simpler: do I want to own everything this model needs to be reliable? + +## Closing + +Self-hosting SDXL showed me how much the vendor API had been handling. Once I owned the model, I owned the runtime, storage, lifecycle, readiness, and observability around it. + +Self-hosting a generative model is not just replacing an API call with a container. It means the model is part of the system, with the same deployment and reliability responsibilities as the rest of the application. + +And once all of that is in place, the model just does its job: + +![Self-hosted SDXL output: a photorealistic mountain lake at sunset with pine trees, still water reflections, and mountains in the background](./media/2026-07-07-surprises-self-hosting-image-model/example-output-mountain-lake.png) diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.mmd new file mode 100644 index 0000000..4d69798 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.mmd @@ -0,0 +1,17 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Provision["azd provision"] --> Placeholder["placeholder image + command
python3 -m http.server 8000"] + Placeholder --> Deploy["azd deploy"] + Deploy --> RealImage["real Flask image
(command preserved)"] + RealImage --> Running["container up
(static server still running)"] + Running -- "GET /" --> Root["200 OK
(static file server)"] + Running -- "/health, /model/status,
/model/pull" --> Missing["404 Not Found
(Message: File not found.)"] + Files["Azure Files share
(model files exist)"] --> Status["/model/status: not_started
(process cold; SMB has no flock)"] + Running --> Status + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Provision,Deploy purple; + class Placeholder,RealImage,Running,Root,Missing pink; + class Files,Status orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.png new file mode 100644 index 0000000..6edd04f Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/actual-behavior-404.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.mmd new file mode 100644 index 0000000..5e90e28 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.mmd @@ -0,0 +1,15 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Code["Local SDXL Python code"] --> Flask["Flask API wrapper"] + Flask --> Docker["Docker image"] + Docker --> ACA["Azure Container Apps"] + ACA --> Files["Azure Files share
(cached model weights)"] + Hook["azd postdeploy hook"] -- "POST /model/pull" --> ACA + ACA -- "model ready" --> Generate["Generate images"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Code,Flask purple; + class Docker,ACA pink; + class Files,Hook,Generate orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.png new file mode 100644 index 0000000..d2f4f67 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/architecture-expected.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-coffee-shop.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-coffee-shop.png new file mode 100644 index 0000000..d0a6de8 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-coffee-shop.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-mountain-lake.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-mountain-lake.png new file mode 100644 index 0000000..4e44c2e Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/example-output-mountain-lake.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.mmd new file mode 100644 index 0000000..0d08e32 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.mmd @@ -0,0 +1,16 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Logs["Container App logs"] --> Verify["Verify source of truth"] + YAML["Supported docker block"] --> Build["Correct image builds"] + CMD["Dockerfile CMD default"] --> Start["Flask starts by default"] + Bicep["Break Bicep cycle"] --> Deploy["Observable correct deploy"] + Verify --> Deploy + Build --> Deploy + Start --> Deploy + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Logs,YAML,CMD,Bicep purple; + class Verify,Build,Start pink; + class Deploy orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.png new file mode 100644 index 0000000..6706809 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/final-architecture-fixes.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.mmd new file mode 100644 index 0000000..bee86da --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.mmd @@ -0,0 +1,13 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Files["Azure Files share
model files persist"] --> Revision["New revision
new process"] + Revision --> Memory["Process memory
model not loaded"] + Memory -- "GET /model/status" --> Status["state: not_started"] + Files --> Status + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Revision,Memory purple; + class Status pink; + class Files orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.png new file mode 100644 index 0000000..040ba45 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-cold-process-state.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.mmd new file mode 100644 index 0000000..a0383d4 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.mmd @@ -0,0 +1,13 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Gate["Deployment gate"] --> Check["Check readiness state"] + Cache["Cached Azure Files share"] --> Load["Warm load
about 48 sec"] + Check -- "state: ready" --> Ready["Ready to serve"] + Load --> Ready + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Gate,Cache purple; + class Check,Load pink; + class Ready orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.png new file mode 100644 index 0000000..b27e6da Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise1-warm-load-fix.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.mmd new file mode 100644 index 0000000..950d39a --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.mmd @@ -0,0 +1,14 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Hook["postdeploy hook"] --> Pull["POST /model/pull"] + Pull --> Download["Download without flock"] + Xet["hf_xet over Azure backbone"] --> Download + Download --> Poll["Poll /model/status"] + Poll -- "state: ready" --> Ready["Ready after cold pull"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Hook,Xet,Poll purple; + class Pull,Download pink; + class Ready orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.png new file mode 100644 index 0000000..50b1bc7 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-cold-pull-fix.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.mmd new file mode 100644 index 0000000..f82b4a1 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.mmd @@ -0,0 +1,14 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + AVD["azd up provisions infra"] --> Share["Empty model share"] + Weights["7GB FP16 / ~14GB FP32 weights"] --> Missing["Weights do not appear"] + Lock["Download assumes POSIX flock"] --> SMB["Azure Files uses SMB"] + SMB --> Broken["Download logic broke"] + Share --> Missing + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class AVD,Weights,Lock purple; + class Missing,Broken pink; + class Share,SMB orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.png new file mode 100644 index 0000000..92ca868 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise2-smb-flock-broken.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.mmd new file mode 100644 index 0000000..ce8bd4b --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.mmd @@ -0,0 +1,13 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Runtime["CPU runtime"] --> ToCPU["pipe.to(cpu)"] + ToCPU --> Attention["Attention slicing"] + Attention --> VAE["VAE slicing and tiling"] + VAE --> Ready["Runs within memory limits"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Runtime purple; + class ToCPU,Attention,VAE pink; + class Ready orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.png new file mode 100644 index 0000000..78018ff Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-memory-fix.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.mmd new file mode 100644 index 0000000..d29a815 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.mmd @@ -0,0 +1,11 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Container["Pure-CPU container"] --> Helper["enable_model_cpu_offload"] + Helper --> Contract["Offloads to CPU
from accelerator"] + Contract --> Error["requires accelerator
but not found"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Container,Helper,Contract purple; + class Error pink; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.png new file mode 100644 index 0000000..4de7925 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise3-cpu-offload-error.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.mmd new file mode 100644 index 0000000..5430a13 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.mmd @@ -0,0 +1,13 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Image["Real Flask image"] --> Heal["Reset command
python3 app.py"] + Heal --> Flask["Flask starts"] + Flask -- "GET /health" --> Health["Health responds"] + Health --> ModelWork["Continue model work"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Image purple; + class Heal pink; + class Flask,Health,ModelWork orange; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.png new file mode 100644 index 0000000..e08a482 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-reset-command-heal.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.mmd new file mode 100644 index 0000000..1297b39 --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.mmd @@ -0,0 +1,14 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Provision["azd provision"] --> Placeholder["Placeholder command"] + Deploy["azd deploy swaps image"] --> Preserve["Command is preserved"] + Placeholder --> Preserve + Preserve --> Static["Static server running"] + Static -- "GET /" --> Root["200 from /"] + Static -- "/health and model routes" --> Missing["404 Not Found
(Message: File not found.)"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Provision,Deploy,Root purple; + class Placeholder,Preserve,Static,Missing pink; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.png new file mode 100644 index 0000000..b622560 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise4-static-server-404.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.mmd b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.mmd new file mode 100644 index 0000000..52c0bda --- /dev/null +++ b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.mmd @@ -0,0 +1,13 @@ +%%{init: {'theme':'base', 'themeVariables': {'primaryColor':'#7B2FA0','primaryTextColor':'#FFFFFF','primaryBorderColor':'#52186F','lineColor':'#E84E9C','edgeLabelBackground':'#F7883B','fontFamily':'inherit'}}}%% +flowchart TD + Hook["azd hook running"] --> Quiet["stdout invisible"] + Config["Invalid azure.yaml keys"] --> Defaults["Default image path used"] + Bicep["Circular Bicep dependency"] --> Validation["Template validation failed"] + Quiet --> Confusing["Deployment looked idle"] + Defaults --> Wrong["Wrong image built"] + + classDef purple fill:#7B2FA0,stroke:#52186F,color:#FFFFFF; + classDef pink fill:#E84E9C,stroke:#B12A6E,color:#FFFFFF; + classDef orange fill:#F7883B,stroke:#C25E15,color:#FFFFFF; + class Hook,Config,Bicep purple; + class Quiet,Defaults,Validation,Confusing,Wrong pink; diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.png new file mode 100644 index 0000000..cb4cbf1 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/surprise5-silent-failures.png differ diff --git a/website/blog/media/2026-07-07-surprises-self-hosting-image-model/web-ui-api-console.png b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/web-ui-api-console.png new file mode 100644 index 0000000..f8bea35 Binary files /dev/null and b/website/blog/media/2026-07-07-surprises-self-hosting-image-model/web-ui-api-console.png differ