Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
763 changes: 763 additions & 0 deletions website/blog/2026-07-04-image-generation.md

Large diffs are not rendered by default.

564 changes: 564 additions & 0 deletions website/blog/2026-07-05-containerized-model-generation.md

Large diffs are not rendered by default.

105 changes: 105 additions & 0 deletions website/blog/2026-07-05-local-vs-cloud-image-generation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
title: "Three Ways to Run Models: API, Rented Hardware, or Your Own Machine"
date: 2026-07-05
author: Niobe
description: "I've been thinking about the choices for running models—image generation, LLMs, speech, all of it. Here's how I think about the three paths and when each makes sense."
draft: true
---

# Three Ways to Run Models: API, Rented Hardware, or Your Own Machine

I keep running into the same question from different angles: Should I use an API? Rent a GPU? Buy my own hardware?

The question sounds simple until you realize there's no universal answer. It depends entirely on what you're actually doing—how often, at what scale, and what you care about.

So I stopped and pulled it apart. Here's what I found.

---

## The Three Real Paths

Whether you're working with image generation (DALL-E, Stable Diffusion), language models (Claude, Llama), voice synthesis, or anything else, you have three genuine options.

### Path 1: Cloud APIs

This is the straightforward path. You use DALL-E, ChatGPT, Midjourney, or any API-based service. You authenticate, make a request, and get a result back in seconds. Setup takes minutes—literally just signing up and getting an API key.

The appeal is obvious: zero infrastructure to manage, and the latest models automatically. You don't think about hardware or drivers or disk space. Someone else handles all of that.

The cost structure is clean: you pay for what you use. A few cents per request or per image. If you're experimenting or building something small, this feels like almost nothing.

But here's the thing: cost scales linearly with volume. Use more, pay more. There's no pricing ceiling. And when the provider updates a model, you get the update whether you want it or not. If something breaks or changes, you adapt or you're stuck.

### Path 2: Rented Hardware

This is the middle path. You rent GPU time or a container from someone like Runpod, Modal, AWS, or Azure. You have full control over which model you run and which version you use. You don't have to buy hardware.

Setup takes a bit longer—maybe 10 or 20 minutes to authenticate and configure a container. But once that's done, you can run whatever model you want on whatever schedule you want. If a new model comes out and you're skeptical, you don't have to adopt it. You stay on what works.

The cost is usually cheaper than cloud APIs if you're doing any serious volume. You pay for compute time. At moderate scale, this often hits a sweet spot: cheaper than APIs, but without the upfront hardware investment.

The catch is that you need some technical comfort with containers and cloud infrastructure. It's not as hands-off as an API. You're managing something, even if it's someone else's server.

### Path 3: Your Own Hardware

This is the longest-term path. You buy a GPU—probably a few hundred to a couple thousand dollars depending on what you want—and run models locally on your machine. After that initial investment, the cost per use is just electricity. Pennies.

You have complete control. You choose which models to run, which versions to keep, when to upgrade. Nothing leaves your machine. If you care about privacy, this is the only path where you truly have it.

But inference is slower. Depending on your hardware, you might be waiting minutes instead of seconds. You manage the infrastructure: drivers, updates, troubleshooting when things break. You're responsible for keeping current with model versions.

And there's the upfront cost, which matters. If you're only running models occasionally, paying for a GPU doesn't make sense. But if you're planning to use this for years and doing it at any real volume, it pays for itself.

---

## What Actually Matters

I think the comparison gets clearer when you stop looking at each in isolation and start thinking about what you're optimizing for.

**Privacy** is one thing. Cloud APIs log your requests. Rented hardware keeps things in a container that's still on someone else's server. Only your own hardware keeps everything completely local. If you're working with confidential data or you simply prefer not to send things to external servers, there's only one real path.

**Control** is another. Cloud APIs force you to adapt to whatever the provider decides. If they change a model's behavior, you get the change. Rented hardware and your own machine both give you control over which versions you run and when you upgrade. That matters if consistency is important or if you've built workflows around specific model behavior.

**Speed** is obvious but worth stating clearly: cloud is fastest (seconds), rented hardware is middle ground, your own machine is slowest. But slowness is relative. If your model takes a few minutes to run, that might be completely acceptable depending on the context.

**Cost** depends on volume. At low volume, cloud is cheapest because there's no upfront investment. As volume increases, rented hardware becomes more economical. At high volume over several years, your own hardware probably wins.

But here's what I actually think matters most: **understanding the direction each path pulls you**. Cloud APIs pull you toward convenience and hands-off simplicity. Rented hardware pulls you toward balance. Your own hardware pulls you toward independence and control.

---

## Examples Across Different Model Types

These three paths apply to everything. The trade-offs don't change just because you're switching from images to language models.

If you're generating images, you might use DALL-E (cloud API), run Flux on Runpod (rented hardware), or install Stable Diffusion locally via ollama (your machine).

If you're working with language models, you might use ChatGPT's API (cloud), run Llama on Together AI or Replicate (rented compute), or use ollama running Mistral or Llama locally (your machine).

Speech and voice synthesis work the same way: cloud APIs like Eleven Labs, containers running open-source TTS, or local Whisper running on your hardware.

The specific tools change. The trade-offs stay identical.

---

## Thinking Through Your Actual Situation

Here's how I'd think through the decision.

**First, how often are you actually running this?** If you're using it a few times a week and experimenting, a cloud API is probably the right move. If you're running it hundreds of times a month, the economics shift dramatically. If you're running it thousands of times a month, your own hardware starts looking inevitable.

**Second, what's your priority?** If you need instant results and don't want to manage infrastructure, cloud is the answer. If you want model control without hardware investment, rented compute is worth exploring. If you're planning for the long term and privacy or control matters, your own hardware makes sense.

**Third, what constraints actually matter to you?** Speed might not matter if you're batch processing. Privacy might not matter if your work is public. Cost might not be the deciding factor if it's a side project and convenience is worth it.

**Fourth, do you already have suitable hardware?** If you've got a GPU sitting there, the economics of your own hardware immediately become more attractive. If you don't, factor in the upfront cost.

---

## The Honest Answer

There's no objectively "best" option here. I keep coming back to that.

Cloud APIs are genuinely the right choice for a lot of situations. I use them regularly when I want simplicity or when I'm exploring something new. Rented hardware makes sense if you're doing enough volume that the setup effort pays off. Your own hardware is right if you're building something you plan to maintain for years.

I'm genuinely interested in how people are actually handling this trade-off. Where does the friction appear? What's the decision point where you switched from one path to another? I'm learning as I go here, same as everyone else.
73 changes: 73 additions & 0 deletions website/blog/2026-07-05-model-deployment-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: "Three Ways to Run Models: Pick the One That Fits"
date: 2026-07-05
author: Niobe
description: "A straightforward look at cloud APIs, rented hardware, and your own machine—and when each one actually makes sense."
draft: true
---

# Three Ways to Run Models: Pick the One That Fits

I keep running into the same question from different angles: Should I use an API? Rent a GPU? Buy my own hardware?

The question sounds simple until you realize there's no universal answer. It depends entirely on what you're actually doing—how often, at what scale, and what you care about.

So I stopped and pulled it apart. Here's what I found.

## The Three Real Paths

Whether you're working with image generation, language models, speech synthesis, or anything else, you have three genuine options.

**Cloud APIs** are the straightforward path. You use ChatGPT, DALL-E, Midjourney, or any API-based service. Setup takes minutes—just sign up and get an API key. The appeal is obvious: zero infrastructure to manage, and the latest models automatically. You pay for what you use. A few cents per request. If you're experimenting or building something small, this feels like almost nothing.

The catch: cost scales linearly with volume. Use more, pay more. There's no pricing ceiling. And when the provider updates a model, you get the update whether you want it or not.

**Rented hardware** is the middle path. You rent GPU time from services like Runpod, Modal, AWS, or Azure. Setup takes longer—maybe 15 minutes to configure a container—but once that's done, you have full control over which model you run and which version you use. You don't have to adopt every update. You stay on what works.

The cost is usually cheaper than cloud APIs at moderate volume. The catch is that you need some technical comfort with containers and cloud infrastructure. It's not as hands-off as an API.

**Your own hardware** is the longest-term path. You buy a GPU—a few hundred to a couple thousand dollars—and run models locally on your machine. After that initial investment, the cost per use is just electricity. You have complete control. Nothing leaves your machine. If privacy matters, this is the only path where you truly have it.

But inference is slower—you might be waiting minutes instead of seconds. You manage the infrastructure: drivers, updates, troubleshooting. And there's that upfront cost, which matters if you're only running models occasionally. But if you're planning to use this for years and doing it at any real volume, it pays for itself.

## What Actually Matters

**Privacy** is one thing. Cloud APIs log your requests. Rented hardware keeps things in a container on someone else's server. Only your own hardware keeps everything completely local. If you're working with confidential data or you simply prefer not to send things to external servers, there's only one real path.

**Control** is another. Cloud APIs force you to adapt to whatever the provider decides. Rented hardware and your own machine both give you control over which versions you run and when you upgrade. That matters if consistency is important or if you've built workflows around specific model behavior.

**Speed** is obvious: cloud is fastest (seconds), rented hardware is middle ground, your own machine is slowest. But slowness is relative. If your model takes a few minutes to run, that might be completely acceptable depending on the context.

**Cost** depends on volume. At low volume, cloud is cheapest because there's no upfront investment. As volume increases, rented hardware becomes more economical. At high volume over several years, your own hardware probably wins.

But here's what actually matters most: understanding the direction each path pulls you. Cloud APIs pull you toward convenience and hands-off simplicity. Rented hardware pulls you toward balance. Your own hardware pulls you toward independence and control.

## When to Pick Each One

**Use cloud APIs if:** You're experimenting. You need instant results. You don't want to manage infrastructure. You're okay with using whatever model the provider currently recommends. You're running this a few times a week at most.

**Use rented hardware if:** You need model control without a large hardware investment. You're running models hundreds of times a month. You want to stick with a specific version or model. You have the technical comfort to configure containers. You want something cheaper than APIs but don't want to buy a GPU.

**Use your own hardware if:** You care about privacy. You're running this thousands of times a month. You're building something you plan to maintain for years. You already own suitable hardware. You want complete independence from a provider's decisions.

## The Framework Applies Everywhere

These three paths apply to everything. The trade-offs don't change just because you're switching from images to language models.

If you're generating images, you might use DALL-E (cloud API), run Flux on Runpod (rented hardware), or install Stable Diffusion locally via ollama (your machine).

If you're working with language models, you might use ChatGPT's API (cloud), run Llama on Together AI (rented compute), or use ollama running Llama locally (your machine).

Speech and voice synthesis work the same way: cloud APIs, containers running open-source TTS, or local Whisper on your hardware.

The specific tools change. The trade-offs stay identical.

## What Changed My Thinking

I spent a lot of time looking at this as a pure economics problem—break-even points, amortization, per-use costs. That's useful, but it misses something important.

The real question isn't which is cheapest. It's which one aligns with how you actually work and what you actually care about.

Some people will always prefer cloud APIs because infrastructure stress isn't worth the savings. Some people need local hardware for privacy and are willing to manage it. Most people probably sit somewhere in the middle.

I'm genuinely interested in how people are actually handling this trade-off. Where does the friction appear? What's the decision point where you switched from one path to another? I'm learning as I go here, same as everyone else.
Loading