Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,15 @@ uv pip install -e ".[gpu-cu13]"

The app auto-detects GPU availability at runtime and falls back to CPU if anything goes wrong — no configuration needed. The CPU sklearn path is auto-accelerated by [scikit-learn-intelex](https://github.com/uxlfoundation/scikit-learn-intelex)[^1]. You can also manually select backends (`cuML`, `sklearn`) in the sidebar.

### Reproducibility

To get reproducible projections and clusters, enable **Use fixed seed** in the sidebar and pin the backend instead of `auto`: the GPU backend is `cuML`, the CPU backend is `sklearn` (auto-accelerated by `scikit-learn-intelex` on x86 CPUs). With a seed and a pinned backend, results are identical across app restarts on both backends, with one exception: cuML t-SNE, which never reproduces exactly.

- **PCA** is a deterministic decomposition, with no stochastic optimization involved. cuML PCA uses a full eigendecomposition and always returns the same result, seed or no seed. sklearn can auto-select a randomized SVD solver, so the app passes the seed to make it reproducible.
- **UMAP** and **KMeans** reproduce exactly on both backends when a seed is set. (Seeded UMAP trades some speed for determinism.)
- **t-SNE** reproduces on `sklearn` when a seed is set. cuML's implementation is highly parallelized and documented as [not completely deterministic between runs, even with the same `random_state`](https://docs.rapids.ai/api/cuml/stable/api/generated/cuml.manifold.tsne/) (see [rapidsai/cuml#2980](https://github.com/rapidsai/cuml/issues/2980)). Select `sklearn` when t-SNE results need to be reproducible.
- `auto` chooses a backend from data size and hardware, so the same seed can run different algorithms on different machines. Exact coordinates may also differ across library versions and hardware; a seed guarantees repeatability within one environment, not across environments.

## Usage

### Standalone Apps
Expand Down
6 changes: 5 additions & 1 deletion shared/utils/clustering.py
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,9 @@ def _reduce_dim_sklearn(embeddings: np.ndarray, method: str, seed: Optional[int]
effective_workers = -1 if n_workers > 1 else n_workers

if method.upper() == "PCA":
reducer = PCA(n_components=2)
# Pass random_state so the randomized SVD solver (auto-selected for
# large inputs) is reproducible when a seed is set; None keeps it random.
reducer = PCA(n_components=2, random_state=seed)
elif method.upper() == "TSNE":
# Adjust perplexity to be valid for the sample size
n_samples = embeddings.shape[0]
Expand Down Expand Up @@ -244,6 +246,8 @@ def _reduce_dim_cuml(embeddings: np.ndarray, method: str, seed: Optional[int], n

if method.upper() == "PCA":
from cuml.decomposition import PCA as cuPCA
# cuML PCA takes no random_state and needs none: its full-SVD solver
# is deterministic, so results are already reproducible run-to-run.
reducer = cuPCA(n_components=2)
elif method.upper() == "TSNE":
from cuml.manifold import TSNE as cuTSNE
Expand Down
Loading