COVET: make preprocessing explicit and batch_size numerically inert - #92
COVET: make preprocessing explicit and batch_size numerically inert#92Marius1311 wants to merge 2 commits into
Conversation
Three things in compute_covet surprise callers who want COVET on data they have already prepared, which the README advertises as a supported use. 1. `X` is log-transformed a second time. The only test for "already log-transformed" is `spatial_data.X.min() < 0`. Log-normalized data is non-negative, so it is read as raw counts and gets `log(x + 1)` applied on top. Measured on log-normalized input, this moves COVET by 152% relative -- same genes, same graph, same estimator. Adds `log_transform=None|True|False`. `None` keeps the current heuristic, so existing results stay reproducible, but warns and says how to override. `use_obsm`/`use_layer` keep meaning "take this at face value". The upstream test suite itself triggers the warning, in test_covet_with_batches. 2. A missing `batch_key` silently pools the kNN across samples. `if batch_key not in spatial_data.obs.columns: batch_key = -1` cannot tell a typo from an absent default. Passing `batch_key="smaple"` builds one graph over all sections and returns niches that straddle samples, with nothing to indicate it. Now only the default `"batch"` falls back; anything else raises and lists the available columns. `batch_key=-1` is unaffected. 3. `batch_size` changes the numbers. The accumulator is a fixed float32 buffer while the unbatched branch takes its dtype from `np.matmul` on the centered data. Passing `batch_size` therefore accumulated in single precision before the regularization term (derived from this stack) and before the matrix square root. Both outputs are cast to float32 on return, so the dtype difference is invisible, but the values differ: ~7e-8 relative on COVET and ~5e-6 absolute on COVET_SQRT. A memory knob should be numerically inert. The buffer now matches the unbatched dtype. Adds tests/test_covet_preprocessing.py covering all three. Ten of its eleven tests fail against the current implementation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| # Initialize the output covariance matrices. The dtype has to match what the | ||
| # unbatched branch below produces (np.matmul on the centered data). With a fixed | ||
| # float32 buffer, passing batch_size instead accumulated in single precision -- | ||
| # before the regularization term, which is derived from this stack, and before | ||
| # the matrix square root -- so a pure memory knob moved the numbers. |
There was a problem hiding this comment.
Match dtypes to reduce errors due to differences in precision
would probably suffice
| :param log_transform: (bool) whether to apply log(x + 1) to the selected expression data. | ||
| None (default) keeps the historical behaviour: data taken from `X` is log-transformed | ||
| unless it contains negative values, and data taken from `use_obsm`/`use_layer` is used | ||
| as is. Pass True or False to decide explicitly. | ||
|
|
There was a problem hiding this comment.
Behaviour also described in code. Maybe enough to say: None (default): estimate if 'X' is log-transformed, else True/False do decide expliciately
|
Looks good to me! |
Tobiaspk
left a comment
There was a problem hiding this comment.
Good to merge otherwise from my side.
Both per review on dpeerlab#92. The dtype comment retold the whole bug; one line naming the invariant (batch_size must not move the numbers) is enough -- the reasoning that motivated it is in the commit that introduced it. The log_transform docstring restated what the code comment at the heuristic already says, so it is cut back to what the parameter does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks — both applied. The dtype comment is down to one line naming the invariant ( One thing to flag before you merge, since it is not this PR: the red which is why collection errors out before any test in this PR runs. I have opened |
Independent of #91 (that one is packaging, this one is behaviour); they touch different parts of
utils.pyand can be merged in either order.Three things in
compute_covetsurprise callers who want COVET on data they have already prepared — which the README advertises as a supported use:1.
Xgets log-transformed a second timeThe only test for "already log-transformed" is:
Log-normalized data is non-negative, so the common case —
sc.pp.normalize_total+sc.pp.log1p— is read as raw counts and log-transformed on top. Measured on log-normalized input with everything else held fixed (same genes, same graph, same estimator), that moves COVET by 152% relative.Adds
log_transform=None|True|False:None(default) keeps the existing heuristic exactly, so published results stay reproducible, but emits aUserWarningsaying what it decided and how to override it.True/Falsedecide explicitly and warn about nothing.use_obsm/use_layerkeep meaning "take this at face value" (log_transform=False), and can now be log-transformed explicitly if wanted.Worth noting the existing test suite triggers the warning in
test_covet_with_batches, on data that is genuinely counts — the heuristic is right there and wrong for normalized input.2. A missing
batch_keysilently pools the kNN across samplesThis cannot distinguish "the default
batchcolumn isn't there" from "the caller asked for a column that doesn't exist".compute_covet(adata, batch_key="smaple")builds a single kNN graph over all sections and returns niches that straddle samples, with nothing in the output to indicate it.Now only the default
"batch"falls back. Anything else raises and lists the available columns.batch_key=-1is unaffected.3.
batch_sizechanges the numbersThe accumulator is a fixed
float32buffer:while the unbatched branch rebinds
CovMatsto the result ofnp.matmulon the centered data, which is float64 for float64 input. So passingbatch_sizeaccumulates in single precision — beforereg_termis derived from this stack, and before the matrix square root.Both outputs are cast to float32 on return, so the dtype difference is invisible, but the values are not identical:
COVETCOVET_SQRTbatch_sizeis documented as a memory knob and should be numerically inert. The buffer now takes the same dtype as the unbatched path, after which the two agree exactly.Testing
Adds
tests/test_covet_preprocessing.py(11 tests). Against the current implementation 10 of the 11 fail; the one that passes is theuse_obsmcase, which already behaves correctly. Full suite with theenviextra: 19 passed.