Skip to content

Design a composable lazy Selection object for VCSCAnnData #25

Description

@fishidaho

VCSCAnnData.__getitem__ (src/vsparse/_anndata_class.py:154-182) is always an eager copy. This means every subset operation pays a full materialization cost, and there's no way to compose several filters (e.g. an obs mask, then a var mask, then a further obs mask) without paying for each step separately.

This matters for the target end-to-end experience as a chain like:

va = va[va.obs["condition"] == "control", :]
va = va.sample_obs(n=5000, by="patient", random_state=42)  # ISSUE-28
X = va.normalize()                                          # ISSUE-29

should compose cheaply and only touch data once, at the point something
actually needs values.

Proposed scope

  • A Selection object that records row/column filters compositionally without touching data until something actually needs values.
  • VCSCAnnData gains a lazy indexing path that returns a Selection-backed view instead of an eager copy; .compute()/.materialize() (naming TBD) forces it when a caller actually needs a real VCSCAnnData.
  • Once IVCSC/IVCSR on-disk format with a byte-offset index (proposed v2 format) #24 's byte index exists, filters expressible entirely in terms of obs/var metadata should resolve to zero reads until materialization.
  • Decide how this interacts with the NormalizedView fix as a Selection composed before constructing a NormalizedView is the clean way to guarantee post-selection statistics.

Acceptance criteria

  • A multi-step subsetting chain (e.g. six filters in sequence) composes in under a millisecond with no data read.
  • Materializing a composed Selection performs exactly one read, not one per step.
  • Existing VCSCAnnData indexing behavior (obs/var labels, ints, slices, boolean masks) is preserved for callers that don't opt into laziness.

References

  • src/vsparse/_anndata_class.py:154-182 (class docstring, __getitem__).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions