feat(core)!: add the typed index usage by vytautas-astrauskas-sensmetry · Pull Request #566 · sensmetry/sysand

vytautas-astrauskas-sensmetry · 2026-09-24T07:32:05Z

Adds a fourth usage kind, the index usage:

{ "publisher": "Acme Labs", "name": "My Lib", "versionConstraint": "^1.2" }

It names a project by its publisher and name, spelled as that project spells them, and resolves through everything that resolves by identity: the configured indexes, .sysand, [[project]] overrides and workspace members. Resolution itself goes by the normalized identifier (pkg:sysand/acme-labs/my-lib); the exact spelling is checked on top. It replaces the Python-only index view, which showed a pkg:sysand resource usage as an index usage and lost the spelling.

What changes for users

sysand add <publisher>/<name> writes an index usage (sysand add "Acme Labs/My Lib" ^1). It used to write a pkg:sysand resource usage. A full pkg:sysand/... IRI still writes a resource usage.
add takes two spellings: the project's own, which it checks, or the fully normalized one (acme-labs/my-lib), from which it takes the project's own spelling. Any other spelling fails.
- When locking, a normalized usage is resolved by identifier first, then written with the spelling of the project locked.
- With --no-lock, the spelling is checked against, or taken from, the versions installed in .sysand that match the constraint, with no network request. add fails when none is installed, or when the installed ones disagree on the spelling, with a hint to leave out --no-lock.
- A normalized spelling of a project already declared as an index usage takes the declared spelling.
An index usage always gets a constraint in the manifest. Without one, add writes ^ the version that locking chose, as cargo add does. With --no-lock, a constraint must be given.
The spelling must match exactly. After the solve, locking checks each index usage's publisher and name against the project it resolved to, and fails on any difference, case included. The error says the usage resolved but was rejected because of its spelling, gives the spelling to use, names the declaring project, and says whether the user can fix it. sync from an existing lockfile does not re-solve, so it skips this check.
sysand remove <publisher>/<name> removes the index usage spelled exactly so. When a usage of the same project exists but doesn't match, the error says what is there instead: another spelling ("did you mean"), a legacy PURL, or another kind, with the command that removes it.
Existing pkg:sysand resource usages stay as they are. Adding an index usage over one is refused, with a hint to remove the legacy usage first.
--strict-index-versions: by default, the solver skips a version offered for an index usage when that version's own metadata is invalid, and picks among the others. With the flag, such a version fails the solve. It applies to index usages only.
sysand index add keeps one spelling per project. It refuses a version spelled differently from the versions already in the index (yanked ones included, removed ones left out). It also refuses any version of a project whose existing versions already disagree.
Python:
- add locks and syncs, as sysand add does, unless no_lock=True or no_sync=True, and restores .project.json if either fails. It takes no_lock, no_sync, no_prune, resolution and auth, and checks and recovers spellings the same way as the CLI. This breaks callers that relied on add only editing the manifest: they now pass no_lock=True. version_constraint stays required;
- remove/set_usage_constraint(publisher=, name=) work on index usages by exact spelling;
- iri= is always a resource usage, and a pkg:sysand IRI reads back as InterchangeProjectUsageResource;
- InterchangeProjectUsageIndex.version_constraint is required;
- Resolution(strict_index_versions=True) affects lock.
Java: a new InterchangeProjectUsageIndex class.

Compatibility

Server: sysand.com has to accept index usages in uploaded KPARs and serve them in versions.json. build keeps the usage typed in the KPAR. Index protocol §11 now also requires an index to serve one spelling per project and refuse a version spelled differently, which sysand.com already does.
Older clients break permanently. A sysand release before this cannot
parse a manifest that contains an index usage. The same goes for the
versions.json of any index project that has one in any version. Published
usage never changes (protocol §11), so older clients lose every version of
such a project for good.
Unknown keys: an index usage entry with any key besides publisher,
name and versionConstraint is rejected rather than ignored, so that a
future kind, a typo, or an index-selecting key is never read as a plain
index usage. RELEASE.md and index protocol §14 record the exception.

Follow-ups

sysand index add and the sysand.com server should refuse a version
whose index usages misspell a project the index already has. Until then,
a spelling mismatch in a published dependency fails every downstream
lock, and no downstream user can fix it.
Diagnostics for unknown keys in usage entries.
A sysand.toml setting for strict index versions.

andrius-puksta-sensmetry · 2026-09-24T07:52:34Z

sysand add <publisher>/<name> writes an index usage, keeping the
spelling as given (sysand add "Acme Labs/My Lib" ^1). It used to write a
pkg:sysand resource usage. A full pkg:sysand/... IRI still writes a
resource usage.

Normalized values must not appear in typed usages, so the actual publisher/name has to be looked up in the index. This exposes a not-so-nice tradeoff: if we want to retain support for shorthand notation publisher/name (which may or may not be normalized), we must always look up the actual values in the index. Solutions:

require non-normalized values in shorthand notation: this is done here, but also requires:
- updating the index UI, which currently suggests adding with the normalized values
- checking that the actual values match expected on resolution or when generating the lockfile
tolerate shorthand, look up in index

vytautas-astrauskas-sensmetry · 2026-09-25T11:55:47Z

sysand add <publisher>/<name> writes an index usage, keeping the
spelling as given (sysand add "Acme Labs/My Lib" ^1). It used to write a
pkg:sysand resource usage. A full pkg:sysand/... IRI still writes a
resource usage.

Normalized values must not appear in typed usages, so the actual publisher/name has to be looked up in the index. This exposes a not-so-nice tradeoff: if we want to retain support for shorthand notation publisher/name (which may or may not be normalized), we must always look up the actual values in the index. Solutions:

Yes, the tradeoff is not nice.

* require non-normalized values in shorthand notation: this is done here, but also requires:

Non-normalized values may contain spaces and, therefore, could be non-ergonomic.

* tolerate shorthand, look up in index

This breaks with --locked, which, currently, makes no network calls.

An index usage names a project by publisher and name, and has to spell them exactly as the project does. `add` now accepts two spellings: the project's own, which is checked, and the fully normalized one (lowercase, hyphens for spaces), which takes the project's spelling. - When locking, a normalized usage is first resolved by identifier as a placeholder, then written with the spelling of the project locked. Any other spelling is checked by the lock, as before. - With `--no-lock`, the spelling is checked against, or taken from, the versions installed in the local environment that match the version constraint, without any network request. It fails when none is installed, or when those installed disagree on the spelling. - A clash with a usage of another kind is reported before anything is looked up. The lock's spelling error now says outright that the usage resolved but was rejected for its spelling, and which spelling to use. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

An index usage has to spell a project's publisher and name exactly as the project does, so every version of a project in an index has to spell them the same way. `sysand index add` now refuses a version spelled differently from the versions already in the index (yanked ones included, removed ones left out), and any version of a project whose existing versions already disagree. The index protocol records one spelling per project as a server obligation. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

`add` now runs `sysand add` itself: after declaring the usage it locks and syncs, unless `no_lock` or `no_sync`, and restores `.project.json` if either fails. It takes `no_lock`, `no_sync`, `no_prune`, `resolution` and `auth`, and checks and recovers the spelling of index usages the way the CLI does. With `no_lock=True`, it only edits `.project.json`, and an index usage needs a matching version installed in the environment. `version_constraint` stays required. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

consideRatio · 2026-09-28T07:33:55Z

@andrius-puksta-sensmetry I haven't acted on discussion about name and publisher normalization yet in sysand.com. @vytautas-astrauskas-sensmetry there is a related issue for sysand.com about forcing sysand.com users to use the same name etc.

With a sysand.com biased perspective, I'd like to avoid a change here, as for sysand.com there is no upside but a downside: users may want a change for some reason -- an initial typo or SENSmetry renaming to Sensmetry etc.

@andrius-puksta-sensmetry confirm again and I'll follow through with sysand.com directly or in the future if you don't consider it high prio.

andrius-puksta-sensmetry · 2026-09-28T09:54:17Z

Python:

add locks and syncs, as sysand add does, unless no_lock=True or no_sync=True, and restores .project.json if either fails. It takes no_lock, no_sync, no_prune, resolution and auth, and checks and recovers spellings the same way as the CLI. This breaks callers that relied on add only editing the manifest: they now pass no_lock=True. version_constraint stays required;

I think it would make more sense for Python bindings to take lock, sync and prune instead to avoid double negation confusion.

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com> Signed-off-by: vytautas-astrauskas-sensmetry <vytautas.astrauskas@sensmetry.com>

- Python `add` takes `lock`, `sync` and `prune` (all `True` by default) instead of `no_lock`, `no_sync` and `no_prune`, avoiding double negation. - Removing by publisher and name, in Python and in `sysand remove <publisher>/<name>`, removes the project's usages of every kind: index, directory and KPAR usages, and a `pkg:sysand` resource usage. Each field matches spelled as the usage spells it, or normalized, as #574 does for the CLI. Python can now remove directory and KPAR usages, so the Python special cases for "cannot remove them yet" are gone. - Every typed usage kind (directory, KPAR, index) now rejects keys it does not define, not only index usages, so a future field such as a constraint on a directory usage is never silently dropped. Resource usages, the shape KerML specifies, keep ignoring unknown keys, since existing manifests carry extra keys in them. Index protocol §14 and RELEASE.md say so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

vytautas-astrauskas-sensmetry · 2026-09-29T09:35:59Z

Python:

add locks and syncs, as sysand add does, unless no_lock=True or no_sync=True, and restores .project.json if either fails. It takes no_lock, no_sync, no_prune, resolution and auth, and checks and recovers spellings the same way as the CLI. This breaks callers that relied on add only editing the manifest: they now pass no_lock=True. version_constraint stays required;

I think it would make more sense for Python bindings to take lock, sync and prune instead to avoid double negation confusion.

Changed in c38d6c2

Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com> Signed-off-by: vytautas-astrauskas-sensmetry <vytautas.astrauskas@sensmetry.com>

andrius-puksta-sensmetry · 2026-09-30T05:27:12Z

+    fn of(usage: &'a InterchangeProjectUsage) -> Self {
+        match usage {
+            // Resource usages may produce invalid candidates that should not fail
+            // the whole resolution (e.g. both src and kpar variants for paths and http)
+            InterchangeProjectUsage::Resource { .. } => Self::Skip,
+            // Path usages name one project outright, so it must be a valid one
+            InterchangeProjectUsage::Directory { .. }
+            | InterchangeProjectUsage::KparPath { .. } => Self::Fail,
+            // A version an index offers is one its publisher published, so a
+            // broken one is a fault of the index to report, not to paper over
+            // by quietly picking another version
+            InterchangeProjectUsage::Index(IndexUsage {
+                version_constraint, ..
+            }) => Self::FailIfSelectable(version_constraint),
+        }
+    }


Suggested change

fn of(usage: &'a InterchangeProjectUsage) -> Self {

match usage {

// Resource usages may produce invalid candidates that should not fail

// the whole resolution (e.g. both src and kpar variants for paths and http)

InterchangeProjectUsage::Resource { .. } => Self::Skip,

// Path usages name one project outright, so it must be a valid one

InterchangeProjectUsage::Directory { .. }

| InterchangeProjectUsage::KparPath { .. } => Self::Fail,

// A version an index offers is one its publisher published, so a

// broken one is a fault of the index to report, not to paper over

// by quietly picking another version

InterchangeProjectUsage::Index(IndexUsage {

version_constraint, ..

}) => Self::FailIfSelectable(version_constraint),

}

}

fn of(usage: &'a InterchangeProjectUsage) -> Self {

// Candidates from env don't change the logic here

match usage {

// Resource usages may produce invalid candidates that should not fail

// the whole resolution (e.g. both src and kpar variants for paths and http)

InterchangeProjectUsage::Resource { .. } => Self::Skip,

// Path usages name one project outright, so it must be a valid one

InterchangeProjectUsage::Directory { .. }

| InterchangeProjectUsage::KparPath { .. } => Self::Fail,

// A version an index offers is one its publisher published, so a

// broken one is a fault of the index to report, not to paper over

// by quietly picking another version

InterchangeProjectUsage::Index(IndexUsage {

version_constraint, ..

}) => Self::FailIfSelectable(version_constraint),

}

}

andrius-puksta-sensmetry · 2026-09-30T05:44:13Z

+    fn describe(&self) -> String {
+        match self {
+            Self::Resolved(e) => format!("is error: {}", format_err(e)),
+            Self::InvalidVersion { version, source } => {
+                format!("has invalid version `{version}`: {}", format_err(source))
+            }
+            Self::MissingVersion => "did not expose a version".to_owned(),
+            Self::VersionObtain(e) => format!("failed to get version: {}", format_err(e)),
+            Self::InvalidProject { version, source } => {
+                format!("{version} has invalid usage: {}", format_err(source))
+            }
+            Self::MissingUsage => "did not expose usages".to_owned(),
+            Self::UsageObtain(e) => format!("failed to get usages: {}", format_err(e)),
+        }
+    }


Self already implements Display; not worth duplicating for minor wording differences.

andrius-puksta-sensmetry · 2026-09-30T05:45:23Z

+/// Displayed as what is wrong with the candidate, for
+/// [`InternalSolverError::BrokenIndexVersion`] to report.
+#[derive(Error, Debug)]
+pub enum CandidateFault<R: ResolveRead> {


Suggested change

pub enum CandidateFault<R: ResolveRead> {

pub enum CandidateError<R: ResolveRead> {

Consistency

andrius-puksta-sensmetry · 2026-09-30T05:45:48Z

+                        match read_candidate::<R>(alternative) {
+                            Ok(candidate) => found.push(candidate),
+                            Err(fault) if on_broken.may_skip(&fault) => {
+                                log::debug!("candidate project for {resolve} {}", fault.describe());


Suggested change

log::debug!("candidate project for {resolve} {}", fault.describe());

log::debug!("skipping candidate project for {resolve}: {}", format_err(fault));

andrius-puksta-sensmetry · 2026-09-30T06:12:21Z

+    /// The project `publisher`/`name` from the configured indexes, or from
+    /// any other source that resolves by identity (the local environment,
+    /// `[[project]]` overrides, workspace members).
+    Index(IndexUsage<VersionReq>),
+}
+
+/// An index usage (see [`InterchangeProjectUsageG::Index`]). `publisher` and
+/// `name` must match the resolved project's, without any normalization.
+// `deny_unknown_fields`: see `Usage`
+#[derive(Eq, Clone, PartialEq, Serialize, Deserialize, Hash, Debug)]
+#[cfg_attr(
+    feature = "python",
+    derive(FromPyObject, IntoPyObject),
+    pyo3(from_item_all)
+)]
+#[serde(rename_all = "camelCase", deny_unknown_fields)]
+pub struct IndexUsage<VersionReq> {
+    pub publisher: String,
+    pub name: String,
+    pub version_constraint: VersionReq,
+}


Why is index usage a struct, but KparPath/Directory are not? Also, statement about publisher/name exact matching applies to all types usages.

andrius-puksta-sensmetry · 2026-09-30T06:22:28Z

+                // Says all there is to say about which usage failed, and why
+                DependencyIdentifier::Requested(_)
+                    if matches!(source, InternalSolverError::BrokenIndexVersion { .. }) =>
+                {
+                    write!(f, "{source}")
+                }
                DependencyIdentifier::Requested(_) => {
                    write!(f, "failed to retrieve project(s): {source}")
                }
                DependencyIdentifier::Remote(iri) => {
                    write!(f, "failed to retrieve usages of `{iri}`: {source}")
                }


Why special case for ::Requested, but not for ::Remote?

andrius-puksta-sensmetry · 2026-09-30T06:29:24Z

+impl<R: ResolveRead + fmt::Debug + 'static> std::error::Error for SolverError<R> {
+    /// `Display` already includes the solver's own error, so this skips to
+    /// what caused it
+    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+        match self.inner.as_ref() {
+            pubgrub::PubGrubError::ErrorRetrievingDependencies { source, .. }
+            | pubgrub::PubGrubError::ErrorChoosingVersion { source, .. } => {
+                std::error::Error::source(source)
+            }
+            pubgrub::PubGrubError::NoSolution(_)
+            | pubgrub::PubGrubError::ErrorInShouldCancel(_) => None,
+        }
+    }
+}


Suggested change

impl<R: ResolveRead + fmt::Debug + 'static> std::error::Error for SolverError<R> {

/// `Display` already includes the solver's own error, so this skips to

/// what caused it

fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {

match self.inner.as_ref() {

pubgrub::PubGrubError::ErrorRetrievingDependencies { source, .. }

| pubgrub::PubGrubError::ErrorChoosingVersion { source, .. } => {

std::error::Error::source(source)

}

pubgrub::PubGrubError::NoSolution(_)

| pubgrub::PubGrubError::ErrorInShouldCancel(_) => None,

}

}

}

impl<R: ResolveRead + fmt::Debug + 'static> std::error::Error for SolverError<R> {

fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {

Some(self.inner.as_ref())

}

}

And change Display to not print error source.

andrius-puksta-sensmetry · 2026-09-30T06:30:32Z

+    /// A version offered for an index usage is not a valid project. It is
+    /// not skipped for another version, which would make what is locked
+    /// depend on which versions happen to be broken


Suggested change

/// A version offered for an index usage is not a valid project. It is

/// not skipped for another version, which would make what is locked

/// depend on which versions happen to be broken

/// A version offered for an index usage is not a valid project. It is

/// not skipped for another version, which would make the choice of

/// projects depend on which versions happen to be broken

andrius-puksta-sensmetry · 2026-09-30T07:59:59Z

+    /// An index usage of the same project is already declared, but spelled
+    /// differently. Only one of the spellings can match the project's own.
+    #[error(
+        "`{new}` is already declared as the index usage `{existing}`;\n\
+        an index usage must spell the publisher and name exactly as the project does"
+    )]
+    IndexUsageSpelledDifferently { existing: String, new: String },


Applies to all typed usages.

andrius-puksta-sensmetry · 2026-09-30T08:00:52Z

+/// Whether `publisher` and `name` are both in normalized form, that is,
+/// what [`normalize_field`] makes of them. An index usage given this way names
+/// the project by its identifier only, and its actual spelling has to be
+/// recovered (see [`spell_index_usage`]); any other spelling has to be the
+/// project's own.
+pub fn is_normalized_spelling(publisher: &str, name: &str) -> bool {
+    normalize_field(publisher) == publisher && normalize_field(name) == name
+}


Suggested change

/// Whether `publisher` and `name` are both in normalized form, that is,

/// what [`normalize_field`] makes of them. An index usage given this way names

/// the project by its identifier only, and its actual spelling has to be

/// recovered (see [`spell_index_usage`]); any other spelling has to be the

/// project's own.

pub fn is_normalized_spelling(publisher: &str, name: &str) -> bool {

normalize_field(publisher) == publisher && normalize_field(name) == name

}

/// Whether `publisher` and `name` are both in normalized form, that is,

/// what [`normalize_field`] makes of them. A typed usage given this way names

/// the project by its identifier only, and its actual spelling has to be

/// recovered (see [`spell_index_usage`]); any other spelling has to be the

/// project's own.

pub fn is_normalized_spelling(publisher: &str, name: &str) -> bool {

normalize_field(publisher) == publisher && normalize_field(name) == name

}

For non-index typed usages are given by their source only, but specifying. publisher+name for them will be allowed in the future.

andrius-puksta-sensmetry · 2026-09-30T09:21:57Z

+        Some(InterchangeProjectUsageRaw::Resource { .. }) | None => {
+            Err(RemoveError::ExpUsageNotFound {
+                publisher: publisher.to_owned(),
+                name: name.to_owned(),
+            })
+        }


Suggested change

Some(InterchangeProjectUsageRaw::Resource { .. }) | None => {

Err(RemoveError::ExpUsageNotFound {

publisher: publisher.to_owned(),

name: name.to_owned(),

})

}

Some(InterchangeProjectUsageRaw::Resource { .. }) | None => {

unreachable!()

}

andrius-puksta-sensmetry · 2026-09-30T09:44:50Z

        )
-    resolved = project_iri("add", iri, publisher, name)
-    return sysand_rs.do_add_py(str(project_dir), resolved, version_constraint)  # type: ignore
+    check_named("add", iri, publisher, name)


This now duplicates the check in do_add_py. Since all the arguments are passed to it anyway, remove the check here.

andrius-puksta-sensmetry · 2026-09-30T09:48:17Z

+pub struct IndexUsageMismatchError {
+    pub usage_publisher: String,
+    pub usage_name: String,
+    pub declared_by: DeclaredBy,
+    pub version: String,
+    pub publisher: Option<String>,
+    pub name: String,
+}


Generalize to all typed usages.

andrius-puksta-sensmetry · 2026-09-30T09:49:10Z

+    let mut index_usages: Vec<(IndexUsage<VersionReq>, DeclaredBy)> = inputs
+        .iter()
+        .zip(declared_by)
+        .filter_map(|(usage, declared_by)| match usage {
+            InterchangeProjectUsage::Index(index) => Some((index.clone(), declared_by)),
+            _ => None,
+        })
+        .collect();


This applies to all typed usages.

Either unify checks from src/kpar readers here, or move this check to index resolver.

andrius-puksta-sensmetry · 2026-09-30T09:56:31Z

        dependencies.push((identifier, project));
    }

+    check_index_usages(index_usages, &solved).map_err(LockError::IndexUsageMismatch)?;


What about transitive usages? This requirement applies to them equally.

andrius-puksta-sensmetry

Didn't pay much attention to CLI crate due to the looming rebase changes.

vytautas-astrauskas-sensmetry and others added 4 commits September 27, 2026 09:20

feat(core)!: add the typed index usage

28da0d5

vytautas-astrauskas-sensmetry force-pushed the feat/index-usage branch from 5547a90 to e5cd21f Compare September 27, 2026 15:45

fix: remove strict checking flag

144616d

vytautas-astrauskas-sensmetry requested a review from andrius-puksta-sensmetry September 28, 2026 05:11

vytautas-astrauskas-sensmetry marked this pull request as ready for review September 28, 2026 05:11