fix: LMOTS pattern, add HSS variant (LMS family)

[Mehrn0ush]

Hi,
LMOTS had a treeHeight parameter it shouldn't — LM-OTS doesn't use a Merkle tree, so there's no height involved. Swapped it for the Winternitz parameter w, which is what actually distinguishes the RFC's parameter sets (LMOTS_SHA256_N32_W1/W2/W4/W8).

Also added HSS, which was missing entirely. It's just the multi-level LMS construction, parameterized by number of levels (L), so the pattern is intentionally minimal — no hash/M/H, since those come from whichever LMS parameter set each level uses.

LMS itself was already correct.

"LMOTS[_{hashAlgorithm}][_N{bytesPerNode}][_W{winternitzParameter}]"
"HSS[_L{levels}]"

One question — LMOTS still reuses {bytesPerNode} for its N parameter, but RFC 8554 calls this n — the LM-OTS hash output length, not a node value (LM-OTS has no tree nodes). Should this be renamed to something like {hashOutputLength} on LMOTS, keeping {bytesPerNode} only on LMS where it's accurate? Didn't want to bundle a naming change into this fix, so leaving it open — happy to push a follow-up if there's agreement either way.

Closes #1028.

[jvdsn]

Also added HSS, which was missing entirely. It's just the multi-level LMS construction, parameterized by number of levels (L), so the pattern is intentionally minimal — no hash/M/H, since those come from whichever LMS parameter set each level uses.

Would there be any way to express that? Dependencies?

Note for reviewers, RFC 8554 explicitly allows a lot of flexibility:

We specifically allow different LMS levels to use different parameter
sets. For example, the 0-th LMS public key (the root) may use the
LMS_SHA256_M32_H15 parameter set, while the 1-th public key may use
LMS_SHA256_M32_H10.

[Mehrn0ush]

Hi,
@jvdsn thank you for reviewing this PR.
In this schema, variant only has pattern/primitive/standard, no dependency field, so there's no structural way to express this today. XMSSMT handles its multi-tree case by encoding {h}/{d} directly in the pattern string, but that only works because XMSS-MT levels share one parameter set. HSS can't use the same trick since RFC 8554 allows each level to pick an independent LMS parameter set. Left HSS[_L{levels}] as-is for now — a real fix would need a schema change to support per-level composition, which is bigger than this PR. (For what it's worth, I also checked the v2 cryptography schema — it moves to a free-text parameterSetIdentifier per algorithm instance rather than family-level patterns, and doesn't add a composition mechanism either, so this looks like a genuine open gap rather than something already solved elsewhere.)
Happy to adjust further if maintainers have thoughts on this — let me know if any additional changes are needed here.

[bhess]

If I understand correctly, we would like to describe an ordered sequence of LMS parameter sets for HSS, one per level, including the associated LMOTS parameter sets. Is that correct?

What we could do is extending the naming-pattern convention to support repetition. For example, * for zero or more occurrences or + for one or more, and references to other patterns, such as <LMS> and <LMOTS>.
This would require documenting the new notation, but no structural change to the JSON schema, since pattern is already a string. It could keep this within the existing naming-pattern approach.

[Mehrn0ush]

Yes, that’s exactly it — HSS is an ordered sequence of L levels, each with its own LMS and LMOTS parameter set (RFC 8554 secion 6; SP 800-208 narrows the allowed sets but keeps the per-level structure).

I like extending the naming-pattern notation rather than changing the schema. Three things I’d want covered when we document it:

• is already a literal in SPAKE2+…. Either restrict quantifiers to follow a ) group only, or define an escape.
  needs a resolution rule: a reference to the variant pattern whose leading literal is LMS (not the family — the LMS family also holds LMOTS), with leading literals unique across the registry.
  The pattern can’t tie the repeat count to {levels}; that would be a stated rule, like the join rule in CBOM: add protocol registry schema #1010.
  Something like HSS[L{levels}][(_)+], root level first.

Since this touches the convention for the whole registry, do you suggest keeping this PR as the LMOTS/HSS correctness fix and doing the notation + full HSS pattern as a follow-up (happy to open it). Fine to fold it in here instead if you prefer. @bhess

[Mehrn0ush]

Pushed the extended approach following @bhess's suggestion. pattern now documents the notation ({} placeholders, | alternation pattern-wide or in a group, ( ) grouping, [ ] optional, ( )+ repetition, references resolved by leading literal; unique and non-cyclic).
Only + is new, and it is literal elsewhere, so SPAKE2+ is unaffected. All existing patterns parse under the documented grammar.
HSS is HSS[L{levels}][(_)+], root level first, pairs = L when both present;
HSS_L2 and pair-only names remain valid partial identifications; omitted parameters are not inherited between pairs. LMOTS N → {hashOutputLength} (bytes); hash tokens (SHA256|SHAKE) per RFC 8554/9858; RFC 9858 added.
Compatibility: JSON structure unchanged; consumers interpreting pattern strings may need updates.

One thing @bhess to confirm: under the written grammar FFDH(E)[-{namedGroup}] reads as a required E (FFDHE…); if FFDH[E] was intended, I can open a separate PR.

[stevespringett]

@bhess Can you please review when you get a chance.

[bhess]

Thanks for this update!

One thing @bhess to confirm: under the written grammar FFDH(E)[-{namedGroup}] reads as a required E (FFDHE…); if FFDH[E] was intended, I can open a separate PR.

Right, the E is optional (stands for ephemeral). Thanks for the catch, and for offering to fix this!

[bhess]

Thanks a lot for writing this up, and for checking that all existing patterns parse under it, that was really helpful!

One thought, and I'm curious what you think: the description is what implementers see in the generated docs, and I wonder whether the full grammar might be a bit heavy there. Maybe we could keep a short list of the constructs with examples in the schema, and move the detailed grammar into the updated CBOM guide, where there's more room for edge cases and examples?

As a starting point:

Defines the pattern used to construct the algorithm name. {name} is a placeholder for an algorithm-specific value, [ ] marks optional content, | separates alternatives of which exactly one is used, across the whole pattern or within ( ), and ( )+ repeats a group one or more times. <Name> stands for a name built from the variant pattern whose text before the first notation character is Name, for example <LMS>. All other characters, such as -, _ and the + in SPAKE2+, are part of the name. An omitted optional part means the value is unspecified, except for a fixed literal such as [-PKCS7], whose absence selects the variant without it. For HSS, the LMS/LMOTS pairs are listed per level, starting at the root; when both {levels} and pairs are given, their numbers shall be equal.

@stevespringett, curious to hear what you think.