[feature] support for pixi lock and toml files

[jpfeuffer]

What this adds

Two new multi-stage build templates that build containers from existing Pixi artifacts instead of re-solving a Conda definition:

• conda/pixi-lock:v1 — builds from a provided pixi.lock via pixi install --frozen. No re-solve, so the image matches the lock byte-for-byte (fully reproducible). When no manifest is supplied, a minimal pixi.toml is reconstructed from the lock; a user-provided manifest can be passed through and used instead.
• conda/pixi-toml:v1 — builds from a provided pixi.toml manifest.

Both support lock/manifest delivered as a remote URL or as inline file content, in Dockerfile and Singularity flavors.

How it differs from the existing Pixi support

The existing conda/pixi:v1 template builds from a Conda definition (conda.yml / package list): Pixi imports and re-solves it at build time (pixi init --import conda.yml), generating a fresh lock during the build — so resolved versions depend on when/where the build runs. The new templates build from already-resolved Pixi artifacts the user supplies, making builds reproducible and bringing native pixi.toml/pixi.lock workflows to Wave.

Base-packages peculiarity

For conda/pixi:v1, base packages (default conda-forge::procps-ng, providing the ps binary Nextflow needs) are added straight into the env via pixi add — fine, because the lock is generated during the build. For lock builds that would invalidate the frozen lock, so base packages are installed into an isolated pixi global prefix and copied into the final image, leaving the locked env unchanged. This only covers CLI tools like ps; Python-level deps (e.g. psutil) must be in the user's pixi.toml before locking.

Changes

• New PixiLockHelper and PixiTomlHelper; ContainerHelper dispatch.
• TemplateUtils rendering + pixi global install base-packages path.
• 8 templates under conda-pixi-lock-v1/ and conda-pixi-toml-v1/ (Dockerfile + Singularity, file + url).
• BuildTemplate constants and PixiOpts (manifest field).
• Unit/controller tests.

I also have a wave-cli branch ready.

• feat: add pixi lock file support with conda/pixi-lock:v1 template
• add test and support for user pixi.toml with fallback to autogenerate
• better support for pixi lock and lock/toml combinations
• allow basepackages again but put them in global env

[jpfeuffer]

Hi everyone, especially @pditommaso

I will have some more time to pick this up again. I'll obviously resolve merge conflicts first. Was there anything you would like me to add/change?
I think pixi lock files are going to be the reproducibility standard in the near future. IMHO resolving a pixi toml cannot guarantee full reproducibility of a pipeline and conda lock files are kind of outdated and were only added as a workaround to the limitations of conda, while pixi supports this natively.

I think it would be a great addition to wave.

[pditommaso]

IMHO resolving a pixi toml cannot guarantee full reproducibility

Then let's go with lock for pixi classic. Please name it conda/pixi:v1-lock

[jpfeuffer]

Cool! Happy to do that. There is one thing that I want to mention and your opinion on.
So, in theory pixi (--locked) is designed to always take both the toml "manifest" and the lockfile. In theory the toml can have activation commands or environment variables that are defined when activating the env. Also the toml could have different subenvironments (e.g. prod vs dev).

In my opinion for simple tomls this is kind of overkill and that is why I included a mechanism for using only a lock file where it creates a simple dummy toml with all the package names from the toml. This might also be seen as hacky so I want to mention that.

Maybe the toml + lock is cleaner but it makes the CLI etc a bit more clumsy since I think there isn't any precedent of a consumer with two required files yet.

Let me know if you want both (current state) or only one of those methods of using pixi with lock files. We could even make adding lockfiles a default or option for the existing pixi image. But I didn't want to be too invasive in the beginning.

[pditommaso]

I'd like to initiate with conda/pixi:v1-lock as independent template only, and eventually transition as default into conda/pixi:v2 later on

[jpfeuffer]

Thanks @pditommaso, I've reworked the PR along those lines and rebased it on current master:

• Renamed the template to conda/pixi:v1-lock (BuildTemplate.CONDA_PIXI_V1_LOCK) and dropped the pixi.toml-only template. conda/pixi:v1 and conda/pixi:v1-fast are unchanged.
  The lock file is sent inline (packages.environment) or as a URL in packages.entries, and works for Docker and Singularity.

• A pixi.toml can optionally be passed as pixiOpts.manifest, which keeps activation scripts and env vars. Without one, a minimal manifest is generated from the lock file.

One thing I need from you: the current default public.cr.seqera.io/wave/pixi:0.61.0-noble can only read lock files up to version 6. Pixi 0.80+ writes version 7, so lock files from a current pixi fail on that image. For conda/pixi:v1-lock I set the default to ghcr.io/prefix-dev/pixi:0.81.0-noble (PixiOpts.DEFAULT_PIXI_LOCK_IMAGE). It's only used when the request has no pixi image or the v1 default, so conda/pixi:v1 build IDs don't change. Could you copy 0.81.0-noble (or a newer tag) to public.cr.seqera.io/wave/pixi? After that I could switch the default back again.

The wave-cli options (--pixi-lock-file, --pixi-manifest) and the Nextflow side (pixi.lock in the conda directive) are ready on separate branches. I'll open them once a wave-api release includes this change, since wave-cli builds against the published wave-api.