From 9628b22dd2447667705571e61c0269bb83a16447 Mon Sep 17 00:00:00 2001 From: Myasnikov Daniil Date: Wed, 12 Aug 2026 15:19:49 +0500 Subject: [PATCH 1/2] docs(decisions): add a decision record log alongside design proposals Design proposals are intent, written before the work. When implementation contradicts the design the proposal gets revised in place, so it ends up reading as though it always said the current thing -- and the reasoning that changed our minds survives only in a pull-request diff. Add decisions/ for that reasoning: one page per settled decision, never edited after merge, superseded rather than rewritten. Reviewed for accuracy by one maintainer rather than by consensus, because the decision has already been made. Wire it into the "Where do I file?" table and rewrite Proposal Drift to say what to do when the implementation diverges. Assisted-By: Claude Signed-off-by: Myasnikov Daniil --- README.md | 2 + decisions/README.md | 91 ++++++++++++++++++++++++++++++++++++++ decisions/template.md | 59 ++++++++++++++++++++++++ design-proposals/README.md | 4 +- 4 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 decisions/README.md create mode 100644 decisions/template.md diff --git a/README.md b/README.md index ae43aef..f1da810 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ This repository serves as the home for everything related to how we collaborate, This repo contains: - [Design proposals](./design-proposals/README.md) — cross-cutting and architectural changes +- [Decision records](./decisions/README.md) — what we decided and why, once a question is settled - [The community meeting](./community_meeting.md) — when it happens, how to join, how to add a topic - [The meeting archive](./meetings/README.md) — every past meeting, its topics and its recording - Community resources and helpful links (below) @@ -26,6 +27,7 @@ Cozystack work is split across two repositories. This repo is for **how we decid | A **bug** or a **concrete, scoped feature** | [cozystack/cozystack issues](https://github.com/cozystack/cozystack/issues/new/choose) | | A **usage question** or an early idea | [cozystack/cozystack Discussions](https://github.com/cozystack/cozystack/discussions) | | A **cross-cutting / architectural change** — affects multiple components or APIs, or needs a decision before code | A [design proposal](./design-proposals/README.md) (a PR) **here** | +| A **decision already made** whose reasoning would otherwise survive only in a pull-request thread | A [decision record](./decisions/README.md) (a PR) **here** | | **Governance, process, or community** matters | An issue **here** | ## Community Resources diff --git a/decisions/README.md b/decisions/README.md new file mode 100644 index 0000000..8a26b86 --- /dev/null +++ b/decisions/README.md @@ -0,0 +1,91 @@ +# Cozystack Decision Records + +This folder holds **decision records**: short, dated notes on architectural decisions the project has actually made, and why. + +A [design proposal](../design-proposals/README.md) is intent — what we think we should build, written before the work. A decision record is history — what we settled on, written once the question is closed. Both are useful; the second is the one we have been missing. + +## Why a separate folder + +Design proposals get edited. When implementation contradicts the design — and it regularly does — the proposal is revised in place, so it ends up reading as though it always said the current thing. The reasoning that changed our minds (*we tried X, hit a concrete constraint, moved to Y*) then survives only in a pull-request diff that nobody will find in a year. + +That reasoning is the most expensive thing we produce and the easiest to lose. A decision record is where it goes. + +| | Design proposal | Decision record | +|---|---|---| +| Written | before the work, to get agreement | once the question is settled | +| Answers | "should we, and how might we?" | "what did we decide, and why not the alternatives?" | +| Edited after merge | yes, as the design evolves | never — superseded by a new record | +| Length | as long as it needs to be | one page | +| Review | consensus from maintainers | one maintainer checks it for accuracy | + +Both have an alternatives section, and they are not the same list. A proposal weighs the options we could imagine *before* building. A record names the option that lost *during* building — frequently the proposal's own original design. + +## When to write one + +Write a decision record when a future contributor would otherwise have to reconstruct the reasoning from a pull-request thread. In practice: + +- Implementation contradicted an accepted proposal and the design changed course. +- Two viable approaches existed and we picked one for reasons that are not visible in the resulting code. +- We hit a constraint that now shapes the design — an upstream limitation, a Kubernetes semantic, a vulnerability class. +- The code enforces a contract it cannot explain: a field is immutable, a release name is load-bearing, an ordering is required. +- We deliberately decided *not* to do something, and the question keeps coming back. + +Do not write one for: + +- Routine code choices that the diff and the tests already explain. +- Operator-facing how-to — that is user documentation, and it belongs on [the website](https://cozystack.io/docs/). +- The mechanics of the product repository's own workflow (release process, changelog conventions) — those live next to the code in [cozystack/cozystack](https://github.com/cozystack/cozystack). +- A decision that has not been made yet. That is a design proposal, or an open question inside one. + +## How to write one + +1. Copy [`template.md`](./template.md) to `decisions/NNNN-short-slug.md`, taking the next free number: + + ``` + ./decisions/0007-etcd-is-per-cluster-not-per-tenant.md + ``` + +1. **Title the decision, not the topic.** `storageClass is immutable after creation`, not `storageClass immutability`. Someone scanning the folder should learn what we decided from the filename alone. + +1. **Keep it to a page.** Link out to the proposal, the code and the pull requests for detail. A record that grows into a second design document will not get read. + +1. **Link both ways.** The record links its proposal and the pull requests that implemented it; the proposal's `Decisions` section links back to the record. + +1. As with all commits in CNCF projects, sign the commit for the DCO check: + + ```bash + git commit --signoff + ``` + +Two pull requests can claim the same number. Whoever merges second renumbers on rebase — a file rename and a couple of link fixes. + +## Status and immutability + +| Status | Meaning | +|---|---| +| `Accepted` | Current. The decision stands. | +| `Superseded by NNNN` | A later record replaced it. The original text stays exactly as written. | +| `Reverted` | We undid it. Say what we do instead. | + +Once merged, **the body of an accepted record is not edited.** Fix typos and broken links; change nothing else. If the decision changes, write a new record and mark the old one `Superseded by NNNN`. + +This one rule is what makes the folder a history instead of a second set of documents to keep current. A record that gets quietly rewritten is worth no more than the proposal it was meant to supplement. + +## Review + +A decision record documents a decision that has already been made. Review therefore checks that the record is **accurate** — not whether the reviewer agrees with it. One maintainer's approval is enough, and it should be quick. + +If review turns into re-litigating the decision, that is a signal the decision was not actually settled. Close the pull request and open a design proposal or an issue instead. + +## Where this sits relative to everything else + +| The question you are answering | Where it is answered | +|---|---| +| "Why is it built this way, and not the obvious other way?" | A decision record, here | +| "What should we build?" | A [design proposal](../design-proposals/README.md), here | +| "How do I use it?" | [User documentation](https://cozystack.io/docs/) on the website | +| "How does this repository release, or write changelogs?" | [cozystack/cozystack](https://github.com/cozystack/cozystack), next to the code | + +## Inspiration + +The format follows [Michael Nygard's architecture decision records](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) and [MADR](https://adr.github.io/madr/), trimmed to the sections we will actually fill in. diff --git a/decisions/template.md b/decisions/template.md new file mode 100644 index 0000000..ae64a11 --- /dev/null +++ b/decisions/template.md @@ -0,0 +1,59 @@ + +# NNNN. State the decision here, as a fact + +- **Number:** `NNNN` +- **Date:** `YYYY-MM-DD` +- **Status:** Accepted | Superseded by `NNNN` | Reverted +- **Deciders:** `@your-github-handle, @codecider` +- **Proposal:** `design-proposals//README.md` — or `none` +- **Implemented in:** `cozystack/cozystack#NNNN` — or `not yet` + + + +## Context + + + +## Decision + + + +## Why not the alternatives + + + +## Consequences + + + +## Revisit if + + + +--- + + diff --git a/design-proposals/README.md b/design-proposals/README.md index 83fba99..e473ce3 100644 --- a/design-proposals/README.md +++ b/design-proposals/README.md @@ -58,7 +58,9 @@ Proposals are reviewed in GitHub PRs. Once a proposal receives consensus from co We understand that real-world implementation may diverge from initial designs. This is normal and expected. -While it's helpful to update the proposal if the divergence is significant, the **codebase and user documentation are the final source of truth**. +Update the proposal when the divergence is significant. When the divergence came from a decision worth remembering — an approach that failed, a constraint you discovered, an alternative you picked instead — also write a [decision record](../decisions/README.md) and link it from the proposal's `Decisions` section. Editing the proposal alone loses the reasoning: the revised text reads as though it always said the current thing, and *why* the design changed course survives only in the pull-request diff. + +The **codebase and user documentation remain the final source of truth** for what the system does. The decision records are the source of truth for why it does it that way. ## Inspiration From efdf056c55971e32440561fc1363f3d348f51900 Mon Sep 17 00:00:00 2001 From: Myasnikov Daniil Date: Wed, 12 Aug 2026 15:19:55 +0500 Subject: [PATCH 2/2] docs(decisions): backfill 0001, ComputePlane as an operator-owned module The ComputePlane delivery shape was decided three times: a tenant module with an inline cluster HelmRelease (#17), preset fields on kind: Kubernetes (#27, closed), and finally an operator-owned extra module wrapping the unchanged apps/kubernetes (#33). Only the last one is visible in the proposal today; why the preset-field model lost lives in the #33 body. Record it as the first decision, and add the Decisions section to the proposal template so later records get linked from the proposal they amend. Assisted-By: Claude Signed-off-by: Myasnikov Daniil --- ...plane-ships-as-an-operator-owned-module.md | 44 +++++++++++++++++++ design-proposals/compute-plane/README.md | 4 ++ design-proposals/template.md | 12 +++++ 3 files changed, 60 insertions(+) create mode 100644 decisions/0001-computeplane-ships-as-an-operator-owned-module.md diff --git a/decisions/0001-computeplane-ships-as-an-operator-owned-module.md b/decisions/0001-computeplane-ships-as-an-operator-owned-module.md new file mode 100644 index 0000000..0936623 --- /dev/null +++ b/decisions/0001-computeplane-ships-as-an-operator-owned-module.md @@ -0,0 +1,44 @@ +# 0001. ComputePlane ships as an operator-owned module, not as preset fields on `kind: Kubernetes` + +- **Number:** `0001` +- **Date:** `2026-07-18` +- **Status:** Accepted +- **Deciders:** `@kvaps, @myasnikovdaniil` +- **Proposal:** [`design-proposals/compute-plane/README.md`](../design-proposals/compute-plane/README.md) +- **Implemented in:** not yet — see the proposal's Rollout section + +## Context + +The first ComputePlane revision ([#17](https://github.com/cozystack/community/pull/17), merged 2026-06-30) delivered a hardened, operator-controlled Kubernetes cluster for running code-executing catalog apps — notebooks, workflow code nodes, plugin systems — behind a per-VM kernel boundary instead of as shared-kernel pods on the management cluster. It was shaped as a tenant module selected by a single string (`computePlane: ""`), with the cluster's HelmRelease rendered inline in `apps/tenant/templates`. + +Two reviews converged on the same objection ([#26](https://github.com/cozystack/community/issues/26), and Timofei Larkin on [#17](https://github.com/cozystack/community/pull/17)): the cluster a ComputePlane runs on *is* an ordinary managed `kind: Kubernetes` — Kamaji control plane, KubeVirt-VM workers, operator-held kubeconfig — so ComputePlane should not become a distinct kind with a parallel cluster implementation, reconcile loop and RBAC surface. + +Revision [#27](https://github.com/cozystack/community/pull/27) answered that by putting the posture directly on the existing app as two orthogonal user-facing preset fields: `isolationProfile` (`standard` | `sandbox` | `cluster-meshed`) × `componentProfile` (`minimal` | `standard-addons` | `gpu`). A ComputePlane would then be nothing more than `kind: Kubernetes` with `isolationProfile: sandbox`. + +The isolation mechanism itself was never in question at any point — one-way remote Flux apply via `HelmRelease.spec.kubeConfig.secretRef`, untrusted code behind a disposable guest kernel, single-tenant, scoped egress. Only the delivery surface was contested. + +## Decision + +ComputePlane ships as a Cozystack-owned **tenant module** (`packages/extra/computeplane`) that provisions a hardened `kind: Kubernetes` by wrapping the unchanged `apps/kubernetes` chart with operator-fixed values, sourced through the existing PackageSource source-only-chart mechanism that NATS and SeaweedFS already use. + +Because a tenant module is registered through an `ApplicationDefinition`, ComputePlane does get its own `apps.cozystack.io` kind, with its own API endpoint and input schema — the same CRD-free way every managed service is registered. So the accurate claim is not "no new kind": it is **no new CRD, no new controller or reconcile path, and no new fields on `kind: Kubernetes`**. Catalog apps route onto the cluster through a `placement: ManagementPlane | ComputePlane` field on `ApplicationDefinition`. + +## Why not the alternatives + +- **User-facing preset fields on `kind: Kubernetes` ([#27](https://github.com/cozystack/community/pull/27)).** Putting the posture on the app object turns an operator's choice into tenant-editable API surface. The entire value of a sandbox is that the tenant *cannot* weaken it, and any field that configures the hardening is a field that can relax it. It also loads a hardened posture onto the one `ApplicationDefinition` that has to stay tenant-generic. +- **A distinct, heavyweight `kind: ComputePlane` with its own cluster implementation ([#17](https://github.com/cozystack/community/pull/17) as merged).** Duplicates the cluster reconcile and RBAC path, which was the substance of the #26 / #17-review objection. Note this decision does still register a `ComputePlane` kind — it departs from #26's literal "no new kind at all" — but as a thin wrapper over the unchanged app, which is what the objection was actually about. +- **The single-string `computePlane:` module rendered inline in `apps/tenant/templates` (the first revision's implementation).** Right delivery shape, wrong packaging: the cluster HelmRelease was inlined into the tenant chart instead of shipping as a first-class chart with its own PackageSource, the way `extra/seaweedfs` does. +- **Hardening containers in the tenant namespace instead.** Rejected as the primary boundary: hardening does not make container isolation multi-tenant, and it breaks the apps in scope. +- **gVisor or a sandboxed runtime as the primary boundary.** Rejected for incomplete syscall coverage and no blast-radius containment on a kernel panic. Still valid as an *inner* layer for per-task sandboxes inside the cluster, and tracked as a future runtime option rather than dismissed. + +## Consequences + +- The hardening is tamper-proof by construction. The cluster's values live entirely in an operator-owned chart and the tenant holds no admin kubeconfig, so "withhold admin" and "own the configuration" become one fact rather than two features to keep in sync. +- `kind: Kubernetes` gains no fields and stays tenant-generic. Node-provisioning and addon changes to the app apply to ComputePlane transparently. +- The cost is composability: several differently-hardened postures per tenant now require separate module variants rather than a combination of two fields. The `placement: ` path that would give a tenant N sandbox clusters is deferred. +- The `computeplane` release name is load-bearing (proposal, Design §1). Any later move of the chart between directories must preserve it. +- [#39](https://github.com/cozystack/community/pull/39) (folding `extra` into `apps` as declarative `ApplicationDefinition` capabilities) composes with this rather than conflicting: ComputePlane becomes a directory move plus `visibility: module` and `cardinality: {scope: tenant, max: 1}`, and the two-piece structure carries over unchanged. + +## Revisit if + +[#39](https://github.com/cozystack/community/pull/39) lands and changes how module kinds are registered, or a concrete need appears for several simultaneous, differently-hardened compute clusters per tenant — the case the single-module shape does not serve and the deferred `placement: ` path would. diff --git a/design-proposals/compute-plane/README.md b/design-proposals/compute-plane/README.md index a64f978..27710f8 100644 --- a/design-proposals/compute-plane/README.md +++ b/design-proposals/compute-plane/README.md @@ -36,6 +36,10 @@ The capability is generic and intended to live in Cozystack core as a reusable p - **`design-proposals/kubernetes-nodes-split`** / **`kubernetes-nodes-hybrid-clusters`** (PR #8/#9): the substrate is the existing managed-`kubernetes` app (Kamaji + CAPI/KubeVirt); node-provisioning changes apply transparently. - **Deferred:** billing/metering of cluster resource and API consumption; secret delivery of managed-service connection strings into sandbox workloads; the per-instance/label granularity of the visibility control (Design §6). (Cross-tenant *sharing* of a cluster is **not** deferred — it is rejected by design; see Non-goals.) +## Decisions + +- [0001. ComputePlane ships as an operator-owned module, not as preset fields on `kind: Kubernetes`](../../decisions/0001-computeplane-ships-as-an-operator-owned-module.md) — why the preset-field revision (#27) was set aside. + ## Context Today Cozystack already has every primitive needed *except* the glue that ties them into "deploy this catalog app onto a hardened, operator-controlled `kind: Kubernetes` the tenant does not administer": diff --git a/design-proposals/template.md b/design-proposals/template.md index 176743a..ae2541c 100644 --- a/design-proposals/template.md +++ b/design-proposals/template.md @@ -20,6 +20,18 @@ A reader who stops here should know what you're asking for and roughly why. --> Link them by repo path or URL. If this proposal must land before or after another, say so. Omit the section if there are no related proposals. --> +## Decisions + + + ## Context