Skip to content

feat(spec): GF-T softmax+cross-entropy gradient — the backward pass (Refs #1764) - #1806

Merged
gHashTag merged 1 commit into
feat/gft-log2-nllfrom
feat/gft-softmax-grad
Aug 6, 2026
Merged

feat(spec): GF-T softmax+cross-entropy gradient — the backward pass (Refs #1764)#1806
gHashTag merged 1 commit into
feat/gft-log2-nllfrom
feat/gft-softmax-grad

Conversation

@gHashTag

@gHashTag gHashTag commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Stacked on #1805. Base retargets to master as the stack merges.

gft_softmax_grad4.t27 — the softmax + cross-entropy gradient

For a one-hot target t, the classic closed form:

∂L/∂l_i = p_i − y_i

So the entire backward pass is the softmax forward p_i minus the one-hot label — grad = (i==t) ? sadd(p, −1.0) : p. Composes the verified softmax (max, sadd+neg, exp2, recip, RNE mul).

  • Bit-exact to the integer oracle 1600/1600 (iverilog).
  • Gradient accuracy ≤0.0018 abs vs true p_i − y_i.
  • Spot: uniform logits, target 0 → grad₀ = 0.25 − 1 = −0.75 exact, grad₁ = 0.25 exact.

Forward + backward now complete on GF-T: logits → softmax → prob → NLL loss, and → gradient p−y. With the exp2/log2 inverse pair (#1805), the per-sample training step — forward loss and the backprop signal into the logits — is fully expressible spec-first and iverilog-verified. Foundation for on-device training, not just inference.

Fresh seal (seal --verify → MATCH). No compiler change.

Refs #1764

⚠️ GitHub Actions runner availability still degraded (jobs queue). Verified locally (iverilog + seals).

🤖 Generated with Claude Code

gft_softmax_grad4.t27 (GftSoftmaxGrad4): the softmax + cross-entropy gradient
over four signed GF-T16 logits. For a one-hot target t, dL/dl_i = p_i - y_i (the
classic closed form) -- the backward pass is just the softmax forward p_i minus
the one-hot label (grad = (i==t) ? sadd(p, -1.0) : p). Composes the verified
softmax primitives. Bit-exact to the integer oracle 1600/1600 (iverilog);
gradient accuracy <=0.0018 abs vs true p_i-y_i. Uniform logits, target 0 ->
grad0 = -0.75 exact, grad1 = 0.25 exact.

Forward + backward now complete on GF-T: logits -> softmax -> prob -> NLL loss,
and -> gradient p-y. With the exp2/log2 inverse pair, the per-sample training
step is fully expressible spec-first and iverilog-verified.

Fresh seal for GftSoftmaxGrad4 (seal --verify MATCH). No compiler change.

Refs #1764

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gHashTag
gHashTag merged commit 7128d2f into feat/gft-log2-nll Aug 6, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant