Skip to content

Propagate a NaN through xLAE2 and xLAEV2 instead of returning finite eigenvalues - #1405

Open
rmlarsen wants to merge 2 commits into
Reference-LAPACK:masterfrom
rmlarsen:lae2-nan-propagation
Open

Propagate a NaN through xLAE2 and xLAEV2 instead of returning finite eigenvalues#1405
rmlarsen wants to merge 2 commits into
Reference-LAPACK:masterfrom
rmlarsen:lae2-nan-propagation

Conversation

@rmlarsen

@rmlarsen rmlarsen commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Disclaimer: The initial changes were prepared using Claude Code; review and subsequent test fixes used Codex.

Summary

xLAE2 and xLAEV2 compute the eigenvalues of the symmetric 2 by 2 matrix [a b; b c] through a three-way comparison of |a-c| with |2b|, whose last branch is meant for the equal case. For a NaN diagonal entry both comparisons are false, the last branch computes rt = |2b| sqrt(2), finite, and the sign test on a+c that follows is false too, so the routines returned rt1 = -rt2 = |b| sqrt(2) with no trace of the NaN. Every 2 by 2 tridiagonal eigenproblem goes through this code: xSTEQR, xSTERF, xSTEV, xSTEVD, xSTEDC and xSTEMR returned INFO = 0 and finite eigenvalues for a 2 by 2 matrix with a NaN on its diagonal (a NaN off the diagonal was propagated). This PR takes the equal-case branch only when the operands are equal and propagates a NaN otherwise. The numerical changes are in {s,d}lae2.f and {s,d}laev2.f, with regression coverage in {s,d,c,z}chkst.f; cLAEV2 and zLAEV2 call the real routine.

Description

      IF( ADF.GT.AB ) THEN
         RT = ADF*SQRT( ONE+( AB / ADF )**2 )
      ELSE IF( ADF.LT.AB ) THEN
         RT = AB*SQRT( ONE+( ADF / AB )**2 )
      ELSE
*        Includes case AB=ADF=0
         RT = AB*SQRT( TWO )
      END IF
      IF( SM.LT.ZERO ) THEN
         RT1 = HALF*( SM-RT )
      ELSE IF( SM.GT.ZERO ) THEN
         RT1 = HALF*( SM+RT )
      ELSE
*        Includes case RT1 = RT2 = 0
         RT1 = HALF*RT
         RT2 = -HALF*RT
      END IF

With a a NaN, sm = a + c and adf = |a - c| are NaN, ab = |2b| is finite, both branches of the first test are skipped, rt = ab sqrt(2), and the sm test lands in its last branch as well.

Fix. The last branch of the first test is taken only for ADF.EQ.AB; otherwise one of the two is a NaN and RT = ADF + AB propagates it, after which sm is a NaN too (a NaN diagonal entry makes both sm and adf NaN), so RT1 and RT2 come out as NaN. Finite arguments take exactly the branch they took before and get the same value.

Minimal reproducer

! DSTEQR('N') and DSTERF on the 2x2 matrix [NaN 1; 1 2].
program minimal
  implicit none
  double precision :: d(2), e(2), z(2,2), work(4), zero
  integer :: info
  zero = 0d0
  d = [zero / zero, 2d0]; e = [1d0, 0d0]
  call dsteqr('N', 2, d, e, z, 2, work, info)
  print '(a,i0,a,2es12.4)', 'DSTEQR: info = ', info, '  eigenvalues =', d
  d = [zero / zero, 2d0]; e = [1d0, 0d0]
  call dsterf(2, d, e, info)
  print '(a,i0,a,2es12.4)', 'DSTERF: info = ', info, '  eigenvalues =', d
end program
BEFORE (master):     DSTEQR: info = 0  eigenvalues = -1.4142E+00  1.4142E+00
                     DSTERF: info = 0  eigenvalues = -1.4142E+00  1.4142E+00
AFTER (this branch): DSTEQR: info = 0  eigenvalues =         NaN         NaN
                     DSTERF: info = 0  eigenvalues =         NaN         NaN

+/- sqrt(2) = +/- |b| sqrt(2): the diagonal, one entry of which is a NaN, played no part in the result.

Regression test. xCHKST calls xSTEQR('N'), xSTERF, xSTEMR('N','A'), xSTEQR('I'), and xSTEMR('V','A') on the 2 by 2 matrix [1 1; 1 2] with a NaN in either diagonal entry. The eigenvalue-only calls exercise xLAE2, and the eigenvector-producing calls exercise xLAEV2. A successful call must return NaN eigenvalues; diagnostics identify the routine and job option. Reverting only SLAEV2 and DLAEV2 makes the new vector cases fail in every precision, while the complete fix passes.

Validation

  • Current tests: gfortran 13.3 on x86-64, reference BLAS, -O2 -fcheck=all; all 8 focused sep.in/se2.in driver runs pass.
  • 8 AddressSanitizer cases pass with small caller arrays and compact/padded leading dimensions.
  • The updated regression still fails against the parent numerical kernels in every affected precision. It also fails when only SLAEV2 and DLAEV2 are reverted.
  • The supplied minimal reproducer was independently checked against the parent and the numerical fix and matches the output above.
  • The full suite and cross-architecture/compiler matrix have not been rerun for this test-only follow-up. Earlier validation of the unchanged numerical kernels included the full LAPACK suite and the special-value/finite sweeps described in the original submission.

Found while auditing the symmetric tridiagonal eigensolvers for the NaN and overflow handling of #1377-#1391.

Update: Codex review identified that the eigenvalue-only regression calls did not reach LAEV2. STEQR(I) and STEMR(V,A) now exercise that path, and reverting only LAEV2 makes the added cases fail.

…eigenvalues

xLAE2 and xLAEV2 compute the eigenvalues of the symmetric 2 by 2
matrix [a b; b c] from sm = a + c, adf = |a - c| and ab = |2b|:

   IF( ADF.GT.AB ) THEN
      RT = ADF*SQRT( ONE+( AB / ADF )**2 )
   ELSE IF( ADF.LT.AB ) THEN
      RT = AB*SQRT( ONE+( ADF / AB )**2 )
   ELSE
      RT = AB*SQRT( TWO )
   END IF

The last branch is meant for adf = ab, but a NaN diagonal entry makes
adf a NaN, both comparisons false, and rt = |2b| sqrt(2), finite.  The
sign test on sm that follows is false for a NaN as well, so the
routines returned rt1 = -rt2 = |b| sqrt(2) with no trace of the NaN.
Every 2 by 2 tridiagonal eigenproblem goes through this code: xSTEQR,
xSTERF, xSTEV, xSTEVD, xSTEDC and xSTEMR returned INFO = 0 and finite
eigenvalues for a 2 by 2 matrix with a NaN on the diagonal, while a
NaN off the diagonal was propagated.

Take the last branch only for adf = ab and let a NaN in either operand
propagate, rt = adf + ab, so that the eigenvalues, and in xLAEV2 the
eigenvector, come out as NaN.  Finite arguments are not affected: the
branch they take is unchanged and so is the value it computes.

xCHKST gets the case as a regression test: it calls xSTEQR, xSTERF
and xSTEMR on the 2 by 2 matrix [1 1; 1 2] with a NaN in place of
either diagonal entry and reports a finite eigenvalue as a failure.
On the parent commit all six calls return INFO = 0 and finite
eigenvalues in every precision; here every eigenvalue is a NaN.  Over
the 1 by 1 and 2 by 2 NaN cases of a sweep of every tridiagonal solver
the parent returned an all-finite result in 62 of 111 cases and this
branch in 16, all of them calls that return no eigenvalue at all
(RANGE = 'V' or 'I' with M = 0) or bisection paths, which do the same
for larger n; every finite case is bit-identical.

The full LAPACK test suite passes: 0 numerical errors, 0 other errors,
120 tests more than the parent from the new calls.

The regression test fails on the parent and passes with the fix, and the
reproducer prints the same before and after output, with gfortran 13
(x86-64 Release and Debug with -fcheck=all, and under QEMU on aarch64,
ppc64le, s390x and riscv64), flang-19 and Intel ifx 2025.3.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.59459% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.37%. Comparing base (9eaccc1) to head (349f09f).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
TESTING/EIG/cchkst.f 94.28% 2 Missing ⚠️
TESTING/EIG/dchkst.f 94.28% 2 Missing ⚠️
TESTING/EIG/schkst.f 94.28% 2 Missing ⚠️
TESTING/EIG/zchkst.f 94.28% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           master    #1405    +/-   ##
========================================
  Coverage   69.36%   69.37%            
========================================
  Files        6122     6122            
  Lines      486337   486485   +148     
  Branches    23268    23268            
========================================
+ Hits       337330   337476   +146     
- Misses     148569   148571     +2     
  Partials      438      438            
Components Coverage Δ
BLAS 97.94% <ø> (ø)
CBLAS 96.98% <ø> (ø)
LAPACK 82.39% <100.00%> (+<0.01%) ⬆️
LAPACKE 2.17% <ø> (ø)
TMGLIB 55.69% <ø> (ø)
BLAS testing 88.33% <ø> (ø)
CBLAS testing 89.63% <ø> (ø)
LAPACK testing 82.25% <94.28%> (+0.01%) ⬆️
LAPACKE testing ∅ <ø> (∅)
Files with missing lines Coverage Δ
SRC/dlae2.f 100.00% <100.00%> (ø)
SRC/dlaev2.f 100.00% <100.00%> (+6.25%) ⬆️
SRC/slae2.f 100.00% <100.00%> (ø)
SRC/slaev2.f 100.00% <100.00%> (+6.25%) ⬆️
TESTING/EIG/cchkst.f 64.98% <94.28%> (+1.83%) ⬆️
TESTING/EIG/dchkst.f 65.01% <94.28%> (+1.85%) ⬆️
TESTING/EIG/schkst.f 65.01% <94.28%> (+1.85%) ⬆️
TESTING/EIG/zchkst.f 64.98% <94.28%> (+1.83%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9eaccc1...349f09f. Read the comment docs.

Add STEQR(I) and STEMR(V,A) cases for both diagonal NaN positions in
all four precisions. Include the job option in diagnostics and check
the real workspace capacity for the complex drivers.

Validation: 8 sep/se2 driver runs and 8 AddressSanitizer cases passed.
All four precisions detect reverting only SLAEV2/DLAEV2, and all four
also retain the expected failures against the parent kernels.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant