Add the solver subtrack, and gate its page on what the kernel is - #27
Merged
Merged
Conversation
A padded allocation extent (mg_vcycle's `flat`, amg_setup's `npad`/`zpad`) is never subscripted, so the Fortran emitter's usage-role inference never saw it and declared it real(c_double) -- which gfortran rejects as "Legacy Extension: REAL array index" at every allocate under -std=f2018. The C backend already typed such a local from its VALUE; that inference moves to numpyto_common.lowering as `integer_valued_locals` and the Fortran classifier consults it, so the two backends share one rule instead of drifting. With the compile error gone amg_setup's fortran leg returned a wrong level_n: A_indices carried `index_array: true`, but the kernel also COMPARES those column ids against 0-based row ids (`a_indices[k] == i`), so the one-based seam shifted a buffer that has no single base. The tag is dropped, which changes nothing for c/cpp (both are zero-based). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eader owns A pinned config knob was declared as a file-scope constexpr under its own name, so rk45_ensemble's `atol` landed beside <stdlib.h>'s `long atol(const char *)`: gcc -std=c23 called it an underspecified declaration of a name already in scope, g++ a redeclaration as a different kind of entity, and the C++ leg then failed again on `atol + rtol`. The declaration now takes an emitter-owned name when a standard header already declares that identifier, with a #define mapping the reference's own spelling onto it, so every use -- body, helper, VLA bound in the signature -- still reads as the kernel wrote it. `atol` is the only collision across the 118 knob names in the corpus, so every other emitted source is byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Emitting a kept helper as its own @dc.program moved 36 generated programs from parsing to refused and 10 from agreeing with numpy to disagreeing. Every cause is in the shape a helper's parameters are DECLARED with, and each is fixed where it arises: - two parameters bound to ONE caller symbol aliased only the first, so `_conv2d(.., kh, kw)` called with `kernel_size` twice declared its return in `kh` while its body computed in `kw`; - a scalar divisor (`acc / (kh * kw)`) counted as an array the sweep could not size, so the out-param fell back to a broadcast join over the ARGUMENTS and took the pool's INPUT shape; - integer scalar PARAMETERS were not symbols to the slice-span inliner, leaving `span_h = (oh - 1) * stride + 1` un-spliced against `ceiling(__sym_span_h/__sym_stride)`; - an out-param declared in the caller's vocabulary while the body allocated in the helper's own; - a symbol that CANCELS out of an extent (`(ci + 1) * 4 - ci * 4`) still demanded as an argument; - `np.newaxis` CONSUMED a source dimension instead of inserting one, so `x1[:, None, :]` over a rank-2 array inferred rank 1; - the caller's recipe for an extent and the helper's parameter for it left standing as two names; - DivisibleStridedSpan matched `lower + span` on the top node only, so the left-associative `oy0 + (h - 1) * stride + 1` carried the idiom past the rewrite that exists for it. Two are not DaCe's alone. The divisor sized `_avgpool2d`'s out-param off the pool's input, and the newaxis walk mis-ranked every helper argument written with one: C and Fortran allocate from the same descriptors and reported neither. conv_standard_2d_square_input_square_kernel pins its stride/dilation/groups as config knobs, which is what its `out` extent already assumed by spelling the stride as a literal 4. Separately, four gates that the taxonomy retirement in 80fbd17 left asserting what is gone. `min_precision: fp64` was nested inside the deleted `taxonomy:` block, so both mandelbrots and mixed_precision_ir lost the fp64 floor they are chaotic enough to need and were being graded at fp32; the key is still schema-legal, so it moves to the top level. The solver roster selected on a `spec.tags` that no longer exists, for a tag now spelled `solvers`. cp2k_grid_integrate's manifest test asserted the retired `kind` beside the `level` that replaced it. And the annotation baseline counted a gitignored file, which no checkout can ever clear, so the ratchet now counts only what git tracks -- 5896 across 356 files, lib_nodes.py's 485 among what came off. A generated module now names its kernel program. Kept helpers are @dc.programs too, so a reader can no longer take the sole one, and the name matches neither the file stem (lenet -> lenet5) nor a fixed word (nussinov -> kernel): three resolvers were answering "no program" for modules that have one. Left refused, with causes rather than excuses: two conv_transpose kernels still spell one extent two ways; jfnk_bratu and sgs_pcg cannot infer their symbols; and hotspot_rodinia passes float scalars computed from a symbol (`0.016 / N`) into a nested program, where DaCe's dtype mapping has no entry for sympy.Float -- upstream, not an emitter bug. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pinned config knob whose name a standard header already declares (rk45_ensemble's atol against <stdlib.h>'s long atol(const char *)) is now declared under an emitter-owned name, with a #define mapping the reference's own spelling onto it. atol is the only collision across the corpus's 118 knob names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A padded allocation extent is never subscripted, so the Fortran emitter's usage-role inference never saw it and declared it real(c_double) -- a REAL array index gfortran rejects. The C backend's value-based inference moves to numpyto_common.lowering so both backends share one rule. Behind that compile error, amg_setup's A_indices carried index_array: true while the kernel also compares those column ids against 0-based row ids, so the one-based seam shifted a buffer with no single base. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… and fortran Four causes, three of them in the shared translator path: a contiguous sub-array handed to a kept helper now becomes a pointer offset instead of meeting a bare ast.Slice; a reduction helper (array in, scalar out) is no longer mistaken for an array-returning one and broadcast over its own operand; the Krylov basis is indexed leading-axis-first, because a trailing-axis slice is a strided view no pointer can carry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…native legs Four causes behind one reported Slice refusal. A contiguous sub-array argument to a kept helper (qu[k, :, :]) met the general expression emitter and now emits a pointer offset. A reduction helper -- array in, scalar out, which is every 2-norm and dot product in both kernels -- was typed as array-returning, so the caller allocated an operand-shaped buffer and broadcast the call over it, one invocation per element reduced; the rank-0 decision is now gated on a PROVABLE rank-0 body rather than on None, which also means 'could not size'. jfnk_bratu's Q is transposed to match the convention bdf already used. And bdf's lagrange_weights divided a row by a pivot it was overwriting mid-loop -- numpy reads the pivot once, the scalarized C loop did not -- which is why that kernel reported a timeout rather than a compile error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
conv3d_softmax_max_pool_max_pool and densenet121_transition_layer were listed as broadcast refusals and both return verdict ok. The ratchet runs both ways -- an entry that no longer refuses is slack a real regression can hide in -- so the gate fails on a stale entry exactly as it fails on a new one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A nested @dc.program call is bound by SOLVING the callee's symbols from the shapes the call site passes, and these two kernels sat on opposite failures of that solve. sgs_apply's three CSR parameters were declared (3*NX-2)*(3*NY-2)*(3*NZ-2) and NX*NY*NZ+1 over a body naming only N: one equation per extent however many symbols it spells, so three equations for four unknowns and sympy answered with a one-parameter family of quadratics. with_solvable_extents gives an extent naming anything nothing else supplies one symbol of its own, and retires what that leaves in no shape. bratu_jvp(u, Q[:, :, k], ...) handed a rank-3 array's trailing-index view -- strides (N*(m+1), m+1) -- to a parameter declared [N, N], whose strides are (N, 1). dace equates strides as well as shapes, so the system said __SOLVE_N = N and __SOLVE_N = 51*N at once and had no solution; the view would have read the wrong elements had it been accepted. materialize_strided_helper_args copies such an argument through a contiguous temp, and copies it back when the callee writes it. A leading-index plane is already contiguous and is left alone. Fixing that exposed a third defect, in the shared frontend: bratu_dot returns a scalar accumulator, _helper_return_shape_from_body spells "carries no extent" and "could not size" both as (None, None), and the caller kept the call-site guess in the second case -- a broadcast join over the call's own (N, N) arguments, so a helper returning one number got an (N, N) out-param and the caller stored a whole grid into H[p, k]. helper_returns_a_scalar_local proves the by-value case instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sgs_apply's CSR parameters were declared as compound extents over a body that names only N, so dace had three equations for four unknowns and sympy returned a one-parameter family; an extent whose names the callee never reads now becomes one minted symbol and the system is square. bratu_jvp was handed Q[:, :, k] -- a trailing-index view whose strides are not the declared parameter's -- and dace equates strides as well as shapes, so the system had no solution at all; the argument is copied through a contiguous temp, because accepting the view would have walked the wrong elements rather than refused. Both agents that touched this independently found the same defect underneath: a body that returns a scalar and a body that could not be sized were both reported as None, so a reduction helper got an operand-shaped out-param. One predicate is kept -- the more general one, which handles an expression return and not only a bare name -- and its call into _extent_operands_resolved is corrected to pass the scalar set that function now requires. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A kept helper's descriptors are spelled in the caller's vocabulary while its body speaks its own parameter names, and the two can use one word twice or two words once. Collapse a by-value extent parameter onto the caller symbol the descriptors already carry (_conv_transpose2d's `stride` against `conv_transpose_stride`), never adopt a helper name the descriptors already spell for the caller's own quantity, and refuse a helper whose captured name has no other spelling so the existing inline fallback emits it instead. conv_transpose2d_max_pool_hardtanh_mean_tanh, conv_transpose3d_leaky_relu_multiply_leaky_relu_max and conv_transpose3d_batch_norm_avg_pool_avg_pool go fail -> ok; 669 of the 679 corpus kernels emit byte-identical source and no kernel goes ok -> fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…d rather than mis-emitted A helper that receives one quantity twice -- as a by-value scalar parameter and as the caller symbol its descriptor is written in -- computed its extents in the scalars while declaring them in the symbols, and dace had no equation relating the two. A scalar parameter now collapses onto the caller symbol its own argument already names. The 3d case is not that. Its descriptor spells the CONV kernel and stride while the call passes the POOL ones, so the module-level dc.symbol is shadowed by the callee's same-named parameter and one symbol stands for two extents. Neither name has an un-captured spelling, so the emitter refuses the kept-helper form and the existing inline fallback emits the flattened program -- the refusal is skipped where helpers are already inlined, so a helper that resists inlining keeps its emission rather than losing the kernel. Evidence for no regression is a source diff, not a re-parse: all 679 kernels were emitted before and after and 669 are byte-identical, so their verdicts cannot move. The 10 that changed were probed both ways -- three fail -> ok, none ok -> fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ash agree with numpy again Two causes, both opened by emitting a kept helper as its own @dc.program instead of inlining it. Both were silent: the five parse, compile, run, and return different numbers. The first four are ONE defect, and it is DACE'S, not the emitter's -- what lands here is a rewrite that structures around it, not a fix for it. DaCe's frontend lowers any `return` into a ReturnBlock, and its codegen emits a nested program's blocks INLINE in the caller's generated function, so a helper's bare return becomes a literal `return;` out of __program_<kernel>_internal. The helper's own write lands; everything in the CALLER after the call site is skipped. eigh_test came back with wout/vout at their input values because the whole eigensolve after the first hermitian_from_triangle call never ran. Measured standalone: a caller-local temp is correct without the return and wrong with it, and writing the caller's own parameter is ALSO wrong with it as soon as one statement follows the call -- so the buffer is not the variable, the return is. Survives simplify=True and simplify=False; the SDFG is well-formed either way. That return is not gratuitous. _rewrite_returns_to_outparam adds it so the C and Fortran legs emit the helper as a void out-param procedure, and it must keep working there -- so render_program structures it away for the dace leg alone, with two exact source-level rewrites: a tail-position bare return is dropped (it says what falling off the end already says), and a guard that exits (`if c: <A>; return` with <B> after it) becomes `if c: <A> else: <B>`. A return inside a loop, or under a guard that already carries an else, is left alone rather than guessed at. spgemm_hash is a second, unrelated cause. A helper's scalar parameter is typed from the CALL-SITE argument, and an element of a kernel LOCAL array resolves to nothing and falls to float64 -- row_bin[row] is int64 and arrived as a double. Inlined that cost nothing, because the body was spliced into the caller and the value kept its own type; as a kept @dc.program the body counts and subscripts with it, and g++ refuses the generated code with `invalid types 'int64_t*[double]' for array subscript`. widen_counting_scalar_params reads the helper's own body instead: a range() bound and a subscript index are integers in all three languages. C and Fortran emit correctly with it, gfortran adding INT(..., c_int64_t) at the call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… exits the caller A return inside a nested @dace.program returns from the CALLER. dace lowers it to a ReturnBlock and codegen emits a nested program's blocks inline in the caller's function, so the bare return becomes a C return; out of __program_<kernel>_internal and every statement after the call site is skipped. The helper's write lands -- the rest of the kernel never runs, which is why eigh_test came back with its outputs still at their input values. This is a DaCe defect. The emitter STRUCTURES THE CONSTRUCT AWAY for the dace leg rather than fixing it: a tail-position bare return is dropped, and a guard that exits becomes an else. The construct is not gratuitous -- _rewrite_returns_to_outparam adds it for the C and Fortran void-out-param form and must keep working there. spgemm_hash was a second, unrelated cause: a helper scalar the body uses as a range bound or a subscript index was typed from a call-site argument that resolves to nothing and fell to float64, giving int64_t*[double] subscripts. Such a parameter is an integer in all three languages. Worth recording for whoever reads this next: 89 corpus kernels emit a second @dc.program, all in machine_learning/, and test_dace_numeric_agreement excludes that track by design -- so every one of them with a promoted return was miscompiling in silence and nothing was measuring it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lence again 89 corpus kernels emit a second @dc.program and every one is in machine_learning, which this gate excludes by design. So when a nested return started returning from the CALLER, layer_norm was out by 2.45e+01, max_pooling_2d by 1.54e+01 and instance_norm by 9.43e+00, and nothing said so -- the parse ratchet stayed green the whole time, because parsing was never the property that mattered for them. The four added here are witnesses for the construct, not a sample of the track: each emits a kept helper, and all three of the named ones go RED with the return rewrite disabled and green with it (verified in place, not argued). They cost 28 s together, against the ten minutes the docstring says the full track would. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…her than only read 492a367 broke tests/test_dace_helper_programs.py and I shipped it, because two agents reported that test as pre-existing and I took their word: pre-existing at THEIR base, which was already downstream of the break. It passes at 9505e6a and fails from 492a367 on. caller_side_recipe only recognised a name it could trace to a single assignment, and a free symbol has none -- so a helper whose extent argument is a bare symbol got no recipe, kept the caller's spelling in its descriptors and its own parameter's in its body, and refused to parse: declared [N] and computed in n. A name the owner never assigns stands for itself. The convention test could not have caught it -- the broken emission satisfies every assertion it makes, and I checked rather than assumed. What caught it was the parse, so the parse is now a test: it goes red on the old emission naming the offending call site. The keyword assertion is also made exact, because demanding that SOMETHING be passed said more than the contract does -- a helper all of whose symbols are inferable needs no keyword at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…hat is waiting Full-corpus sweep at this head: 679 kernels, one refusal nothing on the list excuses, and nothing on the list that parses. This is that one. It is not the emitter. DaCe folds a scalar whose symbols CANCEL -- hotspot computes Rx from two grid spacings that are both chip_extent / N -- into a sympy.Float carrying no free symbols, which issymbolic reports as not-symbolic and dtype_to_typeclass holds no key for. The fix demotes such a value where it is PRODUCED (patching the consumer only moves the refusal to the next gate, which asks the same two questions); it is committed as hotspot-const-fold 0bd65f83c with 765 frontend tests green, and is not pushed because local extended carries 25 commits belonging to other work. The entry comes off the moment that reaches the tip CI installs from, and the ratchet fails until it does -- which is the property that makes this list worth keeping rather than a place to put things. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
170adec made a bare caller symbol its own recipe, so a helper whose extent argument is one would collapse its parameter name onto it. That fixes test_dace_helper_programs, and it stops max_pooling_2d and conv_transpose2d_max_pool_hardtanh_mean_tanh from parsing: the rename reaches the body and only some of the descriptors, leaving one shape spelled two ways on either side of a write. Narrowing it to symbols the callee's own array shapes already mention did not help -- measured, both still failed identically. The underlying emitter bug is real, and test_dace_helper_programs stays red until it is fixed properly, with the rename landing on descriptors and body at once. A red test naming a real defect is worth more than a green one bought by breaking two kernels that were right. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ogether 170adec made caller_side_recipe answer for a bare free Name, and that answer landed in the SAME map the alias pass writes, in the opposite direction. The alias pass keeps the helper's own name and retires the caller's (channels -> c); the recipe pass then read the same pair the other way round (c -> channels). with_helper_vocabulary sends aliases plus collapse through respell() onto the DESCRIPTORS and collapse alone through RenameNames onto the BODY, so a map holding both edges moves the two sides apart: max_pooling_2d declared [batch_size, c, h, w] over a body computing in channels/height/width, and the frontend refused it. The seam is the one the alias pass was missing rather than a better predicate for when to try. Its scalar half already existed and already had the right guards; what stood between it and _scale was an exclusion for a symbol the descriptors state OUTRIGHT, on the reading that dace solves such a symbol from the argument while the body's own name rides along as a keyword. Both are bound, but bound is not EQUAL -- _scale declared [N] over a body allocating np.empty(n) and its closing write was refused -- so the exclusion goes and caller_side_recipe returns to declining a free Name. Measured over the whole corpus by emitting every kernel before and after and parse-probing only what changed: 66 kernels move, all machine_learning, 0 regressions, 37 fail -> ok, 29 fail -> fail and every one of those already on REFUSED. The tip was broken far past the two kernels that were known: 28 of the 37 died on the broadcast this fixes. Two REFUSED kernels change their cause and keep failing (conv_depthwise_2d_square_input_asymmetric_kernel broadcast -> reassign, conv_depthwise_separable_2d broadcast -> a dace const-fold KeyError on sympy Zero); the labels are left as they stand rather than relabelled off one probe. The fixture that catches this class gets a second shape: a helper whose own parameters SHADOW the caller's names for the same dimensions, which is what the corpus carries. Both new tests go red on the old emission -- the structural one because the annotations and the body then name two different module symbols per dimension, the parse one because dace says so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The alias pass and the recipe loop were both answering for the same (parameter, argument) pair, in opposite directions. The alias pass keeps the helper's own name and retires the caller's; the recipe loop, once a bare free Name was made its own recipe, recorded the inverse edge. with_helper_vocabulary sends the merged map through respell() onto the DESCRIPTORS and collapse alone through RenameNames onto the BODY -- so a map holding both directions is not a function, and the two sides moved apart: max_pooling_2d declared one vocabulary over a body computing in the other. That is why narrowing the predicate could not help. The predicate was never the problem; the second authority was. A free symbol has no recipe again, and the scalar half of the alias pass drops an exclusion that assumed dace would solve a bare declared dimension from the argument while the body's own name rode along as a keyword -- bound is not equal, and dace refused the closing write. The tip was broken well past the two kernels that surfaced it: of 66 kernels whose emitted text changes, 37 go from refused to parsing and 28 of those died on exactly this. None of the 37 is on REFUSED, so the shrink direction is untouched and nothing was added to any list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A fused slice assignment stores one element per iteration, so an RHS read of a scalar element of the array it writes could be served from a slot an earlier iteration had already overwritten -- A[k, k:] = A[k, k:] / A[k, k] divided by the 1.0 it had just stored, and c / cpp / fortran all disagreed with numpy by 1.86e+00 on a 6x6. The pivot is now staged into a temp ahead of the loop nest, which is where numpy reads it. Invariance is structural (full rank, no Slice, no newaxis, no index array), so a non-aliasing kernel keeps its value exactly and trades a trip count's worth of loads for one: cholesky / lu / ludcmp / gaussian are the only four corpus kernels whose lowered source moves, all four still numerically green on c / cpp / fortran.
…ze a shape-only name WORK IN PROGRESS, built against cff3913 and not yet re-verified on the current tip. conv_depthwise_2d_square_input_asymmetric_kernel and conv_depthwise_separable_2d were both recorded as "broadcast" refusals, but the cause had drifted. Two repairs, one kernel each: version_rebound_names -- an accumulator's `+=` no longer declines the reshape that rebinds it, so the name carries one shape per version instead of two shapes under one name. freeze_shape_only_parameters -- a manifest name that only a DECLARED shape spells is frozen to its pinned value rather than promoted to a dc.symbol the body can never mention, which is what left two extents the frontend could not prove equal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SliceFusion stores one element per iteration, so an RHS read of a scalar element of the array the statement writes could be served from a slot an earlier iteration had already stored to. A[k, k:] = A[k, k:] / A[k, k] divided by the 1.0 it had just written, and c, cpp and fortran all disagreed with numpy by 1.86e+00 on a 6x6. numpy evaluates the whole RHS against the pre-assignment array, so the pivot is now staged into a temp ahead of the loop nest. Invariance is decided structurally -- full rank, no Slice, no newaxis, no index array -- so a non-aliasing kernel keeps its value exactly and trades a trip count's worth of loads for one. cholesky, lu, ludcmp and gaussian are the only four corpus kernels whose lowered source moves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The nine tests merged with the pivot-aliasing fix carried no return annotation, so test_no_file_gains_an_unannotated_function reported the file at (0, 9). Annotated rather than re-baselined: the baseline records debt that predates the rule, not new debt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…versions `version_rebound_names` counted `acc += tap` as a foreign store, which declined every accumulator a later `acc = acc.reshape(..)` rebinds: one dace descriptor asked to hold two shapes, reported as `Cannot reassign value to variable`. An `+=` reads and writes the buffer the name already holds without touching its shape, so it is owned like a read and renamed like one. Two kernels come off the refusal list on that repair -- conv_depthwise_2d_square_input_asymmetric_kernel and conv2d_min_tanh_tanh. Both were recorded as `broadcast` and neither was; re-measured at the tip they raised `Cannot reassign value to variable`, so a stale label came off with them. conv_depthwise_separable_2d stays refused under its measured cause, `dace_const_fold`: at this tip it raises `KeyError: sympy.core.numbers.Zero` out of dace's own dtype mapping, which is not ours to fix here. Measured by rendering all 679 corpus kernels before and after and parse-probing every kernel whose text moved, one process each: 42 changed, 0 ok -> fail, 2 fail -> ok. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An `acc += tap` was counted as a foreign store, so every accumulator a later `acc = acc.reshape(..)` rebinds had its versioning declined. An `+=` reads and writes the buffer the name already holds without touching its shape, so it is now owned like a read and renamed like one. conv_depthwise_2d_square_input_asymmetric_kernel and conv2d_min_tanh_tanh come off the refusal list; both were recorded as broadcast and both were really refusing on a reassignment. conv_depthwise_separable_2d stays refused, now under the cause that was measured rather than the one inherited: dace const-folds an extent whose symbols cancel to sympy.Zero and its dtype mapping has no key for it. Same DaCe defect already recorded for hotspot_rodinia. The refusal tally in the header was wrong before this change -- it claimed 64 entries and 52 broadcast against a list holding 63 and 50 -- and is corrected to the true numbers. 42 of 679 kernels change emitted text, 0 ok to fail, 2 fail to ok. The other 40 are uniform renames of a scalar accumulator rebound after its loop, verdict unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The JFNK step eps = sqrt(macheps)*(1+||u||)/||v|| is a round-off bound, so it has to follow the width the solve runs at; bratu_jvp pinned macheps to np.finfo(np.float64).eps. The translators already fold np.finfo(...).eps to the emitted precision, but the numpy reference runs as plain Python with no such rewrite, so the fp32 sweep compared a correct native solve against a reference using an eps ~23000x too small -- which divides the residual difference by 1.5e-08 and amplifies u's own fp32 representation error into the Jacobian-vector product. The reference DIVERGED (||F|| 1.8e+02 -> 3.7e+03, |u|max 15.5 against a true 0.795); that gap, not fp32 noise, was the reported d=1.09e+01 on c, cpp and fortran alike. Not fp32-undecidable: with the bound read off u.dtype, exact-arithmetic reorderings of the reference (row-sum forward vs reversed, C- vs F-order) agree to 9.5e-07 and 8.9e-07, and the fp32 solve lands 1.4e-06 from the fp64 answer. Under the pinned bound those same reorderings diverge by 2.95e+01 and 1.63e+01, so the apparent undecidability was manufactured by the defect. Newton step count is identical (20) at both precisions and was never the cause. The new corpus lint keeps the next kernel from pinning a round-off bound the same way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
WORK IN PROGRESS, NOT VERIFIED A nested @dc.program's symbols are solved by sympy from the shapes its call site passes, one equation per declared extent. `//` reaches that solver as `int_floor(a, b)`, a two-argument Function head sympy cannot invert -- and it does not decline: matched against the caller's own `int_floor` it raises `NotImplementedError: equal function with more than 1 argument` and the parse dies. `render_program` redeclares a kept helper's out-param with the extent its own body allocates, which is where such an extent enters the signature. Those equations are redundant anyway (every symbol inside the floor division is already determined by an input parameter's own extent), so the whole floor-divided extent now carries a minted symbol, substituted into the body as well as the declaration so the closing `hret[:] = out` still compares one expression with itself. NOT RUN: the corpus-wide emit-diff regression check. The BEFORE sweep (all 679 kernels emitted at efdca8c) completed; the AFTER sweep and the diff did not, so kernels-changed and any ok->fail are UNMEASURED. No test was added either. Verified so far, against a dace whose relax_int_floor is neutered to match the spcl/dace@extended tip CI installs (upstream has no relax_int_floor at all -- the local /home/primrose/Work/dace tree carries that fix unpushed): conv_standard_1d_dilated_strided and conv_transpose2d_max_pool_hardtanh_mean_tanh both FAIL before and parse ok after, and both return `ok` from the S-preset numeric oracle. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The solve count is decided by the arithmetic ordering, not by the algorithm. Every GMRES call exits on `rel < gmres_tol` and every Newton sweep exits on `resnorm < 1.0`; both are hard thresholds on a floating-point residual, and reassociating the row dot products -- which every backend is free to do -- carries those residuals across a threshold several times per run. Measured at N=64 over four orderings of the same dot product (BLAS ddot, pairwise sum(a*b), left fold, right fold): the solve count lands on 909, 912, 909, 908 while nsteps, njev, t_final and the whole 188-entry order history come out bit-identical. Tightening gmres_tol shrinks the count without removing the sensitivity -- 719/718/719/720 at 1e-2, 670/670/670/669 at 1e-3 -- and even a gmres_tol the GMRES residual can never reach, the move that makes jfnk_bratu's Newton count deterministic, still gives 616/615/616/616: the flip relocates to the Newton test, where one ordering reads resnorm = 1.0452 and corrects once more while the other reads 0.9949 and stops. A BDF corrector has to stop when the corrector has converged, so that threshold cannot be spent the way an inner tolerance can, and no setting of the knobs makes the solve count a testable output. The sibling jfnk_bratu already exposes no such counter. diagnostics is now [nsteps, njev, t_final], shape (3,). Everything the acceptance gates read stays: the order history (order adaptation engaged, order >= 3 and >= 2 changes), njev against nsteps (the frozen Jacobian reused, not refreshed on a schedule), t_final (the integration reached t_end), and the two solution fields. CI's own failure line named diagnostics alone, so those outputs already agree across c/cpp/fortran at 188 order decisions and two full grids -- the gate keeps its teeth. Verified locally: tests/ports/bdf_newton_krylov 7 passed (57s), and the single e2e leg tests/test_e2e_numerical.py::test_e2e_numerical_correctness [bdf_newton_krylov-c] 1 passed (29s). The cpp and fortran legs were NOT run locally (out of scope for this worktree) and are left to CI. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ls-and-skill # Conflicts: # experiments/merge_results.py # hpcagent_bench/harness/recording.py
…ze symbols Kernels handle float or integer arrays of fixed rank and nothing else. A scalar is a rank-0 tensor passed by copy, and a size symbol is a named integer scalar whose meaning is an extent. The canonical NumPy form, the ABI contract and the add-a-benchmark guide now say so in the same words. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he xdist worker The translator op-suite oracle forked a possibly multithreaded pytest-xdist worker to run jax, which can deadlock the child and let it wedge the session through the execnet pipe. Route it through run_forked(mp_context="spawn") like numerical_oracle's jax leg, keep the verdict strings, the JAX_PLATFORMS=cpu pin and the timeout override, and drop the skip:jax-in-parent escape hatch so that case is now graded.
…ion baseline conv2d_min_add_multiply and conv_transpose3d_avg_pool_clamp_softmax_multiply parse against the current dace extended, so the refusal ratchet (55 of 652, broadcast 44) no longer excuses them. The spawned jax leg annotated _op_oracle.py, 42 -> 34 unannotated functions.
tuple_desugar's fold_list_accumulators and numpy_desugar's curve_fit list prelude fold recognized the same seed/while/cut idiom with duplicated matchers. One fold_list_accumulators now lives in numpy_desugar: DS's segment grammar (append, +=, name + [...], for-range stride, while fill) in every block, TD's mutation-count guard, called from both frontend sites. A while with no cut leaves max(len(seed), E) elements; the old TD fold emitted length E. That case is now refused unless offset <= E is known, or the cut binds a fresh name that is the list's only reader (raman). Also refused or fixed: an element reading the list itself, an int seed grown by a float rule (was int64), and pinned stores past the cut length.
LowerCallsDaceCannotReplace now takes rank_table, so advanced indices broadcast instead of summing (a scatter over J[ia, ib, :] opened a third loop), gather temporaries built by np.empty(x.shape) get a rank, and np.ravel(x) is one axis. expr_rank gains method flatten/conj/conjugate. Emitted dace programs of all 22 kernels that call a lowered function are byte-identical. Drop numpy_desugar's shadowed _const_int import (F811).
# Conflicts: # hpcagent_bench/harness/efficacy.py # hpcagent_bench/harness/harbor_grade.py # hpcagent_bench/harness/metric.py # hpcagent_bench/numpy_translators/src/numpyto_c/emit.py # hpcagent_bench/numpy_translators/src/numpyto_common/lib_nodes.py # hpcagent_bench/numpy_translators/src/numpyto_common/lowering.py # hpcagent_bench/numpy_translators/src/numpyto_common/numpy_desugar.py # hpcagent_bench/stats/figures/signed.py # hpcagent_bench/support/collect/sweep.py # tests/annotation_baseline.json # tests/test_memory_metric.py # tests/test_packaging.py
…added The names guard flagged lines origin/main added or that the merge rewrote: frontend.py and numpy_desugar.py take lib_nodes' public iter_extent_of alias, read_axis_keepdims, slice_axes, parse_einsum_subscripts and the nine desugar *Inline classes drop their underscore, and the discarded bindings in efficacy, ablation_stats and two tests get names or index the one value read.
…_common.statement_desugar dace, native lowering and jax now share DesugarArrayIteration and SplitChainedAssign. A chained value is evaluated once: a scalar goes through a temp, dace still repeats a numeric literal (issue 05), and an array binds one name that the other targets are renamed to, up to a rebinding. Native lowering repeated the right-hand side, which gave each name its own buffer and re-read rebound targets.
…elect regression, pass 3-4x slower)
# Conflicts: # hpcagent_bench/numpy_translators/src/numpyto_common/numpy_desugar.py # hpcagent_bench/numpy_translators/src/numpyto_common/tuple_desugar.py # tests/annotation_baseline.json
…PF route tests Box and rule-line comments (# --- title --- #, # ====) become single # text lines across the tree; every rewritten file parses to an identical AST. The CPF route tests typed the make_judge fixture as Any (three ANN401 hits over a baseline of 2), now a Callable alias.
A mask bound to a local has no shape-table entry, so index_rank reported it as one position and tab[1, :nb, 0][match] composed the mask into the view's kept axis as an integer index.
…ed axis xsbench's num_nucs[mat][:, None] flattened to num_nucs[mat, None], which the C scalarizer read as a second gathered axis (and a leftover None literal it could not emit). Moving the newaxis into the index array, num_nucs[mat[:, None]], still indexed mat with the column iterator on the np.where path. The chain now stays as it was before the flatteners were fused; a 26-kernel census emits byte-identical C, dace and jax to the pre-fusion base.
# Conflicts: # hpcagent_bench/numpy_translators/src/numpyto_common/lowering.py
# Conflicts: # hpcagent_bench/numpy_translators/src/numpyto_common/lowering.py
ChainedSubscriptFlattener.visit finds the outermost A[i][j] chains with one leaf-skipping stack scan and hands only those subtrees to the transformer. 99.7% of the pass was NodeTransformer dispatch over ~195k nodes for 57 chains. 46 kernels (11 with ][ + vexx_k + 34 others), lower() x3, depth-0 visit timers, 5 interleaved rounds, medians: pass: A (5ca74a3, three old passes) 0.903 s, B-before (5bf69a9) 0.836 s, B-after 0.217 s (4.2x faster than A). lower() x3 total: A 51.5 s, B-before 47.0 s, B-after 52.6 s (box shared, +-15% noise; the pass is ~2% of lower). Equality: sha1 of emit_c(lower), emit_dace, emit_jax identical to 5bf69a9 on 46 kernels x 3 backends. CC: outermost_chains 10, visit 5.
… they replaced DesugarArrayIteration and SplitChainedAssign now walk statements only (StatementTransformer): no statement sits inside an expression, so the per-node NodeTransformer visit of every expression is gone. rank_table indexes its single-Name bindings and first body bindings once per call (name_binding_index, one walk) instead of re-walking the tree twice per fixpoint round, and reads() stops once every queried name is found. Interleaved, 5 rounds, 57 kernels (all 12 with a chained assignment or `for x in arr`, plus 45 others), medians in seconds, A = 19830c5, B-before = f65ef3a: pass-level sum A 0.577 B-before 1.107 B-after 0.290 dace chain 0.098 0.329 0.065 dace iter 0.097 0.096 0.007 lower chain 0.091 0.304 0.049 lower iter 0.088 0.087 0.006 jax chain 0.111 0.210 0.079 lower+emit total 53.27 52.47 48.70 emit_c(lower), emit_dace and emit_jax sha1 identical to f65ef3a on all 57 kernels, every round.
# Conflicts: # docs/DESIGN_perf_protocol_configs_shapes.md # hpcagent_bench/benchmarks/scientific_computing/map_reduce/mandelbrot2/mandelbrot2_numpy.py # hpcagent_bench/harness/recording.py # hpcagent_bench/harness/scoring.py # hpcagent_bench/harness/timing.py # hpcagent_bench/numpy_translators/src/numpyto_c/dace_emit.py # hpcagent_bench/numpy_translators/src/numpyto_common/numpy_desugar.py # hpcagent_bench/numpy_translators/tests/_op_oracle.py # hpcagent_bench/numpy_translators/tests/test_jax_semantics_fixes.py # hpcagent_bench/numpy_translators/tests/test_meshgrid_ix.py # hpcagent_bench/osinfo.py # scripts/plot_single_shot_score.py # tests/test_canonical_parallel_form.py # tests/test_harness_hot_paths.py # tests/test_inference_audit.py # tests/test_recording.py # tests/test_timing_backend.py
The per_arm_summary.csv values for qwen38-c, qwen38-fortran and kimi27sglang-c against numba (8.648 / 5.886 / 7.852) do not reproduce from the llr40 observations with either the current or main's reduction; the observations give 7.511 / 4.601 / 8.289, and the five c cells match exactly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thirteen solver kernels (fourteen manifests -- the Runge-Kutta pair splits, since only the adaptive variant carries a data-dependent step count) land under a new
solverssubtrack, each with a Canonical NumPy Form reference, an initializer, a manifest and acceptance gates that assert the property rather than the output: preconditioner iteration ratios, per-cycle residual drop, grid-independence, basis orthogonality, order of accuracy, operator complexity, pivot positivity and an unchanged sparsity pattern. The 244-line solver skill ships only to those kernels through a new subtrack gate -- the first gate keyed on what a kernel IS rather than on how its answer is written -- and two size paths that could not see constraints living only ininitialize()are fixed along the way, plus theconstraints:/fuzzed:TypeError that no manifest had previously triggered.🤖 Generated with Claude Code