Make the adjoint design-region gradient independent of the MPI chunk division

[jin-castle]

Fixes #2578 (adjoint gradients inconsistent with finite differences in 3D).

Symptom. The adjoint gradient of a MaterialGrid design region depends on
the number of MPI processes while the objective value does not. Measured on a
20×20×6 design grid (3D waveguide, both with and without subpixel smoothing):

• f0 identical to 8+ digits at every process count;
• gradient max-norm difference between -np 1 and -np 8: 57% of
  |g|_max (smoothing off) / 18% (smoothing on), spread over ~80% of the
  design nodes in chunk-sized patches;
• directional finite-difference checks that pass at -np 1 (ratio 1.0002)
  fail under MPI (ratios 0.86–1.22 at -np 16).

Root causes — all in material_grids_addgradient(), which computed the
result from per-chunk DFT views:

1. chunks of different components were paired by list index
   (forward_dft_chunks[ci_forward][cur_chunk]); the per-component chunk sets
   can legitimately differ near the monitor edges, after which every
   subsequent pair associates the wrong regions, and cross terms were dropped
   entirely when the counts differed (cur_chunk >= num_f_chunks → continue);
2. neighboring-point lookups bounds-checked only the chunk-local linear
   index, so an out-of-range point in one dimension could alias to a valid
   index of a different point instead of returning zero;
3. the ±1-pixel padding of persist dft chunks is clamped to the owning
   fields chunk and cannot cross process boundaries, so the data those
   lookups need does not exist at rank boundaries;
4. the same-component term indexed the forward chunk with the adjoint loop
   index, relying on the chunk-alignment assumption of (1).

Fix. Gather each monitor component onto its full (padded) design-region
grid first — monitor-owned points authoritatively (the is_old..ie_old
decomposition is disjoint), the padding ring as a multiplicity-average of the
ghost copies so its coverage does not depend on the chunk division — then run
the same per-point math on this chunk-independent view, with the work strided
across processes and per-dimension bounds checks on neighbor lookups. Memory
cost is the gathered design-region DFTs (6 complex arrays over the monitor
grid per frequency), negligible for realistic design regions.

Validation.

check	before	after
-np 8 vs -np 1, 20×20×6, smoothing off	57%	7×10⁻¹²
-np 8/16/32/128 vs -np 1 (both smoothing cases)	up to 57%	≤7×10⁻¹²
FD-vs-adjoint at -np 16 and -np 128 (six grid configurations incl. incommensurate/offset/coarse/fine grids)	ratios 0.86–1.22	identical to serial (1.000 ± FD noise)
serial behavior	—	preserved (test_subpixel_3d, adjoint E2E unchanged)

🤖 Generated with Claude Code

[stevengj]

Failing single precision tests?

[stevengj]

@lxvm, you are running a lot of Meep MPI adjoint jobs, can you look at this?

[lxvm]

I tested this PR on my Meep adjoint jobs and can confirm that it dramatically improves the accuracy of the adjoint gradient. I also noticed that changing the number of MPI ranks changed the value of the objective function I calculated and therefore I cannot comment on whether the PR fixes the design-region gradient's dependence on the chunk division because there could be another bug present.