diff --git a/Documentation/ArchitectureDecisions/0027-3d-rendering-pipeline-gap-roadmap.md b/Documentation/ArchitectureDecisions/0027-3d-rendering-pipeline-gap-roadmap.md index 7b09df364..31703c4b9 100644 --- a/Documentation/ArchitectureDecisions/0027-3d-rendering-pipeline-gap-roadmap.md +++ b/Documentation/ArchitectureDecisions/0027-3d-rendering-pipeline-gap-roadmap.md @@ -294,18 +294,25 @@ Each item needs its own short requirements note before implementation. - [x] Stage 1: depth of field. - [ ] Stage 1: motion blur (needs a velocity buffer that exists without temporal upscaling). - [ ] Stage 1: quality tiers for depth of field, bloom and auto-exposure. -- [ ] Stage 2: `KHR_texture_transform`, IOR, specular and clearcoat material extensions. +- [x] Stage 2: `KHR_texture_transform`, `KHR_materials_ior` and `KHR_materials_specular` factors (see "Stage 2, first slice"). +- [x] Stage 2: `KHR_materials_clearcoat` factors (see "Stage 2, second slice"). - [x] Stage 2: parallax occlusion mapping from `PBRMaterial.heightTexture` (world-unit `heightScale`, per-pixel ray march in `flat3d.glsl` that derives the UV shift from screen-space derivatives so no tangents are needed; the height map is read with the base color sampler because Metal has no free sampler slot; depth, shadows and silhouettes are not displaced; no glTF import yet because no displacement extension is ratified). -- [ ] Stage 2: sheen, anisotropy and `KHR_lights_punctual` import. -- [ ] Stage 2: unsupported-extension degradation policy and documented supported list. -- [ ] Stage 3: morph-target asset, glTF import and GPU evaluation. -- [ ] Stage 3: morph with skinning, shadows, motion vectors, bounds and animation weights. +- [x] Stage 2: sheen and anisotropy factors (see "Stage 2, third slice"). +- [x] Stage 2: clearcoat, clearcoat roughness and normal, sheen color and roughness, anisotropy and specular textures (see "Stage 2, fifth slice"). +- [ ] Stage 2: `KHR_lights_punctual` import. +- [x] Stage 2: unsupported-extension degradation policy and documented supported list (`NativeGLTFLoader.supportedExtensions`, `GLTFImportResult.warnings`). +- [x] Stage 3: morph-target asset, glTF import and GPU evaluation (see "Stage 3, first slice"). +- [x] Stage 3: morph with skinning, shadows, motion vectors and bounds. +- [x] Stage 3: animated weights (clips, crossfades and the animation graph) and AdaScript access (see "Stage 3, second slice"). +- [x] Stage 3: Editor panel for morph weights (see "Stage 3, third slice"). - [ ] Stage 4: OIT mode with sorted fallback. - [ ] Stage 4: transmission/volume pass. -- [ ] Stage 5: distance/height fog. +- [x] Stage 5: distance/height fog (`Fog3D` in `Environment3D`, applied in the composite pass; see "Stage 5, first slice"). - [ ] Stage 5: physically based sky feeding IBL and time-of-day driver. -- [ ] Stage 5: volumetric fog and planar reflections. -- [ ] Stage 6: clustered or tiled light assignment with measured limits. +- [x] Stage 5: sun light shafts through the fog (half-resolution raymarch, `Fog3D.volumetric`; see "Stage 5, second slice"). +- [ ] Stage 5: froxel volumetric fog for local lights, fog volumes and planar reflections. +- [x] Stage 6: tiled local-light assignment with measured limits (2D tiles, CPU masks; depth slicing is not done; see "Stage 6, first slice"). +- [x] Stage 6: reuse of unchanged local shadow faces (see "Stage 6, second slice"). - [ ] Stage 6: contact shadows, soft shadows, area lights and cookies. - [ ] Stage 7: decals, static-mesh LOD, terrain, water, foliage, 3D particles (each with its own requirements note). - [ ] Stage 7: Hi-Z/indirect-draw parity on WebGPU and Vulkan or documented fallback. @@ -629,6 +636,348 @@ press" test, and `Physics3DTests.fixedSchedulerAdvancesAndWritesBackDynamicBody` fails under parallel load but passes alone. Not verified: Apple mobile, Windows (the GPU suites are macOS-only), the windowed presentation notification path. +### Stage 5, first slice: distance and height fog (2026-10-18) + +`Environment3D.fog` (`Fog3D`) adds exponential height fog with distance start, maximum opacity, optional sky +influence and sun in-scattering. It is evaluated in `screen_space_reflection.glsl`, in linear HDR on the lit +color and before exposure, bloom add, grading and tone mapping, so it follows exposure like any other light. +Optical depth is the analytic integral of `density * exp(-heightFalloff * (y - baseHeight))` along the view +ray (uniform distance fog when `heightFalloff` is `0`). It costs one extra branch per pixel when off and +allocates nothing. Sky pixels are untouched unless `skyInfluence` is above zero. Sun in-scattering reads the +primary directional light's radiance and direction. SSR-traced reflection hits are not fogged separately +(the fog applies to the final composited pixel). Tests: `Render3DFogSettingsTests` and +`Render3DGPUTests/Fog` (Metal passes, with the `pbr-spheres-fog` golden frame recorded; WebGPU references are not yet recorded because they need `script/validate_webgpu.sh` with Swan/Dawn and Swift 6.3.2). + +### Stage 5, second slice: volumetric light shafts (2026-10-18) + +The ADR first named froxel volumetrics. This slice uses the approach Bevy ships (a full-screen raymarch that +samples shadow maps) instead of Godot's compute froxel grid: it needs no compute path, so it works on every +backend the main pass does, and it reuses the cascaded sun shadows already rendered. + +`Fog3D.volumetric` (`VolumetricFog3D`: intensity, anisotropy, steps, maximum distance) enables +`VolumetricFog3DRenderNode`, which runs after `Main3D` and before the composite. Per half-resolution pixel it +marches the view ray to the surface (or to `maxDistance` for sky pixels) with interleaved-gradient jitter, +integrates the fog medium (the same density and height falloff as the analytic fog), looks up the covering +sun cascade and accumulates `density * transmittance * visibility`, scaled by a Henyey-Greenstein phase and the +sun radiance. The composite adds the bilinearly upsampled result in linear HDR, and the unshadowed analytic +`sunInscattering` term switches off while shafts run. A camera without active shafts allocates no texture +(`VolumetricFog3DViews`) and records no pass. + +The upsample weights the four surrounding half-resolution texels by bilinear distance and by how close their +view depth is to the full-resolution pixel's (sky counts as very far), so shafts do not bleed across silhouettes. + +**Cost (Apple M-class GPU, 1080p, release, 400-sphere benchmark scene, best of 3 rounds).** Analytic fog is free +(`free` 2.43 ms bare, 2.45 ms with fog). Light shafts add 0.26 ms GPU for the pass at 16 steps (`free` 2.69 ms) +and 0.65 ms at 48 steps (`free` 3.09 ms). Reproduce with `ADAENGINE_PERF_FILTER='bare,fog,shafts'`. + +Limits of this slice: sun only (no point or spot lights, no fog volumes), one shadow tap per step, no temporal +accumulation (the jitter shows as fine noise), and no WebGPU validation yet. Tests: `Render3DGPUTests/Fog` (additive light, shadows carving the shafts, texture +release, `pbr-spheres-light-shafts` golden frame on Metal). + +### Stage 6, first slice: tiled local lights (2026-10-18) + +Measured before building: on the benchmark scene the per-pixel loop over 8 local lights cost 0.47 ms (geometry +1.71 ms against 1.24 ms without local lights) and over 32 lights 1.33 ms (2.57 ms). The local shadow pass is +a separate 1.10 ms for 13 faces, which this slice does not touch. + +The CPU now bins the (up to 32) local lights into screen tiles of at least 32 pixels (the size doubles until +there are at most 4,096 tiles, so 1080p is 60 by 34 tiles and 4K is 2,040 to 8,160 at 64 pixels). +Each tile holds a 32-bit light mask; a light sets the bits of the tiles its view-space bounding box projects +onto, plus one pixel of margin. A light whose sphere reaches the camera plane marks every tile. The main pass +projects the fragment's view-space position with the same matrix to find its tile (so the lookup does not depend +on a backend's fragment-coordinate origin), then loops only over the set bits. The masks are conservative: a +light outside its range already contributed zero, so the lit image is unchanged. `Environment3D.tiledLocalLights` +(on by default) falls back to the flat loop, and foveated rendering keeps the flat loop. + +**Results (same machine and method as Performance pass 1).** With 32 local lights geometry went from 2.57 ms to +2.09 ms and `free` from 3.30 ms to 2.81 ms. With 8 lights the change is within noise (1.71 ms to 1.69 ms), +so this helps scenes with many lights, not typical ones. The remaining 0.83 ms at 32 lights is mostly real +shading: the benchmark lights overlap heavily (range 5 at 4 unit spacing), so each pixel still evaluates several. + +Not done: depth slicing into clusters (a tile still lists lights that are in front of or behind the surface), +tighter bounds for spot cones, and a limit above 32 local lights. Those only pay off with many overlapping +lights at different depths. Tests: `Render3DLightTilesMathTests` (bounds, near-plane handling, viewport offset, +32-light cap), `Render3DGPUTests/LightTiles` (tiled and flat frames match; the test fails when the mask is +broken) and the `32 lights` benchmark configuration. + +### Stage 6, second slice: reusing unchanged local shadow faces (2026-10-19) + +Every frame the local shadow pass cleared the atlas and redrew all faces (13 faces, 1.10 ms in the benchmark scene) +even when nothing near a light had moved. Now each face keeps what it drew: + +- `VisibleMesh3DLists` fingerprints every shadow pass it prepares (`shadowDigest`): the batch key of each batch + (mesh part buffers, material parameter buffer, base color texture, skin), each caster's transform, color factor and + fade. A pass with a skinned caster has no fingerprint, because a pose changes without any visible change on + the CPU, so those faces are always redrawn. +- `LocalShadow3DRenderNode` combines it with the face's view-projection matrix (the light's position, direction, range and + cone) and compares it with the one stored for the same atlas tile. If every face matches, the pass is skipped + entirely. Otherwise the atlas is loaded rather than cleared, each dirty tile is reset with a small + scissored clear draw and then redrawn, and clean tiles are left alone. A new atlas or a different budget starts + from scratch. `LocalShadowSettings3D.reusesStaticFaces` (on by default) turns it off, and + `Render3DLightStatistics.Snapshot.reusedShadowFaces` reports how many faces the last frame kept. + +**Result (same machine and method).** On the static benchmark scene the local shadow pass goes from 1.10 ms to +0 ms and `free` from 2.39 ms to 1.90 ms. Preparing the per-face visible lists still runs on the CPU every frame +because the fingerprint comes from it. A moving light dirties all of its faces; a moving caster dirties only the +faces whose frustum it enters or leaves. + +**CPU follow-up.** Fingerprinting a face needed its visible list, so the node still prepared every face each frame. +`ShadowSceneDigest3D` now fingerprints the shadow casters once per frame (transform, mesh part buffers, material +identity and revision, base color texture). While that digest and a face's matrix are both unchanged, the face +is kept without building its list; when the digest changes, every face is prepared and compared by content as +before, so a caster that moves still dirties only the faces it touches. A scene with skinned casters, or with +level-of-detail alternatives while LOD is on (the chosen mesh follows the camera), has no digest and keeps the +per-face path. Measured on 4,900 meshes the node's CPU time per frame went from 0.49 ms to 0.21 ms; the frame +itself is GPU-bound there (`free` 7.4 to 7.5 ms either way), so the saving shows on CPU-bound machines and +scenes, not in the benchmark's frame time. The digest is now the main remaining cost (about 40 ns per caster); an +extraction-time change counter would remove it. + +Limits: meshes edited in place (same buffers, new vertices) are not noticed; a caster that animates only through +skinning keeps its faces dirty every frame; the directional cascades (0.53 ms) are not cached, since they follow +the camera. Tests: `Render3DShadowFaceCacheSettingsTests`, and `Render3DGPUTests/ShadowFaceCache` (a static scene +keeps all faces and looks identical to a redraw, a moved caster dirties some but not all faces and the partly +reused atlas equals a full redraw, a moved light dirties every face). + +### Stage 2, first slice: texture transforms, IOR, specular and the extension policy (2026-10-19) + +**Correction to G2.** The audit said the loader "rejects" extensions it does not know. It only rejects those a file +lists in `extensionsRequired`, which is what the glTF specification asks for; extensions listed only in +`extensionsUsed` were already ignored, silently. The gap was the silence, not a rejection. + +- **Policy.** `NativeGLTFLoader.supportedExtensions` is the public list (`KHR_mesh_quantization`, + `KHR_materials_emissive_strength`, `KHR_texture_transform`, `KHR_materials_ior`, `KHR_materials_specular`). A file that + *requires* anything else still fails with `unsupportedRequiredExtension`. An extension a file merely *uses* is + ignored and reported in `GLTFImportResult.warnings`, as is a `KHR_materials_specular` texture (only the factors are + read). `ModelAsset3D.importWarnings` carries them to the engine; nothing is logged yet because `AdaScene` has no + logging dependency. +- **`KHR_texture_transform`** for base color, metallic/roughness, normal, occlusion and emission: offset, rotation, scale and + the `texCoord` override. `PBRMaterial.textureTransforms` holds five `PBRTextureTransform`s; the material uniform stores each + as a 2x2 matrix and an offset, and `flat3d.glsl` applies it in `materialUV`. The directional-shadow and motion shaders + apply the base-color transform to their alpha test, so cut-out shadows and motion vectors follow the texture. A normal map + whose transform rotates or flips axes stops using authored tangents (the derivative frame follows the transformed UVs). +- **`KHR_materials_ior` and `KHR_materials_specular` (factors).** Non-metal reflectance at normal incidence is + `min(((ior - 1) / (ior + 1))^2 * specularColor, 1) * specularFactor`, computed on the CPU into `PBR3DUniform.dielectric` + and used in place of the fixed 0.04 in the direct, local-light and image-based terms. The defaults give 0.04, so + existing frames are unchanged (all golden frames still match). The glTF specular `F90` term is not modeled (full + white at grazing angles, as before), and the SSR composite keeps its own fixed reflectivity. +- **Validation.** Malformed transforms (wrong vector length, UV set above 1) and out-of-range reflectance (IOR outside + 1...100, negative or short specular values) fail with `invalidMaterial`. + +Cost: the per-material uniform grew from 80 to 272 bytes and the deduplication key from five to sixteen vectors; the benchmark +frame (`bare`, 1.91 ms) is unchanged. Tests: `GLTFMaterialExtensionTests` (import, validation, warnings, the required-extension +failure), `GLTFMaterialExtensionMappingTests` (importer to `PBRMaterial`), `Render3DMaterialExtensionConstantTests` and +`Render3DGPUTests/MaterialExtensions` (a transform moves, scales and rotates the texture, a full turn is the identity, higher IOR +brightens and zero specular dims the highlight, a red tint shifts it, defaults change nothing). Not validated on WebGPU (no +Swan/Dawn here). Still open in Stage 2: clearcoat, sheen, anisotropy, specular textures and `KHR_lights_punctual`. + +### Stage 2, second slice: clearcoat (2026-10-19) + +`KHR_materials_clearcoat` adds a clear lacquer layer over the base surface (car paint, varnished wood, nails). +`PBRMaterial.clearcoatFactor` and `clearcoatRoughnessFactor` come from the glTF factors (0 to 1, validated on import) and +reach the shader through `PBR3DUniform.coat`. Following the glTF layering model, in `flat3d.glsl`: + +- the coat is a dielectric layer (reflectance 0.04) shaded with its own GGX lobe and roughness on the **geometric normal** + (the normal map belongs to the base layer); +- every light (the sun, extra directional lights and local lights, through `evaluateDirect`) adds the coat's highlight and + dims the base layer under it by `1 - strength * F(N.V)`, so energy is conserved; +- with image-based lighting the coat also reflects the environment along the geometric normal, using the BRDF + lookup at the coat's roughness, and dims the base IBL by the same transmission; +- with only the flat ambient, the ambient term is dimmed by the same transmission. + +A material with no coat takes one branch on `g_Clearcoat.x`. Measured against the merge commit without this change (1080p, Apple +M-class, release): `bare` geometry 1.77 to 1.82 ms and 2.18 to 2.26 ms with 32 lights, about 3 to 4 percent. For reference +the parallax merge from `main` had already moved `bare` geometry from 1.69 to 1.77 ms. + +Limits: the clearcoat, clearcoat roughness and clearcoat normal **textures are not read** (the importer reports them in +`GLTFImportResult.warnings`), because the main pass has no free texture slots on Metal. The screen-space reflection composite +reflects the procedural sky from the base layer's roughness only, so without image-based lighting the coat shows light +highlights but not a sky reflection. There is no clearcoat debug view and no WebGPU validation. Tests: +`GLTFMaterialExtensionTests` (import, ranges, texture warning), `Render3DMaterialExtensionConstantTests` (uniform), +`Render3DGPUTests/MaterialExtensions` (a smooth coat adds a highlight to rough paint, a rougher coat dims it, the highlight grows +with strength, turning the coat off restores the frame) and the `pbr-sphere-clearcoat` golden frame (Metal). + +### Stage 2, third slice: sheen and anisotropy (2026-10-19) + +`KHR_materials_sheen` (cloth) and `KHR_materials_anisotropy` (brushed metal, satin) factors. `PBRMaterial` gains +`sheenColorFactor`, `sheenRoughnessFactor`, `anisotropyStrength` and `anisotropyRotation`, imported with range +validation (sheen color and roughness 0 to 1, strength 0 to 1) and carried in `PBR3DUniform.sheen` and `.anisotropy`. + +- **Sheen.** The Charlie distribution with the Neubelt visibility term, as in the glTF specification, evaluated per light in + `evaluateDirect`; the base layer under it is dimmed by `1 - max(sheenColor) * 0.4`. The specification's sheen albedo lookup + table is replaced by that constant (0.4), and under image-based lighting sheen is a diffuse lobe from the irradiance map + with the same albedo. Without image-based lighting only the lights produce sheen. +- **Anisotropy.** Anisotropic GGX (distribution and height-correlated visibility) with `alphaT = mix(alpha, 1, strength^2)` + along the stretch direction and `alpha` across it, so the highlight streaks perpendicular to the direction. The frame + is the authored tangent when the mesh has tangents and the UV-derivative frame otherwise, rotated by + `anisotropyRotation`. Image-based reflections bend the lookup normal toward the stretch (the Filament approximation). +- **Layering order** is base, then sheen, then clearcoat. The sun now goes through `evaluateDirect` like every other light, which + removed the duplicated inline copy of the lighting model. + +**Cost (same machine and method).** With every layer switched off, `bare` main-pass geometry went from 1.77 ms at the merge +with `main` to 1.82 ms with clearcoat and to 1.92 ms with sheen and anisotropy (32 lights: 2.18, 2.26, 2.40 ms), so the +three layered-material features together cost about 8 percent of main-pass geometry on scenes that use none of them. The +branches are uniform across a draw, so the cost is most likely register pressure from the larger lighting function. The fix +is shader permutations selected by the material (the `HAS_...` define mechanism exists on `PBRMaterial`, but the PBR +pipeline is keyed by vertex layout only), which is not done. + +Limits: the sheen color, sheen roughness and anisotropy textures are not read (reported as import warnings). Meshes +without tangents get a tangent frame from UV derivatives, which is constant per triangle and shows faceting along an +anisotropic highlight on coarse meshes (visible on the primitive sphere in `pbr-sphere-anisotropy`) and at UV poles; assets with +authored tangents do not show it. No WebGPU validation. Tests: import, ranges and warnings in `GLTFMaterialExtensionTests`, +the uniform in `Render3DMaterialExtensionConstantTests`, and in `Render3DGPUTests/MaterialExtensions` sheen adds light that +grows with its color and tints it, turning it off restores the frame, anisotropy changes and rotates the highlight and a half +turn looks the same, plus the `pbr-sphere-sheen` and `pbr-sphere-anisotropy` golden frames (Metal). + +### Stage 2, fourth slice: main shader specialized by shading layer (2026-10-19) + +The third slice left layered materials costing about 8 percent of main-pass geometry even when no material used a layer. +The main shader is now specialized by material: + +- `PBRMaterial.shaderFeatures` is the set of layers a material uses (`clearcoat`, `sheen`, `anisotropy`, derived from the + factors, so editing a factor at run time changes it). `PBR3DConfiguration.features` joins the pipeline key, and + `Flat3DPipeline` takes the matching fragment shader from `PBR3DShaderVariants`. +- In `flat3d.glsl`, `hasClearcoat()`, `hasSheen()` and `hasAnisotropy()` are constant `false` unless `ADA_CLEARCOAT`, + `ADA_SHEEN` or `ADA_ANISOTROPY` is defined, so the plain shader has none of the layer code. Names that enable a layer, not + ones that remove it, so the file with no macros is the plain shader. +- `ShaderModule.createBundled(at:from:defines:)` compiles a variant from the file's text. That bypasses the on-disk shader + cache, which is keyed by file and stage (its manifest, reflection and device-compiled entries would otherwise be overwritten + by every variant), so a variant compiles in the process on first use, and about 1 second on this machine. The plain shader, + used by almost every material, keeps the disk cache. The first attempt had the macros remove the layers, which made + the common variant a runtime compile too, and the GPU test suite went from 12 s to 51 s; it was reversed. + +**Result (1080p, Apple M-class, release).** `bare` main-pass geometry 1.92 ms with the layers in every material back to 1.77 ms, +the level before any layer (32 lights: 2.40 to 2.17 ms). Layered materials pay the full layer cost only where they are drawn. + +Limits: WebAssembly builds ship precompiled WGSL and cannot specialize, so on the web there are no layers (the plain module is +used); the motion and shadow passes do not use layers and are not split. The first draw of each new layer combination compiles +a shader. Tests: `shaderFeaturesFollowTheLayersInUse`, `pipelineKeysSplitByFeaturesAndDefaultToThePlainShader`, and the existing +clearcoat, sheen and anisotropy GPU tests, which now run through the variants (turning a layer off restores the plain frame). + +### Stage 2, fifth slice: layer textures (2026-10-19) + +The clearcoat, sheen, anisotropy and specular extensions now read their textures, which the earlier slices skipped. Eight textures +(`PBRLayerTexture`): clearcoat strength (red), clearcoat roughness (green), clearcoat normal, sheen color (RGB, sRGB), sheen +roughness (alpha), anisotropy direction (red and green) with strength (blue), specular strength (alpha) and specular color +(RGB, sRGB). Each multiplies its factor, as the extension texts require. + +- **Import.** Each reference keeps its texture, UV set (0 or 1, or the `texCoord` of its own `KHR_texture_transform`), transform + and, for the clearcoat normal map, its scale (`GLTFImportResult.LayerTextures`). A bad index, UV set or transform fails with + `invalidMaterial`. Import warnings for skipped layer textures are gone because none are skipped. +- **Binding.** Textures bind at 32 to 39, and only in the shader variant of a material that has them: `PBRMaterial.shaderFeatures` + gained one bit per texture (macros `ADA_CLEARCOAT_TEXTURE` and so on), and a texture selects its bit only while its layer is + on, because it scales a factor that would otherwise be zero. The specular textures belong to no layer. So a material pays for the + textures it has, and the plain shader and the bind groups of other materials are unchanged (`bare` geometry is still 1.77 ms). +- **Sampling.** The main pass has no free sampler slot (Metal allows 16), so every layer texture is read with the base color + texture's sampler: a layer texture whose glTF sampler differs from the base color's (for example another wrap mode) takes the + base color's. UVs come from `layerUV`, which applies the parallax shift of the same UV set and then the texture's transform. The + material uniform carries each texture's UV set and transform (about 330 more bytes per material). +- **Shading.** Clearcoat strength and roughness are scaled by their textures, the clearcoat normal bends the coat on its own + frame (derivative frame; the base normal map is untouched), sheen color and roughness are scaled, the anisotropy texture sets + the direction (rotated by `anisotropyRotation`) and scales the strength, and the specular textures scale the dielectric + reflectance `g_Dielectric`, which replaced the uniform value in every lighting term. + +Limits: the layer textures share the base color sampler; the clearcoat normal map uses the UV-derivative frame (no authored +tangents), so it shares the faceting of that frame on coarse meshes; there is no web build of any layer (variants cannot be compiled +there). Tests: import (`importsEveryLayerTextureWithItsOwnUVSetAndTransform` and the rejection cases), the mapping from an +imported material to `PBRMaterial`, the feature bits and uniform, and one GPU test per texture. They include equivalences (an +anisotropy texture of (1, 0.5, 1) equals the factor alone, and (0.5, 1, 1) equals a quarter turn; a specular alpha of zero equals a +specular factor of zero; a flat clearcoat normal changes nothing) and a transform test that samples either half of a two-tone mask. +Each of the shader reads was checked by removing it and watching its test fail. + +### Stage 3, first slice: morph targets (2026-10-19) + +**Import.** Primitives carry `targets` (POSITION, NORMAL and TANGENT deltas per vertex; sparse accessors expand to dense), meshes carry +the default `weights` and the `extras.targetNames` convention, nodes may override the weights, and animation channels with the +`weights` path import with one value per target and key. Inconsistent data fails with `invalidMorphTargets`; morphed +attributes other than the three deltas are reported as warnings. + +**GPU evaluation.** `MorphTargetSet` packs every target's deltas into one RGBA16F texture (`Part.morphTargets`): texel +`(target * vertices + vertex) * attributes + attribute`, 2048 texels wide, up to 8192 rows (about 16.7 million texels of +targets x vertices x attributes; a mesh past that fails to load with an explicit error). A draw applies at most 8 targets, the +ones with the largest weights (`MorphTargetSet.activeTargets`), and its vertex shader fetches only those deltas with `texelFetch` +(`Morph.glsl`, behind `ADA_MORPH`, so unmorphed meshes keep the plain shaders), so cost follows the active targets and memory follows all +of them. Deltas are half precision (portable bit conversion, no `Float16`). The weights live in `Morph3DUniforms`, one buffer per entity +and mesh part (parts differ in vertex count), holding the current and the previous frame's active targets. + +**Passes.** The morph feature (`PBRMaterial.ShaderFeatures.morph`, a property of the draw, not the material) selects a vertex +shader variant of the main shader, the motion shader and the directional and local shadow shaders (`Shadow3DConfiguration` joins the +pipeline key) and the occlusion prepass; weights and the delta texture bind at 21 and 40. Order is morph, then skin (the +deltas move the rest pose before the joint matrices). The motion pass moves a vertex from where the previous weights put it. + +**Culling, batching and caches.** The source bounds grow by the most the active weights can displace (the largest delta of each +active target times its weight, times the largest joint scale for skinned meshes), so frustum culling, shadow culling and the occlusion +candidates see the morphed shape. A morphed mesh is its own batch (like a skinned one), has no level-of-detail +alternatives, and counts as dynamic for the shadow face cache. `MorphWeights3DComponent` holds the weights (with `setWeight(_:named:)`), +and `ModelAsset3D.instantiate` adds it with the file's default weights. + +**Tests.** Import (`GLTFMorphTargetImportTests`, 11 cases); packing, selection and uniform layout (`Render3DMorphTargetSelectionTests`); +and GPU tests (`Render3DGPUTests/MorphTargets`): weight zero equals the unmorphed mesh, a weighted shift equals moving the entity (sun and +local-light shadows included), only the heaviest eight targets apply, weight changes update the shadow faces on the first frame, a mesh +resting off screen is not culled once morphed, normal deltas change shading but not the silhouette, the occlusion prepass and the +temporal motion pass draw the morphed shape, the uniform keeps the previous frame's weights, and a glTF skinned to a joint turned a quarter turn +applies its morph before the skin. Each shader read was checked by breaking it and watching a test fail. + +Limits: deltas are dense and half precision (no sparse or compacted storage, so a face with many targets costs texture memory +in proportion to targets x vertices); the selection is made per frame on the CPU; only POSITION, NORMAL and TANGENT morph; layered materials +and morph share the shader-variant mechanism, so the first draw of a morphed mesh compiles its variants (about a second each), and there is no web +build. Not done at this slice: playing `weights` channels, AdaScript and the Editor (see the second slice). + +### Stage 3, second slice: animated weights and script access (2026-10-19) + +**Clips.** glTF `weights` channels import as `MorphWeightsTrack` (a validated track of `targetCount` weights per key, linear, +step or cubic spline with per-target tangents scaled by the key span, sampled without allocating) in `SkeletalAnimationClip.morphTracks`; +they count toward the clip duration and are valid on matrix-placed nodes. A rig node carries `morphRestWeights` (the node's weights, else the +mesh's defaults), and a player rejects a track that addresses a node with a different target count. + +**Playback.** `SkeletalAnimationPlayer.morphWeights` holds the weights by rig node (empty where the node has no targets). Without a graph a +clip's tracks overwrite the rest weights at its local time, looping and clamping like poses; a crossfade (`transitionDuration`) +interpolates from the weights at the time of the switch. In a graph, blend nodes mix weights by input weight and mask with the rest weights +filling a gap (a mask is read at the mesh node's rig index), and additive nodes add each further input's offset from the rest weights, +the same rules the poses follow. + +**Scene.** `SkeletalAnimation3DSystem` writes `player.morphWeights` into the node's `MorphWeights3DComponent` whenever the pose generation +changes, only for nodes the animation drives (matching, non-empty weights), so hand-set weights on other nodes are not touched. The +renderer already treats weight changes as dynamic (shadow faces, bounds, previous-frame weights), so animated morphs need nothing more. + +**Scripts.** `MorphWeights3DComponent` is a reflected `@Component`, registered with the model plugin and Codable (a missing key +decodes to its default). Reflection exposes scalars only, so arrays show as read-only fields; scripts and the inspector steer one target at a +time through `selectedTarget`, `selectedTargetName` (assigning a name selects that target) and `selectedTargetWeight` (assigning a finite +value sets the weight), and code uses `weights` and `setWeight(_:named:)`. + +**Tests.** 17 CPU tests (`MorphWeightsAnimationTests`): sampling and rejection, the player (rest weights, clip, loop and clamp, independent +players, crossfade, track validation, duration), and the graph (blend with rest fill, mask on the mesh node, additive, clips without +weight tracks). A GPU test loads a glTF with an animated weights channel through `ModelAsset3D`, seeks to half a second, checks that the +entity holds `[0.5]`, that the frame equals the same weight set by hand, and that the end of the clip applies the full shift; reflection +and Codable tests cover the component. + +Not done: an Editor panel (the Editor package keeps its own component catalog and builds separately) and animating weights from keyframe +`AdaAnimation` clips (only glTF clips and the skeletal graph drive weights); an AdaScript sample. + +### Stage 3, third slice: authored weights and the Editor panel (2026-10-19) + +**Where weights live in a scene.** A model in a scene is a `Model3DSource` that instantiates a hierarchy at run time, so the weight +components exist only on generated children. `Model3DSource.morphWeights` (target name to weight, absent in older scenes) is applied at +instantiation to every mesh with a target of that name, which is what an Editor can author. They are the weights the instance +starts with and keeps unless a clip animates that mesh's weights: `SkeletalAnimation3DSystem` now writes player weights only to +nodes some clip animates (`SkeletalAnimationPlayer.animatedMorphNodes`). Before that, a rigged model's rest weights from the file +would have overwritten authored ones on the first frame. + +**Editor.** The Animator panel has a Morph Targets mode next to the skeletal graph and keyframes. It loads the model, lists its *named* +targets (a model's unnamed targets are counted and reported as not addressable from a scene) with one slider each from 0 to 1, +shows the result live in the preview (`EditorModelAssetPreviewModel.morphWeights`), and applies it as one undoable scene transaction +(`setMorphWeights`), stored as a structured `morphWeights` map on the Model 3D component (a weight equal to the file's default is not an +override, and an empty map removes the key). Reset clears the overrides. Names the model does not have are dropped when it loads. + +**Tests (Editor package, built with Xcode's toolchain).** Six run and pass: the model (targets, defaults, clamping, unnamed targets), the payload, +a scene transaction that saves, reopens, decodes through the component registry, undoes, redoes and clears, the preview applying and +undoing weights, and the weight list drawing one named slider per target. A seventh (`loaderAppliesAuthoredWeights`) opens the saved scene +through `EditorSceneFileLoader` and checks the instantiated entity's weights; it did not run here, because on this machine any Editor test that +goes through that loader ends the test process silently, the existing `EditorAnimationGraphTests` scene round trip included. The same behavior is +covered on the runtime side by `authoredWeightsStartTheInstanceAndYieldToAnimation`. + +Not done: slider interaction is covered through the model and workbench, not by driving the slider with simulated pointer events; the Environment 3D inspector does not yet list the fog, light-shaft, tiled-light or shadow-cache settings added earlier in this work; weights +cannot be keyframed in the Editor's keyframe animator (only glTF clips and the skeletal graph animate them). + ## Validation requirements Use Swift Testing for logic (settings decoding, extension gating, budgets, diff --git a/Editor/Sources/AdaEditor/Assets/Localization/en.json b/Editor/Sources/AdaEditor/Assets/Localization/en.json index 721a5f691..9b80a0dc8 100644 --- a/Editor/Sources/AdaEditor/Assets/Localization/en.json +++ b/Editor/Sources/AdaEditor/Assets/Localization/en.json @@ -1655,5 +1655,14 @@ "Save annotation": "Save annotation", "Edit annotation #{0}: {1}": "Edit annotation #{0}: {1}", "Ready to send together: {0}": "Ready to send together: {0}", - "This note was edited in the composer. Edit its text there or remove it before replacing it.": "This note was edited in the composer. Edit its text there or remove it before replacing it." + "This note was edited in the composer. Edit its text there or remove it before replacing it.": "This note was edited in the composer. Edit its text there or remove it before replacing it.", + "Morph Targets": "Morph Targets", + "MORPH TARGETS": "MORPH TARGETS", + "Loading morph target model…": "Loading morph target model…", + "This model has no named morph targets.": "This model has no named morph targets.", + "Targets without a name cannot be set from a scene.": "Targets without a name cannot be set from a scene.", + "Applied to scene. Save the scene to keep these weights.": "Applied to scene. Save the scene to keep these weights.", + "Morph weights cleared.": "Morph weights cleared.", + "Morph weights applied": "Morph weights applied", + "Morph weights cleared": "Morph weights cleared" } diff --git a/Editor/Sources/AdaEditor/Assets/Localization/ru.json b/Editor/Sources/AdaEditor/Assets/Localization/ru.json index 69ec835ec..810a0e3c8 100644 --- a/Editor/Sources/AdaEditor/Assets/Localization/ru.json +++ b/Editor/Sources/AdaEditor/Assets/Localization/ru.json @@ -1655,5 +1655,14 @@ "Save annotation": "Сохранить аннотацию", "Edit annotation #{0}: {1}": "Редактировать аннотацию №{0}: {1}", "Ready to send together: {0}": "Будут отправлены вместе: {0}", - "This note was edited in the composer. Edit its text there or remove it before replacing it.": "Текст замечания изменён в композере. Отредактируйте его там или удалите аннотацию перед заменой." + "This note was edited in the composer. Edit its text there or remove it before replacing it.": "Текст замечания изменён в композере. Отредактируйте его там или удалите аннотацию перед заменой.", + "Morph Targets": "Цели морфинга", + "MORPH TARGETS": "ЦЕЛИ МОРФИНГА", + "Loading morph target model…": "Загрузка модели с целями морфинга…", + "This model has no named morph targets.": "У этой модели нет именованных целей морфинга.", + "Targets without a name cannot be set from a scene.": "Цели без имени нельзя задать из сцены.", + "Applied to scene. Save the scene to keep these weights.": "Применено к сцене. Сохраните сцену, чтобы сохранить эти веса.", + "Morph weights cleared.": "Веса морфинга сброшены.", + "Morph weights applied": "Веса морфинга применены", + "Morph weights cleared": "Веса морфинга сброшены" } diff --git a/Editor/Sources/AdaEditor/EditorComponentRegistry.swift b/Editor/Sources/AdaEditor/EditorComponentRegistry.swift index a9eb6d2a8..51125bfd4 100644 --- a/Editor/Sources/AdaEditor/EditorComponentRegistry.swift +++ b/Editor/Sources/AdaEditor/EditorComponentRegistry.swift @@ -322,6 +322,7 @@ enum EditorComponentRegistry { AssetsManager.registerAssetType(SpineSkeleton.self) RuntimeTypeRegistry.registerComponent(AudioSource.self, names: ["AudioSource"], makeDefault: { AudioSource() }) ModelAnimationPlayback3DComponent.registerComponent() + MorphWeights3DComponent.registerComponent() CharacterController3DComponent.registerComponent() CameraCollision3DComponent.registerComponent() RuntimeTypeRegistry.registerComponent(UIComponent.self, names: ["UIComponent"]) diff --git a/Editor/Sources/AdaEditor/EditorModel3DComponentDescriptor.swift b/Editor/Sources/AdaEditor/EditorModel3DComponentDescriptor.swift index 2c13ad60e..aab35049a 100644 --- a/Editor/Sources/AdaEditor/EditorModel3DComponentDescriptor.swift +++ b/Editor/Sources/AdaEditor/EditorModel3DComponentDescriptor.swift @@ -29,7 +29,8 @@ extension EditorComponentRegistry { if case .null = value { return nil } return try JSONDecoder().decode(Model3DSourceLOD.self, from: Data(value.jsonString.utf8)) }, - voxel: voxelSettings(in: payload) + voxel: voxelSettings(in: payload), + morphWeights: EditorMorphWeightsPayload.decode(payload["morphWeights"]) ) } ) diff --git a/Editor/Sources/AdaEditor/EditorMorphTargetsModel.swift b/Editor/Sources/AdaEditor/EditorMorphTargetsModel.swift new file mode 100644 index 000000000..bf3fe07e9 --- /dev/null +++ b/Editor/Sources/AdaEditor/EditorMorphTargetsModel.swift @@ -0,0 +1,107 @@ +@_spi(AdaEngine) import AdaEngine +import Foundation +import Observation + +/// Scene payloads keep morph weights as a structured map from target name to weight, the shape `Model3DSource.morphWeights` has. +enum EditorMorphWeightsPayload { + static func decode(_ value: EditorSceneValue?) -> [String: Float] { + guard case let .object(entries)? = value else { + return [:] + } + var weights: [String: Float] = [:] + for (name, entry) in entries { + if let number = entry.doubleValue, number.isFinite { weights[name] = Float(number) } + } + return weights + } + + static func encode(_ weights: [String: Float]) -> EditorSceneValue { + .object(weights.mapValues { .double(Double($0)) }) + } +} + +/// The named morph targets of a model and the weights an author has set for them. +@Observable +@MainActor +final class EditorMorphTargetsModel { + struct Target: Equatable, Identifiable { + let name: String + /// The weight the file gives the target when nothing overrides it. + let defaultWeight: Float + var id: String { name } + } + + /// The slider range. glTF weights are normally 0 to 1; the renderer extrapolates outside that. + static let weightRange: ClosedRange = 0...1 + + private(set) var targets: [Target] = [] + /// Targets of the model that have no name; they cannot be addressed from a scene. + private(set) var unnamedTargetCount = 0 + private(set) var weights: [String: Float] = [:] + + func configure(asset: ModelAsset3D, authored: [String: Float]) { + var seen: [String: Float] = [:] + var order: [String] = [] + var unnamed = 0 + for (index, mesh) in asset.meshes.enumerated() where mesh.hasMorphTargets { + let names = mesh.morphTargetNames + let defaults = asset.meshMorphWeights.indices.contains(index) ? asset.meshMorphWeights[index] : [] + for target in 0.. Float { + weights[target.name] ?? target.defaultWeight + } + + func setWeight(_ value: Float, for target: Target) { + guard value.isFinite else { return } + let clamped = min(max(value, Self.weightRange.lowerBound), Self.weightRange.upperBound) + if clamped == target.defaultWeight { + weights.removeValue(forKey: target.name) + } else { + weights[target.name] = clamped + } + } + + /// Clears every authored weight, so the model's own defaults apply. + func reset() { + weights.removeAll() + } + + var hasAuthoredWeights: Bool { !weights.isEmpty } +} + +extension EditorWorkbenchViewModel { + /// One scene-history transaction that sets the weights a Model 3D reference starts with; an empty map removes them. + @discardableResult + func setMorphWeights(_ weights: [String: Float], target: EditorSkeletalAnimationTarget) -> Bool { + guard let document = sceneDocument(id: target.documentID), !document.isReadOnly, + document.sceneModel?.entities.contains(where: { $0.id == target.entityID }) == true + else { return false } + updateSceneModelDocument(id: target.documentID, status: weights.isEmpty ? "Morph weights cleared" : "Morph weights applied") { scene in + guard let index = scene.entities.firstIndex(where: { $0.id == target.entityID }), + scene.entities[index].components[EditorBuiltInComponentType.model3DSource] != nil + else { return } + if weights.isEmpty { + scene.entities[index].components[EditorBuiltInComponentType.model3DSource]?.removeValue(forKey: "morphWeights") + } else { + scene.entities[index].components[EditorBuiltInComponentType.model3DSource]?["morphWeights"] = EditorMorphWeightsPayload.encode(weights) + } + } + return true + } +} diff --git a/Editor/Sources/AdaEditor/UI/Editor/EditorAnimationPanel.swift b/Editor/Sources/AdaEditor/UI/Editor/EditorAnimationPanel.swift index e1d715dab..70f67ddbc 100644 --- a/Editor/Sources/AdaEditor/UI/Editor/EditorAnimationPanel.swift +++ b/Editor/Sources/AdaEditor/UI/Editor/EditorAnimationPanel.swift @@ -4,7 +4,8 @@ import Math struct EditorAnimationPanel: View { let viewModel: EditorViewModel - @State private var usesSkeletalGraph = true + private enum Mode { case skeletalGraph, morphTargets, keyframes } + @State private var mode = Mode.skeletalGraph @Environment(\.metrics) private var metrics @Environment(\.theme) private var theme @@ -13,12 +14,15 @@ struct EditorAnimationPanel: View { VStack(alignment: .leading, spacing: 0) { if let target = viewModel.skeletalAnimationTarget { HStack(spacing: 8) { - cycleButton(title: "Skeletal Graph", identifier: "AdaEditor.Animator.GraphMode") { usesSkeletalGraph = true } - cycleButton(title: "Keyframes", identifier: "AdaEditor.Animator.KeyframeMode") { usesSkeletalGraph = false } + cycleButton(title: "Skeletal Graph", identifier: "AdaEditor.Animator.GraphMode") { mode = .skeletalGraph } + cycleButton(title: "Morph Targets", identifier: "AdaEditor.Animator.MorphMode") { mode = .morphTargets } + cycleButton(title: "Keyframes", identifier: "AdaEditor.Animator.KeyframeMode") { mode = .keyframes } Spacer() }.padding(6) - if usesSkeletalGraph { + if mode == .skeletalGraph { EditorSkeletalAnimatorPanel(viewModel: viewModel, target: target).id(target.documentID + target.entityID) + } else if mode == .morphTargets { + EditorMorphTargetsPanel(viewModel: viewModel, target: target).id(target.documentID + target.entityID) } else { panelToolbar panelContent diff --git a/Editor/Sources/AdaEditor/UI/Editor/EditorModelAssetPreview.swift b/Editor/Sources/AdaEditor/UI/Editor/EditorModelAssetPreview.swift index b0ca62a74..11c24f0fc 100644 --- a/Editor/Sources/AdaEditor/UI/Editor/EditorModelAssetPreview.swift +++ b/Editor/Sources/AdaEditor/UI/Editor/EditorModelAssetPreview.swift @@ -14,6 +14,9 @@ final class EditorModelAssetPreviewModel { var playsAnimation = true { didSet { updatePlayback() } } var animationGraph: AnimationGraph? { didSet { updatePlayback() } } var graphEventLog: [String] = [] + /// Morph target weights by name that the preview shows; every target the entity has is set to its entry, so a target removed from + /// the map needs its default put back by the caller. + var morphWeights: [String: Float] = [:] { didSet { applyMorphWeights() } } @ObservationIgnored private var loadTask: Task? @ObservationIgnored private var normalizationCache: (ObjectIdentifier, Transform, AABB)? @ObservationIgnored private weak var world: World? @@ -56,6 +59,19 @@ final class EditorModelAssetPreviewModel { animationRoot = world.getEntities().first { $0.components[ModelAnimation3DComponent.self] != nil }?.id voxelRoot = world.getEntities().first { $0.components[VoxelAnimationPlayback3DComponent.self] != nil }?.id updatePlayback() + applyMorphWeights() + } + + private func applyMorphWeights() { + guard let world, !morphWeights.isEmpty else { return } + for entity in world.getEntities() { + guard var component = entity.components[MorphWeights3DComponent.self] else { continue } + var changed = false + for (name, weight) in morphWeights where weight.isFinite { + if component.weight(named: name) != weight, component.setWeight(weight, named: name) { changed = true } + } + if changed { entity.components[MorphWeights3DComponent.self] = component } + } } /// Whether the previewed asset is a MagicaVoxel scene with animation to play. diff --git a/Editor/Sources/AdaEditor/UI/Editor/EditorMorphTargetsPanel.swift b/Editor/Sources/AdaEditor/UI/Editor/EditorMorphTargetsPanel.swift new file mode 100644 index 000000000..c2de3c9b9 --- /dev/null +++ b/Editor/Sources/AdaEditor/UI/Editor/EditorMorphTargetsPanel.swift @@ -0,0 +1,144 @@ +@_spi(AdaEngine) import AdaEngine +import Foundation + +/// One slider row per named morph target. Shared by the Animator panel and its tests. +struct EditorMorphWeightList: View { + let model: EditorMorphTargetsModel + var onChange: () -> Void = {} + @Environment(\.theme) private var theme + + var body: some View { + ScrollView { + VStack(alignment: .leading, spacing: 4) { + ForEach(model.targets, id: \.name) { target in + row(target) + } + } + .padding(4) + } + .accessibilityIdentifier("AdaEditor.Morph.List") + } + + private func row(_ target: EditorMorphTargetsModel.Target) -> some View { + HStack(spacing: 8) { + Text(target.name) + .font(.system(size: 11, weight: model.weights[target.name] == nil ? .regular : .semibold)) + .frame(width: 130, alignment: .leading) + .accessibilityIdentifier("AdaEditor.Morph.Name.\(target.name)") + Slider( + value: Binding( + get: { Double(model.weight(of: target)) }, + set: { value in + model.setWeight(Float(value), for: target) + onChange() + } + ), + in: Double(EditorMorphTargetsModel.weightRange.lowerBound)...Double(EditorMorphTargetsModel.weightRange.upperBound) + ) + .accessibilityIdentifier("AdaEditor.Morph.Slider.\(target.name)") + Text(String(format: "%.2f", model.weight(of: target))) + .font(.system(size: 11)).foregroundColor(theme.editorColors.muted) + .frame(width: 38, alignment: .trailing) + } + } +} + +/// Sets the morph weights a Model 3D reference starts with, with a live preview. Applying is one undoable scene transaction. +struct EditorMorphTargetsPanel: View { + let viewModel: EditorViewModel + let target: EditorSkeletalAnimationTarget + @State private var model = EditorMorphTargetsModel() + @State private var preview = EditorModelAssetPreviewModel() + @State private var camera = EditorModelPreviewCamera() + @State private var status = "" + @Environment(\.theme) private var theme + + var body: some View { + VStack(alignment: .leading, spacing: 6) { + toolbar + if preview.isLoading { + Text(L10n.text("Loading morph target model…")).font(.system(size: 12)) + } else if let error = preview.error { + Text(error).font(.system(size: 12)).foregroundColor(.red) + } else if preview.asset != nil { + if model.targets.isEmpty { + Text(L10n.text("This model has no named morph targets.")) + .font(.system(size: 12)).foregroundColor(theme.editorColors.muted) + .accessibilityIdentifier("AdaEditor.Morph.Empty") + } else { + HStack(spacing: 8) { + EditorModelPreviewViewport(model: preview, camera: camera) + .frame(width: 220, height: 150) + EditorMorphWeightList(model: model, onChange: { syncPreview() }) + } + .frame(maxHeight: .infinity) + } + if model.unnamedTargetCount > 0 { + Text(L10n.text("Targets without a name cannot be set from a scene.")) + .font(.system(size: 10)).foregroundColor(theme.editorColors.muted) + } + } + if !status.isEmpty { Text(status).font(.system(size: 11)).foregroundColor(theme.editorColors.muted) } + } + .padding(8) + .onAppear { load() } + .onDisappear { preview.cancel() } + .onChange(of: target.source) { old, new in + if old.source != new.source { load() } else { configure() } + } + .onChange(of: preview.asset.map(ObjectIdentifier.init)) { _, _ in configure() } + .accessibilityIdentifier("AdaEditor.Animator.MorphTargets") + } + + private var readOnly: Bool { viewModel.workbench.activeSceneDocument?.isReadOnly == true } + + private var toolbar: some View { + HStack(spacing: 8) { + Text(L10n.text("MORPH TARGETS")).font(.system(size: 11, weight: .semibold)) + Spacer() + action("Apply to Scene", id: "Apply") { + if viewModel.workbench.setMorphWeights(model.weights, target: target) { + status = L10n.tr("Applied to scene. Save the scene to keep these weights.") + } + } + .disabled(readOnly || model.targets.isEmpty || model.weights == target.source.morphWeights) + action("Reset", id: "Reset") { + model.reset() + syncPreview() + if viewModel.workbench.setMorphWeights([:], target: target) { + status = L10n.tr("Morph weights cleared.") + } + } + .disabled(readOnly || (!model.hasAuthoredWeights && target.source.morphWeights.isEmpty)) + } + } + + private func action(_ title: String, id: String, _ action: @escaping () -> Void) -> some View { + Button(action: action) { Text(L10n.tr(title)).font(.system(size: 11)).padding(4) } + .buttonStyle(DefaultButtonStyle()) + .accessibilityIdentifier("AdaEditor.Morph.\(id)") + } + + private func load() { + do { + let url = try EditorModelResource.resolve(target.source.source, sourceURL: target.sceneURL, resourceRootURL: viewModel.projectAssetsURL) + preview.load(path: url.path) + } catch { + preview.isLoading = false + preview.error = error.localizedDescription + } + } + + private func configure() { + guard let asset = preview.asset else { + return + } + model.configure(asset: asset, authored: target.source.morphWeights) + syncPreview() + } + + /// Every target gets its shown weight, so a reset puts the defaults back. + private func syncPreview() { + preview.morphWeights = Dictionary(uniqueKeysWithValues: model.targets.map { ($0.name, model.weight(of: $0)) }) + } +} diff --git a/Editor/Tests/AdaEditorTests/EditorMorphTargetTests.swift b/Editor/Tests/AdaEditorTests/EditorMorphTargetTests.swift new file mode 100644 index 000000000..33ec2e414 --- /dev/null +++ b/Editor/Tests/AdaEditorTests/EditorMorphTargetTests.swift @@ -0,0 +1,232 @@ +import Foundation +import Math +import Testing + +@_spi(Internal) @testable import AdaApp +@testable import AdaEditor +@_spi(AdaEngine) @testable import AdaEngine +@_spi(Internal) @testable import AdaRender +@_spi(Internal) import AdaUI +import AdaUtils + +@MainActor +@Suite(.serialized) +struct EditorMorphTargetTests { + init() throws { + if unsafe RenderEngine.shared == nil { + unsafe RenderEngine.configurations.preferredBackend = .headless + try RenderEngine.setupRenderEngine() + } + GLTFLoaderResolver.shared.setLoader(NativeGLTFLoader()) + EditorComponentRegistry.registerBuiltIns() + } + + /// A one-quad glTF with two named morph targets ("smile" resting at 0.25, "blink" at 0) written into `directory`. + private func writeModel(in directory: URL, targetNames: [String]? = ["smile", "blink"]) throws -> URL { + var bytes = Data() + var views: [[String: Any]] = [] + func add(_ data: Data) { + views.append(["buffer": 0, "byteOffset": bytes.count, "byteLength": data.count]) + bytes.append(data) + while bytes.count % 4 != 0 { bytes.append(0) } + } + func floats(_ values: [Float]) -> Data { + var data = Data() + for value in values { withUnsafeBytes(of: value.bitPattern.littleEndian) { data.append(contentsOf: $0) } } + return data + } + add(floats([-0.5, 0, 0, 0.5, 0, 0, -0.5, 1, 0, 0.5, 1, 0])) + var indices = Data() + for index in [0, 1, 2, 2, 1, 3] as [UInt16] { withUnsafeBytes(of: index.littleEndian) { indices.append(contentsOf: $0) } } + add(indices) + add(floats([0.3, 0, 0, 0.3, 0, 0, 0.3, 0, 0, 0.3, 0, 0])) + add(floats([0, 0.3, 0, 0, 0.3, 0, 0, 0.3, 0, 0, 0.3, 0])) + func accessor(_ view: Int, _ type: String, _ count: Int, _ component: Int = 5126) -> [String: Any] { + ["bufferView": view, "componentType": component, "count": count, "type": type] + } + var positions = accessor(0, "VEC3", 4) + positions["min"] = [-0.5, 0, 0] + positions["max"] = [0.5, 1, 0] + var mesh: [String: Any] = [ + "weights": [0.25, 0], + "primitives": [["attributes": ["POSITION": 0], "indices": 1, "targets": [["POSITION": 2], ["POSITION": 3]]]], + ] + if let targetNames { mesh["extras"] = ["targetNames": targetNames] } + let document: [String: Any] = [ + "asset": ["version": "2.0"], + "buffers": [["byteLength": bytes.count, "uri": "data:application/octet-stream;base64," + bytes.base64EncodedString()]], + "bufferViews": views, + "accessors": [positions, accessor(1, "SCALAR", 6, 5123), accessor(2, "VEC3", 4), accessor(3, "VEC3", 4)], + "meshes": [mesh], + "nodes": [["mesh": 0]], + "scenes": [["nodes": [0]]], + "scene": 0, + ] + let url = directory.appendingPathComponent("Face.gltf") + try JSONSerialization.data(withJSONObject: document).write(to: url) + return url + } + + private func makeDirectory() throws -> URL { + let directory = FileManager.default.temporaryDirectory.appendingPathComponent("MorphTargets-\(UUID().uuidString)") + try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) + return directory + } + + private func loadAsset(_ url: URL) throws -> ModelAsset3D { + try #require(AssetsManager.loadSync(ModelAsset3D.self, at: url.path).asset) + } + + @Test("The model lists named targets with the file's default weights and drops authored names the model lacks") + func listsTargets() throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let asset = try loadAsset(writeModel(in: directory)) + let model = EditorMorphTargetsModel() + model.configure(asset: asset, authored: ["smile": 0.7, "unknown": 1]) + #expect(model.targets == [.init(name: "smile", defaultWeight: 0.25), .init(name: "blink", defaultWeight: 0)]) + #expect(model.unnamedTargetCount == 0) + #expect(model.weights == ["smile": 0.7], "A name the model does not have is dropped.") + #expect(model.weight(of: model.targets[0]) == 0.7 && model.weight(of: model.targets[1]) == 0) + + let unnamed = EditorMorphTargetsModel() + unnamed.configure(asset: try loadAsset(writeModel(in: directory, targetNames: nil)), authored: [:]) + #expect(unnamed.targets.isEmpty && unnamed.unnamedTargetCount == 2, "Targets without names cannot be addressed from a scene.") + } + + @Test("Weights clamp, and returning a target to its default removes the override") + func editsWeights() throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let model = EditorMorphTargetsModel() + model.configure(asset: try loadAsset(writeModel(in: directory)), authored: [:]) + let smile = model.targets[0], blink = model.targets[1] + #expect(!model.hasAuthoredWeights) + model.setWeight(5, for: blink) + model.setWeight(.nan, for: smile) + #expect(model.weights == ["blink": 1], "Weights clamp to the slider range and non-finite values are ignored.") + model.setWeight(0.25, for: smile) + #expect(model.weights == ["blink": 1], "A weight equal to the file's default is not an override.") + model.setWeight(0.5, for: smile) + model.setWeight(0, for: blink) + #expect(model.weights == ["smile": 0.5]) + model.reset() + #expect(model.weights.isEmpty && !model.hasAuthoredWeights) + } + + @Test("The payload round trips and ignores non-numbers") + func payloadRoundTrip() { + let weights: [String: Float] = ["smile": 0.5, "blink": 1] + #expect(EditorMorphWeightsPayload.decode(EditorMorphWeightsPayload.encode(weights)) == weights) + #expect(EditorMorphWeightsPayload.decode(nil).isEmpty && EditorMorphWeightsPayload.decode(.string("x")).isEmpty) + #expect(EditorMorphWeightsPayload.decode(.object(["a": .string("no"), "b": .int(1), "c": .double(.nan)])) == ["b": 1]) + } + + /// A scene with one Model 3D entity that points at `Face.gltf` in `directory`, open in a workbench. + private func makeWorkbench(in directory: URL) throws -> (workbench: EditorWorkbenchViewModel, target: EditorSkeletalAnimationTarget, entityID: String, sceneURL: URL) { + _ = try writeModel(in: directory) + let sceneURL = directory.appendingPathComponent("Main.ascn") + var scene = EditorSceneModel.default(projectName: "Morph") + let entity = scene.addEntity(template: .importedModel3D, parentID: scene.rootEntityID) + let index = try #require(scene.entities.firstIndex { $0.id == entity.id }) + scene.entities[index].components[EditorBuiltInComponentType.model3DSource] = [ + "source": .string("@res://Face.gltf"), "animation": .string(""), "autoplay": .bool(false), "repeats": .bool(true), + ] + let content = try scene.encodedYAML() + try content.write(to: sceneURL, atomically: true, encoding: .utf8) + let document = EditorSceneDocument( + id: "Scene", title: "Main", relativePath: "Main.ascn", absolutePath: sceneURL.path, + content: content, lastSavedContent: content, sceneModel: scene, isDirty: false, + loadSummary: .init(entityCount: 0, warnings: []) + ) + let workbench = EditorWorkbenchViewModel(openDocuments: [.scene(document)], activeDocumentID: "Scene") + let target = EditorSkeletalAnimationTarget(documentID: "Scene", entityID: entity.id, source: .init(source: "@res://Face.gltf"), sceneURL: sceneURL) + return (workbench, target, entity.id, sceneURL) + } + + @Test("Applying saves structured scene data and supports undo, redo and clearing") + func sceneTransaction() throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let (workbench, target, entityID, sceneURL) = try makeWorkbench(in: directory) + + #expect(workbench.setMorphWeights(["smile": 0.8, "blink": 1], target: target)) + #expect(workbench.saveAllDocuments()) + let reopened = try EditorSceneModel.decode(from: String(contentsOf: sceneURL, encoding: .utf8)) + let saved = try #require(reopened.entities.first { $0.id == entityID }?.components[EditorBuiltInComponentType.model3DSource]) + #expect(EditorMorphWeightsPayload.decode(saved["morphWeights"]) == ["smile": 0.8, "blink": 1]) + let decoded = try #require(EditorComponentRegistry.decode(typeName: EditorBuiltInComponentType.model3DSource, payload: saved) as? Model3DSource) + #expect(decoded.morphWeights == ["smile": 0.8, "blink": 1]) + + #expect(workbench.performDocumentHistory(redo: false)) + let undone = try #require(workbench.activeSceneDocument?.sceneModel?.entities.first { $0.id == entityID }) + #expect(undone.components[EditorBuiltInComponentType.model3DSource]?["morphWeights"] == nil) + #expect(workbench.performDocumentHistory(redo: true)) + let redone = try #require(workbench.activeSceneDocument?.sceneModel?.entities.first { $0.id == entityID }) + #expect(EditorMorphWeightsPayload.decode(redone.components[EditorBuiltInComponentType.model3DSource]?["morphWeights"]) == ["smile": 0.8, "blink": 1]) + + // Clearing removes the key rather than leaving an empty map. + #expect(workbench.setMorphWeights([:], target: target)) + let cleared = try #require(workbench.activeSceneDocument?.sceneModel?.entities.first { $0.id == entityID }) + #expect(cleared.components[EditorBuiltInComponentType.model3DSource]?["morphWeights"] == nil) + let missing = EditorSkeletalAnimationTarget(documentID: "Scene", entityID: "missing", source: .init(source: "x"), sceneURL: sceneURL) + #expect(!workbench.setMorphWeights(["smile": 1], target: missing), "A target that no longer exists changes nothing.") + } + + /// Needs the scene file loader, which the Editor tests of this package cannot run in every environment (the existing + /// `EditorAnimationGraphTests` scene round trip ends the test process the same way where it fails). + @Test("A saved scene opens in the runtime with the authored weights") + func loaderAppliesAuthoredWeights() throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let (workbench, target, _, sceneURL) = try makeWorkbench(in: directory) + #expect(workbench.setMorphWeights(["smile": 0.8, "blink": 1], target: target)) + #expect(workbench.saveAllDocuments()) + let reopened = try EditorSceneModel.decode(from: String(contentsOf: sceneURL, encoding: .utf8)) + let world = World() + let result = EditorSceneFileLoader.load(content: try reopened.encodedYAML(), into: world, sourceURL: sceneURL, resourceRootURL: directory) + #expect(result.warnings.isEmpty, Comment(rawValue: result.warnings.joined(separator: "\n"))) + let morphed = try #require(world.getEntities().first { $0.components[MorphWeights3DComponent.self] != nil }) + #expect(morphed.components[MorphWeights3DComponent.self]?.weights == [0.8, 1]) + } + + @Test("The preview applies the weights to the model it shows, including defaults put back") + func previewShowsWeights() throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let asset = try loadAsset(writeModel(in: directory)) + let world = World() + _ = asset.instantiate(in: world) + let preview = EditorModelAssetPreviewModel() + preview.asset = asset + preview.bind(to: world) + func weights() throws -> [Float]? { + try #require(world.getEntities().first { $0.components[MorphWeights3DComponent.self] != nil }).components[MorphWeights3DComponent.self]?.weights + } + #expect(try weights() == [0.25, 0], "The file's defaults show before anything is authored.") + preview.morphWeights = ["smile": 0.9, "blink": 0.4] + #expect(try weights() == [0.9, 0.4]) + preview.morphWeights = ["smile": 0.25, "blink": 0] + #expect(try weights() == [0.25, 0], "Giving every target its default weight undoes the preview.") + } + + @Test("The weight list draws one named slider per target") + func listShowsSliders() async throws { + let directory = try makeDirectory() + defer { try? FileManager.default.removeItem(at: directory) } + let model = EditorMorphTargetsModel() + model.configure(asset: try loadAsset(writeModel(in: directory)), authored: ["smile": 0.5]) + let container = UIContainerView(rootView: EditorMorphWeightList(model: model)) + container.frame = Rect(x: 0, y: 0, width: 480, height: 160) + container.bounds.size = container.frame.size + for _ in 0..<3 { + try await Task.sleep(for: .milliseconds(1)) + container.update(1 / 60) + container.layoutIfNeeded() + } + for name in ["smile", "blink"] { + _ = try container.uiNode(matching: .accessibilityIdentifier("AdaEditor.Morph.Name.\(name)")) + _ = try container.uiNode(matching: .accessibilityIdentifier("AdaEditor.Morph.Slider.\(name)")) + } + } +} diff --git a/Sources/AdaAnimation/Skeletal/CompiledAnimationGraph.swift b/Sources/AdaAnimation/Skeletal/CompiledAnimationGraph.swift index 4bbfd55fe..654b4e96f 100644 --- a/Sources/AdaAnimation/Skeletal/CompiledAnimationGraph.swift +++ b/Sources/AdaAnimation/Skeletal/CompiledAnimationGraph.swift @@ -10,6 +10,9 @@ struct CompiledAnimationGraph: Sendable { let clipIndices: [Int?] let rest: [SkeletalJointPose] var poses: [[SkeletalJointPose]] + /// Morph weights per graph node and rig node (empty for rig nodes without morph targets). + var morph: [[[Float]]] + let restMorph: [[Float]] var contributions: [Float] let duration: Double let loops: Bool @@ -76,6 +79,9 @@ struct CompiledAnimationGraph: Sendable { self.clipIndices = clipIndices self.rest = rest self.poses = graph.nodes.map { _ in rest } + let restMorph = rig.nodes.map { $0.morphRestWeights ?? [] } + self.restMorph = restMorph + self.morph = graph.nodes.map { _ in restMorph } self.contributions = [Float](repeating: 0, count: graph.nodes.count) var duration: Double = 0 var loops = false @@ -132,9 +138,18 @@ struct CompiledAnimationGraph: Sendable { case .scale: poses[index][track.nodeIndex].scale = value.xyz } } + for joint in restMorph.indices where !restMorph[joint].isEmpty { + for target in restMorph[joint].indices { morph[index][joint][target] = restMorph[joint][target] } + } + for track in clip.morphTracks { + track.sample(at: local, into: &morph[index][track.nodeIndex]) + } } else { for joint in rest.indices { poses[index][joint] = node.kind == .blend ? blend(index, joint: joint) : additive(index, joint: joint) + if !restMorph[joint].isEmpty { + morph[index][joint] = node.kind == .blend ? blendMorph(index, joint: joint) : additiveMorph(index, joint: joint) + } } } } @@ -173,6 +188,38 @@ struct CompiledAnimationGraph: Sendable { return SkeletalJointPose(translation: translation / denominator, rotation: quaternion.squaredLength > 0.000001 ? quaternion.normalized : reference, scale: scale / denominator) } + /// Blends morph weights by the same rules as poses: weighted by input weight and mask, with the rest weights filling any gap. + private func blendMorph(_ index: Int, joint: Int) -> [Float] { + let node = graph.nodes[index] + var total: Float = 0 + for edge in node.inputs.indices { total += node.inputs[edge].weight * masks[index][edge][joint] } + let denominator = max(1, total) + let restWeight = max(0, 1 - total) + var result = restMorph[joint].map { $0 * restWeight } + for edge in node.inputs.indices { + let weight = node.inputs[edge].weight * masks[index][edge][joint] + let input = morph[inputs[index][edge]][joint] + for target in result.indices { result[target] += input[target] * weight } + } + for target in result.indices { result[target] /= denominator } + return result + } + + /// Additive morph weights: the first input is the base, and each further input adds its offset from the rest weights. + private func additiveMorph(_ index: Int, joint: Int) -> [Float] { + let node = graph.nodes[index] + let rest = restMorph[joint] + let base = morph[inputs[index][0]][joint] + let amount = min(1, node.inputs[0].weight * masks[index][0][joint]) + var result = rest.indices.map { rest[$0] + (base[$0] - rest[$0]) * amount } + for edge in node.inputs.indices.dropFirst() { + let weight = min(1, node.inputs[edge].weight * masks[index][edge][joint]) + let input = morph[inputs[index][edge]][joint] + for target in result.indices { result[target] += (input[target] - rest[target]) * weight } + } + return result + } + private func additive(_ index: Int, joint: Int) -> SkeletalJointPose { let node = graph.nodes[index] let base = poses[inputs[index][0]][joint] diff --git a/Sources/AdaAnimation/Skeletal/MorphWeightsTrack.swift b/Sources/AdaAnimation/Skeletal/MorphWeightsTrack.swift new file mode 100644 index 000000000..0b88f87e7 --- /dev/null +++ b/Sources/AdaAnimation/Skeletal/MorphWeightsTrack.swift @@ -0,0 +1,78 @@ +import Foundation + +/// A validated glTF-style track of morph target weights for one node: `targetCount` weights per key. +/// +/// Cubic keys store the in-tangents, the values and the out-tangents of every target, in that order. +public struct MorphWeightsTrack: Sendable { + public let nodeIndex: Int + public let interpolation: SkeletalAnimationTrack.Interpolation + public let times: [Double] + public let targetCount: Int + /// `times.count * targetCount` values, or three times that for cubic keys. + public let values: [Float] + + public init( + nodeIndex: Int, + interpolation: SkeletalAnimationTrack.Interpolation, + times: [Double], + targetCount: Int, + values: [Float] + ) throws(SkeletalAnimationError) { + let multiplier = interpolation == .cubicSpline ? 3 : 1 + guard nodeIndex >= 0, targetCount > 0, !times.isEmpty, times.allSatisfy({ $0.isFinite && $0 >= 0 }), + zip(times, times.dropFirst()).allSatisfy({ $0 < $1 }), + values.count == times.count * targetCount * multiplier, values.allSatisfy(\.isFinite), + interpolation != .cubicSpline || times.count >= 2 + else { throw .invalidTrack } + self.nodeIndex = nodeIndex + self.interpolation = interpolation + self.times = times + self.targetCount = targetCount + self.values = values + } + + /// Writes the weights at `time` into `output` (which must hold `targetCount` values) without allocating. + public func sample(at time: Double, into output: inout [Float]) { + guard output.count >= targetCount else { return } + guard time.isFinite, time > times[0] else { + for target in 0.. 1 { + let middle = (lower + upper) / 2 + if times[middle] <= time { lower = middle } else { upper = middle } + } + let span = Float(times[upper] - times[lower]) + let amount = Float((time - times[lower]) / (times[upper] - times[lower])) + let squared = amount * amount + let cubed = squared * amount + for target in 0.. Float { + interpolation == .cubicSpline ? values[(key * 3 + 1) * targetCount + target] : values[key * targetCount + target] + } +} diff --git a/Sources/AdaAnimation/Skeletal/SkeletalAnimation.swift b/Sources/AdaAnimation/Skeletal/SkeletalAnimation.swift index 1501d5790..d96e5897b 100644 --- a/Sources/AdaAnimation/Skeletal/SkeletalAnimation.swift +++ b/Sources/AdaAnimation/Skeletal/SkeletalAnimation.swift @@ -32,8 +32,11 @@ public struct SkeletalRig: Sendable { public let parentIndex: Int? public let restPose: SkeletalJointPose? public let restMatrix: Transform3D + /// The morph target weights of the mesh on this node when nothing animates them; `nil` for a node without morph targets. + public let morphRestWeights: [Float]? - public init(parentIndex: Int?, restPose: SkeletalJointPose?, restMatrix: Transform3D, name: String? = nil) { + public init(parentIndex: Int?, restPose: SkeletalJointPose?, restMatrix: Transform3D, name: String? = nil, morphRestWeights: [Float]? = nil) { + self.morphRestWeights = morphRestWeights self.name = name self.parentIndex = parentIndex self.restPose = restPose @@ -162,11 +165,14 @@ public struct SkeletalAnimationTrack: Sendable { public struct SkeletalAnimationClip: Sendable { public let name: String public let tracks: [SkeletalAnimationTrack] + /// Morph target weight tracks, which play alongside the pose tracks. + public let morphTracks: [MorphWeightsTrack] public let duration: Double - public init(name: String, tracks: [SkeletalAnimationTrack]) { + public init(name: String, tracks: [SkeletalAnimationTrack], morphTracks: [MorphWeightsTrack] = []) { self.name = name self.tracks = tracks - self.duration = tracks.compactMap { $0.times.last }.max() ?? 0 + self.morphTracks = morphTracks + self.duration = (tracks.compactMap { $0.times.last } + morphTracks.compactMap { $0.times.last }).max() ?? 0 } } diff --git a/Sources/AdaAnimation/Skeletal/SkeletalAnimationPlayer.swift b/Sources/AdaAnimation/Skeletal/SkeletalAnimationPlayer.swift index 4afbd55cb..e9cc830b0 100644 --- a/Sources/AdaAnimation/Skeletal/SkeletalAnimationPlayer.swift +++ b/Sources/AdaAnimation/Skeletal/SkeletalAnimationPlayer.swift @@ -17,7 +17,12 @@ public struct SkeletalAnimationPlayer: Sendable { public private(set) var poses: [SkeletalJointPose] public private(set) var localTransforms: [Transform3D] public private(set) var globalTransforms: [Transform3D] + /// Morph target weights by rig node: the weights of the mesh on that node, or an empty array for a node without morph targets. + public private(set) var morphWeights: [[Float]] + /// The rig nodes whose morph weights any clip animates. Other nodes keep whatever weights their entity was given. + public let animatedMorphNodes: Set private var transitionSource: [SkeletalJointPose] = [] + private var transitionMorphSource: [[Float]] = [] private var transitionDuration: Double = 0 private var transitionTime: Double = 0 private var graphState: CompiledAnimationGraph? @@ -30,6 +35,8 @@ public struct SkeletalAnimationPlayer: Sendable { guard clips.allSatisfy({ clip in clip.tracks.allSatisfy { rig.nodes.indices.contains($0.nodeIndex) && rig.nodes[$0.nodeIndex].restPose != nil } + // A weights track must address a node whose mesh has exactly that many targets. + && clip.morphTracks.allSatisfy { rig.nodes.indices.contains($0.nodeIndex) && rig.nodes[$0.nodeIndex].morphRestWeights?.count == $0.targetCount } }) else { throw .invalidTrack } self.rig = rig @@ -37,6 +44,8 @@ public struct SkeletalAnimationPlayer: Sendable { self.poses = rig.nodes.map { $0.restPose ?? SkeletalJointPose() } self.localTransforms = rig.nodes.map(\.restMatrix) self.globalTransforms = Array(repeating: .identity, count: rig.nodes.count) + self.morphWeights = rig.nodes.map { $0.morphRestWeights ?? [] } + self.animatedMorphNodes = Set(clips.flatMap { $0.morphTracks.map(\.nodeIndex) }) evaluate() } @@ -52,6 +61,7 @@ public struct SkeletalAnimationPlayer: Sendable { graphState = nil pendingGraphEvents.removeAll(keepingCapacity: true) self.transitionSource = poses + self.transitionMorphSource = morphWeights self.transitionDuration = transitionDuration.isFinite ? max(0, transitionDuration) : 0 self.transitionTime = 0 self.clipIndex = index @@ -113,6 +123,7 @@ public struct SkeletalAnimationPlayer: Sendable { graphState = compiled clipIndex = nil transitionSource = poses + transitionMorphSource = morphWeights self.transitionDuration = transitionDuration.isFinite ? max(0, transitionDuration) : 0 transitionTime = 0 time = 0 @@ -156,13 +167,22 @@ public struct SkeletalAnimationPlayer: Sendable { } private mutating func evaluate() { - for index in rig.nodes.indices { poses[index] = rig.nodes[index].restPose ?? SkeletalJointPose() } + for index in rig.nodes.indices { + poses[index] = rig.nodes[index].restPose ?? SkeletalJointPose() + if let rest = rig.nodes[index].morphRestWeights { + // Element-wise, so the buffer is reused rather than reallocated every frame. + for target in rest.indices { morphWeights[index][target] = rest[target] } + } + } if graphState != nil { let graphTime = time let graphClips = clips graphState?.evaluate(time: graphTime, clips: graphClips) if let root = graphState?.root { for index in poses.indices { poses[index] = graphState?.poses[root][index] ?? poses[index] } + for index in morphWeights.indices where !morphWeights[index].isEmpty { + morphWeights[index] = graphState?.morph[root][index] ?? morphWeights[index] + } } } else if let clipIndex { let clip = clips[clipIndex] @@ -176,6 +196,9 @@ public struct SkeletalAnimationPlayer: Sendable { case .scale: poses[track.nodeIndex].scale = value.xyz } } + for track in clip.morphTracks { + track.sample(at: localTime, into: &morphWeights[track.nodeIndex]) + } } if transitionDuration > 0 && transitionTime < transitionDuration { let amount = Float(transitionTime / transitionDuration) @@ -185,8 +208,15 @@ public struct SkeletalAnimationPlayer: Sendable { poses[index].scale = source.scale + (poses[index].scale - source.scale) * amount poses[index].rotation = slerpQuat(source.rotation, poses[index].rotation, t: amount) } + for index in morphWeights.indices where transitionMorphSource.indices.contains(index) && transitionMorphSource[index].count == morphWeights[index].count { + for target in morphWeights[index].indices { + let from = transitionMorphSource[index][target] + morphWeights[index][target] = from + (morphWeights[index][target] - from) * amount + } + } } else { transitionSource.removeAll(keepingCapacity: true) + transitionMorphSource.removeAll(keepingCapacity: true) transitionDuration = 0 } rebuildTransforms() diff --git a/Sources/AdaAssets/GLTF/GLTFLoader.swift b/Sources/AdaAssets/GLTF/GLTFLoader.swift index bd7a5b6a6..34c81d1e0 100644 --- a/Sources/AdaAssets/GLTF/GLTFLoader.swift +++ b/Sources/AdaAssets/GLTF/GLTFLoader.swift @@ -24,8 +24,19 @@ public struct GLTFImportResult: Sendable { public let skinIndex: Int? /// Authored local TRS; nil for nodes authored with a matrix. public let restPose: NodeTRS? - - public init(name: String?, transform: Transform3D, children: [Int], meshIndex: Int?, skinIndex: Int? = nil, restPose: NodeTRS? = nil) { + /// Morph target weights of this node's mesh instance; they replace the mesh's default weights. + public let weights: [Float]? + + public init( + name: String?, + transform: Transform3D, + children: [Int], + meshIndex: Int?, + skinIndex: Int? = nil, + restPose: NodeTRS? = nil, + weights: [Float]? = nil + ) { + self.weights = weights self.name = name self.transform = transform self.children = children @@ -35,13 +46,36 @@ public struct GLTFImportResult: Sendable { } } + /// Per-vertex displacement of one morph target of a primitive. A target that leaves an attribute alone has an empty + /// array for it; otherwise each array has one entry per vertex. Tangent deltas are directions only (xyz). + public struct MorphTarget: Sendable, Equatable { + public let positionDeltas: [Vector3] + public let normalDeltas: [Vector3] + public let tangentDeltas: [Vector3] + + public init(positionDeltas: [Vector3] = [], normalDeltas: [Vector3] = [], tangentDeltas: [Vector3] = []) { + self.positionDeltas = positionDeltas + self.normalDeltas = normalDeltas + self.tangentDeltas = tangentDeltas + } + } + public struct Mesh: Sendable { public let name: String? public let primitives: [Primitive] + /// Default morph target weights, one per target; empty when the mesh has no morph targets. + public let defaultWeights: [Float] + /// Morph target names (from `extras.targetNames`); empty or one per target. + public let targetNames: [String] + + /// Number of morph targets every primitive of this mesh has. + public var morphTargetCount: Int { primitives.first?.morphTargets.count ?? 0 } - public init(name: String?, primitives: [Primitive]) { + public init(name: String?, primitives: [Primitive], defaultWeights: [Float] = [], targetNames: [String] = []) { self.name = name self.primitives = primitives + self.defaultWeights = defaultWeights + self.targetNames = targetNames } } @@ -51,8 +85,18 @@ public struct GLTFImportResult: Sendable { public let materialIndex: Int? public let mode: PrimitiveMode public let skinning: Skinning? - - public init(attributes: [Attribute: Accessor], indices: [UInt32]?, materialIndex: Int?, mode: PrimitiveMode, skinning: Skinning? = nil) { + /// Morph targets of this primitive, in target order. + public let morphTargets: [MorphTarget] + + public init( + attributes: [Attribute: Accessor], + indices: [UInt32]?, + materialIndex: Int?, + mode: PrimitiveMode, + skinning: Skinning? = nil, + morphTargets: [MorphTarget] = [] + ) { + self.morphTargets = morphTargets self.attributes = attributes self.indices = indices self.materialIndex = materialIndex @@ -159,6 +203,8 @@ public struct GLTFImportResult: Sendable { public let defaultScene: Int? public let skins: [Skin] public let animations: [Animation] + /// Content the importer read but could not honor, such as an optional extension it does not implement. + public let warnings: [String] public init( nodes: [Node], @@ -170,7 +216,8 @@ public struct GLTFImportResult: Sendable { defaultScene: Int?, skins: [Skin] = [], animations: [Animation] = [], - samplers: [Sampler] = [] + samplers: [Sampler] = [], + warnings: [String] = [] ) { self.nodes = nodes self.meshes = meshes @@ -182,6 +229,7 @@ public struct GLTFImportResult: Sendable { self.skins = skins self.animations = animations self.samplers = samplers + self.warnings = warnings } } diff --git a/Sources/AdaAssets/GLTF/GLTFMaterialData.swift b/Sources/AdaAssets/GLTF/GLTFMaterialData.swift index 42547f57a..8f608f9b5 100644 --- a/Sources/AdaAssets/GLTF/GLTFMaterialData.swift +++ b/Sources/AdaAssets/GLTF/GLTFMaterialData.swift @@ -7,6 +7,72 @@ public extension GLTFImportResult { case blend = "BLEND" } + /// `KHR_texture_transform`: the UV of a texture is scaled, then rotated (counter-clockwise, radians), then offset. + struct TextureTransform: Sendable, Equatable { + public var offset: Vector2 + public var rotation: Float + public var scale: Vector2 + + public static let identity = TextureTransform() + + public init(offset: Vector2 = .zero, rotation: Float = 0, scale: Vector2 = .one) { + self.offset = offset + self.rotation = rotation + self.scale = scale + } + } + + /// A texture used by a material extension: which texture, the UV set and transform it is read with, and a scale + /// (only the clearcoat normal map has one). + struct TextureReference: Sendable, Equatable { + public let index: Int + public let texCoord: Int + public let transform: TextureTransform + public let scale: Float + + public init(index: Int, texCoord: Int = 0, transform: TextureTransform = .identity, scale: Float = 1) { + self.index = index + self.texCoord = texCoord + self.transform = transform + self.scale = scale + } + } + + /// Textures read by `KHR_materials_clearcoat`, `KHR_materials_sheen`, `KHR_materials_anisotropy` and + /// `KHR_materials_specular`. Channels follow the extension texts: clearcoat in red, clearcoat roughness in green, + /// sheen color in RGB (sRGB), sheen roughness in alpha, anisotropy direction in RG and strength in blue, specular + /// strength in alpha and specular color in RGB (sRGB). + struct LayerTextures: Sendable, Equatable { + public var clearcoat: TextureReference? + public var clearcoatRoughness: TextureReference? + public var clearcoatNormal: TextureReference? + public var sheenColor: TextureReference? + public var sheenRoughness: TextureReference? + public var anisotropy: TextureReference? + public var specular: TextureReference? + public var specularColor: TextureReference? + + public init( + clearcoat: TextureReference? = nil, + clearcoatRoughness: TextureReference? = nil, + clearcoatNormal: TextureReference? = nil, + sheenColor: TextureReference? = nil, + sheenRoughness: TextureReference? = nil, + anisotropy: TextureReference? = nil, + specular: TextureReference? = nil, + specularColor: TextureReference? = nil + ) { + self.clearcoat = clearcoat + self.clearcoatRoughness = clearcoatRoughness + self.clearcoatNormal = clearcoatNormal + self.sheenColor = sheenColor + self.sheenRoughness = sheenRoughness + self.anisotropy = anisotropy + self.specular = specular + self.specularColor = specularColor + } + } + /// glTF metallic/roughness material values; color factors are linear. struct Material: Sendable { public let name: String? @@ -27,6 +93,24 @@ public extension GLTFImportResult { public let doubleSided: Bool /// UV set indices for base color, metallic/roughness, normal, occlusion, emission. public let textureCoordinates: [Int] + /// `KHR_texture_transform` for base color, metallic/roughness, normal, occlusion, emission. + public let textureTransforms: [TextureTransform] + /// `KHR_materials_ior`. The glTF default is 1.5, which gives the 4% dielectric reflectance. + public let ior: Float + /// `KHR_materials_specular` strength and tint of the dielectric reflection. Specular textures are in `layerTextures`. + public let specularFactor: Float + public let specularColorFactor: Vector3 + /// `KHR_materials_clearcoat` strength and roughness. Clearcoat textures are in `layerTextures`. + public let clearcoatFactor: Float + public let clearcoatRoughnessFactor: Float + /// `KHR_materials_sheen` color (linear) and roughness. Sheen textures are in `layerTextures`. + public let sheenColorFactor: Vector3 + public let sheenRoughnessFactor: Float + /// `KHR_materials_anisotropy` strength and rotation. The anisotropy texture is in `layerTextures`. + public let anisotropyStrength: Float + public let anisotropyRotation: Float + /// The textures of the extensions above. + public let layerTextures: LayerTextures public init( name: String?, @@ -45,7 +129,18 @@ public extension GLTFImportResult { alphaMode: AlphaMode = .opaque, alphaCutoff: Float = 0.5, doubleSided: Bool = false, - textureCoordinates: [Int] = [0, 0, 0, 0, 0] + textureCoordinates: [Int] = [0, 0, 0, 0, 0], + textureTransforms: [TextureTransform] = Array(repeating: .identity, count: 5), + ior: Float = 1.5, + specularFactor: Float = 1, + specularColorFactor: Vector3 = .one, + clearcoatFactor: Float = 0, + clearcoatRoughnessFactor: Float = 0, + sheenColorFactor: Vector3 = .zero, + sheenRoughnessFactor: Float = 0, + anisotropyStrength: Float = 0, + anisotropyRotation: Float = 0, + layerTextures: LayerTextures = LayerTextures() ) { self.name = name self.baseColorFactor = baseColorFactor @@ -64,6 +159,17 @@ public extension GLTFImportResult { self.alphaCutoff = alphaCutoff self.doubleSided = doubleSided self.textureCoordinates = textureCoordinates + self.textureTransforms = textureTransforms + self.ior = ior + self.specularFactor = specularFactor + self.specularColorFactor = specularColorFactor + self.clearcoatFactor = clearcoatFactor + self.clearcoatRoughnessFactor = clearcoatRoughnessFactor + self.sheenColorFactor = sheenColorFactor + self.sheenRoughnessFactor = sheenRoughnessFactor + self.anisotropyStrength = anisotropyStrength + self.anisotropyRotation = anisotropyRotation + self.layerTextures = layerTextures } } diff --git a/Sources/AdaAssets/GLTF/GLTFSkeletalData.swift b/Sources/AdaAssets/GLTF/GLTFSkeletalData.swift index 0a8ef9432..56d88f63f 100644 --- a/Sources/AdaAssets/GLTF/GLTFSkeletalData.swift +++ b/Sources/AdaAssets/GLTF/GLTFSkeletalData.swift @@ -54,6 +54,8 @@ extension GLTFImportResult { case translation case rotation case scale + /// Morph target weights of the node's mesh: one value per target for each key. + case weights } public struct AnimationSampler: Sendable { diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Animations.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Animations.swift index ab00a452d..f9a678167 100644 --- a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Animations.swift +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Animations.swift @@ -33,7 +33,8 @@ extension NativeGLTFLoader { guard let path = GLTFImportResult.AnimationPath(rawValue: channel.target.path) else { throw GLTFError.unsupportedAnimationPath(channel.target.path) } - guard let node = channel.target.node, nodes.indices.contains(node), nodes[node].matrix == nil, + // Weights animate a mesh's morph targets, so they are valid on a node that is placed by a matrix too. + guard let node = channel.target.node, nodes.indices.contains(node), path == .weights || nodes[node].matrix == nil, samplers.indices.contains(channel.sampler), targets.insert("\(node):\(path.rawValue)").inserted else { throw GLTFError.invalidAnimation(index) @@ -41,8 +42,14 @@ extension NativeGLTFLoader { let sampler = samplers[channel.sampler] let output = try checkedAccessor(animation.samplers[channel.sampler].output, gltf: gltf) let isCubic = sampler.interpolation == .cubicSpline - guard output.type == (path == .rotation ? "VEC4" : "VEC3"), - sampler.output.count == sampler.times.count * (isCubic ? 3 : 1), + // A weights sampler holds one value per target and key (three of each per key for cubic splines). + let valuesPerKey = path == .weights ? morphTargetCount(ofNode: nodes[node], in: gltf) : 1 + guard path != .weights || valuesPerKey > 0 else { + throw GLTFError.invalidAnimation(index) + } + let expectedType = path == .rotation ? "VEC4" : (path == .weights ? "SCALAR" : "VEC3") + guard output.type == expectedType, + sampler.output.count == sampler.times.count * valuesPerKey * (isCubic ? 3 : 1), !isCubic || sampler.times.count >= 2 else { throw GLTFError.invalidAnimation(index) diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Materials.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Materials.swift index 1f9797beb..91f1ce37d 100644 --- a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Materials.swift +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Materials.swift @@ -20,56 +20,186 @@ extension NativeGLTFLoader { } func importMaterials(_ gltf: GLTF) throws -> [GLTFImportResult.Material] { - try (gltf.materials ?? []).enumerated().map { index, material in - let pbr = material.pbrMetallicRoughness - let base = pbr?.baseColorFactor ?? [1, 1, 1, 1] - let emissive = material.emissiveFactor ?? [0, 0, 0] - let metallic = pbr?.metallicFactor ?? 1 - let roughness = pbr?.roughnessFactor ?? 1 - let normalScale = material.normalTexture?.scale ?? 1 - let occlusion = material.occlusionTexture?.strength ?? 1 - let cutoff = material.alphaCutoff ?? 0.5 - let strength = material.extensions?.KHR_materials_emissive_strength?.emissiveStrength ?? 1 - let coordinates = [ - pbr?.baseColorTexture?.texCoord ?? 0, - pbr?.metallicRoughnessTexture?.texCoord ?? 0, - material.normalTexture?.texCoord ?? 0, - material.occlusionTexture?.texCoord ?? 0, - material.emissiveTexture?.texCoord ?? 0, - ] - let textures = [ - pbr?.baseColorTexture?.index, - pbr?.metallicRoughnessTexture?.index, - material.normalTexture?.index, - material.occlusionTexture?.index, - material.emissiveTexture?.index, - ] - guard base.count == 4, emissive.count == 3, - (base + emissive + [metallic, roughness, occlusion]).allSatisfy({ $0.isFinite && $0 >= 0 && $0 <= 1 }), - normalScale.isFinite, cutoff.isFinite, cutoff >= 0, strength.isFinite, strength >= 0, - coordinates.allSatisfy({ $0 == 0 || $0 == 1 }), - textures.compactMap({ $0 }).allSatisfy({ (gltf.textures ?? []).indices.contains($0) }), - let alpha = GLTFImportResult.AlphaMode(rawValue: material.alphaMode ?? "OPAQUE") - else { throw GLTFError.invalidMaterial(index) } - return .init( - name: material.name, - baseColorFactor: Vector4(base[0], base[1], base[2], base[3]), - baseColorTextureIndex: textures[0], - metallicFactor: metallic, - roughnessFactor: roughness, - metallicRoughnessTextureIndex: textures[1], - normalTextureIndex: textures[2], - normalScale: normalScale, - occlusionTextureIndex: textures[3], - occlusionStrength: occlusion, - emissiveTextureIndex: textures[4], - emissiveFactor: Vector3(emissive[0], emissive[1], emissive[2]), - emissiveStrength: strength, - alphaMode: alpha, - alphaCutoff: cutoff, - doubleSided: material.doubleSided ?? false, - textureCoordinates: coordinates - ) + var result: [GLTFImportResult.Material] = [] + for (index, material) in (gltf.materials ?? []).enumerated() { + result.append(try importMaterial(material, index: index, textureCount: (gltf.textures ?? []).count)) + } + return result + } + + /// The `KHR_texture_transform` of every texture slot (base color, metallic/roughness, normal, occlusion, emission). + private func textureTransformInfos(_ material: GLTF.Material) -> [GLTF.TextureExtensions.Transform?] { + let pbr = material.pbrMetallicRoughness + var infos: [GLTF.TextureExtensions.Transform?] = [] + infos.append(pbr?.baseColorTexture?.extensions?.KHR_texture_transform) + infos.append(pbr?.metallicRoughnessTexture?.extensions?.KHR_texture_transform) + infos.append(material.normalTexture?.extensions?.KHR_texture_transform) + infos.append(material.occlusionTexture?.extensions?.KHR_texture_transform) + infos.append(material.emissiveTexture?.extensions?.KHR_texture_transform) + return infos + } + + private func isValid(_ info: GLTF.TextureExtensions.Transform?) -> Bool { + guard let info else { return true } + let offset: [Float] = info.offset ?? [0, 0] + let scale: [Float] = info.scale ?? [1, 1] + let values: [Float] = offset + scale + [info.rotation ?? 0] + return offset.count == 2 && scale.count == 2 && values.allSatisfy { $0.isFinite } + } + + /// Reads one extension texture reference. Returns `.absent` when there is none and `.invalid` when its index, UV set or + /// transform cannot be used. + private enum LayerReference { + case absent + case invalid + case value(GLTFImportResult.TextureReference) + } + + private func layerReference(_ reference: GLTF.Material.Extensions.TextureReference?, textureCount: Int) -> LayerReference { + guard let reference else { return .absent } + let transformInfo = reference.extensions?.KHR_texture_transform + let texCoord = transformInfo?.texCoord ?? reference.texCoord ?? 0 + let scale: Float = reference.scale ?? 1 + guard reference.index >= 0, reference.index < textureCount, texCoord == 0 || texCoord == 1, scale.isFinite, isValid(transformInfo) else { + return .invalid + } + var transform = GLTFImportResult.TextureTransform.identity + if let transformInfo { + let offset: [Float] = transformInfo.offset ?? [0, 0] + let scaling: [Float] = transformInfo.scale ?? [1, 1] + transform = .init(offset: Vector2(offset[0], offset[1]), rotation: transformInfo.rotation ?? 0, scale: Vector2(scaling[0], scaling[1])) } + return .value(.init(index: reference.index, texCoord: texCoord, transform: transform, scale: scale)) + } + + private func importMaterial(_ material: GLTF.Material, index: Int, textureCount: Int) throws -> GLTFImportResult.Material { + let pbr = material.pbrMetallicRoughness + let base: [Float] = pbr?.baseColorFactor ?? [1, 1, 1, 1] + let emissive: [Float] = material.emissiveFactor ?? [0, 0, 0] + let metallic: Float = pbr?.metallicFactor ?? 1 + let roughness: Float = pbr?.roughnessFactor ?? 1 + let normalScale: Float = material.normalTexture?.scale ?? 1 + let occlusion: Float = material.occlusionTexture?.strength ?? 1 + let cutoff: Float = material.alphaCutoff ?? 0.5 + let strength: Float = material.extensions?.KHR_materials_emissive_strength?.emissiveStrength ?? 1 + let transformInfos = textureTransformInfos(material) + // KHR_texture_transform may name another UV set for its texture. + let authoredCoordinates: [Int] = [ + pbr?.baseColorTexture?.texCoord ?? 0, + pbr?.metallicRoughnessTexture?.texCoord ?? 0, + material.normalTexture?.texCoord ?? 0, + material.occlusionTexture?.texCoord ?? 0, + material.emissiveTexture?.texCoord ?? 0, + ] + var coordinates: [Int] = [] + for slot in authoredCoordinates.indices { + coordinates.append(transformInfos[slot]?.texCoord ?? authoredCoordinates[slot]) + } + let transformsAreValid = transformInfos.allSatisfy { isValid($0) } + var transforms: [GLTFImportResult.TextureTransform] = [] + for info in transformInfos { + guard let info, transformsAreValid else { + transforms.append(.identity) + continue + } + let offset: [Float] = info.offset ?? [0, 0] + let scale: [Float] = info.scale ?? [1, 1] + transforms.append(.init(offset: Vector2(offset[0], offset[1]), rotation: info.rotation ?? 0, scale: Vector2(scale[0], scale[1]))) + } + let ior: Float = material.extensions?.KHR_materials_ior?.ior ?? 1.5 + let specular = material.extensions?.KHR_materials_specular + let specularFactor: Float = specular?.specularFactor ?? 1 + let specularColor: [Float] = specular?.specularColorFactor ?? [1, 1, 1] + let coat = material.extensions?.KHR_materials_clearcoat + let clearcoatFactor: Float = coat?.clearcoatFactor ?? 0 + let clearcoatRoughness: Float = coat?.clearcoatRoughnessFactor ?? 0 + let sheen = material.extensions?.KHR_materials_sheen + let sheenColor: [Float] = sheen?.sheenColorFactor ?? [0, 0, 0] + let sheenRoughness: Float = sheen?.sheenRoughnessFactor ?? 0 + let anisotropy = material.extensions?.KHR_materials_anisotropy + let layerInputs: [LayerReference] = [ + layerReference(coat?.clearcoatTexture, textureCount: textureCount), + layerReference(coat?.clearcoatRoughnessTexture, textureCount: textureCount), + layerReference(coat?.clearcoatNormalTexture, textureCount: textureCount), + layerReference(sheen?.sheenColorTexture, textureCount: textureCount), + layerReference(sheen?.sheenRoughnessTexture, textureCount: textureCount), + layerReference(material.extensions?.KHR_materials_anisotropy?.anisotropyTexture, textureCount: textureCount), + layerReference(material.extensions?.KHR_materials_specular?.specularTexture, textureCount: textureCount), + layerReference(material.extensions?.KHR_materials_specular?.specularColorTexture, textureCount: textureCount), + ] + var layerValues: [GLTFImportResult.TextureReference?] = [] + var layerTexturesAreValid = true + for input in layerInputs { + switch input { + case .absent: layerValues.append(nil) + case .invalid: + layerTexturesAreValid = false + layerValues.append(nil) + case let .value(value): layerValues.append(value) + } + } + let anisotropyStrength: Float = anisotropy?.anisotropyStrength ?? 0 + let anisotropyRotation: Float = anisotropy?.anisotropyRotation ?? 0 + let textures: [Int?] = [ + pbr?.baseColorTexture?.index, + pbr?.metallicRoughnessTexture?.index, + material.normalTexture?.index, + material.occlusionTexture?.index, + material.emissiveTexture?.index, + ] + let unitValues: [Float] = base + emissive + [metallic, roughness, occlusion] + let texturesExist = textures.compactMap { $0 }.allSatisfy { $0 >= 0 && $0 < textureCount } + let factorsAreValid = base.count == 4 && emissive.count == 3 && unitValues.allSatisfy { $0.isFinite && $0 >= 0 && $0 <= 1 } + let scalarsAreValid = normalScale.isFinite && cutoff.isFinite && cutoff >= 0 && strength.isFinite && strength >= 0 + let coatIsValid = clearcoatFactor.isFinite && clearcoatFactor >= 0 && clearcoatFactor <= 1 + && clearcoatRoughness.isFinite && clearcoatRoughness >= 0 && clearcoatRoughness <= 1 + let sheenIsValid = sheenColor.count == 3 && sheenColor.allSatisfy { $0.isFinite && $0 >= 0 && $0 <= 1 } + && sheenRoughness.isFinite && sheenRoughness >= 0 && sheenRoughness <= 1 + let anisotropyIsValid = anisotropyStrength.isFinite && anisotropyStrength >= 0 && anisotropyStrength <= 1 && anisotropyRotation.isFinite + let specularIsValid = ior.isFinite && ior >= 1 && ior <= 100 && specularFactor.isFinite && specularFactor >= 0 + && specularColor.count == 3 && specularColor.allSatisfy { $0.isFinite && $0 >= 0 } + guard factorsAreValid, scalarsAreValid, specularIsValid, coatIsValid, sheenIsValid, anisotropyIsValid, layerTexturesAreValid, texturesExist, transformsAreValid, + coordinates.allSatisfy({ $0 == 0 || $0 == 1 }), + let alpha = GLTFImportResult.AlphaMode(rawValue: material.alphaMode ?? "OPAQUE") + else { throw GLTFError.invalidMaterial(index) } + return .init( + name: material.name, + baseColorFactor: Vector4(base[0], base[1], base[2], base[3]), + baseColorTextureIndex: textures[0], + metallicFactor: metallic, + roughnessFactor: roughness, + metallicRoughnessTextureIndex: textures[1], + normalTextureIndex: textures[2], + normalScale: normalScale, + occlusionTextureIndex: textures[3], + occlusionStrength: occlusion, + emissiveTextureIndex: textures[4], + emissiveFactor: Vector3(emissive[0], emissive[1], emissive[2]), + emissiveStrength: strength, + alphaMode: alpha, + alphaCutoff: cutoff, + doubleSided: material.doubleSided ?? false, + textureCoordinates: coordinates, + textureTransforms: transforms, + ior: ior, + specularFactor: specularFactor, + specularColorFactor: Vector3(specularColor[0], specularColor[1], specularColor[2]), + clearcoatFactor: clearcoatFactor, + clearcoatRoughnessFactor: clearcoatRoughness, + sheenColorFactor: Vector3(sheenColor[0], sheenColor[1], sheenColor[2]), + sheenRoughnessFactor: sheenRoughness, + anisotropyStrength: anisotropyStrength, + anisotropyRotation: anisotropyRotation, + layerTextures: GLTFImportResult.LayerTextures( + clearcoat: layerValues[0], + clearcoatRoughness: layerValues[1], + clearcoatNormal: layerValues[2], + sheenColor: layerValues[3], + sheenRoughness: layerValues[4], + anisotropy: layerValues[5], + specular: layerValues[6], + specularColor: layerValues[7] + ) + ) } } diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Morph.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Morph.swift new file mode 100644 index 000000000..90cd2709f --- /dev/null +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Morph.swift @@ -0,0 +1,77 @@ +import Foundation +import Math + +extension NativeGLTFLoader { + /// Reads the morph targets of one primitive. Each target lists the attributes it displaces (POSITION, NORMAL, + /// TANGENT); every accessor must be a float-vector per vertex of the primitive. Other morphed attributes are + /// ignored and reported by `warnings(for:)`. + func importMorphTargets( + _ primitive: GLTF.Mesh.Primitive, + attributes: [GLTFImportResult.Attribute: GLTFImportResult.Accessor], + gltf: GLTF, + buffers: [Data], + meshIndex: Int + ) throws -> [GLTFImportResult.MorphTarget] { + guard let targets = primitive.targets, !targets.isEmpty else { + return [] + } + guard let positions = attributes[.position] else { + throw GLTFError.invalidMorphTargets(meshIndex) + } + func deltas(_ name: String, in target: [String: Int]) throws -> [Vector3] { + guard let accessorIndex = target[name] else { return [] } + let accessor = try checkedAccessor(accessorIndex, gltf: gltf) + let decoded = try decodeAccessor(accessorIndex, gltf: gltf, buffers: buffers) + guard accessor.type == "VEC3", decoded.componentCount == 3, decoded.values.count == positions.count * 3, + decoded.values.allSatisfy({ $0.isFinite }) + else { + throw GLTFError.invalidMorphTargets(meshIndex) + } + var vectors: [Vector3] = [] + vectors.reserveCapacity(positions.count) + for offset in stride(from: 0, to: decoded.values.count, by: 3) { + let x = Float(decoded.values[offset]), y = Float(decoded.values[offset + 1]), z = Float(decoded.values[offset + 2]) + vectors.append(Vector3(x, y, z)) + } + return vectors + } + return try targets.map { target in + GLTFImportResult.MorphTarget( + positionDeltas: try deltas("POSITION", in: target), + normalDeltas: try deltas("NORMAL", in: target), + tangentDeltas: try deltas("TANGENT", in: target) + ) + } + } + + /// Checks that every primitive of a mesh has the same number of targets and that the default weights and names match. + func importMorphMetadata( + _ mesh: GLTF.Mesh, + primitives: [GLTFImportResult.Primitive], + meshIndex: Int + ) throws -> (weights: [Float], names: [String]) { + let count = primitives.first?.morphTargets.count ?? 0 + guard primitives.allSatisfy({ $0.morphTargets.count == count }) else { + throw GLTFError.invalidMorphTargets(meshIndex) + } + guard count > 0 else { + // Weights or names without targets mean nothing; an authored non-empty list is an error. + if mesh.weights?.isEmpty == false { throw GLTFError.invalidMorphTargets(meshIndex) } + return ([], []) + } + let weights = mesh.weights ?? [Float](repeating: 0, count: count) + let names = mesh.extras?.targetNames ?? [] + guard weights.count == count, weights.allSatisfy(\.isFinite), names.isEmpty || names.count == count else { + throw GLTFError.invalidMorphTargets(meshIndex) + } + return (weights, names) + } + + /// The number of morph targets of the mesh a node instantiates. + func morphTargetCount(ofNode node: GLTF.Node, in gltf: GLTF) -> Int { + guard let meshIndex = node.mesh, let meshes = gltf.meshes, meshes.indices.contains(meshIndex) else { + return 0 + } + return meshes[meshIndex].primitives.first?.targets?.count ?? 0 + } +} diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Schema.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Schema.swift index 2af814cc6..42984b394 100644 --- a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Schema.swift +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Schema.swift @@ -9,6 +9,7 @@ extension NativeGLTFLoader { case invalidSkinningAttributes case unsupportedJointSet(Int) case invalidAnimation(Int) + case invalidMorphTargets(Int) case unsupportedAnimationPath(String) case invalidGLB case missingJSONChunk @@ -77,16 +78,36 @@ extension NativeGLTFLoader { let indices: Int? let material: Int? let mode: Int? + /// Morph targets: per target, the accessor of each displaced attribute (POSITION, NORMAL, TANGENT). + let targets: [[String: Int]]? + } + + /// Application data glTF leaves open; `targetNames` is the widely used convention for naming morph targets. + struct Extras: Codable { + let targetNames: [String]? } let name: String? let primitives: [Primitive] + let weights: [Float]? + let extras: Extras? + } + + struct TextureExtensions: Codable { + struct Transform: Codable { + let offset: [Float]? + let rotation: Float? + let scale: [Float]? + let texCoord: Int? + } + let KHR_texture_transform: Transform? } struct Material: Codable { struct TextureInfo: Codable { let index: Int let texCoord: Int? + let extensions: TextureExtensions? } struct PBR: Codable { @@ -101,17 +122,55 @@ extension NativeGLTFLoader { let index: Int let texCoord: Int? let scale: Float? + let extensions: TextureExtensions? } struct OcclusionTextureInfo: Codable { let index: Int let texCoord: Int? let strength: Float? + let extensions: TextureExtensions? } struct Extensions: Codable { struct EmissiveStrength: Codable { let emissiveStrength: Float } + struct IOR: Codable { let ior: Float? } + struct TextureReference: Codable { + let index: Int + let texCoord: Int? + let scale: Float? + let extensions: TextureExtensions? + } + struct Specular: Codable { + let specularFactor: Float? + let specularColorFactor: [Float]? + let specularTexture: TextureReference? + let specularColorTexture: TextureReference? + } + struct Clearcoat: Codable { + let clearcoatFactor: Float? + let clearcoatTexture: TextureReference? + let clearcoatRoughnessFactor: Float? + let clearcoatRoughnessTexture: TextureReference? + let clearcoatNormalTexture: TextureReference? + } + struct Sheen: Codable { + let sheenColorFactor: [Float]? + let sheenColorTexture: TextureReference? + let sheenRoughnessFactor: Float? + let sheenRoughnessTexture: TextureReference? + } + struct Anisotropy: Codable { + let anisotropyStrength: Float? + let anisotropyRotation: Float? + let anisotropyTexture: TextureReference? + } let KHR_materials_emissive_strength: EmissiveStrength? + let KHR_materials_sheen: Sheen? + let KHR_materials_anisotropy: Anisotropy? + let KHR_materials_clearcoat: Clearcoat? + let KHR_materials_ior: IOR? + let KHR_materials_specular: Specular? } let name: String? @@ -154,6 +213,7 @@ extension NativeGLTFLoader { let mesh: Int? let camera: Int? let skin: Int? + let weights: [Float]? } struct Skin: Codable { diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Skins.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Skins.swift index 564e75788..fa3af5017 100644 --- a/Sources/AdaAssets/GLTF/NativeGLTFLoader+Skins.swift +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader+Skins.swift @@ -55,13 +55,20 @@ extension NativeGLTFLoader { restPose = pose transform = pose.matrix } + if let weights = node.weights { + // Node weights override the mesh's; there must be one per target of the node's mesh. + guard weights.count == morphTargetCount(ofNode: node, in: gltf), !weights.isEmpty, weights.allSatisfy(\.isFinite) else { + throw GLTFError.invalidNode(index) + } + } return GLTFImportResult.Node( name: node.name, transform: transform, children: node.children ?? [], meshIndex: node.mesh, skinIndex: node.skin, - restPose: restPose + restPose: restPose, + weights: node.weights ) } } diff --git a/Sources/AdaAssets/GLTF/NativeGLTFLoader.swift b/Sources/AdaAssets/GLTF/NativeGLTFLoader.swift index 05ea7a346..6c269f8f6 100644 --- a/Sources/AdaAssets/GLTF/NativeGLTFLoader.swift +++ b/Sources/AdaAssets/GLTF/NativeGLTFLoader.swift @@ -17,6 +17,34 @@ public struct NativeGLTFLoader: GLTFLoader { } /// Loads a glTF or GLB document from memory. + /// The glTF extensions this loader implements. A file that *requires* any other extension fails to load; an + /// extension it merely *uses* is ignored and reported in ``GLTFImportResult/warnings``. + public static let supportedExtensions: Set = [ + "KHR_mesh_quantization", + "KHR_materials_emissive_strength", + "KHR_texture_transform", + "KHR_materials_ior", + "KHR_materials_specular", + "KHR_materials_clearcoat", + "KHR_materials_sheen", + "KHR_materials_anisotropy", + ] + + /// Optional content the importer cannot honor: extensions the file lists as used that are not implemented. + private func warnings(for gltf: GLTF) -> [String] { + var result: [String] = [] + for name in (gltf.extensionsUsed ?? []).sorted() where !Self.supportedExtensions.contains(name) { + result.append("The extension \(name) is not supported and was ignored.") + } + for (index, mesh) in (gltf.meshes ?? []).enumerated() { + let ignored = Set(mesh.primitives.flatMap { $0.targets ?? [] }.flatMap(\.keys)).subtracting(["POSITION", "NORMAL", "TANGENT"]) + if !ignored.isEmpty { + result.append("Mesh \(index) morphs \(ignored.sorted().joined(separator: ", ")); only POSITION, NORMAL and TANGENT deltas were applied.") + } + } + return result + } + public func load(data: Data, baseURL: URL? = nil) throws -> GLTFImportResult { let resourceBaseURL = baseURL ?? URL(fileURLWithPath: FileManager.default.currentDirectoryPath) @@ -32,8 +60,7 @@ public struct NativeGLTFLoader: GLTFLoader { binaryBuffer = nil } - let supportedExtensions: Set = ["KHR_mesh_quantization", "KHR_materials_emissive_strength"] - if let unsupportedExtension = gltf.extensionsRequired?.first(where: { !supportedExtensions.contains($0) }) { + if let unsupportedExtension = gltf.extensionsRequired?.first(where: { !Self.supportedExtensions.contains($0) }) { throw GLTFError.unsupportedRequiredExtension(unsupportedExtension) } @@ -142,8 +169,8 @@ public struct NativeGLTFLoader: GLTFLoader { } let materials = try importMaterials(gltf) - let meshes = try (gltf.meshes ?? []) - .map { mesh -> GLTFImportResult.Mesh in + let meshes = try (gltf.meshes ?? []).enumerated() + .map { meshIndex, mesh -> GLTFImportResult.Mesh in let primitives = try mesh.primitives.map { primitive -> GLTFImportResult.Primitive in var attributes = [GLTFImportResult.Attribute: GLTFImportResult.Accessor]() @@ -177,11 +204,13 @@ public struct NativeGLTFLoader: GLTFLoader { indices: indices, materialIndex: primitive.material, mode: GLTFImportResult.PrimitiveMode(rawValue: primitive.mode ?? 4) ?? .triangles, - skinning: importSkinning(primitive, attributes: attributes, gltf: gltf) + skinning: importSkinning(primitive, attributes: attributes, gltf: gltf), + morphTargets: importMorphTargets(primitive, attributes: attributes, gltf: gltf, buffers: buffers, meshIndex: meshIndex) ) } - return GLTFImportResult.Mesh(name: mesh.name, primitives: primitives) + let morph = try importMorphMetadata(mesh, primitives: primitives, meshIndex: meshIndex) + return GLTFImportResult.Mesh(name: mesh.name, primitives: primitives, defaultWeights: morph.weights, targetNames: morph.names) } let nodes = try importNodes(gltf) @@ -201,7 +230,8 @@ public struct NativeGLTFLoader: GLTFLoader { defaultScene: gltf.scene, skins: skins, animations: animations, - samplers: samplers + samplers: samplers, + warnings: warnings(for: gltf) ) } diff --git a/Sources/AdaCorePipelines/Assets/Shaders/directional_shadow_3d.glsl b/Sources/AdaCorePipelines/Assets/Shaders/directional_shadow_3d.glsl index 2c47ceaa2..0b50a0106 100644 --- a/Sources/AdaCorePipelines/Assets/Shaders/directional_shadow_3d.glsl +++ b/Sources/AdaCorePipelines/Assets/Shaders/directional_shadow_3d.glsl @@ -1,5 +1,10 @@ #version 450 core #pragma stage : vert +#ifdef ADA_MORPH +#extension GL_EXT_samplerless_texture_functions : require +#endif + +#include layout (location = 13) in vec4 a_JointIndices; layout (location = 14) in vec4 a_JointWeights; @@ -39,13 +44,17 @@ layout (binding = 2) uniform DirectionalShadowViewUniform { [[main]] void directional_shadow_3d_vertex() { + vec3 localPosition = a_Position; +#ifdef ADA_MORPH + applyMorphPosition(localPosition); +#endif mat4 model = mat4(a_Model0, a_Model1, a_Model2, a_Model3) * skinTransform(); v_Fade = a_ShadowFlags.w; v_TileBounds = u_TileBounds; v_UV0 = a_TextureCoordinate; v_UV1 = a_TextureCoordinate1; v_Alpha = a_Color.a * a_VertexColor.a; - gl_Position = u_ShadowViewProjection * model * vec4(a_Position, 1.0); + gl_Position = u_ShadowViewProjection * model * vec4(localPosition, 1.0); } #version 450 core @@ -65,6 +74,9 @@ layout (binding = 15) uniform PBR3DUniform { vec4 u_SurfaceFlags; vec4 u_UVSets; vec4 u_Emission; + vec4 u_Parallax; + vec4 u_TextureTransforms[10]; + vec4 u_Dielectric; }; [[main]] @@ -79,6 +91,7 @@ void directional_shadow_3d_fragment() if (u_SurfaceFlags.x < 0.5 && !gl_FrontFacing) { discard; } if (u_SurfaceProperties.w > 0.5 && u_SurfaceProperties.w < 1.5) { vec2 uv = u_UVSets.x > 0.5 ? v_UV1 : v_UV0; + uv = vec2(dot(u_TextureTransforms[0].xy, uv), dot(u_TextureTransforms[0].zw, uv)) + u_TextureTransforms[1].xy; float alpha = texture(sampler2D(u_BaseColorTexture, u_BaseColorSampler), uv).a * v_Alpha; if (alpha < u_SurfaceProperties.z) { discard; } } diff --git a/Sources/AdaCorePipelines/Assets/Shaders/flat3d.glsl b/Sources/AdaCorePipelines/Assets/Shaders/flat3d.glsl index f2e4db2a5..4faf15f5c 100644 --- a/Sources/AdaCorePipelines/Assets/Shaders/flat3d.glsl +++ b/Sources/AdaCorePipelines/Assets/Shaders/flat3d.glsl @@ -1,5 +1,8 @@ #version 450 core #pragma stage : vert +#ifdef ADA_MORPH +#extension GL_EXT_samplerless_texture_functions : require +#endif #ifdef ADA_METAL_RENDER_LAYER #extension GL_ARB_shader_viewport_layer_array : require @@ -9,6 +12,7 @@ layout (binding = 30) uniform RenderTargetLayer { #endif #include +#include layout (binding = 1) uniform DirectionalLight3DUniform { vec4 u_LightDirectionIntensity; @@ -75,13 +79,20 @@ void flat3d_vertex() { #ifdef ADA_METAL_RENDER_LAYER gl_Layer = int(u_RenderTargetLayer); +#endif + // Morph targets move the rest pose first; skinning then moves the morphed pose. + vec3 localPosition = a_Position; + vec3 localNormal = a_Normal; + vec3 localTangent = a_Tangent.xyz; +#ifdef ADA_MORPH + applyMorph(localPosition, localNormal, localTangent); #endif mat4 model = mat4(a_Model0, a_Model1, a_Model2, a_Model3) * skinTransform(); mat3 basis = mat3(model); mat3 normalMatrix = abs(determinant(basis)) > 0.000001 ? transpose(inverse(basis)) : mat3(1.0); - vec3 normal = normalize(normalMatrix * a_Normal); - vec4 worldPosition = model * vec4(a_Position, 1.0); - vec3 worldTangent = normalize(mat3(model) * a_Tangent.xyz); + vec3 normal = normalize(normalMatrix * localNormal); + vec4 worldPosition = model * vec4(localPosition, 1.0); + vec3 worldTangent = normalize(mat3(model) * localTangent); Output.Color = a_Color * a_VertexColor; Output.WorldPosition = worldPosition.xyz; Output.ViewPosition = (u_ViewMatrix * worldPosition).xyz; @@ -140,7 +151,51 @@ layout (binding = 15) uniform PBR3DUniform { vec4 u_UVSets; vec4 u_Emission; vec4 u_Parallax; + // Per texture slot (base color, metallic/roughness, normal, occlusion, emission): xyzw is a row-major 2x2 + // matrix, then xy is the offset (glTF KHR_texture_transform). + vec4 u_TextureTransforms[10]; + vec4 u_Dielectric; // rgb: reflectance of non-metals at normal incidence (0.04 for the default IOR of 1.5) + vec4 u_Coat; // x clearcoat strength, y clearcoat roughness (KHR_materials_clearcoat) + vec4 u_Sheen; // rgb sheen color, w sheen roughness (KHR_materials_sheen) + vec4 u_Anisotropy; // x strength, y rotation in radians (KHR_materials_anisotropy) + // UV set of each layer texture (clearcoat, clearcoat roughness, clearcoat normal, sheen color, sheen roughness, + // anisotropy, specular, specular color), and for each a row-major 2x2 matrix then an offset. + vec4 u_LayerSets[2]; + vec4 u_LayerTransforms[16]; + vec4 u_LayerParameters; // x clearcoat normal scale }; + +// Layer textures exist only in the shader variants of materials that have them. They are read with the base color +// sampler, because the main pass has no free sampler slot. +#ifdef ADA_CLEARCOAT_TEXTURE +layout (binding = 32) uniform texture2D u_ClearcoatTexture; +#endif +#ifdef ADA_CLEARCOAT_ROUGHNESS_TEXTURE +layout (binding = 33) uniform texture2D u_ClearcoatRoughnessTexture; +#endif +#ifdef ADA_CLEARCOAT_NORMAL_TEXTURE +layout (binding = 34) uniform texture2D u_ClearcoatNormalTexture; +#endif +#ifdef ADA_SHEEN_COLOR_TEXTURE +layout (binding = 35) uniform texture2D u_SheenColorTexture; +#endif +#ifdef ADA_SHEEN_ROUGHNESS_TEXTURE +layout (binding = 36) uniform texture2D u_SheenRoughnessTexture; +#endif +#ifdef ADA_ANISOTROPY_TEXTURE +layout (binding = 37) uniform texture2D u_AnisotropyTexture; +#endif +#ifdef ADA_SPECULAR_TEXTURE +layout (binding = 38) uniform texture2D u_SpecularTexture; +#endif +#ifdef ADA_SPECULAR_COLOR_TEXTURE +layout (binding = 39) uniform texture2D u_SpecularColorTexture; +#endif + +vec2 applyTextureTransform(vec2 uv, int slot) { + vec4 matrix = u_TextureTransforms[slot * 2]; + return vec2(dot(matrix.xy, uv), dot(matrix.zw, uv)) + u_TextureTransforms[slot * 2 + 1].xy; +} layout (binding = 16) uniform texture2D u_OcclusionTexture; layout (binding = 17) uniform texture2D u_HeightTexture; layout (binding = 0) uniform sampler u_OcclusionSampler; @@ -190,6 +245,27 @@ layout (binding = 28) uniform LocalShadow3DUniform { vec4 u_LocalAtlas; }; layout (binding = 26) uniform texture2D u_LocalShadowTexture; +layout (binding = 27) uniform LightTiles3DUniform { + mat4 u_TileProjection; + vec4 u_TileViewport; // xy pixel origin, zw pixel size of the viewport the projection maps to + vec4 u_TileInfo; // x tile size in pixels, y columns, z rows, w 1 when tile masks are valid +}; +layout (binding = 29) uniform LightTileMasks3DUniform { + uvec4 u_TileMasks[1024]; +}; + +// Bit mask of the local lights that may reach this fragment. The tile comes from projecting the view-space +// position the same way the CPU bounded each light, so it holds for any backend's fragment coordinate origin. +uint localLightMask() { + uint reachable = int(u_LightCounts.y) >= 32 ? 0xFFFFFFFFu : ((1u << uint(max(int(u_LightCounts.y), 0))) - 1u); + if (u_TileInfo.w < 0.5) { return reachable; } + vec4 clip = u_TileProjection * vec4(Input.ViewPosition, 1.0); + if (clip.w <= 0.0001) { return reachable; } + vec2 pixel = u_TileViewport.xy + vec2(clip.x / clip.w * 0.5 + 0.5, 0.5 - clip.y / clip.w * 0.5) * u_TileViewport.zw; + ivec2 tile = clamp(ivec2(floor(pixel / u_TileInfo.x)), ivec2(0), ivec2(int(u_TileInfo.y) - 1, int(u_TileInfo.z) - 1)); + int index = tile.y * int(u_TileInfo.y) + tile.x; + return u_TileMasks[index >> 2][index & 3] & reachable; +} const float PI = 3.14159265359; @@ -220,12 +296,77 @@ vec3 fresnelSchlick(float cosine, vec3 reflectanceAtNormal) { return reflectanceAtNormal + (vec3(1.0) - reflectanceAtNormal) * pow(clamp(1.0 - cosine, 0.0, 1.0), 5.0); } +// Clearcoat layer: x strength, y roughness. It sits on the geometric normal, not on the normal map. Set once per fragment. +vec2 g_Clearcoat = vec2(0.0); +vec3 g_CoatNormal = vec3(0.0, 0.0, 1.0); + +// The optional layers are compiled in only for materials that use them (ADA_CLEARCOAT, ADA_SHEEN, ADA_ANISOTROPY), which +// keeps the common path as small as it was before the layers existed. +bool hasClearcoat() { +#ifdef ADA_CLEARCOAT + return g_Clearcoat.x > 0.0; +#else + return false; +#endif +} + +// Share of the base layer's light that survives the coat at this view angle. +float coatTransmission(float normalView) { + return 1.0 - g_Clearcoat.x * fresnelSchlick(normalView, vec3(0.04)).x; +} + +// Reflectance of non-metals at normal incidence for this fragment: the material value times its specular textures. +vec3 g_Dielectric = vec3(0.04); + +// Sheen layer (cloth): rgb color, w roughness. A black color turns it off. +vec4 g_Sheen = vec4(0.0); +// Average share of light the sheen lobe takes from the base layer. The glTF sheen albedo lookup is replaced by this constant. +const float SHEEN_ALBEDO = 0.4; + +bool hasSheen() { +#ifdef ADA_SHEEN + return max(g_Sheen.r, max(g_Sheen.g, g_Sheen.b)) > 0.0; +#else + return false; +#endif +} + +// Share of the base layer's light that survives the sheen. +float sheenTransmission() { + return 1.0 - max(g_Sheen.r, max(g_Sheen.g, g_Sheen.b)) * SHEEN_ALBEDO; +} + +// Anisotropy: strength, and the view-space tangent and bitangent of the stretched highlight. +float g_Anisotropy = 0.0; +vec3 g_AnisotropyT = vec3(1.0, 0.0, 0.0); +vec3 g_AnisotropyB = vec3(0.0, 1.0, 0.0); + +bool hasAnisotropy() { +#ifdef ADA_ANISOTROPY + return g_Anisotropy > 0.0; +#else + return false; +#endif +} + // UV shift from parallax occlusion mapping, applied to every material texture that uses the height map's UV set. vec2 g_ParallaxDelta = vec2(0.0); -vec2 materialUV(float setIndex) { +vec2 materialUV(float setIndex, int slot) { vec2 uv = setIndex > 0.5 ? Input.TextureCoordinate1 : Input.TextureCoordinate; - return abs(setIndex - u_Parallax.y) < 0.5 ? uv + g_ParallaxDelta : uv; + // The parallax shift lives in mesh UV space, so it applies before the texture's own transform. + if (abs(setIndex - u_Parallax.y) < 0.5) { uv += g_ParallaxDelta; } + + return applyTextureTransform(uv, slot); +} + +// UV of a layer texture (slot 0 to 7 in PBRLayerTexture order): its UV set, with the parallax shift, then its own transform. +vec2 layerUV(int slot) { + float setIndex = u_LayerSets[slot >> 2][slot & 3]; + vec2 uv = setIndex > 0.5 ? Input.TextureCoordinate1 : Input.TextureCoordinate; + if (abs(setIndex - u_Parallax.y) < 0.5) { uv += g_ParallaxDelta; } + vec4 matrix = u_LayerTransforms[slot * 2]; + return vec2(dot(matrix.xy, uv), dot(matrix.zw, uv)) + u_LayerTransforms[slot * 2 + 1].xy; } // Ray-marches the height map along the view ray. Returns the UV shift that makes the surface appear to have depth. @@ -295,15 +436,37 @@ vec3 environmentLighting(vec3 normal, vec3 viewDirection, vec3 baseColor, float vec3 worldNormal = normalize(mat3(u_EnvironmentInverseView) * normal); vec3 worldView = normalize(mat3(u_EnvironmentInverseView) * viewDirection); float ndotv = max(dot(normal, viewDirection), 0.0); - vec3 f0 = mix(vec3(0.04), baseColor, metallic); + vec3 f0 = mix(g_Dielectric, baseColor, metallic); vec3 fresnel = f0 + (max(vec3(1.0 - roughness), f0) - f0) * pow(1.0 - ndotv, 5.0); vec3 diffuseWeight = (vec3(1.0) - fresnel) * (1.0 - metallic); vec3 irradiance = texture(sampler2D(u_Irradiance, u_IBLSampler), environmentUV(worldNormal)).rgb; - vec3 reflected = samplePrefiltered(reflect(-worldView, worldNormal), roughness); + vec3 reflectionNormal = worldNormal; + if (hasAnisotropy()) { + // Bend the reflection toward the stretched direction so the environment smears along it (Filament). + vec3 anisotropicTangent = cross(g_AnisotropyB, viewDirection); + vec3 anisotropicNormal = cross(anisotropicTangent, g_AnisotropyB); + float bend = g_Anisotropy * clamp(5.0 * roughness, 0.0, 1.0); + reflectionNormal = normalize(mat3(u_EnvironmentInverseView) * normalize(mix(normal, anisotropicNormal, bend))); + } + vec3 reflected = samplePrefiltered(reflect(-worldView, reflectionNormal), roughness); vec2 lutSize = vec2(textureSize(sampler2D(u_BRDF, u_IBLSampler), 0)); vec2 lutUV = clamp(vec2(ndotv, roughness), 0.5 / lutSize, vec2(1.0) - 0.5 / lutSize); vec2 lut = texture(sampler2D(u_BRDF, u_IBLSampler), lutUV).rg; - return (diffuseWeight * irradiance * baseColor / PI + reflected * (f0 * lut.x + lut.y)) * u_IBLParameters.y; + vec3 result = (diffuseWeight * irradiance * baseColor / PI + reflected * (f0 * lut.x + lut.y)) * u_IBLParameters.y; + if (hasSheen()) { + // Sheen under image-based lighting is approximated as a diffuse lobe with the average sheen albedo. + result = result * sheenTransmission() + g_Sheen.rgb * SHEEN_ALBEDO * irradiance / PI * u_IBLParameters.y; + } + if (hasClearcoat()) { + // The coat reflects the environment along the geometric normal and dims the base layer beneath it. + vec3 coatWorldNormal = normalize(mat3(u_EnvironmentInverseView) * g_CoatNormal); + float coatNormalView = max(dot(g_CoatNormal, viewDirection), 0.0); + vec2 coatLutUV = clamp(vec2(coatNormalView, g_Clearcoat.y), 0.5 / lutSize, vec2(1.0) - 0.5 / lutSize); + vec2 coatLut = texture(sampler2D(u_BRDF, u_IBLSampler), coatLutUV).rg; + vec3 coatReflected = samplePrefiltered(reflect(-worldView, coatWorldNormal), g_Clearcoat.y); + result = result * coatTransmission(coatNormalView) + coatReflected * (0.04 * coatLut.x + coatLut.y) * g_Clearcoat.x * u_IBLParameters.y; + } + return result; } mat3 cotangentFrame(vec3 normal, vec3 position, vec2 uv) { @@ -325,15 +488,18 @@ vec3 materialNormal() { return normal; } - vec3 tangentNormal = texture(sampler2D(u_NormalTexture, u_NormalSampler), materialUV(u_UVSets.z)).xyz * 2.0 - 1.0; + vec3 tangentNormal = texture(sampler2D(u_NormalTexture, u_NormalSampler), materialUV(u_UVSets.z, 2)).xyz * 2.0 - 1.0; tangentNormal.xy *= u_SurfaceProperties.x; - if (Input.TextureFlags.w > 0.5 && u_UVSets.z < 0.5) { + // Authored tangents follow the mesh UVs, so they only hold while the normal map's transform keeps its axes. + vec4 normalMatrix = u_TextureTransforms[4]; + bool keepsTangents = abs(normalMatrix.y) < 0.0001 && abs(normalMatrix.z) < 0.0001 && normalMatrix.x > 0.0 && normalMatrix.w > 0.0; + if (Input.TextureFlags.w > 0.5 && u_UVSets.z < 0.5 && keepsTangents) { vec3 tangent = normalize(Input.ViewTangent.xyz - normal * dot(normal, Input.ViewTangent.xyz)); vec3 bitangent = normalize(cross(normal, tangent)) * Input.ViewTangent.w; return normalize(mat3(tangent, bitangent, normal) * tangentNormal); } - return normalize(cotangentFrame(normal, Input.ViewPosition, materialUV(u_UVSets.z)) * tangentNormal); + return normalize(cotangentFrame(normal, Input.ViewPosition, materialUV(u_UVSets.z, 2)) * tangentNormal); } float cascadeVisibility(int cascade, vec3 normal, vec3 lightDirection) { @@ -380,16 +546,66 @@ float directionalShadow(vec3 normal, vec3 lightDirection) { return visibility; } -vec3 evaluateDirect(vec3 normal, vec3 viewDirection, vec3 lightDirection, vec3 baseColor, float metallic, float roughness, vec3 radiance) { +// The coat's own highlight for one light (rgb, already scaled by radiance and n.l) and the transmission of the +// base layer under it (a). +vec4 clearcoatDirect(vec3 viewDirection, vec3 lightDirection, vec3 radiance) { + vec3 normal = g_CoatNormal; float nl = max(dot(normal, lightDirection), 0.0), nv = max(dot(normal, viewDirection), 0.0); - if (nl <= 0.0 || nv <= 0.0) { return vec3(0.0); } + float transmission = coatTransmission(nv); + if (nl <= 0.0 || nv <= 0.0) { return vec4(0.0, 0.0, 0.0, transmission); } vec3 halfway = normalize(viewDirection + lightDirection); - vec3 f0 = mix(vec3(0.04), baseColor, metallic); - vec3 fresnel = fresnelSchlick(max(dot(halfway, viewDirection), 0.0), f0); - float distribution = distributionGGX(normal, halfway, roughness); - float geometry = geometrySmith(normal, viewDirection, lightDirection, roughness); - vec3 specular = distribution * geometry * fresnel / max(4.0 * nv * nl, 0.0001); - return ((vec3(1.0) - fresnel) * (1.0 - metallic) * baseColor / PI + specular) * max(radiance, vec3(0.0)) * nl; + float fresnel = fresnelSchlick(max(dot(halfway, viewDirection), 0.0), vec3(0.04)).x * g_Clearcoat.x; + float distribution = distributionGGX(normal, halfway, g_Clearcoat.y); + float geometry = geometrySmith(normal, viewDirection, lightDirection, g_Clearcoat.y); + float specular = distribution * geometry * fresnel / max(4.0 * nv * nl, 0.0001); + return vec4(vec3(specular) * max(radiance, vec3(0.0)) * nl, transmission); +} + +// Charlie sheen distribution (Estevez and Kulla) and the Neubelt visibility term, as in the glTF sheen specification. +float sheenDistribution(float roughness, float normalHalfway) { + float alpha = roughness * roughness; + float inverseAlpha = 1.0 / alpha; + float sine2 = max(1.0 - normalHalfway * normalHalfway, 0.0078125); + return (2.0 + inverseAlpha) * pow(sine2, inverseAlpha * 0.5) / (2.0 * PI); +} + +float sheenVisibility(float normalLight, float normalView) { + return clamp(1.0 / (4.0 * (normalLight + normalView - normalLight * normalView)), 0.0, 1.0); +} + +vec3 evaluateDirect(vec3 normal, vec3 viewDirection, vec3 lightDirection, vec3 baseColor, float metallic, float roughness, vec3 radiance) { + float nl = max(dot(normal, lightDirection), 0.0), nv = max(dot(normal, viewDirection), 0.0); + vec3 base = vec3(0.0); + if (nl > 0.0 && nv > 0.0) { + vec3 halfway = normalize(viewDirection + lightDirection); + vec3 f0 = mix(g_Dielectric, baseColor, metallic); + vec3 fresnel = fresnelSchlick(max(dot(halfway, viewDirection), 0.0), f0); + vec3 specular; + if (hasAnisotropy()) { + // Anisotropic GGX: roughness stretches along the tangent, so the highlight streaks along the bitangent. + float normalHalfway = max(dot(normal, halfway), 0.0); + float alpha = roughness * roughness; + float alphaT = mix(alpha, 1.0, g_Anisotropy * g_Anisotropy), alphaB = alpha; + float tangentHalfway = dot(g_AnisotropyT, halfway), bitangentHalfway = dot(g_AnisotropyB, halfway); + float denominator = tangentHalfway * tangentHalfway / (alphaT * alphaT) + bitangentHalfway * bitangentHalfway / (alphaB * alphaB) + normalHalfway * normalHalfway; + float distribution = 1.0 / max(PI * alphaT * alphaB * denominator * denominator, 0.000001); + float lambdaView = nl * length(vec3(alphaT * dot(g_AnisotropyT, viewDirection), alphaB * dot(g_AnisotropyB, viewDirection), nv)); + float lambdaLight = nv * length(vec3(alphaT * dot(g_AnisotropyT, lightDirection), alphaB * dot(g_AnisotropyB, lightDirection), nl)); + specular = fresnel * distribution * 0.5 / max(lambdaView + lambdaLight, 0.00001); + } else { + float distribution = distributionGGX(normal, halfway, roughness); + float geometry = geometrySmith(normal, viewDirection, lightDirection, roughness); + specular = distribution * geometry * fresnel / max(4.0 * nv * nl, 0.0001); + } + base = ((vec3(1.0) - fresnel) * (1.0 - metallic) * baseColor / PI + specular) * max(radiance, vec3(0.0)) * nl; + if (hasSheen()) { + vec3 sheen = g_Sheen.rgb * sheenDistribution(g_Sheen.w, max(dot(normal, halfway), 0.0)) * sheenVisibility(nl, nv); + base = base * sheenTransmission() + sheen * max(radiance, vec3(0.0)) * nl; + } + } + if (!hasClearcoat()) { return base; } + vec4 coat = clearcoatDirect(viewDirection, lightDirection, radiance); + return base * coat.a + coat.rgb; } int localPointFace(vec3 ray) { @@ -429,8 +645,11 @@ vec3 additionalLighting(vec3 normal, vec3 viewDirection, vec3 baseColor, float m result += evaluateDirect(normal, viewDirection, normalize(u_AdditionalDirections[i].xyz), baseColor, metallic, roughness, u_AdditionalRadiance[i].rgb * max(u_AdditionalDirections[i].w, 0.0)); } - for (int i = 0; i < 32; ++i) { - if (i >= int(u_LightCounts.y)) { break; } + uint mask = localLightMask(); + for (int n = 0; n < 32; ++n) { + if (mask == 0u) { break; } + int i = findLSB(mask); + mask &= mask - 1u; LocalLight3D light = u_LocalLights[i]; vec3 delta = light.positionRange.xyz - Input.ViewPosition; float distance = length(delta), range = light.positionRange.w; @@ -503,7 +722,7 @@ void flat3d_fragment() if (u_SurfaceFlags.x < 0.5 && !gl_FrontFacing) { discard; } vec4 baseColor = Input.Color; if (Input.TextureFlags.x > 0.5) { - vec4 sampledBaseColor = texture(sampler2D(u_BaseColorTexture, u_BaseColorSampler), materialUV(u_UVSets.x)); + vec4 sampledBaseColor = texture(sampler2D(u_BaseColorTexture, u_BaseColorSampler), materialUV(u_UVSets.x, 0)); baseColor *= vec4(srgbToLinear(sampledBaseColor.rgb), sampledBaseColor.a); } @@ -514,7 +733,7 @@ void flat3d_fragment() if (Input.TextureFlags.y > 0.5) { vec4 metallicRoughness = texture( sampler2D(u_MetallicRoughnessTexture, u_MetallicRoughnessSampler), - materialUV(u_UVSets.y) + materialUV(u_UVSets.y, 1) ); roughness = clamp(roughness * metallicRoughness.g, 0.04, 1.0); metallic = clamp(metallic * metallicRoughness.b, 0.0, 1.0); @@ -522,6 +741,59 @@ void flat3d_fragment() vec3 normal = materialNormal(); if (!gl_FrontFacing && u_SurfaceFlags.x > 0.5) { normal = -normal; } + // Shading layers: each factor is multiplied by its texture when the material has one. + g_Dielectric = u_Dielectric.rgb; +#ifdef ADA_SPECULAR_TEXTURE + g_Dielectric *= texture(sampler2D(u_SpecularTexture, u_BaseColorSampler), layerUV(6)).a; +#endif +#ifdef ADA_SPECULAR_COLOR_TEXTURE + g_Dielectric *= srgbToLinear(texture(sampler2D(u_SpecularColorTexture, u_BaseColorSampler), layerUV(7)).rgb); +#endif + + g_Sheen = vec4(max(u_Sheen.rgb, vec3(0.0)), clamp(u_Sheen.w, 0.07, 1.0)); +#ifdef ADA_SHEEN_COLOR_TEXTURE + g_Sheen.rgb *= srgbToLinear(texture(sampler2D(u_SheenColorTexture, u_BaseColorSampler), layerUV(3)).rgb); +#endif +#ifdef ADA_SHEEN_ROUGHNESS_TEXTURE + g_Sheen.w = clamp(u_Sheen.w * texture(sampler2D(u_SheenRoughnessTexture, u_BaseColorSampler), layerUV(4)).a, 0.07, 1.0); +#endif + + g_Anisotropy = clamp(u_Anisotropy.x, 0.0, 1.0); + vec2 anisotropyDirection = vec2(1.0, 0.0); +#ifdef ADA_ANISOTROPY_TEXTURE + // Red and green hold the direction in tangent space, blue scales the strength. + vec3 anisotropySample = texture(sampler2D(u_AnisotropyTexture, u_BaseColorSampler), layerUV(5)).rgb; + g_Anisotropy = clamp(g_Anisotropy * anisotropySample.b, 0.0, 1.0); + vec2 sampledDirection = anisotropySample.rg * 2.0 - 1.0; + if (dot(sampledDirection, sampledDirection) > 0.0001) { anisotropyDirection = normalize(sampledDirection); } +#endif + if (hasAnisotropy()) { + bool authored = Input.TextureFlags.w > 0.5; + vec3 tangent = authored ? Input.ViewTangent.xyz : cotangentFrame(normalize(Input.ViewNormal), Input.ViewPosition, materialUV(u_UVSets.x, 0))[0]; + tangent = normalize(tangent - normal * dot(normal, tangent)); + vec3 bitangent = normalize(cross(normal, tangent)) * (authored ? Input.ViewTangent.w : 1.0); + // The rotation turns the direction, whether it comes from the texture or is the tangent itself. + float rotationCosine = cos(u_Anisotropy.y), rotationSine = sin(u_Anisotropy.y); + vec2 turned = vec2(rotationCosine * anisotropyDirection.x - rotationSine * anisotropyDirection.y, + rotationSine * anisotropyDirection.x + rotationCosine * anisotropyDirection.y); + g_AnisotropyT = normalize(turned.x * tangent + turned.y * bitangent); + g_AnisotropyB = normalize(cross(normal, g_AnisotropyT)); + } + + g_Clearcoat = vec2(u_Coat.x, clamp(u_Coat.y, 0.04, 1.0)); + g_CoatNormal = normalize(Input.ViewNormal); + if (!gl_FrontFacing && u_SurfaceFlags.x > 0.5) { g_CoatNormal = -g_CoatNormal; } +#ifdef ADA_CLEARCOAT_TEXTURE + g_Clearcoat.x *= texture(sampler2D(u_ClearcoatTexture, u_BaseColorSampler), layerUV(0)).r; +#endif +#ifdef ADA_CLEARCOAT_ROUGHNESS_TEXTURE + g_Clearcoat.y = clamp(u_Coat.y * texture(sampler2D(u_ClearcoatRoughnessTexture, u_BaseColorSampler), layerUV(1)).g, 0.04, 1.0); +#endif +#ifdef ADA_CLEARCOAT_NORMAL_TEXTURE + vec3 coatTangentNormal = texture(sampler2D(u_ClearcoatNormalTexture, u_BaseColorSampler), layerUV(2)).xyz * 2.0 - 1.0; + coatTangentNormal.xy *= u_LayerParameters.x; + g_CoatNormal = normalize(cotangentFrame(g_CoatNormal, Input.ViewPosition, layerUV(2)) * coatTangentNormal); +#endif vec3 viewDirection = normalize(-Input.ViewPosition); vec3 lightDirection = normalize(u_LightDirectionIntensity.xyz); gl_FragDepth = gl_FragCoord.z; @@ -559,20 +831,16 @@ void flat3d_fragment() float normalLight = max(dot(normal, lightDirection), 0.0); float normalView = max(dot(normal, viewDirection), 0.0); - vec3 reflectanceAtNormal = mix(vec3(0.04), baseColor.rgb, metallic); - vec3 fresnel = fresnelSchlick(max(dot(halfway, viewDirection), 0.0), reflectanceAtNormal); - float distribution = distributionGGX(normal, halfway, roughness); - float geometry = geometrySmith(normal, viewDirection, lightDirection, roughness); - vec3 specular = distribution * geometry * fresnel / max(4.0 * normalView * normalLight, 0.0001); - vec3 diffuseWeight = (vec3(1.0) - fresnel) * (1.0 - metallic); vec3 radiance = max(u_LightRadianceAmbient.rgb, vec3(0.0)) * max(u_LightDirectionIntensity.w, 0.0); - // The sun term is multiplied by n.l, so surfaces facing away from it skip the cascade lookups. - float shadowVisibility = normalLight > 0.0 ? directionalShadow(normal, lightDirection) : 1.0; - vec3 directLighting = (diffuseWeight * baseColor.rgb / PI + specular) * radiance * normalLight * shadowVisibility; + // The sun term is multiplied by n.l, so surfaces facing away from it skip the cascade lookups. The clearcoat + // follows the geometric normal and can still be lit there. + float shadowVisibility = (normalLight > 0.0 || hasClearcoat()) ? directionalShadow(normal, lightDirection) : 1.0; + vec3 directLighting = evaluateDirect(normal, viewDirection, lightDirection, baseColor.rgb, metallic, roughness, radiance) * shadowVisibility; directLighting += additionalLighting(normal, viewDirection, baseColor.rgb, metallic, roughness); vec3 ambient = baseColor.rgb * (1.0 - metallic) * max(u_LightRadianceAmbient.w, 0.0); if (u_IBLParameters.x > 0.5) { ambient = environmentLighting(normal, viewDirection, baseColor.rgb, metallic, roughness); } - float occlusion = u_SurfaceFlags.y > 0.5 ? texture(sampler2D(u_OcclusionTexture, u_OcclusionSampler), materialUV(u_UVSets.w)).r : 1.0; + else if (hasClearcoat()) { ambient *= coatTransmission(max(dot(g_CoatNormal, viewDirection), 0.0)); } + float occlusion = u_SurfaceFlags.y > 0.5 ? texture(sampler2D(u_OcclusionTexture, u_OcclusionSampler), materialUV(u_UVSets.w, 3)).r : 1.0; ambient *= mix(1.0, occlusion, u_SurfaceProperties.y); float emissiveVisibility = 1.0; if (Input.EmissiveLightThreshold >= 0.0) { @@ -584,7 +852,7 @@ void flat3d_fragment() } vec3 emissive = srgbToLinear(texture( sampler2D(u_EmissiveTexture, u_EmissiveSampler), - materialUV(u_Emission.x) + materialUV(u_Emission.x, 4) ).rgb) * u_EmissiveFactor.rgb * Input.EmissiveStrength * emissiveVisibility; color = vec4(directLighting + ambient + emissive, opacity); diff --git a/Sources/AdaCorePipelines/Assets/Shaders/motion_3d.glsl b/Sources/AdaCorePipelines/Assets/Shaders/motion_3d.glsl index 997c71701..1460a3341 100644 --- a/Sources/AdaCorePipelines/Assets/Shaders/motion_3d.glsl +++ b/Sources/AdaCorePipelines/Assets/Shaders/motion_3d.glsl @@ -1,6 +1,10 @@ #version 450 core #pragma stage : vert +#ifdef ADA_MORPH +#extension GL_EXT_samplerless_texture_functions : require +#endif #include +#include layout (binding = 14) uniform Skinning3DUniform { mat4 u_Joints[128]; }; layout (binding = 24) uniform PreviousSkinning3DUniform { mat4 u_PreviousJoints[128]; }; layout (binding = 23) uniform Temporal3DUniform { @@ -42,8 +46,14 @@ void motion_vertex() { previousSkin = u_PreviousJoints[int(a_Joints.x)] * a_Weights.x + u_PreviousJoints[int(a_Joints.y)] * a_Weights.y + u_PreviousJoints[int(a_Joints.z)] * a_Weights.z + u_PreviousJoints[int(a_Joints.w)] * a_Weights.w; } - vec4 world = mat4(a_Model0, a_Model1, a_Model2, a_Model3) * skin * vec4(a_Position, 1.0); - vec4 previousWorld = mat4(a_PreviousModel0, a_PreviousModel1, a_PreviousModel2, a_PreviousModel3) * previousSkin * vec4(a_Position, 1.0); + vec3 localPosition = a_Position; + vec3 previousLocalPosition = a_Position; +#ifdef ADA_MORPH + applyMorphPosition(localPosition); + previousLocalPosition = previousMorphedPosition(a_Position); +#endif + vec4 world = mat4(a_Model0, a_Model1, a_Model2, a_Model3) * skin * vec4(localPosition, 1.0); + vec4 previousWorld = mat4(a_PreviousModel0, a_PreviousModel1, a_PreviousModel2, a_PreviousModel3) * previousSkin * vec4(previousLocalPosition, 1.0); v_CurrentClip = u_CurrentViewProjection * world; v_PreviousClip = u_Parameters.z > 0.5 ? v_CurrentClip : u_PreviousViewProjection * previousWorld; gl_Position = u_ViewProjection * world; @@ -65,6 +75,9 @@ layout (binding = 4) uniform texture2D u_BaseColor; layout (binding = 7) uniform sampler u_Sampler; layout (binding = 15) uniform PBR3DUniform { vec4 u_EmissiveFactor; vec4 u_SurfaceProperties; vec4 u_SurfaceFlags; vec4 u_UVSets; vec4 u_Emission; + vec4 u_Parallax; + vec4 u_TextureTransforms[10]; + vec4 u_Dielectric; }; [[main]] void motion_fragment() { @@ -75,7 +88,11 @@ void motion_fragment() { if ((values[y * 4 + x] + 0.5) / 16.0 < v_Fade) { discard; } } float alpha = v_ColorFlags.x; - if (v_ColorFlags.y > 0.5) { alpha *= texture(sampler2D(u_BaseColor, u_Sampler), u_UVSets.x > 0.5 ? v_UV1 : v_UV).a; } + if (v_ColorFlags.y > 0.5) { + vec2 uv = u_UVSets.x > 0.5 ? v_UV1 : v_UV; + uv = vec2(dot(u_TextureTransforms[0].xy, uv), dot(u_TextureTransforms[0].zw, uv)) + u_TextureTransforms[1].xy; + alpha *= texture(sampler2D(u_BaseColor, u_Sampler), uv).a; + } if (u_SurfaceProperties.w > 0.5 && u_SurfaceProperties.w < 1.5 && alpha < u_SurfaceProperties.z) { discard; } bool valid = v_CurrentClip.w > 0.00001 && v_PreviousClip.w > 0.00001; vec2 motion = valid ? (v_PreviousClip.xy / v_PreviousClip.w - v_CurrentClip.xy / v_CurrentClip.w) * vec2(0.5, -0.5) : vec2(0.0); diff --git a/Sources/AdaCorePipelines/Assets/Shaders/screen_space_reflection.glsl b/Sources/AdaCorePipelines/Assets/Shaders/screen_space_reflection.glsl index b7f1e0b9b..a70a2dbfd 100644 --- a/Sources/AdaCorePipelines/Assets/Shaders/screen_space_reflection.glsl +++ b/Sources/AdaCorePipelines/Assets/Shaders/screen_space_reflection.glsl @@ -31,6 +31,7 @@ layout (binding = 12) uniform texture2D u_Bloom; layout (binding = 13) uniform texture2D u_GradingLUT; layout (binding = 14) uniform sampler u_GradingSampler; layout (binding = 15) uniform texture2D u_ExposureState; +layout (binding = 16) uniform texture2D u_Volumetric; layout (binding = 4) uniform Environment3DUniform { mat4 u_Projection; @@ -49,6 +50,11 @@ layout (binding = 4) uniform Environment3DUniform { vec4 u_PostProcess; // x exposure scale, y bloom intensity, z tone mapper (see Tonemapping.glsl), w LUT intensity vec4 u_Grading; // x saturation, y contrast, z vignette intensity, w vignette radius vec4 u_Vignette; // x vignette smoothness, y 1 when automatic exposure is bound + vec4 u_FogColor; // rgb sRGB fog color, w density (0 = fog off) + vec4 u_FogShape; // x height falloff, y base height, z start distance, w maximum opacity + vec4 u_FogRange; // x sky influence, y sun exponent, z 1 when volumetric light shafts are bound + vec4 u_FogSun; // xyz world direction to the sun, w sun in-scattering strength + vec4 u_FogSunColor; // rgb sun radiance times intensity }; const float PI = 3.14159265359; @@ -75,6 +81,60 @@ vec3 presentColor(vec3 linearColor) { return adaApplyLut(adaOutputTransform(exposed, u_PostProcess.z), u_PostProcess.w); } +// View-space depth the half-resolution volumetric pass saw at `uv`; sky counts as very far. +float volumetricDepthAt(vec2 uv) { + vec3 position = texture(sampler2D(u_ViewPositionMetallic, u_GBufferSampler), uv).xyz; + return length(position) > 0.0001 ? position.z : 10000.0; +} + +// Shadowed sun light marched through the fog by the half-resolution volumetric pass. The four surrounding +// half-resolution texels are weighted by how close their depth is to this pixel's, so shafts do not bleed +// across silhouettes. +vec3 volumetricScatter(float depth) { + if (u_FogRange.z < 0.5) { + return vec3(0.0); + } + vec2 size = vec2(textureSize(sampler2D(u_Volumetric, u_LinearSampler), 0)); + vec2 base = floor(v_UV * size - 0.5); + vec2 fraction = fract(v_UV * size - 0.5); + vec3 sum = vec3(0.0); + vec3 plain = vec3(0.0); + float weights = 0.0; + for (int y = 0; y < 2; ++y) { + for (int x = 0; x < 2; ++x) { + vec2 uv = clamp((base + vec2(float(x), float(y)) + 0.5) / size, 0.5 / size, vec2(1.0) - 0.5 / size); + float bilinear = (x == 0 ? 1.0 - fraction.x : fraction.x) * (y == 0 ? 1.0 - fraction.y : fraction.y); + vec3 value = texture(sampler2D(u_Volumetric, u_GBufferSampler), uv).rgb; + float weight = bilinear * exp(-abs(volumetricDepthAt(uv) - depth) / max(0.1 * depth, 0.05)); + sum += value * weight; + plain += value * bilinear; + weights += weight; + } + } + return weights > 0.001 ? sum / weights : plain; +} + +// Blends exponential height fog into linear HDR color. `viewPosition` is the view-space position of the surface +// (or a far point along the ray for the sky, flagged by `isSky`) and `skyAmount` scales fog for sky pixels. +vec3 applyFog(vec3 linearColor, vec3 viewPosition, float skyAmount, bool isSky) { + if (u_FogColor.w <= 0.0) { + return linearColor; + } + float travel = max(length(viewPosition) - u_FogShape.z, 0.0); + vec3 worldDirection = normalize((u_InverseView * vec4(viewPosition, 0.0)).xyz); + float falloff = u_FogShape.x; + float cameraHeight = u_InverseView[3].y; + float rise = falloff * worldDirection.y; + // Integral of exp(-falloff * (height - base)) along the ray, with the series limit for a flat ray. + float exponent = clamp(rise * travel, -60.0, 60.0); + float pathFactor = abs(rise * travel) > 0.0001 ? (1.0 - exp(-exponent)) / rise : travel; + float baseDensity = u_FogColor.w * exp(clamp(-falloff * (cameraHeight - u_FogShape.y), -60.0, 60.0)); + float opacity = min(1.0 - exp(-baseDensity * pathFactor), u_FogShape.w) * skyAmount; + float sunAlignment = pow(max(dot(worldDirection, u_FogSun.xyz), 0.0), u_FogRange.y); + vec3 scattered = srgbToLinear(u_FogColor.rgb) + u_FogSunColor.rgb * (u_FogSun.w * sunAlignment); + return mix(linearColor, scattered, opacity) + volumetricScatter(isSky ? 10000.0 : viewPosition.z); +} + vec3 proceduralSky(vec3 direction) { float height = clamp(direction.y, -1.0, 1.0); float horizonBlend = pow(abs(height), 0.55); @@ -241,7 +301,7 @@ void ssr_fragment() { vec2 ndc = vec2(v_UV.x * 2.0 - 1.0, 1.0 - v_UV.y * 2.0); vec4 viewFar = u_InverseProjection * vec4(ndc, 1.0, 1.0); vec3 viewRay = normalize(viewFar.xyz / viewFar.w); - o_Color = vec4(presentColor(sampleSky(viewRay)), 1.0); + o_Color = vec4(presentColor(applyFog(sampleSky(viewRay), viewRay * 1000.0, u_FogRange.x, true)), 1.0); return; } @@ -250,7 +310,7 @@ void ssr_fragment() { vec3 indirect = texture(sampler2D(u_Indirect, u_LinearSampler), v_UV).rgb; baseColor = max(baseColor - indirect * (1.0 - ao), vec3(0.0)); // The PBR pass already integrated diffuse and specular IBL, including AO and material F0. - if (u_IBL.x > 0.5) { o_Color = vec4(presentColor(baseColor), 1.0); return; } + if (u_IBL.x > 0.5) { o_Color = vec4(presentColor(applyFog(baseColor, positionMetallic.xyz, 1.0, false)), 1.0); return; } vec3 normal = normalize(normalRoughness.xyz); float roughness = clamp(normalRoughness.w, 0.04, 1.0); float metallic = clamp(positionMetallic.w, 0.0, 1.0); @@ -275,5 +335,5 @@ void ssr_fragment() { } vec3 ambient = sampleSky(normal) * (0.035 + 0.035 * (1.0 - roughness)); vec3 reflectionResult = mix(environment, reflectedColor, visibility); - o_Color = vec4(presentColor(baseColor + (ambient + reflectionResult * reflectivity) * ao), 1.0); + o_Color = vec4(presentColor(applyFog(baseColor + (ambient + reflectionResult * reflectivity) * ao, positionMetallic.xyz, 1.0, false)), 1.0); } diff --git a/Sources/AdaCorePipelines/Assets/Shaders/shadow_clear_3d.glsl b/Sources/AdaCorePipelines/Assets/Shaders/shadow_clear_3d.glsl new file mode 100644 index 000000000..72537b1f7 --- /dev/null +++ b/Sources/AdaCorePipelines/Assets/Shaders/shadow_clear_3d.glsl @@ -0,0 +1,15 @@ +#version 450 core +#pragma stage : vert +[[main]] +void shadow_clear_vertex() { + vec2 uv = vec2(float(gl_VertexIndex >> 1u), float(gl_VertexIndex & 1u)) * 2.0; + gl_Position = vec4(uv * vec2(2.0, -2.0) + vec2(-1.0, 1.0), 0.0, 1.0); +} + +#version 450 core +#pragma stage : frag +layout (location = 0) out vec4 o_Color; +[[main]] +void shadow_clear_fragment() { + o_Color = vec4(1.0); +} diff --git a/Sources/AdaCorePipelines/Assets/Shaders/volumetric_fog_3d.glsl b/Sources/AdaCorePipelines/Assets/Shaders/volumetric_fog_3d.glsl new file mode 100644 index 000000000..674f60e07 --- /dev/null +++ b/Sources/AdaCorePipelines/Assets/Shaders/volumetric_fog_3d.glsl @@ -0,0 +1,96 @@ +#version 450 core +#pragma stage : vert +layout (location = 0) out vec2 v_UV; +[[main]] +void volumetric_fog_vertex() { + vec2 uv = vec2(float(gl_VertexIndex >> 1u), float(gl_VertexIndex & 1u)) * 2.0; + gl_Position = vec4(uv * vec2(2.0, -2.0) + vec2(-1.0, 1.0), 0.0, 1.0); + v_UV = uv; +} + +#version 450 core +#pragma stage : frag +layout (location = 0) in vec2 v_UV; +layout (location = 0) out vec4 o_Color; +layout (binding = 0) uniform texture2D u_ViewPositionMetallic; +layout (binding = 1) uniform texture2D u_DirectionalShadowTexture; +layout (binding = 2) uniform sampler u_Nearest; +layout (binding = 4) uniform VolumetricFog3DUniform { + mat4 u_InverseView; + mat4 u_InverseProjection; + mat4 u_ShadowViewProjection0; + mat4 u_ShadowViewProjection1; + mat4 u_ShadowViewProjection2; + vec4 u_CascadeSplits; // xyz far view depth of each cascade, w cascade count + vec4 u_ShadowAtlas; // xy shadow texel size, z constant bias, w 1 when shadows are enabled + vec4 u_SunDirection; // xyz world direction to the sun, w anisotropy + vec4 u_SunRadiance; // rgb radiance times intensity times the pass intensity + vec4 u_Medium; // x density, y height falloff, z base height, w start distance + vec4 u_March; // x steps, y maximum distance, z frame jitter +}; + +const float PI = 3.14159265359; + +float interleavedGradientNoise(vec2 pixel) { + return fract(52.9829189 * fract(dot(pixel, vec2(0.06711056, 0.00583715)))); +} + +float henyeyGreenstein(float cosine, float g) { + float denominator = 1.0 + g * g - 2.0 * g * cosine; + return (1.0 - g * g) / (4.0 * PI * pow(max(denominator, 0.0001), 1.5)); +} + +// Whether the sun reaches a world-space point, from the cascade covering its view depth. +float sunVisibility(vec3 world, float viewDepth) { + if (u_ShadowAtlas.w < 0.5 || viewDepth > u_CascadeSplits.z) { return 1.0; } + int count = int(u_CascadeSplits.w); + int cascade = min(viewDepth <= u_CascadeSplits.x ? 0 : (viewDepth <= u_CascadeSplits.y ? 1 : 2), count - 1); + vec4 position; + if (cascade == 0) { position = u_ShadowViewProjection0 * vec4(world, 1.0); } + else if (cascade == 1) { position = u_ShadowViewProjection1 * vec4(world, 1.0); } + else { position = u_ShadowViewProjection2 * vec4(world, 1.0); } + vec3 projected = position.xyz / position.w; + vec2 localUV = projected.xy * vec2(0.5, -0.5) + 0.5; + if (projected.z < 0.0 || projected.z > 1.0 || any(lessThan(localUV, vec2(0.0))) || any(greaterThan(localUV, vec2(1.0)))) { return 1.0; } + float cascadeCount = max(u_CascadeSplits.w, 1.0); + vec2 uv = vec2((localUV.x + float(cascade)) / cascadeCount, localUV.y); + vec2 minimum = vec2(float(cascade) / cascadeCount, 0.0) + u_ShadowAtlas.xy * 0.5; + vec2 maximum = vec2(float(cascade + 1) / cascadeCount, 1.0) - u_ShadowAtlas.xy * 0.5; + float stored = textureLod(sampler2D(u_DirectionalShadowTexture, u_Nearest), clamp(uv, minimum, maximum), 0.0).r; + return projected.z - u_ShadowAtlas.z <= stored ? 1.0 : 0.0; +} + +[[main]] +void volumetric_fog_fragment() { + vec4 positionMetallic = textureLod(sampler2D(u_ViewPositionMetallic, u_Nearest), v_UV, 0.0); + vec2 ndc = vec2(v_UV.x * 2.0 - 1.0, 1.0 - v_UV.y * 2.0); + vec4 viewFar = u_InverseProjection * vec4(ndc, 1.0, 1.0); + vec3 viewRay = normalize(viewFar.xyz / viewFar.w); + bool hasSurface = length(positionMetallic.xyz) > 0.0001; + float surfaceDistance = hasSurface ? length(positionMetallic.xyz) : u_March.y; + float rangeEnd = min(surfaceDistance, u_March.y); + float rangeStart = min(u_Medium.w, rangeEnd); + int steps = int(u_March.x); + float stepLength = (rangeEnd - rangeStart) / float(steps); + if (stepLength <= 0.0) { + o_Color = vec4(0.0); + return; + } + vec3 worldDirection = normalize((u_InverseView * vec4(viewRay, 0.0)).xyz); + vec3 origin = u_InverseView[3].xyz; + float phase = henyeyGreenstein(dot(worldDirection, u_SunDirection.xyz), u_SunDirection.w); + float jitter = fract(interleavedGradientNoise(gl_FragCoord.xy) + u_March.z); + + float transmittance = 1.0; + float scattered = 0.0; + for (int index = 0; index < 64; ++index) { + if (index >= steps) { break; } + float travel = rangeStart + (float(index) + jitter) * stepLength; + vec3 world = origin + worldDirection * travel; + float density = u_Medium.x * exp(clamp(-u_Medium.y * (world.y - u_Medium.z), -60.0, 60.0)); + float step = density * stepLength; + transmittance *= exp(-step); + scattered += step * transmittance * sunVisibility(world, travel * viewRay.z); + } + o_Color = vec4(u_SunRadiance.rgb * (scattered * phase), 1.0); +} diff --git a/Sources/AdaCorePipelines/CorePipelines/Core3DPlugin.swift b/Sources/AdaCorePipelines/CorePipelines/Core3DPlugin.swift index 13fb60101..3b664f575 100644 --- a/Sources/AdaCorePipelines/CorePipelines/Core3DPlugin.swift +++ b/Sources/AdaCorePipelines/CorePipelines/Core3DPlugin.swift @@ -55,6 +55,7 @@ public struct Core3DPlugin: Plugin { graph.addNode(AutoExposure3DRenderNode()) graph.addNode(Bloom3DRenderNode()) graph.addNode(AntiAliasing3DRenderNode(notifiesCompletion: !includes2D)) + graph.addNode(VolumetricFog3DRenderNode()) graph.addNode(ScreenSpaceReflectionRenderNode(notifiesCompletion: !includes2D)) graph.addNode(Temporal3DRenderNode()) graph.addNode(DepthOfField3DRenderNode(notifiesCompletion: !includes2D)) @@ -76,6 +77,8 @@ public struct Core3DPlugin: Plugin { graph.addNodeEdge(from: DirectionalShadow3DRenderNode.name, to: Main3DRenderNode.name) graph.addNodeEdge(from: Main3DRenderNode.name, to: ScreenSpaceReflectionRenderNode.name) graph.addNodeEdge(from: Main3DRenderNode.name, to: ScreenSpaceAO3DRenderNode.name) + graph.addNodeEdge(from: Main3DRenderNode.name, to: VolumetricFog3DRenderNode.name) + graph.addNodeEdge(from: VolumetricFog3DRenderNode.name, to: ScreenSpaceReflectionRenderNode.name) graph.addNodeEdge(from: ScreenSpaceAO3DRenderNode.name, to: AutoExposure3DRenderNode.name) graph.addNodeEdge(from: AutoExposure3DRenderNode.name, to: Bloom3DRenderNode.name) graph.addNodeEdge(from: Bloom3DRenderNode.name, to: ScreenSpaceReflectionRenderNode.name) @@ -105,7 +108,9 @@ public struct Core3DPlugin: Plugin { .insertResource(LocalShadow3DViews()) .insertResource(Render3DLightStatistics()) .insertResource(LocalShadow3DScratch()) + .insertResource(LocalShadow3DClear(device: renderDevice)) .insertResource(LocalLighting3DGPUScratch()) + .insertResource(LightTiles3DScratch()) .insertResource(RenderPipelines(configurator: LocalShadow3DPipeline())) .insertResource(ExtractedMesh3DSources()) .insertResource(VisibleMesh3DLists()) @@ -127,6 +132,7 @@ public struct Core3DPlugin: Plugin { .insertResource(Bloom3DViews(device: renderDevice)) .insertResource(AutoExposure3DViews(device: renderDevice)) .insertResource(DepthOfField3DViews(device: renderDevice)) + .insertResource(VolumetricFog3DViews(device: renderDevice)) .insertResource(ExtractedDebugLines3D()) .insertResource(DebugDraw3DResources()) .addSystem(ExtractDebugDraw3DSystem.self, on: .extract) diff --git a/Sources/AdaCorePipelines/CorePipelines/Environment3D.swift b/Sources/AdaCorePipelines/CorePipelines/Environment3D.swift index 351a9f8f8..204c03ca8 100644 --- a/Sources/AdaCorePipelines/CorePipelines/Environment3D.swift +++ b/Sources/AdaCorePipelines/CorePipelines/Environment3D.swift @@ -124,6 +124,10 @@ public struct Environment3D: Codable, Sendable { public var debugView: DebugView3D = DebugView3D.none /// Exposure and bloom applied before tone mapping. public var postProcess: PostProcess3D = PostProcess3D() + /// Distance and height fog applied in linear HDR before tone mapping. + public var fog: Fog3D = Fog3D() + /// Evaluate only the local lights that reach each screen tile. Disabling it falls back to the flat per-pixel loop. + public var tiledLocalLights: Bool = true /// Flat reflected controls for script and inspector access to render quality settings. public var shadowsEnabled: Bool { @@ -200,6 +204,14 @@ public struct Environment3D: Codable, Sendable { get { postProcess.bloom.threshold } set { postProcess.bloom.threshold = newValue } } + public var fogEnabled: Bool { + get { fog.isEnabled } + set { fog.isEnabled = newValue } + } + public var fogDensity: Float { + get { fog.density } + set { fog.density = newValue } + } public var localShadowsEnabled: Bool { get { localShadows.isEnabled } set { localShadows.isEnabled = newValue } @@ -244,7 +256,7 @@ public struct Environment3D: Codable, Sendable { private enum CodingKeys: CodingKey { case skybox, screenSpaceReflection, imageBasedLighting, shadows, - ambientOcclusion, antiAliasing, meshVisibility, localShadows, debugView, postProcess + ambientOcclusion, antiAliasing, meshVisibility, localShadows, debugView, postProcess, fog, tiledLocalLights } public init(from decoder: Decoder) throws { @@ -259,6 +271,8 @@ public struct Environment3D: Codable, Sendable { localShadows = try container.decodeIfPresent(LocalShadowSettings3D.self, forKey: .localShadows) ?? LocalShadowSettings3D() debugView = try container.decodeIfPresent(DebugView3D.self, forKey: .debugView) ?? DebugView3D.none postProcess = try container.decodeIfPresent(PostProcess3D.self, forKey: .postProcess) ?? PostProcess3D() + fog = try container.decodeIfPresent(Fog3D.self, forKey: .fog) ?? Fog3D() + tiledLocalLights = try container.decodeIfPresent(Bool.self, forKey: .tiledLocalLights) ?? true } } diff --git a/Sources/AdaCorePipelines/CorePipelines/Fog3D.swift b/Sources/AdaCorePipelines/CorePipelines/Fog3D.swift new file mode 100644 index 000000000..89018ed3b --- /dev/null +++ b/Sources/AdaCorePipelines/CorePipelines/Fog3D.swift @@ -0,0 +1,160 @@ +import AdaRender +import AdaUtils +import Foundation +import Math + +/// Shadowed sun light shafts marched through the fog medium. +/// +/// A half-resolution pass marches each view ray, samples the primary directional light's cascaded shadow map and +/// adds the in-scattered light to the frame before tone mapping. It needs `Fog3D` enabled, whose `density` and +/// height falloff define the medium, and replaces the unshadowed `sunInscattering` glow while it runs. +public struct VolumetricFog3D: Codable, Sendable, Equatable { + /// Whether the pass runs. Off by default; a disabled effect allocates nothing. + public var isEnabled: Bool + /// Strength of the in-scattered sun light, as a multiple of the sun's radiance. + public var intensity: Float + /// Henyey-Greenstein anisotropy (`-0.95...0.95`). Positive values brighten shafts when looking toward the sun. + public var anisotropy: Float + /// Samples per ray (`4...64`). More samples reduce noise and banding. + public var steps: Int + /// Farthest distance marched from the camera. Closer geometry ends the ray earlier. + public var maxDistance: Float + + public init(isEnabled: Bool = false, intensity: Float = 1, anisotropy: Float = 0.4, steps: Int = 24, maxDistance: Float = 60) { + self.isEnabled = isEnabled + self.intensity = intensity + self.anisotropy = anisotropy + self.steps = steps + self.maxDistance = maxDistance + } + + private enum CodingKeys: CodingKey { case isEnabled, intensity, anisotropy, steps, maxDistance } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + let defaults = VolumetricFog3D() + isEnabled = try container.decodeIfPresent(Bool.self, forKey: .isEnabled) ?? defaults.isEnabled + intensity = try container.decodeIfPresent(Float.self, forKey: .intensity) ?? defaults.intensity + anisotropy = try container.decodeIfPresent(Float.self, forKey: .anisotropy) ?? defaults.anisotropy + steps = try container.decodeIfPresent(Int.self, forKey: .steps) ?? defaults.steps + maxDistance = try container.decodeIfPresent(Float.self, forKey: .maxDistance) ?? defaults.maxDistance + } + + /// Settings clamped to ranges the shader handles, replacing non-finite values with defaults. + var sanitized: VolumetricFog3D { + let defaults = VolumetricFog3D() + return VolumetricFog3D( + isEnabled: isEnabled, + intensity: finiteRenderValue(intensity, fallback: defaults.intensity, range: 0...16), + anisotropy: finiteRenderValue(anisotropy, fallback: defaults.anisotropy, range: -0.95...0.95), + steps: min(max(steps, 4), 64), + maxDistance: finiteRenderValue(maxDistance, fallback: defaults.maxDistance, range: 1...100_000) + ) + } +} + +/// Distance and exponential height fog, applied to lit geometry in linear HDR before exposure and tone mapping. +/// +/// Optical depth along a view ray is the integral of `density * exp(-heightFalloff * (y - baseHeight))`, so fog +/// is thickest at and below `baseHeight` and thins out above it. A `heightFalloff` of `0` gives uniform +/// distance fog. Fog scatters ambient light in `color` and, when `sunInscattering` is above zero, brightens +/// towards the primary directional light. +public struct Fog3D: Codable, Sendable, Equatable { + /// Whether fog is applied. Off by default. + public var isEnabled: Bool + /// Fog color authored in sRGB, like the skybox colors. + public var color: Color + /// Extinction per world unit at `baseHeight`. `0.02` is light haze; `0.2` is dense fog. + public var density: Float + /// How quickly fog thins with height, per world unit. `0` disables the height term. + public var heightFalloff: Float + /// World-space height where fog has its full `density`. + public var baseHeight: Float + /// Distance from the camera before fog starts. + public var startDistance: Float + /// Largest fraction of a pixel that fog can replace (`0...1`). + public var maximumOpacity: Float + /// How much fog covers the sky (`0...1`). `0` leaves the sky untouched. + public var skyInfluence: Float + /// Strength of the glow towards the directional light, as a multiple of its radiance. + public var sunInscattering: Float + /// Tightness of the sun glow. Larger values give a narrower cone. + public var sunExponent: Float + /// Shadowed light shafts through this fog. + public var volumetric: VolumetricFog3D + + public init( + isEnabled: Bool = false, + color: Color = Color(red: 0.62, green: 0.72, blue: 0.82), + density: Float = 0.02, + heightFalloff: Float = 0, + baseHeight: Float = 0, + startDistance: Float = 0, + maximumOpacity: Float = 1, + skyInfluence: Float = 0, + sunInscattering: Float = 0, + sunExponent: Float = 8, + volumetric: VolumetricFog3D = VolumetricFog3D() + ) { + self.isEnabled = isEnabled + self.color = color + self.density = density + self.heightFalloff = heightFalloff + self.baseHeight = baseHeight + self.startDistance = startDistance + self.maximumOpacity = maximumOpacity + self.skyInfluence = skyInfluence + self.sunInscattering = sunInscattering + self.sunExponent = sunExponent + self.volumetric = volumetric + } + + private enum CodingKeys: CodingKey { + case isEnabled, color, density, heightFalloff, baseHeight, startDistance, maximumOpacity, skyInfluence, sunInscattering, sunExponent, volumetric + } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + let defaults = Fog3D() + isEnabled = try container.decodeIfPresent(Bool.self, forKey: .isEnabled) ?? defaults.isEnabled + color = try container.decodeIfPresent(Color.self, forKey: .color) ?? defaults.color + density = try container.decodeIfPresent(Float.self, forKey: .density) ?? defaults.density + heightFalloff = try container.decodeIfPresent(Float.self, forKey: .heightFalloff) ?? defaults.heightFalloff + baseHeight = try container.decodeIfPresent(Float.self, forKey: .baseHeight) ?? defaults.baseHeight + startDistance = try container.decodeIfPresent(Float.self, forKey: .startDistance) ?? defaults.startDistance + maximumOpacity = try container.decodeIfPresent(Float.self, forKey: .maximumOpacity) ?? defaults.maximumOpacity + skyInfluence = try container.decodeIfPresent(Float.self, forKey: .skyInfluence) ?? defaults.skyInfluence + sunInscattering = try container.decodeIfPresent(Float.self, forKey: .sunInscattering) ?? defaults.sunInscattering + sunExponent = try container.decodeIfPresent(Float.self, forKey: .sunExponent) ?? defaults.sunExponent + volumetric = try container.decodeIfPresent(VolumetricFog3D.self, forKey: .volumetric) ?? defaults.volumetric + } + + /// Whether fog can change the image. A camera that returns `false` pays nothing for fog. + var changesImage: Bool { + let settings = sanitized + return isEnabled && settings.density > 0 && settings.maximumOpacity > 0 + } + + /// Whether the light-shaft pass has work to do. + var volumetricActive: Bool { + changesImage && volumetric.isEnabled && volumetric.sanitized.intensity > 0 + } + + /// Settings clamped to ranges the shader handles, replacing non-finite values with defaults. + var sanitized: Fog3D { + let defaults = Fog3D() + return Fog3D( + isEnabled: isEnabled, + color: color, + density: finiteRenderValue(density, fallback: defaults.density, range: 0...10), + heightFalloff: finiteRenderValue(heightFalloff, fallback: defaults.heightFalloff, range: 0...10), + baseHeight: finiteRenderValue(baseHeight, fallback: defaults.baseHeight, range: -100_000...100_000), + startDistance: finiteRenderValue(startDistance, fallback: defaults.startDistance, range: 0...100_000), + maximumOpacity: finiteRenderValue(maximumOpacity, fallback: defaults.maximumOpacity, range: 0...1), + skyInfluence: finiteRenderValue(skyInfluence, fallback: defaults.skyInfluence, range: 0...1), + sunInscattering: finiteRenderValue(sunInscattering, fallback: defaults.sunInscattering, range: 0...16), + sunExponent: finiteRenderValue(sunExponent, fallback: defaults.sunExponent, range: 1...256), + volumetric: volumetric.sanitized + ) + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/GPUVisibility3D.swift b/Sources/AdaCorePipelines/CorePipelines/GPUVisibility3D.swift index ea435efcd..08d79dd20 100644 --- a/Sources/AdaCorePipelines/CorePipelines/GPUVisibility3D.swift +++ b/Sources/AdaCorePipelines/CorePipelines/GPUVisibility3D.swift @@ -79,7 +79,14 @@ public struct GPUVisibility3DState: Resource { let material = materials.buffer(for: item.material) else { continue } let part = item.mesh.models[item.modelIndex].parts[item.partIndex] - pass.setRenderPipelineState(pipelines.pipeline(for: part.vertexDescriptor, device: device)) + let morphed = item.morphBuffer != nil && part.morphTargets != nil + pass.setRenderPipelineState(pipelines.pipeline(for: Shadow3DConfiguration(vertex: part.vertexDescriptor, morph: morphed), device: device)) + if morphed, let morphBuffer = item.morphBuffer, let morphSet = part.morphTargets { + pass.setResourceSet(.init(bindings: [ + .init(binding: Morph3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(morphBuffer, offset: 0)), + .init(binding: Morph3DUniforms.textureBinding, shaderStages: .vertex, resource: .texture(morphSet.deltaTexture)), + ]), index: 0) + } pass.setVertexBuffer(part.vertexBuffer, offset: 0, slot: 0) pass.setVertexBuffer(wrap(instances), offset: Int(range.lowerBound) * MemoryLayout.stride, slot: 3) pass.setVertexBuffer(defaults, offset: 0, slot: 4) diff --git a/Sources/AdaCorePipelines/CorePipelines/LightTiles3D.swift b/Sources/AdaCorePipelines/CorePipelines/LightTiles3D.swift new file mode 100644 index 000000000..4e05ddf81 --- /dev/null +++ b/Sources/AdaCorePipelines/CorePipelines/LightTiles3D.swift @@ -0,0 +1,140 @@ +import AdaECS +@_spi(Internal) import AdaRender +import Math + +/// Screen-space tiles that each carry a bit mask of the local lights reaching them. +/// +/// The main pass evaluates only the lights set in the pixel's tile instead of every light in the frame. A light +/// reaches a tile when the screen-space bounds of its range sphere overlap it, so the mask is conservative and +/// never drops a light that could contribute: lights outside their range already contribute nothing, which keeps +/// the lit result identical to the flat loop. +enum LightTiles3DMath { + /// Local lights per frame, one bit each in a 32-bit mask. + static let maximumLights = 32 + /// Upper bound on tile count, set by the size of the GPU mask buffer. + static let maximumTiles = 4096 + static let smallestTileSize = 32 + /// Nearest view depth a light sphere may reach before the whole screen counts as covered. + private static let nearDepth: Float = 0.05 + + struct Layout: Equatable { + var tileSize: Int + var columns: Int + var rows: Int + var count: Int { columns * rows } + } + + /// The smallest power-of-two multiple of 32 pixels that keeps the tile count within `maximumTiles`. + static func layout(width: Int, height: Int) -> Layout { + var size = smallestTileSize + func grid(_ size: Int) -> Layout { + Layout(tileSize: size, columns: max(1, (width + size - 1) / size), rows: max(1, (height + size - 1) / size)) + } + while grid(size).count > maximumTiles { + size *= 2 + } + return grid(size) + } + + /// A mask with the first `count` lights set, used when tiles are not in play. + static func allLights(_ count: Int) -> UInt32 { + count >= maximumLights ? UInt32.max : (UInt32(1) << UInt32(max(count, 0))) - 1 + } + + /// One mask per tile for lights given as view-space sphere centers and ranges. + /// + /// `projection` maps view space to clip space (the one the main pass draws with) and `viewport` is the pixel + /// rectangle it renders into. + static func masks( + spheres: [(center: Vector3, range: Float)], + projection: Transform3D, + viewport: Rect, + layout: Layout + ) -> [UInt32] { + var masks = [UInt32](repeating: 0, count: layout.count) + for (index, sphere) in spheres.prefix(maximumLights).enumerated() { + let bit = UInt32(1) << UInt32(index) + guard sphere.range > 0, sphere.center.z + sphere.range > 0 else { + continue + } + guard let bounds = screenBounds(of: sphere, projection: projection, viewport: viewport) else { + for tile in masks.indices { masks[tile] |= bit } + continue + } + let size = Float(layout.tileSize) + let firstColumn = max(0, Int((bounds.minX - 1) / size)), lastColumn = min(layout.columns - 1, Int((bounds.maxX + 1) / size)) + let firstRow = max(0, Int((bounds.minY - 1) / size)), lastRow = min(layout.rows - 1, Int((bounds.maxY + 1) / size)) + guard bounds.maxX >= 0, bounds.maxY >= 0, firstColumn <= lastColumn, firstRow <= lastRow else { + continue + } + for row in firstRow...lastRow { + for column in firstColumn...lastColumn { + masks[row * layout.columns + column] |= bit + } + } + } + return masks + } + + /// Pixel bounds of the sphere's view-space bounding box, or `nil` when it may reach the camera plane and so + /// can cover any pixel. + private static func screenBounds( + of sphere: (center: Vector3, range: Float), + projection: Transform3D, + viewport: Rect + ) -> (minX: Float, minY: Float, maxX: Float, maxY: Float)? { + guard sphere.center.z - sphere.range > nearDepth else { + return nil + } + var bounds = (minX: Float.infinity, minY: Float.infinity, maxX: -Float.infinity, maxY: -Float.infinity) + for corner in 0..<8 { + let point = Vector4( + sphere.center.x + (corner & 1 == 0 ? -sphere.range : sphere.range), + sphere.center.y + (corner & 2 == 0 ? -sphere.range : sphere.range), + sphere.center.z + (corner & 4 == 0 ? -sphere.range : sphere.range), + 1 + ) + let clip = projection * point + guard clip.w > 0.0001 else { + return nil + } + let pixelX = viewport.origin.x + (clip.x / clip.w * 0.5 + 0.5) * viewport.size.width + let pixelY = viewport.origin.y + (0.5 - clip.y / clip.w * 0.5) * viewport.size.height + bounds = (min(bounds.minX, pixelX), min(bounds.minY, pixelY), max(bounds.maxX, pixelX), max(bounds.maxY, pixelY)) + } + return bounds + } +} + +/// Frame-buffered tile masks, one ring per camera. +struct LightTiles3DScratch: Resource { + /// `uvec4` entries in the shader's mask array (`maximumTiles / 4`). + static let maskVectors = LightTiles3DMath.maximumTiles / 4 + + private var buffers: [Entity.ID: (buffers: [BufferData>], index: Int)] = [:] + private let frames = max(1, unsafe RenderEngine.configurations.maxFramesInFlight) + + mutating func retainViews(_ views: [Entity.ID]) { + for view in buffers.keys where !views.contains(view) { buffers.removeValue(forKey: view) } + } + + mutating func write(view: Entity.ID, masks: [UInt32], device: RenderDevice) -> BufferData> { + var entry = buffers.removeValue(forKey: view) ?? ( + (0..](repeating: .zero, count: Self.maskVectors)) }, + 0 + ) + entry.index = (entry.index + 1) % frames + for vector in 0...zero + for lane in 0..<4 { + let tile = vector * 4 + lane + packed[lane] = tile < masks.count ? masks[tile] : 0 + } + entry.buffers[entry.index].elements[vector] = packed + } + entry.buffers[entry.index].write(to: device) + let output = entry.buffers[entry.index] + buffers[view] = entry + return output + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/LocalLighting3D.swift b/Sources/AdaCorePipelines/CorePipelines/LocalLighting3D.swift index ac80556c1..4ebe6cf6d 100644 --- a/Sources/AdaCorePipelines/CorePipelines/LocalLighting3D.swift +++ b/Sources/AdaCorePipelines/CorePipelines/LocalLighting3D.swift @@ -6,10 +6,24 @@ import Math public struct LocalShadowSettings3D: Codable, Sendable { public var isEnabled: Bool public var maximumLights: Int + /// Keep the shadow faces of lights and casters that did not change since the last frame instead of redrawing + /// them. Faces with skinned casters are always redrawn. + public var reusesStaticFaces: Bool - public init(isEnabled: Bool = true, maximumLights: Int = 4) { + public init(isEnabled: Bool = true, maximumLights: Int = 4, reusesStaticFaces: Bool = true) { self.isEnabled = isEnabled self.maximumLights = maximumLights + self.reusesStaticFaces = reusesStaticFaces + } + + private enum CodingKeys: CodingKey { case isEnabled, maximumLights, reusesStaticFaces } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + let defaults = LocalShadowSettings3D() + isEnabled = try container.decodeIfPresent(Bool.self, forKey: .isEnabled) ?? defaults.isEnabled + maximumLights = try container.decodeIfPresent(Int.self, forKey: .maximumLights) ?? defaults.maximumLights + reusesStaticFaces = try container.decodeIfPresent(Bool.self, forKey: .reusesStaticFaces) ?? defaults.reusesStaticFaces } } @@ -210,6 +224,15 @@ struct LocalShadow3DViews: Resource { var color: RenderTexture? var depth: RenderTexture? var matrices = [Transform3D](repeating: .identity, count: 24) + /// Fingerprint of what each atlas face last drew, keyed by `slot * 6 + face`. + var faceDigests: [Int: UInt64] = [:] + /// Fingerprint of the face's view-projection matrix when its entry in `faceDigests` was taken. + var faceMatrixDigests: [Int: UInt64] = [:] + /// Fingerprint of the shadow casters when the faces were last prepared, or `nil` when it was not reusable. + var sceneDigest: UInt64? + /// Faces drawn and faces kept from an earlier frame by the last pass. + var renderedFaces = 0 + var reusedFaces = 0 } var entries: [Entity.ID: Entry] = [:] } diff --git a/Sources/AdaCorePipelines/CorePipelines/MeshVisibility3D.swift b/Sources/AdaCorePipelines/CorePipelines/MeshVisibility3D.swift index 6c62b9bb7..bbd736965 100644 --- a/Sources/AdaCorePipelines/CorePipelines/MeshVisibility3D.swift +++ b/Sources/AdaCorePipelines/CorePipelines/MeshVisibility3D.swift @@ -49,6 +49,13 @@ public struct Mesh3DRenderSource: Sendable { public var castShadows: Bool public var receiveShadows: Bool public var skinningBuffer: (any UniformBuffer)? + /// Morph weight buffers by mesh part (see ``morphKey(model:part:)``); empty for a mesh without morph weights. + public var morphBuffers: [UInt64: any UniformBuffer] + + /// The key of a mesh part in ``morphBuffers``. + public static func morphKey(model: Int, part: Int) -> UInt64 { + UInt64(truncatingIfNeeded: model) << 32 | UInt64(truncatingIfNeeded: part) + } public init( entity: Entity.ID, @@ -66,8 +73,10 @@ public struct Mesh3DRenderSource: Sendable { castShadows: Bool = true, receiveShadows: Bool = true, skinningBuffer: (any UniformBuffer)? = nil, - previousTransform: Transform3D? = nil + previousTransform: Transform3D? = nil, + morphBuffers: [UInt64: any UniformBuffer] = [:] ) { + self.morphBuffers = morphBuffers self.previousTransform = previousTransform ?? transform self.drawPass = drawPass self.entity = entity diff --git a/Sources/AdaCorePipelines/CorePipelines/Morph3DUniforms.swift b/Sources/AdaCorePipelines/CorePipelines/Morph3DUniforms.swift new file mode 100644 index 000000000..30ee3ea58 --- /dev/null +++ b/Sources/AdaCorePipelines/CorePipelines/Morph3DUniforms.swift @@ -0,0 +1,126 @@ +import AdaECS +@_spi(Internal) import AdaRender +import Math + +/// Frame-buffered weights of the active morph targets, one uniform buffer per entity and mesh part. +/// +/// A part's vertex count and layout differ from its siblings', so each part gets its own buffer even though the weights belong to +/// the entity. The buffer holds the current and the previous frame's active targets, so the motion pass can move a vertex from +/// where the previous weights put it. +/// +/// Layout, as 18 `vec4`: `[0]` current target count, previous target count, vertices per target, attributes per vertex; `[1]` +/// attribute flags; `[2..<10]` current (target index, weight); `[10..<18]` previous (target index, weight). +public struct Morph3DUniforms: Resource { + /// The vertex-stage slot of the weights buffer. + public static let binding = 21 + /// The vertex-stage slot of the delta texture. + public static let textureBinding = 40 + static let vectorCount = 18 + + /// Identifies one part of an entity's mesh. + public struct PartKey: Hashable, Sendable { + public let entity: Entity.ID + public let model: Int + public let part: Int + } + + private struct Entry { + var buffers: [(any UniformBuffer)?] + var frame: Int + var previous: [(index: Int, weight: Float)] + } + + private var entries: [PartKey: Entry] = [:] + private var frame = 0 + private var currentIndex = 0 + private let bufferCount: Int + private var upload = [Vector4](repeating: .zero, count: vectorCount) + + public init() { + bufferCount = max(1, unsafe RenderEngine.configurations.maxFramesInFlight) + } + + public mutating func beginFrame() { + for (key, entry) in entries where entry.frame != frame { + entries.removeValue(forKey: key) + } + frame &+= 1 + currentIndex = (currentIndex + 1) % bufferCount + } + + /// Test hooks for the buffer layout, which need no ``MorphTargetSet`` and so no GPU. + static let vectorCountForTests = vectorCount + + static func packForTests( + current: [(index: Int, weight: Float)], + previous: [(index: Int, weight: Float)], + vertexCount: Int, + attributes: Int, + flags: Float, + into vectors: inout [Vector4] + ) { + pack(current: current, previous: previous, header: (vertexCount, attributes, flags), into: &vectors) + } + + /// The vector array a part's buffer holds for these weights; the previous list is the last frame's when there is one. + static func pack( + current: [(index: Int, weight: Float)], + previous: [(index: Int, weight: Float)], + set: MorphTargetSet, + into vectors: inout [Vector4] + ) { + pack(current: current, previous: previous, header: (set.vertexCount, set.attributesPerVertex, set.attributeFlags), into: &vectors) + } + + private static func pack( + current: [(index: Int, weight: Float)], + previous: [(index: Int, weight: Float)], + header: (vertexCount: Int, attributes: Int, flags: Float), + into vectors: inout [Vector4] + ) { + for index in vectors.indices { vectors[index] = .zero } + vectors[0] = Vector4(Float(current.count), Float(previous.count), Float(header.vertexCount), Float(header.attributes)) + vectors[1] = Vector4(header.flags, 0, 0, 0) + for (slot, active) in current.prefix(MorphTargetSet.maximumActiveTargets).enumerated() { + vectors[2 + slot] = Vector4(Float(active.index), active.weight, 0, 0) + } + for (slot, active) in previous.prefix(MorphTargetSet.maximumActiveTargets).enumerated() { + vectors[10 + slot] = Vector4(Float(active.index), active.weight, 0, 0) + } + } + + /// The buffers written for the current frame, for tests. + func debugBuffers() -> [any UniformBuffer] { + entries.values.compactMap { $0.frame == frame ? $0.buffers[currentIndex] : nil } + } + + /// Writes the buffers of every morphed part of `mesh` for an entity and returns them by part. + public mutating func write( + weights: [Float], + mesh: Mesh, + for entity: Entity.ID, + device: RenderDevice + ) -> [PartKey: any UniformBuffer] { + let current = MorphTargetSet.activeTargets(weights: weights) + var result: [PartKey: any UniformBuffer] = [:] + for (modelIndex, model) in mesh.models.enumerated() { + for (partIndex, part) in model.parts.enumerated() { + guard let set = part.morphTargets else { continue } + let key = PartKey(entity: entity, model: modelIndex, part: partIndex) + var entry = entries[key] ?? Entry(buffers: Array(repeating: nil, count: bufferCount), frame: frame, previous: current) + if entry.buffers[currentIndex] == nil { + entry.buffers[currentIndex] = device.createUniformBuffer(Vector4.self, count: Self.vectorCount, binding: Self.binding) + } + Self.pack(current: current, previous: entry.previous, set: set, into: &upload) + entry.buffers[currentIndex]?.setElements(&upload) + entry.previous = current + entry.frame = frame + entries[key] = entry + if let buffer = entry.buffers[currentIndex] { + result[key] = buffer + } + } + } + return result + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/PBR3DUniforms.swift b/Sources/AdaCorePipelines/CorePipelines/PBR3DUniforms.swift index ff33a1ef2..927ce5005 100644 --- a/Sources/AdaCorePipelines/CorePipelines/PBR3DUniforms.swift +++ b/Sources/AdaCorePipelines/CorePipelines/PBR3DUniforms.swift @@ -2,6 +2,24 @@ import AdaECS @_spi(Internal) import AdaRender import Math +/// One texture slot's UV transform: the first vector is the row-major 2x2 matrix, the second holds the offset. +public struct PBR3DTextureTransform: Sendable, Equatable { + public let matrix: Vector4 + public let offset: Vector4 + + init(_ transform: PBRTextureTransform?) { + let transform = transform ?? .identity + // glTF: uv' = T * R * S * uv with R = [[cos, sin], [-sin, cos]]. + let cosine = Math.cos(transform.rotation) + let sine = Math.sin(transform.rotation) + matrix = Vector4( + cosine * transform.scale.x, sine * transform.scale.y, + -sine * transform.scale.x, cosine * transform.scale.y + ) + offset = Vector4(transform.offset.x, transform.offset.y, 0, 0) + } +} + /// Per-draw material parameters; vertex instance data retains color and metal/rough factors. public struct PBR3DUniform: Sendable, Equatable { public let emissiveFactor: Vector4 @@ -11,6 +29,34 @@ public struct PBR3DUniform: Sendable, Equatable { public let emission: Vector4 /// Parallax: height scale, UV set, maximum ray-march layers and fade distance. public let parallax: Vector4 + /// UV transforms for base color, metallic/roughness, normal, occlusion and emission, in that order. + public let transform0: PBR3DTextureTransform + public let transform1: PBR3DTextureTransform + public let transform2: PBR3DTextureTransform + public let transform3: PBR3DTextureTransform + public let transform4: PBR3DTextureTransform + /// RGB is the reflectance of a non-metal at normal incidence, from the index of refraction and `KHR_materials_specular`. + public let dielectric: Vector4 + /// X is the clearcoat strength and y its roughness. + public let coat: Vector4 + /// RGB is the sheen color and w its roughness. + public let sheen: Vector4 + /// X is the anisotropy strength and y its rotation in radians. + public let anisotropy: Vector4 + /// UV sets (0 or 1) of the layer textures in `PBRLayerTexture` order: clearcoat to sheen color, then sheen roughness to specular color. + public let layerSets0: Vector4 + public let layerSets1: Vector4 + /// UV transforms of the layer textures in `PBRLayerTexture` order. + public let layerTransform0: PBR3DTextureTransform + public let layerTransform1: PBR3DTextureTransform + public let layerTransform2: PBR3DTextureTransform + public let layerTransform3: PBR3DTextureTransform + public let layerTransform4: PBR3DTextureTransform + public let layerTransform5: PBR3DTextureTransform + public let layerTransform6: PBR3DTextureTransform + public let layerTransform7: PBR3DTextureTransform + /// X is the clearcoat normal scale. + public let layerParameters: Vector4 public init(material: PBRMaterial?, descriptor _: VertexDescriptor) { func coordinate(_ slot: Int) -> Int { @@ -19,6 +65,9 @@ public struct PBR3DUniform: Sendable, Equatable { } return material.textureCoordinates[slot] } + func transform(_ slot: Int) -> PBR3DTextureTransform { + PBR3DTextureTransform(material?.textureTransforms.indices.contains(slot) == true ? material?.textureTransforms[slot] : nil) + } emissiveFactor = Vector4(material?.emissiveFactor ?? .one, 0) properties = Vector4(material?.normalScale ?? 1, material?.occlusionStrength ?? 1, material?.alphaCutoff ?? 0.5, Float(material?.alphaMode.rawValue ?? 0)) let heightScale = material?.heightTexture != nil ? max(finiteRenderValue(material?.heightScale ?? 0, fallback: 0, range: 0...1000), 0) : 0 @@ -31,6 +80,76 @@ public struct PBR3DUniform: Sendable, Equatable { Float(min(max(material?.heightMaxLayers ?? 32, 4), 64)), finiteRenderValue(material?.heightFadeDistance ?? 30, fallback: 30, range: 0.1...100000) ) + transform0 = transform(0) + transform1 = transform(1) + transform2 = transform(2) + transform3 = transform(3) + transform4 = transform(4) + dielectric = Vector4(Self.dielectricReflectance(material), 0) + coat = Vector4( + finiteRenderValue(material?.clearcoatFactor ?? 0, fallback: 0, range: 0...1), + finiteRenderValue(material?.clearcoatRoughnessFactor ?? 0, fallback: 0, range: 0...1), + 0, + 0 + ) + let sheenColor = material?.sheenColorFactor ?? .zero + sheen = Vector4( + finiteRenderValue(sheenColor.x, fallback: 0, range: 0...1), + finiteRenderValue(sheenColor.y, fallback: 0, range: 0...1), + finiteRenderValue(sheenColor.z, fallback: 0, range: 0...1), + finiteRenderValue(material?.sheenRoughnessFactor ?? 0, fallback: 0, range: 0...1) + ) + anisotropy = Vector4( + finiteRenderValue(material?.anisotropyStrength ?? 0, fallback: 0, range: 0...1), + finiteRenderValue(material?.anisotropyRotation ?? 0, fallback: 0, range: -100...100), + 0, + 0 + ) + func layer(_ texture: PBRLayerTexture) -> PBRTextureBinding? { material?.layerTextures[texture] } + func layerSet(_ texture: PBRLayerTexture) -> Float { Float(min(max(layer(texture)?.texCoord ?? 0, 0), 1)) } + func layerTransform(_ texture: PBRLayerTexture) -> PBR3DTextureTransform { PBR3DTextureTransform(layer(texture)?.transform) } + layerSets0 = Vector4(layerSet(.clearcoat), layerSet(.clearcoatRoughness), layerSet(.clearcoatNormal), layerSet(.sheenColor)) + layerSets1 = Vector4(layerSet(.sheenRoughness), layerSet(.anisotropy), layerSet(.specular), layerSet(.specularColor)) + layerTransform0 = layerTransform(.clearcoat) + layerTransform1 = layerTransform(.clearcoatRoughness) + layerTransform2 = layerTransform(.clearcoatNormal) + layerTransform3 = layerTransform(.sheenColor) + layerTransform4 = layerTransform(.sheenRoughness) + layerTransform5 = layerTransform(.anisotropy) + layerTransform6 = layerTransform(.specular) + layerTransform7 = layerTransform(.specularColor) + layerParameters = Vector4(finiteRenderValue(layer(.clearcoatNormal)?.scale ?? 1, fallback: 1, range: 0...100), 0, 0, 0) + } + + /// Reflectance at normal incidence of the dielectric layer: ((ior - 1) / (ior + 1))^2 tinted and scaled by the + /// specular parameters, capped at 1. The defaults give exactly 0.04. + static func dielectricReflectance(_ material: PBRMaterial?) -> Vector3 { + let ior = material.map { $0.ior.isFinite ? max($0.ior, 1) : 1.5 } ?? 1.5 + let base = ((ior - 1) / (ior + 1)) * ((ior - 1) / (ior + 1)) + let strength = material.map { $0.specularFactor.isFinite ? max($0.specularFactor, 0) : 1 } ?? 1 + let tint = material?.specularColorFactor ?? .one + func channel(_ value: Float) -> Float { + min(base * (value.isFinite ? max(value, 0) : 1), 1) * strength + } + return Vector3(channel(tint.x), channel(tint.y), channel(tint.z)) + } + + /// A bit-exact view of every field, so equal material values share one buffer even for -0 and NaN. + var bitPatterns: [SIMD4] { + func bits(_ vector: Vector4) -> SIMD4 { + SIMD4(vector.x.bitPattern, vector.y.bitPattern, vector.z.bitPattern, vector.w.bitPattern) + } + return [ + bits(emissiveFactor), bits(properties), bits(flags), bits(coordinates), bits(emission), bits(parallax), + bits(transform0.matrix), bits(transform0.offset), bits(transform1.matrix), bits(transform1.offset), + bits(transform2.matrix), bits(transform2.offset), bits(transform3.matrix), bits(transform3.offset), + bits(transform4.matrix), bits(transform4.offset), bits(dielectric), bits(coat), bits(sheen), bits(anisotropy), bits(layerSets0), bits(layerSets1), + bits(layerTransform0.matrix), bits(layerTransform0.offset), bits(layerTransform1.matrix), bits(layerTransform1.offset), + bits(layerTransform2.matrix), bits(layerTransform2.offset), bits(layerTransform3.matrix), bits(layerTransform3.offset), + bits(layerTransform4.matrix), bits(layerTransform4.offset), bits(layerTransform5.matrix), bits(layerTransform5.offset), + bits(layerTransform6.matrix), bits(layerTransform6.offset), bits(layerTransform7.matrix), bits(layerTransform7.offset), + bits(layerParameters), + ] } } @@ -39,23 +158,10 @@ public struct PBR3DUniform: Sendable, Equatable { public struct PBR3DUniforms: Resource { public static let binding = 15 private struct Key: Hashable { - let emissive: SIMD4 - let properties: SIMD4 - let flags: SIMD4 - let coordinates: SIMD4 - let emission: SIMD4 - let parallax: SIMD4 + let words: [SIMD4] init(_ value: PBR3DUniform) { - func bits(_ vector: Vector4) -> SIMD4 { - SIMD4(vector.x.bitPattern, vector.y.bitPattern, vector.z.bitPattern, vector.w.bitPattern) - } - emissive = bits(value.emissiveFactor) - properties = bits(value.properties) - flags = bits(value.flags) - coordinates = bits(value.coordinates) - emission = bits(value.emission) - parallax = bits(value.parallax) + words = value.bitPatterns } } private struct Entry { let buffer: any UniformBuffer; var frame: Int } diff --git a/Sources/AdaCorePipelines/CorePipelines/Render3DLightStatistics.swift b/Sources/AdaCorePipelines/CorePipelines/Render3DLightStatistics.swift index e8453465c..f5ea1efde 100644 --- a/Sources/AdaCorePipelines/CorePipelines/Render3DLightStatistics.swift +++ b/Sources/AdaCorePipelines/CorePipelines/Render3DLightStatistics.swift @@ -15,6 +15,8 @@ public final class Render3DLightStatistics: Resource, Sendable { public var shadowFaces: Int public var atlasWidth: Int public var atlasHeight: Int + /// Shadow faces kept from an earlier frame instead of redrawn. + public var reusedShadowFaces: Int = 0 } private let state = Mutex<[String: Snapshot]>([:]) public init() {} @@ -30,7 +32,8 @@ public final class Render3DLightStatistics: Resource, Sendable { shadowLights: selected.count, shadowFaces: selected.reduce(0) { $0 + ($1.kind == .point ? 6 : 1) }, atlasWidth: entry.color?.size.width ?? 0, - atlasHeight: entry.color?.size.height ?? 0 + atlasHeight: entry.color?.size.height ?? 0, + reusedShadowFaces: entry.reusedFaces ) state.withLock { $0["\(view)"] = value } } diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/DirectionalShadow3DRenderNode.swift b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/DirectionalShadow3DRenderNode.swift index 313d201c9..a8708f3fe 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/DirectionalShadow3DRenderNode.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/DirectionalShadow3DRenderNode.swift @@ -211,7 +211,8 @@ public struct DirectionalShadow3DRenderNode: RenderNode { guard item.castShadows, (item.material as? PBRMaterial)?.alphaMode != .blend, let batchRange = item.batchRange else { continue } - let pipeline = pipelines.pipeline(for: part.vertexDescriptor, device: renderDevice.renderDevice) + let morphed = item.morphBuffer != nil && part.morphTargets != nil + let pipeline = pipelines.pipeline(for: Shadow3DConfiguration(vertex: part.vertexDescriptor, morph: morphed), device: renderDevice.renderDevice) pass.setRenderPipelineState(pipeline) pass.setVertexBuffer(part.vertexBuffer, offset: 0, slot: 0) if let material = materialUniforms.buffer(for: item.material) { @@ -227,6 +228,12 @@ public struct DirectionalShadow3DRenderNode: RenderNode { .init(binding: Skinning3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(palette, offset: 0)), ]), index: 0) } + if morphed, let morphBuffer = item.morphBuffer, let morphSet = part.morphTargets { + pass.setResourceSet(RenderResourceSet(bindings: [ + .init(binding: Morph3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(morphBuffer, offset: 0)), + .init(binding: Morph3DUniforms.textureBinding, shaderStages: .vertex, resource: .texture(morphSet.deltaTexture)), + ]), index: 0) + } if let defaults = buffers.defaults { pass.setVertexBuffer(defaults, offset: 0, slot: 4) } diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/LocalShadow3DRenderNode.swift b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/LocalShadow3DRenderNode.swift index 7cab675fb..62bf0bb7b 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/LocalShadow3DRenderNode.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/LocalShadow3DRenderNode.swift @@ -1,3 +1,4 @@ +import AdaAssets import AdaECS @_spi(Internal) import AdaRender import AdaUtils @@ -8,8 +9,10 @@ struct LocalShadow3DScratch: Resource { } struct LocalShadow3DPipeline: RenderPipelineConfigurator { + typealias Configuration = Shadow3DConfiguration + private let base = DirectionalShadow3DPipeline() - func configurate(with configuration: VertexDescriptor) -> RenderPipelineDescriptor { + func configurate(with configuration: Shadow3DConfiguration) -> RenderPipelineDescriptor { var descriptor = base.configurate(with: configuration) descriptor.debugName = "Local Shadow 3D" descriptor.colorAttachments = [.init(format: .rgba_16f)] @@ -17,6 +20,24 @@ struct LocalShadow3DPipeline: RenderPipelineConfigurator { } } +/// Resets one atlas tile to the far value, so a face can be redrawn while its neighbors keep their contents. +struct LocalShadow3DClear: Resource { + let pipeline: RenderPipeline + + init(device: RenderDevice) { + let shader = CorePipelineShaders.loadRequiredBundled(at: "Shaders/shadow_clear_3d.glsl") + var descriptor = RenderPipelineDescriptor( + vertex: shader.asset.requiredShader(for: .vertex), + fragment: shader.asset.getShader(for: .fragment), + debugName: "Local Shadow Clear 3D", + backfaceCulling: false + ) + descriptor.depthStencilDescriptor = DepthStencilDescriptor(isDepthTestEnabled: false, isDepthWriteEnabled: false, depthCompareOperator: .always) + descriptor.colorAttachments = [.init(format: .rgba_16f)] + pipeline = device.createRenderPipeline(from: descriptor) + } +} + /// One atlas pass per camera covers a bounded set of point/spot shadow faces, including offscreen casters. public struct LocalShadow3DRenderNode: RenderNode { public static let name: RenderNodeLabel = "Main3D.LocalShadows" @@ -33,6 +54,7 @@ public struct LocalShadow3DRenderNode: RenderNode { @Res private var skinning @ResMut> private var pipelines @Res private var device + @Res private var clear @Res private var metrics public init() {} @@ -51,6 +73,7 @@ public struct LocalShadow3DRenderNode: RenderNode { _skinning.update(from: world) _pipelines.update(from: world) _device.update(from: world) + _clear.update(from: world) _metrics.update(from: world) var active: [Entity.ID] = [] cameras.forEach { _, _, _, source in active.append(source.entityId) } @@ -95,24 +118,31 @@ public struct LocalShadow3DRenderNode: RenderNode { samplerDescription: .init(minFilter: .nearest, magFilter: .nearest, mipFilter: .notMipmapped) ) entry.depth = RenderTexture(size: size, scaleFactor: 1, format: .depth_32f_stencil8, debugLabel: "Local Shadow Depth") + // A new atlas holds nothing worth keeping. + entry.faceDigests.removeAll() + entry.faceMatrixDigests.removeAll() + entry.sceneDigest = nil } guard let color = entry.color, let depth = entry.depth else { views.entries[source.entityId] = entry return } - let command = renderContext.commandQueue.makeCommandBuffer() - command.label = "Local Shadows 3D" - let pass = command.beginRenderPass( - .init( - label: "Local Shadows 3D", - colorAttachments: [.init(texture: color, operation: .init(loadAction: .clear, storeAction: .store), clearColor: .white)], - depthStencilAttachment: .init( - texture: depth, - depthOperation: .init(loadAction: .clear, storeAction: .dontCare), - stencilOperation: .init(loadAction: .clear, storeAction: .dontCare) - ) - ) - ) + + // Prepare every face and fingerprint it before encoding, so unchanged faces can be left alone. + struct Face { + var index: Int + var tile: Rect + var isClean: Bool + } + var faces: [Face] = [] + var digests: [Int: UInt64] = [:] + var matrixDigests: [Int: UInt64] = [:] + // With the casters unchanged since the faces were last prepared, a face whose matrix is also unchanged + // draws what it drew before, so its list needs no rebuilding at all. + let sceneDigest = settings.reusesStaticFaces + ? ShadowSceneDigest3D.compute(sources: candidates.meshes, usesLOD: environment.meshVisibility.lod) + : nil + let sceneUnchanged = sceneDigest != nil && sceneDigest == entry.sceneDigest for light in entry.lights { guard let slot = entry.allocator.slots[light.entity] else { continue } let matrices = LocalLighting3DMath.shadowProjections(light) @@ -120,8 +150,14 @@ public struct LocalShadow3DRenderNode: RenderNode { let index = slot * 6 + face entry.matrices[index] = matrices[face] let tile = Rect(x: Float(face * resolution), y: Float(slot * resolution), width: Float(resolution), height: Float(resolution)) - pass.setViewport(tile) - pass.setScissorRect(tile) + var matrixDigest = FrameDigest3D() + matrixDigest.combine(matrices[face]) + matrixDigests[index] = matrixDigest.value + if sceneUnchanged, let kept = entry.faceDigests[index], entry.faceMatrixDigests[index] == matrixDigest.value { + digests[index] = kept + faces.append(Face(index: index, tile: tile, isClean: true)) + continue + } // Light-volume culling is independent of the main camera list and its distance cuts. lists.prepare( sources: candidates.meshes, @@ -134,36 +170,88 @@ public struct LocalShadow3DRenderNode: RenderNode { device: device.renderDevice, materials: &_materials.wrappedValue ) - let buffers = lists.buffers(view: source.entityId, pass: 100 + index) - guard let instances = buffers.instances else { continue } - let constants = scratch.cache.write( - DirectionalShadowViewUniform(viewProjection: matrices[face], tileBounds: [tile.minX, tile.minY, tile.maxX, tile.maxY]), - view: source.entityId, - pass: index, - device: device.renderDevice + var digest: UInt64? + if settings.reusesStaticFaces, let content = lists.shadowDigest(view: source.entityId, pass: 100 + index) { + var combined = FrameDigest3D() + combined.combine(matrices[face]) + combined.combine(content) + digest = combined.value + digests[index] = combined.value + } + faces.append(Face(index: index, tile: tile, isClean: digest != nil && entry.faceDigests[index] == digest)) + } + } + entry.sceneDigest = sceneDigest + entry.faceMatrixDigests = matrixDigests + entry.faceDigests = digests + entry.reusedFaces = faces.filter(\.isClean).count + entry.renderedFaces = faces.count - entry.reusedFaces + guard entry.renderedFaces > 0 else { + // Every face already holds what it would draw again. + statistics.record(view: source.entityId, scene: _lighting.wrappedValue, entry: entry) + views.entries[source.entityId] = entry + return + } + + let keepsContents = entry.reusedFaces > 0 + let command = renderContext.commandQueue.makeCommandBuffer() + command.label = "Local Shadows 3D" + let pass = command.beginRenderPass( + .init( + label: "Local Shadows 3D", + colorAttachments: [.init(texture: color, operation: .init(loadAction: keepsContents ? .load : .clear, storeAction: .store), clearColor: .white)], + depthStencilAttachment: .init( + texture: depth, + depthOperation: .init(loadAction: .clear, storeAction: .dontCare), + stencilOperation: .init(loadAction: .clear, storeAction: .dontCare) ) - pass.setVertexBuffer(constants, offset: 0, slot: GlobalBufferIndex.viewUniform) - for item in lists.items(view: source.entityId, pass: 100 + index) { - guard item.castShadows, let range = item.batchRange, (item.material as? PBRMaterial)?.alphaMode != .blend else { continue } - let part = item.mesh.models[item.modelIndex].parts[item.partIndex] - guard let material = materials.buffer(for: item.material), let palette = item.skinningBuffer ?? skinning.identityBuffer else { continue } - pass.setRenderPipelineState(pipelines.pipeline(for: part.vertexDescriptor, device: device.renderDevice)) - pass.setVertexBuffer(part.vertexBuffer, offset: 0, slot: 0) - pass.setVertexBuffer(instances, offset: Int(range.lowerBound) * MemoryLayout.stride, slot: 3) - if let defaults = buffers.defaults { pass.setVertexBuffer(defaults, offset: 0, slot: 4) } - let base = (item.material as? PBRMaterial)?.baseColorTexture ?? Texture2D.whiteTexture - pass.setResourceSet( - .init(bindings: [ - .init(binding: 15, shaderStages: .fragment, resource: .uniformBuffer(material, offset: 0)), - .init(binding: 4, shaderStages: .fragment, resource: .texture(base)), - .init(binding: 7, shaderStages: .fragment, resource: .sampler(base.sampler)), - .init(binding: Skinning3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(palette, offset: 0)), - ]), - index: 0 - ) - pass.setIndexBuffer(part.indexBuffer, offset: 0) - pass.drawIndexed(indexCount: part.indexCount, indexBufferOffset: 0, instanceCount: Int(range.count)) + ) + ) + for face in faces where !face.isClean { + let index = face.index + let tile = face.tile + pass.setViewport(tile) + pass.setScissorRect(tile) + if keepsContents { + pass.setRenderPipelineState(clear.pipeline) + pass.draw(type: .triangle, vertexStart: 0, vertexCount: 3, instanceCount: 1) + } + let buffers = lists.buffers(view: source.entityId, pass: 100 + index) + guard let instances = buffers.instances else { continue } + let constants = scratch.cache.write( + DirectionalShadowViewUniform(viewProjection: entry.matrices[index], tileBounds: [tile.minX, tile.minY, tile.maxX, tile.maxY]), + view: source.entityId, + pass: index, + device: device.renderDevice + ) + pass.setVertexBuffer(constants, offset: 0, slot: GlobalBufferIndex.viewUniform) + for item in lists.items(view: source.entityId, pass: 100 + index) { + guard item.castShadows, let range = item.batchRange, (item.material as? PBRMaterial)?.alphaMode != .blend else { continue } + let part = item.mesh.models[item.modelIndex].parts[item.partIndex] + guard let material = materials.buffer(for: item.material), let palette = item.skinningBuffer ?? skinning.identityBuffer else { continue } + let morphed = item.morphBuffer != nil && part.morphTargets != nil + pass.setRenderPipelineState(pipelines.pipeline(for: Shadow3DConfiguration(vertex: part.vertexDescriptor, morph: morphed), device: device.renderDevice)) + if morphed, let morphBuffer = item.morphBuffer, let morphSet = part.morphTargets { + pass.setResourceSet(RenderResourceSet(bindings: [ + .init(binding: Morph3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(morphBuffer, offset: 0)), + .init(binding: Morph3DUniforms.textureBinding, shaderStages: .vertex, resource: .texture(morphSet.deltaTexture)), + ]), index: 0) } + pass.setVertexBuffer(part.vertexBuffer, offset: 0, slot: 0) + pass.setVertexBuffer(instances, offset: Int(range.lowerBound) * MemoryLayout.stride, slot: 3) + if let defaults = buffers.defaults { pass.setVertexBuffer(defaults, offset: 0, slot: 4) } + let base = (item.material as? PBRMaterial)?.baseColorTexture ?? Texture2D.whiteTexture + pass.setResourceSet( + .init(bindings: [ + .init(binding: 15, shaderStages: .fragment, resource: .uniformBuffer(material, offset: 0)), + .init(binding: 4, shaderStages: .fragment, resource: .texture(base)), + .init(binding: 7, shaderStages: .fragment, resource: .sampler(base.sampler)), + .init(binding: Skinning3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(palette, offset: 0)), + ]), + index: 0 + ) + pass.setIndexBuffer(part.indexBuffer, offset: 0) + pass.drawIndexed(indexCount: part.indexCount, indexBufferOffset: 0, instanceCount: Int(range.count)) } } pass.endRenderPass() diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/Main3DRenderNode.swift b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/Main3DRenderNode.swift index f00942088..5459549ef 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/Main3DRenderNode.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/Main3DRenderNode.swift @@ -42,6 +42,8 @@ public struct Main3DRenderNode: RenderNode { @ResMut private var localUniforms + @ResMut private var lightTiles + @ResMut private var lightingScratch @@ -71,6 +73,7 @@ public struct Main3DRenderNode: RenderNode { _lightingScratch.update(from: world) _viewUniforms.update(from: world) _localUniforms.update(from: world) + _lightTiles.update(from: world) _environments.update(from: world) _ibl.update(from: world) _renderDevice.update(from: world) @@ -79,6 +82,7 @@ public struct Main3DRenderNode: RenderNode { lightingScratch.cache.retainViews(activeViews) viewUniforms.cache.retainViews(activeViews) ibl.cache.retainViews(activeViews) + lightTiles.retainViews(activeViews) lists.retainViews(activeViews) gpuVisibility.retainViews(activeViews) } @@ -187,6 +191,29 @@ public struct Main3DRenderNode: RenderNode { } let localTexture = localShadows?.color ?? Texture2D.whiteTexture + // Tiles let the main pass skip local lights that cannot reach a pixel. Foveated rendering maps pixels + // through a rate map the CPU bounds do not model, so it keeps the flat loop. + let localLights = localShadows?.lights ?? [] + let tileLayout = LightTiles3DMath.layout(width: sceneColor.size.width, height: sceneColor.size.height) + let tilesActive = (environments.environments[source.entityId]?.tiledLocalLights ?? true) && camera.rasterizationRateMap == nil + let tileMasks = tilesActive + ? LightTiles3DMath.masks( + spheres: localLights.map { ((uniform.viewMatrix * Vector4($0.position, 1)).xyz, $0.range) }, + projection: drawUniform.projectionMatrix, + viewport: camera.viewport.rect, + layout: tileLayout + ) + : [] + let tileConstants = localUniforms.write(view: source.entityId, pass: 2, count: 6, device: renderDevice.renderDevice) { + $0[0] = drawUniform.projectionMatrix.x + $0[1] = drawUniform.projectionMatrix.y + $0[2] = drawUniform.projectionMatrix.z + $0[3] = drawUniform.projectionMatrix.w + $0[4] = [camera.viewport.rect.origin.x, camera.viewport.rect.origin.y, camera.viewport.rect.size.width, camera.viewport.rect.size.height] + $0[5] = [Float(tileLayout.tileSize), Float(tileLayout.columns), Float(tileLayout.rows), tilesActive ? 1 : 0] + } + let tileMaskBuffer = lightTiles.write(view: source.entityId, masks: tileMasks, device: renderDevice.renderDevice) + let commandBuffer = renderContext.commandQueue.makeCommandBuffer() commandBuffer.label = "Main 3d Render Pass" @@ -233,6 +260,8 @@ public struct Main3DRenderNode: RenderNode { renderPass.setFragmentBuffer(lightConstants, offset: 0, slot: 1) renderPass.setFragmentBuffer(localConstants, offset: 0, slot: 25) renderPass.setFragmentBuffer(localMatrices, offset: 0, slot: 28) + renderPass.setFragmentBuffer(tileConstants, offset: 0, slot: 27) + renderPass.setFragmentBuffer(tileMaskBuffer, offset: 0, slot: 29) renderPass.setResourceSet( RenderResourceSet(bindings: [ .init(binding: 26, shaderStages: .fragment, resource: .texture(localTexture)) diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/ScreenSpaceReflectionRenderNode.swift b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/ScreenSpaceReflectionRenderNode.swift index 8eb416332..53347c314 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/ScreenSpaceReflectionRenderNode.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/ScreenSpaceReflectionRenderNode.swift @@ -19,6 +19,9 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { @Res private var environments + @Res + private var lighting + @Res private var pipeline @@ -33,6 +36,7 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { @Res private var metrics @Res private var overlays @Res private var bloomViews + @Res private var volumetricViews @Res private var autoExposureViews public init(notifiesCompletion: Bool = true) { @@ -43,9 +47,11 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { _metrics.update(from: world) _overlays.update(from: world) _bloomViews.update(from: world) + _volumetricViews.update(from: world) _autoExposureViews.update(from: world) query.update(from: world) _environments.update(from: world) + _lighting.update(from: world) _pipeline.update(from: world) _scratch.update(from: world) _renderDevice.update(from: world) @@ -82,6 +88,12 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { let skybox = environment.skybox let reflection = environment.screenSpaceReflection let skyIntensity = max(0, skybox.intensity) + let fog = environment.fog.sanitized + let fogActive = environment.fog.changesImage + let shafts = fogActive && environment.fog.volumetricActive ? volumetricViews.output(view: source.entityId) : nil + let sun = lighting.directionalLight + let fogSunDirection = sun?.directionToLight.normalized ?? Vector3(0, 1, 0) + let fogSunRadiance = sun.map { $0.radiance * $0.intensity } ?? .zero let uniform = Environment3DUniform( projection: jitteredUniform.projectionMatrix, inverseProjection: jitteredUniform.projectionMatrix.inverse, @@ -138,7 +150,12 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { environment.postProcess.vignette.sanitized.intensity, environment.postProcess.vignette.sanitized.radius ), - vignette: Vector4(environment.postProcess.vignette.sanitized.smoothness, autoExposureTexture == nil ? 0 : 1, 0, 0) + vignette: Vector4(environment.postProcess.vignette.sanitized.smoothness, autoExposureTexture == nil ? 0 : 1, 0, 0), + fogColor: Vector4(fog.color.asVector.xyz, fogActive ? fog.density : 0), + fogShape: Vector4(fog.heightFalloff, fog.baseHeight, fog.startDistance, fog.maximumOpacity), + fogRange: Vector4(fog.skyInfluence, fog.sunExponent, shafts == nil ? 0 : 1, 0), + fogSun: Vector4(fogSunDirection, fogActive && shafts == nil ? fog.sunInscattering : 0), + fogSunColor: Vector4(fogSunRadiance, 0) ) let constants = scratch.cache.write(uniform, view: source.entityId, device: renderDevice.renderDevice) @@ -166,6 +183,7 @@ public struct ScreenSpaceReflectionRenderNode: RenderNode { .init(binding: 9, shaderStages: .fragment, resource: .texture(target.ambientOcclusion3DTexture ?? Texture2D.whiteTexture)), .init(binding: 11, shaderStages: .fragment, resource: .sampler(pipeline.nearest)), .init(binding: 12, shaderStages: .fragment, resource: .texture(bloomTexture ?? Texture2D.whiteTexture)), + .init(binding: 16, shaderStages: .fragment, resource: .texture(shafts ?? Texture2D.whiteTexture)), .init(binding: 13, shaderStages: .fragment, resource: .texture(gradingLUT?.texture ?? Texture2D.whiteTexture)), .init(binding: 14, shaderStages: .fragment, resource: .sampler(pipeline.sampler)), .init(binding: 15, shaderStages: .fragment, resource: .texture(autoExposureTexture ?? Texture2D.whiteTexture)), diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/VolumetricFog3DRenderNode.swift b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/VolumetricFog3DRenderNode.swift new file mode 100644 index 000000000..b6528a7d0 --- /dev/null +++ b/Sources/AdaCorePipelines/CorePipelines/RenderNodes3D/VolumetricFog3DRenderNode.swift @@ -0,0 +1,185 @@ +import AdaAssets +import AdaECS +@_spi(Internal) import AdaRender +import AdaUtils +import Math + +/// Marches the primary directional light through the fog medium at half resolution and stores the in-scattered light. +/// +/// Runs after the main pass and before the environment composite, which adds the result to the frame in linear HDR. +/// A camera without active volumetric fog, or one showing a full-screen debug view, records no GPU work and holds +/// no texture. +public struct VolumetricFog3DRenderNode: RenderNode { + public static let name: RenderNodeLabel = "Main3D.VolumetricFog" + + @Query private var views + @Res private var environments + @Res private var lighting + @ResMut private var volumetric + @Res private var device + @Res private var metrics + + public init() {} + + public func update(from world: World) { + views.update(from: world) + _environments.update(from: world) + _lighting.update(from: world) + _volumetric.update(from: world) + _device.update(from: world) + _metrics.update(from: world) + var activeViews: [Entity.ID] = [] + views.forEach { _, _, _, _, source in activeViews.append(source.entityId) } + volumetric.retainViews(activeViews) + } + + public func execute(context: inout Context, renderContext: RenderContext) async throws -> [RenderSlotValue] { + guard let view = context.viewEntity else { + return [] + } + let shadow = context.world.getResource(DirectionalShadow3D.self) + var submitted: CommandBuffer? + views.forEach { entity, camera, target, viewUniform, source in + guard entity == view else { + return + } + guard let environment = environments.environments[source.entityId], environment.fog.volumetricActive, + !environment.debugView.replacesFinalImage, let sun = lighting.directionalLight, + target.rendering3DUsesEnvironmentTargets, let position = target.viewPositionMetallic3DTexture + else { + volumetric.discard(view: source.entityId) + return + } + let fog = environment.fog.sanitized + let settings = fog.volumetric + let output = volumetric.output(view: source.entityId, fullSize: position.size, scaleFactor: position.scaleFactor) + let shadowsEnabled = shadow.map { $0.isEnabled && sun.castsShadows && $0.colorTexture != nil } ?? false + let atlas = shadow?.colorTexture?.size ?? SizeInt(width: 1024, height: 1024) + let jittered = temporalViewUniform(viewUniform, target: target) + let constants = volumetric.uniform.write( + VolumetricFog3DUniform( + inverseView: viewUniform.viewMatrix.inverse, + inverseProjection: jittered.projectionMatrix.inverse, + shadowViewProjection0: shadow?.viewProjection ?? .identity, + shadowViewProjection1: shadow?.viewProjection1 ?? .identity, + shadowViewProjection2: shadow?.viewProjection2 ?? .identity, + cascadeSplits: shadow?.cascadeSplits ?? [1000, 1000, 1000, 1], + shadowAtlas: Vector4( + 1 / Float(max(atlas.width, 1)), + 1 / Float(max(atlas.height, 1)), + max(0, sun.shadowBias), + shadowsEnabled ? 1 : 0 + ), + sunDirection: Vector4(sun.directionToLight.normalized, settings.anisotropy), + sunRadiance: Vector4(sun.radiance * (sun.intensity * settings.intensity), 0), + medium: Vector4(fog.density, fog.heightFalloff, fog.baseHeight, fog.startDistance), + march: Vector4(Float(settings.steps), settings.maxDistance, 0, 0) + ), + view: source.entityId, + device: device.renderDevice + ) + let command = renderContext.commandQueue.makeCommandBuffer() + command.label = "Volumetric Fog 3D" + let pass = command.beginRenderPass(RenderPassDescriptor( + label: "Volumetric Fog 3D", + colorAttachments: [.init(texture: output, operation: .init(loadAction: .clear, storeAction: .store), clearColor: .black)], + depthStencilAttachment: nil + )) + pass.setViewport(Rect(x: 0, y: 0, width: Float(output.size.width), height: Float(output.size.height))) + pass.setRenderPipelineState(volumetric.pipeline) + pass.setFragmentBuffer(constants, offset: 0, slot: 4) + pass.setResourceSet(RenderResourceSet(bindings: [ + .init(binding: 0, shaderStages: .fragment, resource: .texture(position)), + .init(binding: 1, shaderStages: .fragment, resource: .texture(shadow?.colorTexture ?? Texture2D.whiteTexture)), + .init(binding: 2, shaderStages: .fragment, resource: .sampler(volumetric.nearest)), + ]), index: 0) + pass.draw(type: .triangle, vertexStart: 0, vertexCount: 3, instanceCount: 1) + pass.endRenderPass() + if let metrics { + command.addCompletedTimingHandler { metrics.record(pass: "volumetricFog", seconds: $0) } + } + submitted = command + } + if let submitted { + if let metrics { await metrics.commit(submitted) } else { submitted.commit() } + } + return [] + } +} + +struct VolumetricFog3DUniform: Sendable { + var inverseView: Transform3D + var inverseProjection: Transform3D + var shadowViewProjection0: Transform3D + var shadowViewProjection1: Transform3D + var shadowViewProjection2: Transform3D + var cascadeSplits: Vector4 + var shadowAtlas: Vector4 + var sunDirection: Vector4 + var sunRadiance: Vector4 + var medium: Vector4 + var march: Vector4 +} + +/// The light-shaft pipeline and a half-resolution result texture per camera, created only while the effect runs. +struct VolumetricFog3DViews: Resource { + let pipeline: RenderPipeline + let nearest: Sampler + var uniform = FrameUniformCache3D() + private var textures: [Entity.ID: RenderTexture] = [:] + private var retired: [RenderTexture] = [] + + init(device: RenderDevice) { + let shader = CorePipelineShaders.loadRequiredBundled(at: "Shaders/volumetric_fog_3d.glsl") + var descriptor = RenderPipelineDescriptor( + vertex: shader.asset.requiredShader(for: .vertex), + fragment: shader.asset.getShader(for: .fragment), + debugName: "Volumetric Fog 3D", + backfaceCulling: false, + depthPixelFormat: .none + ) + descriptor.colorAttachments = [.init(format: .rgba_16f)] + pipeline = device.createRenderPipeline(from: descriptor) + nearest = device.createSampler(from: .init(minFilter: .nearest, magFilter: .nearest, mipFilter: .notMipmapped)) + } + + /// The light-shaft texture for `view` if the effect ran for it. + func output(view: Entity.ID) -> RenderTexture? { + textures[view] + } + + /// A texture at half the size of the scene targets, rebuilt when that size changes. + mutating func output(view: Entity.ID, fullSize: SizeInt, scaleFactor: Float) -> RenderTexture { + let size = SizeInt(width: max(1, fullSize.width / 2), height: max(1, fullSize.height / 2)) + if let existing = textures[view], existing.size == size { + return existing + } + if let replaced = textures[view] { + retire(replaced) + } + let texture = RenderTexture(size: size, scaleFactor: scaleFactor, format: .rgba_16f, debugLabel: "3D Volumetric Fog") + textures[view] = texture + return texture + } + + mutating func retainViews(_ views: [Entity.ID]) { + for view in textures.keys where !views.contains(view) { + discard(view: view) + } + uniform.retainViews(views) + } + + mutating func discard(view: Entity.ID) { + if let removed = textures.removeValue(forKey: view) { + retire(removed) + } + } + + private mutating func retire(_ texture: RenderTexture) { + retired.append(texture) + let limit = max(1, unsafe RenderEngine.configurations.maxFramesInFlight) + 1 + if retired.count > limit { + retired.removeFirst(retired.count - limit) + } + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/DirectionalShadow3DPipeline.swift b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/DirectionalShadow3DPipeline.swift index 7f119381d..278c4ce06 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/DirectionalShadow3DPipeline.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/DirectionalShadow3DPipeline.swift @@ -2,16 +2,38 @@ import AdaAssets import AdaECS import AdaRender +/// What decides a shadow-pass pipeline: the part's vertex layout and whether its mesh has morph targets. +public struct Shadow3DConfiguration: Hashable, Sendable { + public let vertex: VertexDescriptor + public let morph: Bool + + public init(vertex: VertexDescriptor, morph: Bool = false) { + self.vertex = vertex + self.morph = morph + } +} + /// Pipeline that renders opaque mesh depth into the primary directional-light shadow map. public struct DirectionalShadow3DPipeline: RenderPipelineConfigurator { + public typealias Configuration = Shadow3DConfiguration + private let shader: AssetHandle + private let variants = PBR3DShaderVariants() public init() { self.shader = ShaderModule.loadRequiredBundled(at: "Shaders/directional_shadow_3d.glsl", from: .adaModule) } - public func configurate(with configuration: VertexDescriptor) -> RenderPipelineDescriptor { - var configuration = configuration + /// The pipeline for a part without morph targets. + public func configurate(with vertex: VertexDescriptor) -> RenderPipelineDescriptor { + configurate(with: Shadow3DConfiguration(vertex: vertex)) + } + + public func configurate(with shadowConfiguration: Shadow3DConfiguration) -> RenderPipelineDescriptor { + let shader = shadowConfiguration.morph + ? variants.shader(at: "Shaders/directional_shadow_3d.glsl", for: .morph, fallback: self.shader) + : self.shader + var configuration = shadowConfiguration.vertex configureSkinningAttributes(&configuration) if !configuration.attributes.containsAttribute(by: MeshDescriptor.textureCoordinates.id.name) { configuration.attributes[2] = .attribute(.vector2, name: "defaultTextureCoordinate", bufferIndex: 4, offset: 0) diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Flat3DPipeline.swift b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Flat3DPipeline.swift index 101d9913c..67816822e 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Flat3DPipeline.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Flat3DPipeline.swift @@ -9,6 +9,7 @@ import AdaAssets import AdaECS import AdaRender import AdaUtils +import Foundation import Math /// Per-instance data consumed by the flat 3D vertex pipeline. @@ -158,12 +159,19 @@ public struct Opaque3DInstanceBuffers: Resource { /// Pipeline configurator for a simple unlit 3D mesh pass. public struct Flat3DPipeline: RenderPipelineConfigurator { private let shader: AssetHandle + private let variants = PBR3DShaderVariants() public init() { self.shader = ShaderModule.loadRequiredBundled(at: "Shaders/flat3d.glsl", from: .adaModule) } public func configurate(with configuration: VertexDescriptor) -> RenderPipelineDescriptor { + configurate(with: configuration, features: []) + } + + /// A pipeline whose fragment shader only contains the shading layers in `features`. + public func configurate(with configuration: VertexDescriptor, features: PBRMaterial.ShaderFeatures) -> RenderPipelineDescriptor { + let shader = features.isEmpty ? shader : variants.shader(for: features, fallback: shader) var configuration = configuration if !configuration.attributes.containsAttribute(by: MeshDescriptor.textureCoordinates.id.name) { configuration.attributes[2] = .attribute(.vector2, name: "defaultTextureCoordinate", bufferIndex: 4, offset: 0) @@ -194,7 +202,7 @@ public struct Flat3DPipeline: RenderPipelineConfigurator { var descriptor = RenderPipelineDescriptor(vertex: shader.asset.requiredShader(for: .vertex)) descriptor.fragment = shader.asset.getShader(for: .fragment) - descriptor.debugName = "Flat 3D Pipeline" + descriptor.debugName = features.isEmpty ? "Flat 3D Pipeline" : "Flat 3D Pipeline \(features.rawValue)" descriptor.vertexDescriptor = configuration #if os(Android) descriptor.frontFaceWinding = .clockwise @@ -213,3 +221,38 @@ public struct Flat3DPipeline: RenderPipelineConfigurator { return descriptor } } + + +/// Compiled shader variants by file and the features they include. A variant is compiled on first use and shared by every +/// pipeline that needs it; if it fails to compile, the plain shader is used instead. +final class PBR3DShaderVariants: @unchecked Sendable { + private struct Key: Hashable { + let path: String + let features: PBRMaterial.ShaderFeatures + } + + private let lock = NSLock() + private var variants: [Key: AssetHandle] = [:] + + func shader( + at path: String = "Shaders/flat3d.glsl", + for features: PBRMaterial.ShaderFeatures, + fallback: AssetHandle + ) -> AssetHandle { + lock.lock() + defer { lock.unlock() } + let key = Key(path: path, features: features) + if let existing = variants[key] { + return existing + } + let variant: AssetHandle + do { + variant = AssetHandle(try ShaderModule.createBundled(at: path, from: .adaModule, defines: features.enablingMacros)) + } catch { + assertionFailure("Shader variant \(features.rawValue) of \(path) failed to compile: \(error)") + variant = fallback + } + variants[key] = variant + return variant + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Motion3DPipeline.swift b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Motion3DPipeline.swift index 7d44c4996..ba7972111 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Motion3DPipeline.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/Motion3DPipeline.swift @@ -12,10 +12,14 @@ public struct ActiveMotion3DPass: Resource { /// Motion and reactive mask share the main pass's instance data, UVs and alpha tests. public struct Motion3DPipeline: RenderPipelineConfigurator { private let base = Flat3DPipeline() + private let variants = PBR3DShaderVariants() public init() {} public func configurate(with configuration: PBR3DConfiguration) -> RenderPipelineDescriptor { - let shader = CorePipelineShaders.loadRequiredBundled(at: "Shaders/motion_3d.glsl") + let plain = CorePipelineShaders.loadRequiredBundled(at: "Shaders/motion_3d.glsl") + // Only the morph feature matters to the motion shader; the shading layers do not. + let morph = configuration.features.intersection(.morph) + let shader = morph.isEmpty ? plain : variants.shader(at: "Shaders/motion_3d.glsl", for: morph, fallback: plain) var descriptor = base.configurate(with: configuration.vertex) descriptor.vertex = shader.asset.requiredShader(for: .vertex) descriptor.fragment = shader.asset.getShader(for: .fragment) diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/PBR3DPipeline.swift b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/PBR3DPipeline.swift index 8faa49b7c..3f1a6d0dd 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/PBR3DPipeline.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/PBR3DPipeline.swift @@ -5,8 +5,11 @@ public struct PBR3DConfiguration: Hashable, Sendable { public let vertex: VertexDescriptor public let blended: Bool public let temporalDepth: Bool + /// The shading layers the draw's material uses. The default is none, which is the plain shader. + public let features: PBRMaterial.ShaderFeatures - public init(vertex: VertexDescriptor, blended: Bool, temporalDepth: Bool = false) { + public init(vertex: VertexDescriptor, blended: Bool, temporalDepth: Bool = false, features: PBRMaterial.ShaderFeatures = []) { + self.features = features self.temporalDepth = temporalDepth self.vertex = vertex self.blended = blended @@ -19,7 +22,7 @@ public struct PBR3DPipeline: RenderPipelineConfigurator { public init() {} public func configurate(with configuration: PBR3DConfiguration) -> RenderPipelineDescriptor { - var descriptor = base.configurate(with: configuration.vertex) + var descriptor = base.configurate(with: configuration.vertex, features: configuration.features) descriptor.backfaceCulling = false if configuration.temporalDepth { descriptor.depthPixelFormat = .depth_32f } descriptor.depthStencilDescriptor?.isDepthWriteEnabled = !configuration.blended diff --git a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/ScreenSpaceReflectionPipeline.swift b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/ScreenSpaceReflectionPipeline.swift index 600f15c75..e17e2b326 100644 --- a/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/ScreenSpaceReflectionPipeline.swift +++ b/Sources/AdaCorePipelines/CorePipelines/RenderPipelines3D/ScreenSpaceReflectionPipeline.swift @@ -59,6 +59,11 @@ struct Environment3DUniform: Sendable { var postProcess: Vector4 var grading: Vector4 var vignette: Vector4 + var fogColor: Vector4 + var fogShape: Vector4 + var fogRange: Vector4 + var fogSun: Vector4 + var fogSunColor: Vector4 } public struct ScreenSpaceReflectionScratch: Resource, Sendable { diff --git a/Sources/AdaCorePipelines/CorePipelines/ShadowSceneDigest3D.swift b/Sources/AdaCorePipelines/CorePipelines/ShadowSceneDigest3D.swift new file mode 100644 index 000000000..add79eb60 --- /dev/null +++ b/Sources/AdaCorePipelines/CorePipelines/ShadowSceneDigest3D.swift @@ -0,0 +1,35 @@ +@_spi(Internal) import AdaRender +import Math + +/// A fingerprint of every shadow caster in the extracted scene, used to skip rebuilding per-face draw lists when +/// nothing a shadow face could draw has changed. +enum ShadowSceneDigest3D { + /// The digest of the casters in `sources`, or `nil` when the scene cannot be summarized cheaply: a skinned or morphed + /// caster changes shape without any change visible here, and with level-of-detail on, which mesh a caster draws + /// depends on the camera. + static func compute(sources: [Mesh3DRenderSource], usesLOD: Bool) -> UInt64? { + var digest = FrameDigest3D() + for index in sources.indices where sources[index].castShadows { + if sources[index].skinningBuffer != nil || !sources[index].morphBuffers.isEmpty || (usesLOD && !sources[index].alternatives.isEmpty) { + return nil + } + digest.combine(sources[index].transform) + digest.combine(UInt64(sources[index].forceVisible ? 1 : 0)) + for model in sources[index].mesh.models { + for part in model.parts { + digest.combine(UInt64(UInt(bitPattern: ObjectIdentifier(part.vertexBuffer)))) + digest.combine(UInt64(part.indexCount)) + digest.combine(UInt64(part.materialIndex)) + } + } + for material in sources[index].materials { + digest.combine(UInt64(UInt(bitPattern: ObjectIdentifier(material)))) + if let pbr = material as? PBRMaterial { + digest.combine(UInt64(truncatingIfNeeded: pbr.uniformRevision)) + digest.combine(UInt64(pbr.baseColorTexture.map { UInt(bitPattern: ObjectIdentifier($0)) } ?? 0)) + } + } + } + return digest.value + } +} diff --git a/Sources/AdaCorePipelines/CorePipelines/VisibleMesh3DLists.swift b/Sources/AdaCorePipelines/CorePipelines/VisibleMesh3DLists.swift index 1b7da1924..171811aed 100644 --- a/Sources/AdaCorePipelines/CorePipelines/VisibleMesh3DLists.swift +++ b/Sources/AdaCorePipelines/CorePipelines/VisibleMesh3DLists.swift @@ -12,6 +12,32 @@ public struct Active3DInstanceBuffers: Resource { } /// Compact, frame-buffered draw lists per camera and per shadow cascade. +/// A cheap rolling hash used to tell whether a shadow face would draw the same thing as last frame. +struct FrameDigest3D: Sendable { + private(set) var value: UInt64 = 0xcbf2_9ce4_8422_2325 + + mutating func combine(_ word: UInt64) { + value = (value ^ word) &* 0x0000_0100_0000_01b3 + value ^= value >> 29 + } + + mutating func combine(_ float: Float) { combine(UInt64(float.bitPattern)) } + + mutating func combine(_ vector: Vector4) { + combine(vector.x) + combine(vector.y) + combine(vector.z) + combine(vector.w) + } + + mutating func combine(_ matrix: Transform3D) { + combine(matrix.x) + combine(matrix.y) + combine(matrix.z) + combine(matrix.w) + } +} + public struct VisibleMesh3DLists: Resource { private struct Key: Hashable { var view: Entity.ID @@ -24,6 +50,8 @@ public struct VisibleMesh3DLists: Resource { var count = Visibility3DCount() var records: [Record] = [] var order: [Int32] = [] + var digest = FrameDigest3D() + var isDynamic = false } private struct SelectionKey: Hashable { @@ -37,6 +65,7 @@ public struct VisibleMesh3DLists: Resource { var material: MaterialKey var descriptor: Int32 var skin: ObjectIdentifier? + var morph: ObjectIdentifier? var cast: Bool var receive: Bool } @@ -58,6 +87,8 @@ public struct VisibleMesh3DLists: Resource { var blended: Bool /// Batch order, assigned when the first record of this batch is appended. var rank: Int + /// Hash of the batch key, which names the mesh part, material parameters and skin of every instance in it. + var batchHash: UInt64 = 0 } /// A visible part, kept free of references so building, ordering and reading thousands of them costs no @@ -73,6 +104,8 @@ public struct VisibleMesh3DLists: Resource { } private var entries: [Key: Entry] = [:] + /// Passes prepared since creation, for tests that check which lists a frame rebuilt. + private(set) var preparedPassCount = 0 private var history: [SelectionKey: Int] = [:] public init() {} @@ -92,6 +125,7 @@ public struct VisibleMesh3DLists: Resource { device: RenderDevice, materials: inout PBR3DUniforms ) { + preparedPassCount &+= 1 let key = Key(view: view, pass: pass) var entry = entries.removeValue(forKey: key) ?? Entry() entry.items.removeAll(keepingCapacity: true) @@ -100,6 +134,8 @@ public struct VisibleMesh3DLists: Resource { entry.order.removeAll(keepingCapacity: true) entry.candidates.removeAll(keepingCapacity: true) entry.count = Visibility3DCount() + entry.digest = FrameDigest3D() + entry.isDynamic = false let cameraPosition = viewMatrix.inverse.origin let frustum = MeshVisibilityFrustum3D(viewProjection: cullingProjection) let isShadowPass = pass >= 0 @@ -189,6 +225,7 @@ public struct VisibleMesh3DLists: Resource { material: materialKey, descriptor: Int32(descriptorID), skin: sources[sourceIndex].skinningBuffer.map(ObjectIdentifier.init), + morph: sources[sourceIndex].morphBuffers[Mesh3DRenderSource.morphKey(model: modelIndex, part: partIndex)].map(ObjectIdentifier.init), cast: sources[sourceIndex].castShadows, receive: sources[sourceIndex].receiveShadows ) @@ -200,6 +237,7 @@ public struct VisibleMesh3DLists: Resource { if !isShadowPass { materials.write(material: material, descriptor: descriptor, device: device) } template = Int32(templates.count) templateIndex[batchKey] = template + let templateHash = UInt64(bitPattern: Int64(batchKey.hashValue)) templates.append(Self.makeTemplate( material: material, descriptor: descriptor, @@ -207,6 +245,7 @@ public struct VisibleMesh3DLists: Resource { blended: pbr?.alphaMode == .blend, shadowOnly: isShadowPass )) + templates[Int(template)].batchHash = templateHash } if templates[Int(template)].rank < 0 { templates[Int(template)].rank = rankCount @@ -249,6 +288,14 @@ public struct VisibleMesh3DLists: Resource { let template = templates[Int(record.template)] var shadowFlags = template.shadowFlags shadowFlags.w = 1 - record.fade + if isShadowPass { + // A skinned pose changes without any CPU-visible identity change, so such a face is never reusable. + if sources[sourceIndex].skinningBuffer != nil || !sources[sourceIndex].morphBuffers.isEmpty { entry.isDynamic = true } + entry.digest.combine(template.batchHash) + entry.digest.combine(sources[sourceIndex].transform) + entry.digest.combine((sources[sourceIndex].materials[Int(record.material)] as? PBRMaterial)?.baseColorFactor ?? .one) + entry.digest.combine(shadowFlags.w) + } let instanceIndex = entry.buffers.append(Flat3DInstanceData( modelMatrix: sources[sourceIndex].transform, // Shadow batches can mix materials that only differ in colour, so that comes from the record itself. @@ -276,7 +323,8 @@ public struct VisibleMesh3DLists: Resource { castShadows: sources[sourceIndex].castShadows, receiveShadows: sources[sourceIndex].receiveShadows, batchRange: instanceIndex..<(instanceIndex + 1), - skinningBuffer: sources[sourceIndex].skinningBuffer + skinningBuffer: sources[sourceIndex].skinningBuffer, + morphBuffer: sources[sourceIndex].morphBuffers[Mesh3DRenderSource.morphKey(model: Int(record.model), part: Int(record.part))] ) ) } @@ -298,6 +346,18 @@ public struct VisibleMesh3DLists: Resource { entries[key] = entry } + /// Fingerprint of everything a shadow pass would draw (batches, transforms, alpha inputs), or `nil` when the pass + /// contains skinned meshes whose pose changes without any visible change here. Equal fingerprints mean an + /// identical face. Meshes edited in place keep their buffer identity and are not noticed. + func shadowDigest(view: Entity.ID, pass: Int) -> UInt64? { + guard let entry = entries[Key(view: view, pass: pass)], !entry.isDynamic else { + return nil + } + var digest = entry.digest + digest.combine(UInt64(entry.items.count)) + return digest.value + } + public func gpuCandidates(view: Entity.ID, pass: Int) -> [GPUVisibilityCandidate] { entries[Key(view: view, pass: pass)]?.candidates ?? [] } public func items(view: Entity.ID, pass: Int) -> [Opaque3DRenderItem] { entries[Key(view: view, pass: pass)]?.items ?? [] } diff --git a/Sources/AdaRender/Assets/Shaders/Public/Morph.glsl b/Sources/AdaRender/Assets/Shaders/Public/Morph.glsl new file mode 100644 index 000000000..1fe928f72 --- /dev/null +++ b/Sources/AdaRender/Assets/Shaders/Public/Morph.glsl @@ -0,0 +1,54 @@ +// Morph targets (glTF blend shapes) applied to a vertex before skinning. +// +// Compiled in only for draws of morphed meshes, which define ADA_MORPH; otherwise this file declares nothing. The deltas of +// the few active targets are fetched from a half-float texture (see MorphTargetSet), so the cost depends on the number of +// active targets, not the number of targets the mesh has. +#ifdef ADA_MORPH +layout (binding = 21) uniform Morph3DUniform { + vec4 u_MorphCounts; // x active targets, y previous frame's active targets, z vertices per target, w attributes per vertex + vec4 u_MorphFlags; // x bit 0: normal deltas present, bit 1: tangent deltas present + vec4 u_MorphCurrent[8]; // x target index, y weight + vec4 u_MorphPrevious[8]; // the previous frame's, for motion vectors +}; +layout (binding = 40) uniform texture2D u_MorphDeltas; + +const int MORPH_TEXTURE_WIDTH = 2048; + +vec3 morphDelta(int morphTarget, int channel) { + int texel = (morphTarget * int(u_MorphCounts.z) + gl_VertexIndex) * int(u_MorphCounts.w) + channel; + return texelFetch(u_MorphDeltas, ivec2(texel % MORPH_TEXTURE_WIDTH, texel / MORPH_TEXTURE_WIDTH), 0).xyz; +} + +// Adds the weighted position deltas of the active targets. +void applyMorphPosition(inout vec3 position) { + for (int i = 0; i < 8; ++i) { + if (i >= int(u_MorphCounts.x)) { break; } + position += morphDelta(int(u_MorphCurrent[i].x), 0) * u_MorphCurrent[i].y; + } +} + +// The same for the previous frame's weights. +vec3 previousMorphedPosition(vec3 position) { + for (int i = 0; i < 8; ++i) { + if (i >= int(u_MorphCounts.y)) { break; } + position += morphDelta(int(u_MorphPrevious[i].x), 0) * u_MorphPrevious[i].y; + } + return position; +} + +// Adds the weighted position, normal and tangent deltas of the active targets. +void applyMorph(inout vec3 position, inout vec3 normal, inout vec3 tangent) { + int flags = int(u_MorphFlags.x); + bool hasNormals = (flags & 1) != 0; + bool hasTangents = (flags & 2) != 0; + int tangentChannel = hasNormals ? 2 : 1; + for (int i = 0; i < 8; ++i) { + if (i >= int(u_MorphCounts.x)) { break; } + int morphTarget = int(u_MorphCurrent[i].x); + float weight = u_MorphCurrent[i].y; + position += morphDelta(morphTarget, 0) * weight; + if (hasNormals) { normal += morphDelta(morphTarget, 1) * weight; } + if (hasTangents) { tangent += morphDelta(morphTarget, tangentChannel) * weight; } + } +} +#endif diff --git a/Sources/AdaRender/Materials/PBRMaterial.swift b/Sources/AdaRender/Materials/PBRMaterial.swift index fe6fcc4df..ee145c7ee 100644 --- a/Sources/AdaRender/Materials/PBRMaterial.swift +++ b/Sources/AdaRender/Materials/PBRMaterial.swift @@ -9,6 +9,90 @@ import AdaAssets import AdaUtils import Math +/// Scales, then rotates, then offsets the UV coordinates a PBR texture is sampled with (glTF `KHR_texture_transform`). +public struct PBRTextureTransform: Sendable, Equatable { + public var offset: Vector2 + /// Counter-clockwise rotation in radians. + public var rotation: Float + public var scale: Vector2 + + public static let identity = PBRTextureTransform() + + public init(offset: Vector2 = .zero, rotation: Float = 0, scale: Vector2 = .one) { + self.offset = offset + self.rotation = rotation + self.scale = scale + } +} + +/// A texture read by one of the material's shading layers, with the UV set and transform it is read with. +public struct PBRTextureBinding: Sendable { + public var texture: Texture2D + /// UV set (0 or 1). + public var texCoord: Int + public var transform: PBRTextureTransform + /// Scale of the sampled value; only the clearcoat normal map uses it. + public var scale: Float + + public init(texture: Texture2D, texCoord: Int = 0, transform: PBRTextureTransform = .identity, scale: Float = 1) { + self.texture = texture + self.texCoord = texCoord + self.transform = transform + self.scale = scale + } +} + +/// The textures of the shading layers and of the specular extension, and where they bind in the main shader. +/// +/// They are read with the base color texture's sampler, because the main pass has no free sampler slots. +public enum PBRLayerTexture: Int, CaseIterable, Hashable, Sendable { + /// Clearcoat strength (red). + case clearcoat + /// Clearcoat roughness (green). + case clearcoatRoughness + /// Tangent-space normal of the clearcoat layer. + case clearcoatNormal + /// Sheen color (RGB, sRGB). + case sheenColor + /// Sheen roughness (alpha). + case sheenRoughness + /// Anisotropy direction (red, green) and strength (blue). + case anisotropy + /// Specular strength (alpha). + case specular + /// Specular color (RGB, sRGB). + case specularColor + + /// The texture binding in the main shader. + public var binding: Int { 32 + rawValue } + + /// The feature bit that selects the shader variant declaring this texture. + public var feature: PBRMaterial.ShaderFeatures { PBRMaterial.ShaderFeatures(rawValue: 1 << UInt32(3 + rawValue)) } + + var macro: String { + switch self { + case .clearcoat: "ADA_CLEARCOAT_TEXTURE" + case .clearcoatRoughness: "ADA_CLEARCOAT_ROUGHNESS_TEXTURE" + case .clearcoatNormal: "ADA_CLEARCOAT_NORMAL_TEXTURE" + case .sheenColor: "ADA_SHEEN_COLOR_TEXTURE" + case .sheenRoughness: "ADA_SHEEN_ROUGHNESS_TEXTURE" + case .anisotropy: "ADA_ANISOTROPY_TEXTURE" + case .specular: "ADA_SPECULAR_TEXTURE" + case .specularColor: "ADA_SPECULAR_COLOR_TEXTURE" + } + } + + /// The layer a texture belongs to, if it only matters while that layer is on. + fileprivate var layer: PBRMaterial.ShaderFeatures? { + switch self { + case .clearcoat, .clearcoatRoughness, .clearcoatNormal: .clearcoat + case .sheenColor, .sheenRoughness: .sheen + case .anisotropy: .anisotropy + case .specular, .specularColor: nil + } + } +} + /// A material that uses Physically Based Rendering (PBR) to define its appearance. public class PBRMaterial: Material, @unchecked Sendable { public var baseColorFactor: Vector4 = .one { didSet { uniformRevision &+= 1 } } @@ -46,6 +130,73 @@ public class PBRMaterial: Material, @unchecked Sendable { /// A negative value emits everywhere. Non-negative values fade emission /// out as direct illumination rises above this threshold. public var emissiveLightThreshold: Float = -1 + /// UV transforms for base color, metallic/roughness, normal, occlusion, and emission. + public var textureTransforms: [PBRTextureTransform] = Array(repeating: .identity, count: 5) { didSet { uniformRevision &+= 1 } } + /// Index of refraction of the dielectric layer. The default 1.5 gives the usual 4% reflectance at normal incidence. + public var ior: Float = 1.5 { didSet { uniformRevision &+= 1 } } + /// Strength of the dielectric specular reflection (`KHR_materials_specular`). + public var specularFactor: Float = 1 { didSet { uniformRevision &+= 1 } } + /// Tint of the dielectric specular reflection (`KHR_materials_specular`). + public var specularColorFactor: Vector3 = .one { didSet { uniformRevision &+= 1 } } + /// Strength of a clear lacquer layer over the surface (`KHR_materials_clearcoat`); 0 turns the layer off. + public var clearcoatFactor: Float = 0 { didSet { uniformRevision &+= 1 } } + /// Roughness of the clearcoat layer, independent of the base layer's roughness. + public var clearcoatRoughnessFactor: Float = 0 { didSet { uniformRevision &+= 1 } } + /// Color of a soft fabric sheen over the surface (`KHR_materials_sheen`); black turns the layer off. + public var sheenColorFactor: Vector3 = .zero { didSet { uniformRevision &+= 1 } } + /// Roughness of the sheen layer. + public var sheenRoughnessFactor: Float = 0 { didSet { uniformRevision &+= 1 } } + /// How much highlights stretch (`KHR_materials_anisotropy`, 0 to 1); 0 keeps them round. + public var anisotropyStrength: Float = 0 { didSet { uniformRevision &+= 1 } } + /// Rotation of the stretch direction from the surface tangent, in radians. + public var anisotropyRotation: Float = 0 { didSet { uniformRevision &+= 1 } } + /// Textures of the clearcoat, sheen, anisotropy and specular layers. Each multiplies its layer's factor. + public var layerTextures: [PBRLayerTexture: PBRTextureBinding] = [:] { didSet { uniformRevision &+= 1 } } + + /// The optional shading layers and layer textures a material needs. The main shader is compiled with only what a + /// material uses; the empty set is the plain shader, which is also the only one that has a shader cache and a web build. + public struct ShaderFeatures: OptionSet, Hashable, Sendable { + public let rawValue: UInt32 + public init(rawValue: UInt32) { self.rawValue = rawValue } + + public static let clearcoat = ShaderFeatures(rawValue: 1 << 0) + public static let sheen = ShaderFeatures(rawValue: 1 << 1) + public static let anisotropy = ShaderFeatures(rawValue: 1 << 2) + + /// The draw's mesh has morph targets. A property of the draw, not of the material, so ``PBRMaterial/shaderFeatures`` + /// never contains it. + public static let morph = ShaderFeatures(rawValue: 1 << 11) + + /// Every layer, without textures. + public static let all: ShaderFeatures = [.clearcoat, .sheen, .anisotropy] + + /// Preprocessor names that switch each layer and texture in this set on. Empty for the plain shader. + public var enablingMacros: [String] { + var macros: [String] = [] + if contains(.clearcoat) { macros.append("ADA_CLEARCOAT") } + if contains(.sheen) { macros.append("ADA_SHEEN") } + if contains(.anisotropy) { macros.append("ADA_ANISOTROPY") } + if contains(.morph) { macros.append("ADA_MORPH") } + for texture in PBRLayerTexture.allCases where contains(texture.feature) { + macros.append(texture.macro) + } + return macros + } + } + + /// The layers and textures this material currently uses, from its factor values and textures. + public var shaderFeatures: ShaderFeatures { + var features: ShaderFeatures = [] + if clearcoatFactor > 0 { features.insert(.clearcoat) } + if max(sheenColorFactor.x, max(sheenColorFactor.y, sheenColorFactor.z)) > 0 { features.insert(.sheen) } + if anisotropyStrength > 0 { features.insert(.anisotropy) } + for texture in layerTextures.keys { + // A layer texture scales its layer's factor, so with the layer off it changes nothing. + if let layer = texture.layer, !features.contains(layer) { continue } + features.insert(texture.feature) + } + return features + } /// Changes whenever parameters in the shared PBR uniform snapshot are edited. @_spi(Internal) public private(set) var uniformRevision: UInt64 = 0 diff --git a/Sources/AdaRender/Mesh/Mesh.swift b/Sources/AdaRender/Mesh/Mesh.swift index 919338420..4d35b16ec 100644 --- a/Sources/AdaRender/Mesh/Mesh.swift +++ b/Sources/AdaRender/Mesh/Mesh.swift @@ -69,6 +69,9 @@ extension Mesh { public var indexCount: Int public var vertexBuffer: VertexBuffer + /// Morph targets of this part, if it has any. Weights are applied by the vertex shader before skinning. + public var morphTargets: MorphTargetSet? + @_spi(Internal) public init( id: Int, diff --git a/Sources/AdaRender/Mesh/MorphTargetSet.swift b/Sources/AdaRender/Mesh/MorphTargetSet.swift new file mode 100644 index 000000000..ccc069d31 --- /dev/null +++ b/Sources/AdaRender/Mesh/MorphTargetSet.swift @@ -0,0 +1,191 @@ +// +// MorphTargetSet.swift +// AdaEngine +// + +import Math + +/// The morph targets of one mesh part, packed for the GPU. +/// +/// Every target stores, for every vertex, a position delta and optionally a normal delta and a tangent delta, in one +/// half-float texture. The vertex shader reads the deltas of the few active targets with `texelFetch`, so any number of +/// targets cost memory but only the active ones cost time. Texel `i` is `(target * vertexCount + vertex) * attributes + +/// attribute`, laid out row by row in a texture ``textureWidth`` texels wide; the attributes are position, then normal and +/// tangent when any target has them. +public struct MorphTargetSet: Sendable { + /// Width of the delta texture in texels. + public static let textureWidth = 2048 + /// Tallest delta texture; with ``textureWidth`` this bounds `targets * vertices * attributes` to about 16.7 million texels. + public static let maximumTextureRows = 8192 + /// Most targets a draw applies at once; the heaviest weights win. + public static let maximumActiveTargets = 8 + + public let targetCount: Int + public let vertexCount: Int + /// Texels per vertex and target: 1 for positions only, up to 3 with normals and tangents. + public let attributesPerVertex: Int + public let hasNormals: Bool + public let hasTangents: Bool + public let targetNames: [String] + /// Per target, the longest position delta; an upper bound of how far the target can move any vertex. + public let maximumDisplacements: [Float] + public let deltaTexture: Texture2D + + /// Flags for the shader: bit 0 normals present, bit 1 tangents present. + public var attributeFlags: Float { Float((hasNormals ? 1 : 0) | (hasTangents ? 2 : 0)) } + + /// Packs per-target deltas. An empty array for a target and attribute means no displacement. Returns `nil` when there are + /// no targets or vertices, an array has the wrong length, a value is not finite, or the deltas would not fit the + /// texture. + public init?( + positionDeltas: [[Vector3]], + normalDeltas: [[Vector3]], + tangentDeltas: [[Vector3]], + vertexCount: Int, + targetNames: [String] = [] + ) { + let targetCount = positionDeltas.count + guard targetCount > 0, vertexCount > 0, normalDeltas.count == targetCount, tangentDeltas.count == targetCount else { + return nil + } + func valid(_ deltas: [[Vector3]]) -> Bool { + deltas.allSatisfy { $0.isEmpty || ($0.count == vertexCount && $0.allSatisfy { $0.x.isFinite && $0.y.isFinite && $0.z.isFinite }) } + } + guard valid(positionDeltas), valid(normalDeltas), valid(tangentDeltas) else { + return nil + } + let hasNormals = normalDeltas.contains { !$0.isEmpty } + let hasTangents = tangentDeltas.contains { !$0.isEmpty } + let attributes = 1 + (hasNormals ? 1 : 0) + (hasTangents ? 1 : 0) + let (texels, overflow) = targetCount.multipliedReportingOverflow(by: vertexCount) + let (total, overflowTotal) = texels.multipliedReportingOverflow(by: attributes) + let width = Self.textureWidth + guard !overflow, !overflowTotal, total <= width * Self.maximumTextureRows else { + return nil + } + let rows = (total + width - 1) / width + var halves = [UInt16](repeating: 0, count: width * rows * 4) + var displacements = [Float](repeating: 0, count: targetCount) + func write(_ deltas: [Vector3], target: Int, attribute: Int) { + guard !deltas.isEmpty else { return } + for vertex in 0.. UInt16 { + let bits = value.bitPattern + let sign = UInt16((bits >> 16) & 0x8000) + let exponent = Int((bits >> 23) & 0xFF) - 127 + 15 + let mantissa = bits & 0x7F_FFFF + if exponent >= 31 { + return sign | 0x7BFF + } + if exponent <= 0 { + guard exponent >= -10 else { return sign } + let significand = mantissa | 0x80_0000 + let shift = UInt32(14 - exponent) + var half = UInt16(significand >> shift) + if (significand >> (shift - 1)) & 1 == 1 { half &+= 1 } + return sign | half + } + var half = UInt16(exponent << 10) | UInt16(mantissa >> 13) + if mantissa & 0x1000 != 0 { half &+= 1 } + return sign | half + } + + /// The value of half-precision bits as a float. + @_spi(Internal) + public static func float(fromHalfBits half: UInt16) -> Float { + let sign: Float = half & 0x8000 != 0 ? -1 : 1 + let exponent = Int((half >> 10) & 0x1F) + let mantissa = Float(half & 0x3FF) + if exponent == 0 { return sign * mantissa * Float(sign: .plus, exponent: -24, significand: 1) } + if exponent == 31 { return sign * .infinity } + return sign * (1 + mantissa / 1024) * Float(sign: .plus, exponent: exponent - 15, significand: 1) + } + + /// The most the active weights can move any vertex, for growing culling bounds. Weights past the active limit are ignored + /// the way the shader ignores them. + public func maximumDisplacement(for weights: [Float]) -> Float { + var total: Float = 0 + for (index, weight) in Self.activeTargets(weights: weights) { + total += abs(weight) * maximumDisplacements[index] + } + return total + } + + /// The up to ``maximumActiveTargets`` targets with the largest weights, as (target index, weight). Targets whose weight is + /// zero or not finite are skipped. Ties keep the lower index first. + public static func activeTargets(weights: [Float], limit: Int = maximumActiveTargets) -> [(index: Int, weight: Float)] { + var candidates: [(index: Int, weight: Float)] = [] + for (index, weight) in weights.enumerated() where weight.isFinite && weight != 0 { + candidates.append((index, weight)) + } + candidates.sort { abs($0.weight) == abs($1.weight) ? $0.index < $1.index : abs($0.weight) > abs($1.weight) } + return Array(candidates.prefix(limit)) + } +} + +extension Mesh { + /// Whether any part of this mesh has morph targets. + public var hasMorphTargets: Bool { + models.contains { model in model.parts.contains { $0.morphTargets != nil } } + } + + /// The target names of the first morphed part, empty when there are none. + public var morphTargetNames: [String] { + for model in models { + for part in model.parts { + if let set = part.morphTargets { return set.targetNames } + } + } + return [] + } + + /// The number of morph targets of the first morphed part, zero when there are none. + public var morphTargetCount: Int { + for model in models { + for part in model.parts { + if let set = part.morphTargets { return set.targetCount } + } + } + return 0 + } +} diff --git a/Sources/AdaRender/RenderItems/Opaque3DRenderItem.swift b/Sources/AdaRender/RenderItems/Opaque3DRenderItem.swift index bbc165317..20120f297 100644 --- a/Sources/AdaRender/RenderItems/Opaque3DRenderItem.swift +++ b/Sources/AdaRender/RenderItems/Opaque3DRenderItem.swift @@ -23,6 +23,8 @@ public struct Opaque3DRenderItem: RenderItem { public let castShadows: Bool public let receiveShadows: Bool public let skinningBuffer: (any UniformBuffer)? + /// Weights of the active morph targets, for a part with ``Mesh/Part/morphTargets``. + public let morphBuffer: (any UniformBuffer)? public init( entity: Entity.ID, @@ -36,7 +38,8 @@ public struct Opaque3DRenderItem: RenderItem { castShadows: Bool = true, receiveShadows: Bool = true, batchRange: Range? = nil, - skinningBuffer: (any UniformBuffer)? = nil + skinningBuffer: (any UniformBuffer)? = nil, + morphBuffer: (any UniformBuffer)? = nil ) { self.entity = entity self.drawPass = drawPass @@ -50,5 +53,6 @@ public struct Opaque3DRenderItem: RenderItem { self.receiveShadows = receiveShadows self.batchRange = batchRange self.skinningBuffer = skinningBuffer + self.morphBuffer = morphBuffer } } diff --git a/Sources/AdaRender/Shaders/ShaderModule.swift b/Sources/AdaRender/Shaders/ShaderModule.swift index 684fcc86d..816b520ec 100644 --- a/Sources/AdaRender/Shaders/ShaderModule.swift +++ b/Sources/AdaRender/Shaders/ShaderModule.swift @@ -95,6 +95,39 @@ extension ShaderModule { try ShaderCompiler(from: fileUrl).compileShaderModule() } + /// Compile a bundled shader with every name in `defines` set to `1` in the vertex and fragment stages. + /// + /// The variant is compiled from the file's text rather than the file, so it bypasses the on-disk shader cache, whose + /// entries are keyed by file and stage and would mix up variants of one file. Callers should keep the result. + /// WebAssembly builds ship precompiled modules, so there the plain module is returned and `defines` are ignored. + public static func createBundled(at path: String, from bundle: Bundle, defines: [String]) throws -> ShaderModule { + guard !defines.isEmpty else { + return try loadBundled(at: path, from: bundle).asset + } + #if WASM + return try loadBundled(at: path, from: bundle).asset + #else + guard let resourceURL = bundle.resourceURL else { + throw ShaderModuleError.notFound(path) + } + let url = resourceURL.appendingPathComponent(path) + guard let data = FileSystem.current.readFile(at: url) else { + throw ShaderModuleError.notFound(path) + } + let source = try ShaderSource( + source: String(bytes: data, encoding: .utf8) ?? "", + includeSearchPaths: [.local(url.deletingLastPathComponent())] + ) + let compiler = ShaderCompiler(shaderSource: source) + for name in defines { + for stage in [ShaderStage.vertex, .fragment] { + compiler.setMacro(name, value: "1", for: stage) + } + } + return try compiler.compileShaderModule() + #endif + } + /// Load a bundled shader module without the async asset pipeline. /// /// Browser startup still constructs render pipelines synchronously, so shader diff --git a/Sources/AdaScene/3D/Components/MorphWeights3DComponent.swift b/Sources/AdaScene/3D/Components/MorphWeights3DComponent.swift new file mode 100644 index 000000000..8ba17117d --- /dev/null +++ b/Sources/AdaScene/3D/Components/MorphWeights3DComponent.swift @@ -0,0 +1,76 @@ +// +// MorphWeights3DComponent.swift +// AdaEngine +// + +import AdaECS + +/// The weights of an entity's morph targets (blend shapes), one per target of its mesh. +/// +/// A weight of 1 applies a target's full displacement and 0 none; values outside that range extrapolate. The renderer applies +/// at most eight targets at once, the ones with the largest weights, and moves vertices before skinning. A model instantiated +/// from glTF gets this component with the weights the file defines. +/// +/// Scripts and the inspector cannot read arrays, so they steer one target at a time: pick it with ``selectedTarget`` or +/// ``selectedTargetName`` and set ``selectedTargetWeight``. Code can use ``weights`` and ``setWeight(_:named:)`` directly. +@Component +public struct MorphWeights3DComponent: Codable, Sendable { + public var weights: [Float] + /// Names of the targets in weight order, when the asset has them. + public let targetNames: [String] + /// The target that ``selectedTargetName`` and ``selectedTargetWeight`` address. + public var selectedTarget: Int + + public init(weights: [Float], targetNames: [String] = [], selectedTarget: Int = 0) { + self.weights = weights + self.targetNames = targetNames + self.selectedTarget = selectedTarget + } + + private enum CodingKeys: CodingKey { case weights, targetNames, selectedTarget } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + weights = try container.decodeIfPresent([Float].self, forKey: .weights) ?? [] + targetNames = try container.decodeIfPresent([String].self, forKey: .targetNames) ?? [] + selectedTarget = try container.decodeIfPresent(Int.self, forKey: .selectedTarget) ?? 0 + } + + /// The number of targets. + public var targetCount: Int { weights.count } + + /// The name of the selected target; assigning a name selects that target. Empty when the target has no name. + public var selectedTargetName: String { + get { targetNames.indices.contains(selectedTarget) ? targetNames[selectedTarget] : "" } + set { + if let index = index(ofTarget: newValue) { selectedTarget = index } + } + } + + /// The weight of the selected target (0 when none is selected); assigning a finite value sets it. + public var selectedTargetWeight: Float { + get { weights.indices.contains(selectedTarget) ? weights[selectedTarget] : 0 } + set { + if weights.indices.contains(selectedTarget), newValue.isFinite { weights[selectedTarget] = newValue } + } + } + + /// The index of the target with this name. + public func index(ofTarget name: String) -> Int? { + targetNames.firstIndex(of: name) + } + + /// The current weight of the target with this name. + public func weight(named name: String) -> Float? { + guard let index = index(ofTarget: name), weights.indices.contains(index) else { return nil } + return weights[index] + } + + /// Sets the weight of the target with this name. Returns `false` when there is no such target. + @discardableResult + public mutating func setWeight(_ weight: Float, named name: String) -> Bool { + guard let index = index(ofTarget: name), weights.indices.contains(index) else { return false } + weights[index] = weight + return true + } +} diff --git a/Sources/AdaScene/3D/Model3DPlugin.swift b/Sources/AdaScene/3D/Model3DPlugin.swift index 8cdaec541..5523adc6b 100644 --- a/Sources/AdaScene/3D/Model3DPlugin.swift +++ b/Sources/AdaScene/3D/Model3DPlugin.swift @@ -37,6 +37,7 @@ public struct Model3DPlugin: Plugin { ModelAnimation3DComponent.registerComponent() ModelNode3DComponent.registerComponent() SkinnedMesh3DComponent.registerComponent() + MorphWeights3DComponent.registerComponent() app.addSystem(SkeletalAnimation3DSystem.self, on: .postUpdate) VoxelAnimationPlayback3DComponent.registerComponent() VoxelTransformAnimation3DComponent.registerComponent() @@ -58,6 +59,7 @@ public struct Model3DPlugin: Plugin { .insertResource(AnimatedBounds3D()) .insertResource(Active3DInstanceBuffers()) .insertResource(Skinning3DUniforms()) + .insertResource(Morph3DUniforms()) .insertResource(PBR3DUniforms()) .insertResource(RenderPipelines(configurator: PBR3DPipeline())) .insertResource(RenderPipelines(configurator: Flat3DPipeline())) @@ -82,6 +84,9 @@ func ExtractModel3D( RelationshipComponent>>, _ skins: Extract>, + _ morphs: Extract>, + _ morphUniforms: ResMut, _ sources: ResMut, _ animatedBounds: ResMut, _ skinningUniforms: ResMut, @@ -101,6 +106,11 @@ func ExtractModel3D( lods.wrappedValue.forEach { entity, lod in lodByEntity[entity.id] = lod } var posedBounds: [Entity.ID: AABB] = [:] skinningUniforms.wrappedValue.beginFrame(device: renderDevice.renderDevice) + morphUniforms.wrappedValue.beginFrame() + var morphByEntity: [Entity.ID: [Float]] = [:] + morphs.wrappedValue.forEach { entity, morph in morphByEntity[entity.id] = morph.weights } + // Morph deltas are applied before skinning, so a joint matrix that scales also scales the displacement. + var skinScale: [Entity.ID: Float] = [:] skins.wrappedValue.forEach { entity, skin in _ = skinningUniforms.wrappedValue.write(skin.matrices, for: entity.id, device: renderDevice.renderDevice, revision: .init(id: skin.paletteID, generation: skin.poseGeneration)) if let mesh = entity.components[Mesh3DComponent.self]?.mesh { @@ -111,6 +121,11 @@ func ExtractModel3D( : animatedBounds.wrappedValue.bounds(mesh: mesh, matrices: skin.matrices) posedBounds[entity.id] = bounds } + var largest: Float = 1 + for matrix in skin.matrices { + largest = max(largest, matrix.x.xyz.length, matrix.y.xyz.length, matrix.z.xyz.length) + } + skinScale[entity.id] = largest } materialUniforms.wrappedValue.beginFrame() @@ -134,6 +149,22 @@ func ExtractModel3D( if palette == nil, let lod { for level in lod.alternatives { localBounds = localBounds.map { AABB(min: min($0.min, level.bounds.min), max: max($0.max, level.bounds.max)) } ?? level.bounds } } + // Weights move vertices past the rest-pose bounds, so culling and shadow bounds grow by the most the weights can displace. + var morphBuffers: [UInt64: any UniformBuffer] = [:] + if mesh.hasMorphTargets, let weights = morphByEntity[entity.id] { + let written = morphUniforms.wrappedValue.write(weights: weights, mesh: mesh, for: entity.id, device: renderDevice.renderDevice) + var displacement: Float = 0 + for (key, buffer) in written { + morphBuffers[Mesh3DRenderSource.morphKey(model: key.model, part: key.part)] = buffer + if let set = mesh.models[key.model].parts[key.part].morphTargets { + displacement = max(displacement, set.maximumDisplacement(for: weights)) + } + } + if displacement > 0, let bounds = localBounds { + let growth = Vector3.one * (displacement * (skinScale[entity.id] ?? 1)) + localBounds = AABB(min: bounds.min - growth, max: bounds.max + growth) + } + } sources.meshes.append(Mesh3DRenderSource( entity: entity.id, drawPass: drawPass.wrappedValue, @@ -141,7 +172,8 @@ func ExtractModel3D( materials: mesh3d.materials, transform: transform.matrix, bounds: localBounds.map { MeshVisibility3DMath.transformed($0, by: transform.matrix) }, - alternatives: palette == nil || lod?.skeletonCompatible == true && localBounds != nil ? lod?.alternatives ?? [] : [], + // Level-of-detail meshes have their own vertices, which the morph deltas do not address. + alternatives: morphBuffers.isEmpty && (palette == nil || lod?.skeletonCompatible == true && localBounds != nil) ? lod?.alternatives ?? [] : [], thresholds: lod?.screenThresholds ?? [], hysteresis: lod?.hysteresis ?? 0.15, maximumDistance: lod?.maximumDistance, @@ -150,7 +182,8 @@ func ExtractModel3D( castShadows: mesh3d.castShadows, receiveShadows: mesh3d.receiveShadows, skinningBuffer: palette, - previousTransform: previousTransforms[entity.id] + previousTransform: previousTransforms[entity.id], + morphBuffers: morphBuffers )) } @@ -179,12 +212,15 @@ public final class Model3DDrawPass: DrawPass, @unchecked Sendable { return } + let isMotion = world.getResource(ActiveMotion3DPass.self)?.isEnabled == true + let morphed = item.morphBuffer != nil && part.morphTargets != nil let configuration = PBR3DConfiguration( vertex: part.vertexDescriptor, blended: (item.material as? PBRMaterial)?.alphaMode == .blend, - temporalDepth: view.components[RenderViewTarget.self]?.temporalUpscalingActive == true + temporalDepth: view.components[RenderViewTarget.self]?.temporalUpscalingActive == true, + // The motion shader has no shading layers, so its pipelines are not split by them. + features: (isMotion ? [] : (item.material as? PBRMaterial)?.shaderFeatures ?? []).union(morphed ? .morph : []) ) - let isMotion = world.getResource(ActiveMotion3DPass.self)?.isEnabled == true let pipeline: any RenderPipeline if isMotion { let pipelines = world.getRefResource(RenderPipelines.self) @@ -215,6 +251,20 @@ public final class Model3DDrawPass: DrawPass, @unchecked Sendable { let normalTexture = pbrMaterial?.normalTexture ?? Self.flatNormalTexture let emissiveTexture = pbrMaterial?.emissiveTexture ?? Texture2D.whiteTexture let heightTexture = pbrMaterial?.heightTexture ?? Texture2D.whiteTexture + // The main shader variant declares exactly the layer textures the material has (see PBRMaterial.shaderFeatures). + var layerBindings: [RenderResourceSet.Binding] = [] + if morphed, let morphBuffer = item.morphBuffer, let morphSet = part.morphTargets { + layerBindings.append(.init(binding: Morph3DUniforms.binding, shaderStages: .vertex, resource: .uniformBuffer(morphBuffer, offset: 0))) + layerBindings.append(.init(binding: Morph3DUniforms.textureBinding, shaderStages: .vertex, resource: .texture(morphSet.deltaTexture))) + } + if !isMotion, let pbrMaterial { + let features = pbrMaterial.shaderFeatures + for kind in PBRLayerTexture.allCases where features.contains(kind.feature) { + if let layer = pbrMaterial.layerTextures[kind] { + layerBindings.append(.init(binding: kind.binding, shaderStages: .fragment, resource: .texture(layer.texture))) + } + } + } if isMotion { let previous = world.getResource(Skinning3DUniforms.self)?.previousBuffer(for: item.entity) ?? palette @@ -239,7 +289,7 @@ public final class Model3DDrawPass: DrawPass, @unchecked Sendable { .init(binding: 12, shaderStages: .fragment, resource: .texture(emissiveTexture)), .init(binding: 13, shaderStages: .fragment, resource: .sampler(emissiveTexture.sampler)), .init(binding: 17, shaderStages: .fragment, resource: .texture(heightTexture)), - ] + ] + layerBindings ), index: 0 ) diff --git a/Sources/AdaScene/3D/Model3DSource.swift b/Sources/AdaScene/3D/Model3DSource.swift index b7af50b42..efd18f76a 100644 --- a/Sources/AdaScene/3D/Model3DSource.swift +++ b/Sources/AdaScene/3D/Model3DSource.swift @@ -37,9 +37,14 @@ public struct Model3DSource: Codable, Equatable, Sendable { public var lod: Model3DSourceLOD? /// Import settings for MagicaVoxel (`.vox`) sources; ignored for other formats. public var voxel: VoxelImportSettings? + /// Morph target weights by target name, applied to every mesh of the model that has a target of that name. They are the + /// weights the instance starts with and keeps unless an animation clip animates that mesh's weights. + public var morphWeights: [String: Float] public init(source: String = "", animation: String = "", autoplay: Bool = false, repeats: Bool = true, - animationGraph: AnimationGraph? = nil, lod: Model3DSourceLOD? = nil, voxel: VoxelImportSettings? = nil) { + animationGraph: AnimationGraph? = nil, lod: Model3DSourceLOD? = nil, voxel: VoxelImportSettings? = nil, + morphWeights: [String: Float] = [:]) { + self.morphWeights = morphWeights self.voxel = voxel self.source = source self.animation = animation @@ -49,6 +54,23 @@ public struct Model3DSource: Codable, Equatable, Sendable { self.lod = lod } + private enum CodingKeys: CodingKey { + case source, animation, autoplay, repeats, animationGraph, lod, voxel, morphWeights + } + + public init(from decoder: Decoder) throws { + let container = try decoder.container(keyedBy: CodingKeys.self) + source = try container.decode(String.self, forKey: .source) + animation = try container.decode(String.self, forKey: .animation) + autoplay = try container.decode(Bool.self, forKey: .autoplay) + repeats = try container.decode(Bool.self, forKey: .repeats) + animationGraph = try container.decodeIfPresent(AnimationGraph.self, forKey: .animationGraph) + lod = try container.decodeIfPresent(Model3DSourceLOD.self, forKey: .lod) + voxel = try container.decodeIfPresent(VoxelImportSettings.self, forKey: .voxel) + // Scenes written before morph targets have no weights. + morphWeights = try container.decodeIfPresent([String: Float].self, forKey: .morphWeights) ?? [:] + } + /// The path the model asset is loaded from: ``source`` plus the import settings of a `.vox` file as an asset query. public var assetPath: String { guard let voxel, !voxel.isDefault, !source.isEmpty, !source.contains("#"), @@ -74,6 +96,19 @@ public struct Model3DSource: Codable, Equatable, Sendable { try animationGraph.validate(rig: rig, clips: model.animationClips) } let root = model.instantiate(in: world) + if !morphWeights.isEmpty { + var pending = [root] + while let node = pending.popLast() { + if var component = node.components[MorphWeights3DComponent.self] { + var changed = false + for (name, weight) in morphWeights where weight.isFinite { + if component.setWeight(weight, named: name) { changed = true } + } + if changed { node.components[MorphWeights3DComponent.self] = component } + } + pending.append(contentsOf: node.children) + } + } if var animationComponent = root.components[ModelAnimation3DComponent.self], animationGraph != nil || !animation.isEmpty { if let animationGraph { try animationComponent.player.play(animationGraph, transitionDuration: 0) diff --git a/Sources/AdaScene/3D/ModelAsset3D+Animation.swift b/Sources/AdaScene/3D/ModelAsset3D+Animation.swift index 450b0084c..839309d32 100644 --- a/Sources/AdaScene/3D/ModelAsset3D+Animation.swift +++ b/Sources/AdaScene/3D/ModelAsset3D+Animation.swift @@ -3,7 +3,13 @@ import AdaAssets import Math extension ModelAsset3D { - static func makeRig(nodes: [GLTFImportResult.Node]) throws -> SkeletalRig { + /// The weights a node's mesh has when nothing animates them, or `nil` when it has no morph targets. + private static func restWeights(of node: GLTFImportResult.Node, meshes: [GLTFImportResult.Mesh]) -> [Float]? { + guard let meshIndex = node.meshIndex, meshes.indices.contains(meshIndex), meshes[meshIndex].morphTargetCount > 0 else { return nil } + return node.weights ?? meshes[meshIndex].defaultWeights + } + + static func makeRig(nodes: [GLTFImportResult.Node], meshes: [GLTFImportResult.Mesh] = []) throws -> SkeletalRig { var parents = [Int?](repeating: nil, count: nodes.count) for (index, node) in nodes.enumerated() { for child in node.children { @@ -13,19 +19,31 @@ extension ModelAsset3D { } return try SkeletalRig(nodes: nodes.enumerated().map { index, node in let pose = node.restPose.map { SkeletalJointPose(translation: $0.translation, rotation: $0.rotation, scale: $0.scale) } - return SkeletalRig.Node(parentIndex: parents[index], restPose: pose, restMatrix: node.transform, name: node.name) + return SkeletalRig.Node( + parentIndex: parents[index], + restPose: pose, + restMatrix: node.transform, + name: node.name, + morphRestWeights: restWeights(of: node, meshes: meshes) + ) }) } - static func makeClips(_ clips: [GLTFImportResult.Animation]) throws -> [SkeletalAnimationClip] { + static func makeClips( + _ clips: [GLTFImportResult.Animation], + nodes: [GLTFImportResult.Node] = [], + meshes: [GLTFImportResult.Mesh] = [] + ) throws -> [SkeletalAnimationClip] { try clips.enumerated().map { index, clip in - let tracks = try clip.channels.map { channel in + // Weight channels become morph tracks; the others are pose tracks. + let tracks = try clip.channels.filter { $0.path != .weights }.map { channel in let sampler = clip.samplers[channel.samplerIndex] let path: SkeletalAnimationTrack.Path switch channel.path { case .translation: path = .translation case .rotation: path = .rotation case .scale: path = .scale + case .weights: preconditionFailure("Weight channels are filtered out above.") } let interpolation: SkeletalAnimationTrack.Interpolation switch sampler.interpolation { @@ -40,7 +58,24 @@ extension ModelAsset3D { nodeIndex: channel.nodeIndex, path: path, interpolation: interpolation, times: sampler.times.map(Double.init), values: values ) } - return SkeletalAnimationClip(name: clip.name ?? "Animation \(index)", tracks: tracks) + let morphTracks = try clip.channels.filter { $0.path == .weights }.map { channel in + let sampler = clip.samplers[channel.samplerIndex] + let interpolation: SkeletalAnimationTrack.Interpolation + switch sampler.interpolation { + case .linear: interpolation = .linear + case .step: interpolation = .step + case .cubicSpline: interpolation = .cubicSpline + } + let targets = nodes.indices.contains(channel.nodeIndex) ? Self.restWeights(of: nodes[channel.nodeIndex], meshes: meshes)?.count ?? 0 : 0 + return try MorphWeightsTrack( + nodeIndex: channel.nodeIndex, + interpolation: interpolation, + times: sampler.times.map(Double.init), + targetCount: targets, + values: sampler.output.values + ) + } + return SkeletalAnimationClip(name: clip.name ?? "Animation \(index)", tracks: tracks, morphTracks: morphTracks) } } } diff --git a/Sources/AdaScene/3D/ModelAsset3D+Materials.swift b/Sources/AdaScene/3D/ModelAsset3D+Materials.swift index 2d1c6b836..69fe97327 100644 --- a/Sources/AdaScene/3D/ModelAsset3D+Materials.swift +++ b/Sources/AdaScene/3D/ModelAsset3D+Materials.swift @@ -59,6 +59,18 @@ extension ModelAsset3D { material.alphaCutoff = source.alphaCutoff material.doubleSided = source.doubleSided material.textureCoordinates = source.textureCoordinates + material.textureTransforms = source.textureTransforms.map { + PBRTextureTransform(offset: $0.offset, rotation: $0.rotation, scale: $0.scale) + } + material.ior = source.ior + material.specularFactor = source.specularFactor + material.specularColorFactor = source.specularColorFactor + material.clearcoatFactor = source.clearcoatFactor + material.clearcoatRoughnessFactor = source.clearcoatRoughnessFactor + material.sheenColorFactor = source.sheenColorFactor + material.sheenRoughnessFactor = source.sheenRoughnessFactor + material.anisotropyStrength = source.anisotropyStrength + material.anisotropyRotation = source.anisotropyRotation switch source.alphaMode { case .opaque: material.alphaMode = .opaque case .mask: material.alphaMode = .mask @@ -69,6 +81,26 @@ extension ModelAsset3D { material.normalTexture = try texture(source.normalTextureIndex, sRGB: false) material.occlusionTexture = try texture(source.occlusionTextureIndex, sRGB: false) material.emissiveTexture = try texture(source.emissiveTextureIndex, sRGB: true) + let layers = source.layerTextures + let layerSources: [(PBRLayerTexture, GLTFImportResult.TextureReference?, Bool)] = [ + (.clearcoat, layers.clearcoat, false), + (.clearcoatRoughness, layers.clearcoatRoughness, false), + (.clearcoatNormal, layers.clearcoatNormal, false), + (.sheenColor, layers.sheenColor, true), + (.sheenRoughness, layers.sheenRoughness, false), + (.anisotropy, layers.anisotropy, false), + (.specular, layers.specular, false), + (.specularColor, layers.specularColor, true), + ] + for (kind, reference, isSRGB) in layerSources { + guard let reference, let value = try texture(reference.index, sRGB: isSRGB) else { continue } + material.layerTextures[kind] = PBRTextureBinding( + texture: value, + texCoord: reference.texCoord, + transform: PBRTextureTransform(offset: reference.transform.offset, rotation: reference.transform.rotation, scale: reference.transform.scale), + scale: reference.scale + ) + } return material } return materials.isEmpty ? [PBRMaterial()] : materials diff --git a/Sources/AdaScene/3D/ModelAsset3D.swift b/Sources/AdaScene/3D/ModelAsset3D.swift index 393a51115..ec4882803 100644 --- a/Sources/AdaScene/3D/ModelAsset3D.swift +++ b/Sources/AdaScene/3D/ModelAsset3D.swift @@ -26,6 +26,8 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { public let restPose: GLTFImportResult.NodeTRS? /// Keyed transform and mesh animation of imported voxel scenes. public let voxelAnimation: VoxelNodeAnimation? + /// Morph target weights of this node's mesh instance; they replace the mesh's default weights. + public let morphWeights: [Float]? public init( name: String?, @@ -34,8 +36,10 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { children: [Int], skinIndex: Int? = nil, restPose: GLTFImportResult.NodeTRS? = nil, - voxelAnimation: VoxelNodeAnimation? = nil + voxelAnimation: VoxelNodeAnimation? = nil, + morphWeights: [Float]? = nil ) { + self.morphWeights = morphWeights self.voxelAnimation = voxelAnimation self.name = name self.transform = transform @@ -54,6 +58,11 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { public let skins: [GLTFImportResult.Skin] public let animationRig: SkeletalRig? public let animationClips: [SkeletalAnimationClip] + /// Default morph target weights of each mesh, empty for a mesh without morph targets. + public private(set) var meshMorphWeights: [[Float]] = [] + /// Content of the source file the importer could not honor, such as glTF extensions it does not implement. + /// Empty for formats that report nothing. The model still loads and renders without that content. + public private(set) var importWarnings: [String] = [] public var assetMetaInfo: AssetMetaInfo? @@ -138,13 +147,14 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { let loader = GLTFLoaderResolver.shared.getLoader() let result = try await loader.load(url: assetDecoder.assetMeta.filePath) + importWarnings = result.warnings guard result.skins.allSatisfy({ $0.joints.count <= Skinning3DUniforms.maximumJoints }) else { throw AssetError.message("A skinned mesh supports at most \(Skinning3DUniforms.maximumJoints) joints.") } skins = result.skins - animationRig = result.skins.isEmpty && result.animations.isEmpty ? nil : try Self.makeRig(nodes: result.nodes) - animationClips = try Self.makeClips(result.animations) + animationRig = result.skins.isEmpty && result.animations.isEmpty ? nil : try Self.makeRig(nodes: result.nodes, meshes: result.meshes) + animationClips = try Self.makeClips(result.animations, nodes: result.nodes, meshes: result.meshes) let device = unsafe RenderEngine.shared.renderDevice let materials = try Self.makeMaterials(result) @@ -202,7 +212,7 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { descriptor.indicies = converted.indices descriptor.primitiveTopology = converted.topology - let part = Mesh.Part( + var part = Mesh.Part( id: index, materialIndex: primitive.materialIndex ?? 0, primitiveTopology: descriptor.primitiveTopology, @@ -213,13 +223,28 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { indexCount: descriptor.indicies.count, vertexBuffer: descriptor.getVertexBuffer(renderDevice: device) ) + if !primitive.morphTargets.isEmpty { + guard let morph = MorphTargetSet( + positionDeltas: primitive.morphTargets.map(\.positionDeltas), + normalDeltas: primitive.morphTargets.map(\.normalDeltas), + tangentDeltas: primitive.morphTargets.map(\.tangentDeltas), + vertexCount: descriptor.positions.count, + targetNames: gltfMesh.targetNames + ) else { + throw AssetError.message("The \(primitive.morphTargets.count) morph targets of mesh \"\(gltfMesh.name ?? "")\" do not fit the delta texture (at most \(MorphTargetSet.textureWidth * MorphTargetSet.maximumTextureRows) texels of vertices x targets x attributes).") + } + part.morphTargets = morph + } parts.append(part) } meshes.append(Mesh(models: [Mesh.Model(name: gltfMesh.name ?? "", parts: parts)])) + meshMorphWeights.append(gltfMesh.defaultWeights) } - nodes = result.nodes.map { Node(name: $0.name, transform: $0.transform, meshIndex: $0.meshIndex, children: $0.children, skinIndex: $0.skinIndex, restPose: $0.restPose) } + nodes = result.nodes.map { + Node(name: $0.name, transform: $0.transform, meshIndex: $0.meshIndex, children: $0.children, skinIndex: $0.skinIndex, restPose: $0.restPose, morphWeights: $0.weights) + } self.meshes = meshes self.materials = materials scenes = result.scenes @@ -293,6 +318,11 @@ public final class ModelAsset3D: Asset, @unchecked Sendable { if let meshIndex = node.meshIndex { let mesh = meshes[meshIndex] entity.components[Mesh3DComponent.self] = Mesh3DComponent(mesh: mesh, materials: materials) + if mesh.hasMorphTargets { + let defaults = meshMorphWeights.indices.contains(meshIndex) ? meshMorphWeights[meshIndex] : [] + let weights = node.morphWeights ?? (defaults.count == mesh.morphTargetCount ? defaults : [Float](repeating: 0, count: mesh.morphTargetCount)) + entity.components[MorphWeights3DComponent.self] = MorphWeights3DComponent(weights: weights, targetNames: mesh.morphTargetNames) + } if let skinIndex = node.skinIndex, skins.indices.contains(skinIndex), let animation = modelRoot.components[ModelAnimation3DComponent.self] { var binding = SkinnedMesh3DComponent(modelRoot: modelRoot.id, meshNodeIndex: nodeIndex, skin: skins[skinIndex]) diff --git a/Sources/AdaScene/3D/SkeletalAnimation3D.swift b/Sources/AdaScene/3D/SkeletalAnimation3D.swift index cf2792f20..3e4dc3818 100644 --- a/Sources/AdaScene/3D/SkeletalAnimation3D.swift +++ b/Sources/AdaScene/3D/SkeletalAnimation3D.swift @@ -90,6 +90,7 @@ public struct SkeletalAnimation3DSystem { @Query> private var models @Query> private var nodes @Query> private var skins + @Query> private var morphNodes @Res private var time @EventsSender private var animationEvents @Query private var cameras @@ -108,6 +109,8 @@ public struct SkeletalAnimation3DSystem { // These references are local to this update; no structural world mutations occur here. var nodesByRoot: [Entity.ID: [(Int, Ref)]] = [:] nodes.forEach { binding, transform in nodesByRoot[binding.modelRoot, default: []].append((binding.nodeIndex, transform)) } + var morphByRoot: [Entity.ID: [(Int, Ref)]] = [:] + morphNodes.forEach { binding, weights in morphByRoot[binding.modelRoot, default: []].append((binding.nodeIndex, weights)) } var skinsByRoot: [Entity.ID: [Ref]] = [:] skins.forEach { skin in skinsByRoot[skin.wrappedValue.modelRoot, default: []].append(skin) } models.forEach { root, animation in @@ -134,6 +137,15 @@ public struct SkeletalAnimation3DSystem { let next = Transform(rotation: pose.rotation, scale: pose.scale, position: pose.translation) if transform.wrappedValue != next { transform.wrappedValue = next } } + for (index, component) in morphByRoot[root.id] ?? [] { + guard value.player.morphWeights.indices.contains(index), value.player.animatedMorphNodes.contains(index) else { continue } + let weights = value.player.morphWeights[index] + // Only nodes some clip animates are written, so weights authored on a model (Model3DSource.morphWeights) + // survive on the others. The weights must also match the node's targets. + if !weights.isEmpty, component.wrappedValue.weights.count == weights.count, component.wrappedValue.weights != weights { + component.wrappedValue.weights = weights + } + } for skin in skinsByRoot[root.id] ?? [] { skin.wrappedValue.updateMatrices(using: value.player) } value.appliedGeneration = value.player.poseGeneration value.appliedPlayer = value.player.instanceID diff --git a/Tests/AdaAnimationTests/MorphWeightsAnimationTests.swift b/Tests/AdaAnimationTests/MorphWeightsAnimationTests.swift new file mode 100644 index 000000000..ba79c1732 --- /dev/null +++ b/Tests/AdaAnimationTests/MorphWeightsAnimationTests.swift @@ -0,0 +1,214 @@ +import AdaAnimation +import Math +import Testing + +@Suite +struct MorphWeightsAnimationTests { + // MARK: Tracks + + @Test + func samplesLinearAndStepWeightsAndClampsAtTheEnds() throws { + let linear = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0, 1], targetCount: 2, values: [0, 1, 1, 0.5]) + var output: [Float] = [9, 9] + linear.sample(at: -3, into: &output) + #expect(output == [0, 1]) + linear.sample(at: 0.5, into: &output) + #expect(output == [0.5, 0.75]) + linear.sample(at: 4, into: &output) + #expect(output == [1, 0.5]) + let step = try MorphWeightsTrack(nodeIndex: 0, interpolation: .step, times: [0, 1], targetCount: 2, values: [0, 1, 1, 0.5]) + step.sample(at: 0.99, into: &output) + #expect(output == [0, 1]) + step.sample(at: 1, into: &output) + #expect(output == [1, 0.5]) + } + + @Test + func cubicWeightsUseTangentsScaledByTheKeySpan() throws { + // Two targets, two keys 2 seconds apart. Per key: in-tangent, value, out-tangent (each of both targets). + let values: [Float] = [ + 0, 0, /* in */ 0, 0, /* value */ 0, 0, /* out */ + 0, 0, /* in */ 1, 4, /* value */ 0, 0, /* out */ + ] + let cubic = try MorphWeightsTrack(nodeIndex: 0, interpolation: .cubicSpline, times: [0, 2], targetCount: 2, values: values) + var output: [Float] = [0, 0] + cubic.sample(at: 1, into: &output) + #expect(abs(output[0] - 0.5) < 0.00001 && abs(output[1] - 2) < 0.00001, "Flat tangents give the smoothstep midpoint.") + let sloped: [Float] = [0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0] + let withTangent = try MorphWeightsTrack(nodeIndex: 0, interpolation: .cubicSpline, times: [0, 2], targetCount: 2, values: sloped) + withTangent.sample(at: 1, into: &output) + #expect(abs(output[0] - 0.25) < 0.00001, "An out-tangent of 1 over a 2 second span adds a quarter at the midpoint.") + } + + @Test(arguments: ["noTargets", "noKeys", "times", "valueCount", "nonFinite", "cubicNeedsTwoKeys", "negativeNode"]) + func rejectsInvalidTracks(kind: String) { + #expect(throws: SkeletalAnimationError.invalidTrack) { + switch kind { + case "noTargets": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0], targetCount: 0, values: []) + case "noKeys": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [], targetCount: 1, values: []) + case "times": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [1, 1], targetCount: 1, values: [0, 1]) + case "valueCount": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0, 1], targetCount: 2, values: [0, 1, 2]) + case "nonFinite": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0], targetCount: 1, values: [.nan]) + case "cubicNeedsTwoKeys": _ = try MorphWeightsTrack(nodeIndex: 0, interpolation: .cubicSpline, times: [0], targetCount: 1, values: [0, 0, 0]) + default: _ = try MorphWeightsTrack(nodeIndex: -1, interpolation: .linear, times: [0], targetCount: 1, values: [0]) + } + } + } + + // MARK: Player + + /// Node 0 carries a mesh with two targets resting at [0.2, 0.4]; node 1 has no morph targets. + private func makeRig() throws -> SkeletalRig { + try SkeletalRig(nodes: [ + .init(parentIndex: nil, restPose: SkeletalJointPose(), restMatrix: .identity, morphRestWeights: [0.2, 0.4]), + .init(parentIndex: 0, restPose: SkeletalJointPose(), restMatrix: .identity), + ]) + } + + private func clip(_ name: String, from: [Float], to: [Float], duration: Double = 1) throws -> SkeletalAnimationClip { + SkeletalAnimationClip(name: name, tracks: [], morphTracks: [ + try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0, duration], targetCount: 2, values: from + to), + ]) + } + + private func makePlayer() throws -> SkeletalAnimationPlayer { + try SkeletalAnimationPlayer(rig: makeRig(), clips: [clip("Open", from: [0, 0], to: [1, 0.5]), clip("Close", from: [1, 1], to: [1, 1])]) + } + + @Test + func weightsStartAtTheRestWeightsAndOnlyMorphNodesHaveThem() throws { + let player = try makePlayer() + #expect(player.morphWeights == [[0.2, 0.4], []]) + } + + @Test + func aPlayingClipDrivesTheWeights() throws { + var player = try makePlayer() + try player.play("Open", transitionDuration: 0) + #expect(player.morphWeights[0] == [0, 0]) + player.seek(to: 0.5) + #expect(player.morphWeights[0] == [0.5, 0.25]) + player.advance(by: 0.25) + #expect(player.morphWeights[0] == [0.75, 0.375]) + #expect(player.morphWeights[1].isEmpty) + // The clip loops: 1.25 seconds in is 0.25 into the next cycle. + player.seek(to: 1.25) + #expect(player.morphWeights[0] == [0.25, 0.125]) + player.repeats = false + player.seek(to: 7) + #expect(player.morphWeights[0] == [1, 0.5], "A clip that does not repeat holds its last weights.") + } + + @Test + func playersKeepIndependentWeights() throws { + var first = try makePlayer() + let second = first + try first.play("Open", transitionDuration: 0) + first.repeats = false + first.seek(to: 1) + #expect(first.morphWeights[0] == [1, 0.5] && second.morphWeights[0] == [0.2, 0.4]) + } + + @Test + func aTransitionCrossfadesFromThePreviousWeights() throws { + var player = try makePlayer() + try player.play("Open", transitionDuration: 0) + player.repeats = false + player.seek(to: 1) + #expect(player.morphWeights[0] == [1, 0.5]) + try player.play("Close", transitionDuration: 1) + #expect(player.morphWeights[0] == [1, 0.5], "A fade that has not started shows the old weights.") + player.advance(by: 0.5) + #expect(abs(player.morphWeights[0][0] - 1) < 0.00001 && abs(player.morphWeights[0][1] - 0.75) < 0.00001) + player.advance(by: 1) + #expect(player.morphWeights[0] == [1, 1]) + } + + @Test + func tracksMustAddressNodesWithMatchingTargetCounts() throws { + let rig = try makeRig() + let wrongNode = SkeletalAnimationClip(name: "Bad", tracks: [], morphTracks: [ + try MorphWeightsTrack(nodeIndex: 1, interpolation: .linear, times: [0], targetCount: 2, values: [0, 0]), + ]) + #expect(throws: SkeletalAnimationError.invalidTrack) { try SkeletalAnimationPlayer(rig: rig, clips: [wrongNode]) } + let wrongCount = SkeletalAnimationClip(name: "Bad", tracks: [], morphTracks: [ + try MorphWeightsTrack(nodeIndex: 0, interpolation: .linear, times: [0], targetCount: 3, values: [0, 0, 0]), + ]) + #expect(throws: SkeletalAnimationError.invalidTrack) { try SkeletalAnimationPlayer(rig: rig, clips: [wrongCount]) } + } + + @Test + func onlyNodesSomeClipAnimatesAreReportedAsAnimated() throws { + #expect(try makePlayer().animatedMorphNodes == [0]) + let still = try SkeletalAnimationPlayer(rig: makeRig(), clips: [SkeletalAnimationClip(name: "Still", tracks: [])]) + #expect(still.animatedMorphNodes.isEmpty, "A rig whose clips animate no weights must leave entity weights alone.") + } + + @Test + func weightTracksCountTowardTheClipDuration() throws { + #expect(try clip("Long", from: [0, 0], to: [1, 1], duration: 2.5).duration == 2.5) + } + + // MARK: Graph + + @Test + func aBlendNodeMixesWeightsLikePoses() throws { + var player = try makePlayer() + let graph = AnimationGraph(root: "Mix", nodes: [ + .init(id: "Open", clip: "Open"), .init(id: "Close", clip: "Close"), + .init(id: "Mix", kind: .blend, inputs: [.init(node: "Open", weight: 0.75), .init(node: "Close", weight: 0.25)]), + ]) + try player.play(graph, transitionDuration: 0) + player.seek(to: 0.5) + // Open at 0.5 is [0.5, 0.25] and Close is [1, 1]. + #expect(abs(player.morphWeights[0][0] - (0.5 * 0.75 + 1 * 0.25)) < 0.00001) + #expect(abs(player.morphWeights[0][1] - (0.25 * 0.75 + 1 * 0.25)) < 0.00001) + // Weights that add up to less than one let the rest weights fill the gap. + try player.setGraphWeight(0.5, node: "Mix", input: 0) + try player.setGraphWeight(0, node: "Mix", input: 1) + #expect(abs(player.morphWeights[0][0] - (0.5 * 0.5 + 0.2 * 0.5)) < 0.00001) + } + + @Test + func aMaskOnTheMeshNodeScalesItsWeights() throws { + var player = try makePlayer() + let graph = AnimationGraph(root: "Mix", nodes: [ + .init(id: "Open", clip: "Open"), .init(id: "Close", clip: "Close"), + .init(id: "Mix", kind: .blend, inputs: [ + .init(node: "Open"), + // The second input only reaches node 1, so it must leave the weights of node 0 alone. + .init(node: "Close", mask: .init(joints: [.init(nodeIndex: 1, includesDescendants: false)])), + ]), + ]) + try player.play(graph, transitionDuration: 0) + player.seek(to: 0.5) + #expect(abs(player.morphWeights[0][0] - 0.5) < 0.00001 && abs(player.morphWeights[0][1] - 0.25) < 0.00001) + } + + @Test + func anAdditiveNodeAddsOffsetsFromTheRestWeights() throws { + var player = try makePlayer() + let graph = AnimationGraph(root: "Layer", nodes: [ + .init(id: "Base", clip: "Open"), .init(id: "Extra", clip: "Close"), + .init(id: "Layer", kind: .additive, inputs: [.init(node: "Base"), .init(node: "Extra", weight: 0.5)]), + ]) + try player.play(graph, transitionDuration: 0) + player.seek(to: 0.5) + // Base [0.5, 0.25]; Extra [1, 1] is [0.8, 0.6] above the rest weights [0.2, 0.4], half of which is added. + #expect(abs(player.morphWeights[0][0] - (0.5 + 0.8 * 0.5)) < 0.00001) + #expect(abs(player.morphWeights[0][1] - (0.25 + 0.6 * 0.5)) < 0.00001) + } + + @Test + func aClipWithoutWeightTracksLeavesTheRestWeights() throws { + var player = try SkeletalAnimationPlayer( + rig: makeRig(), + clips: [SkeletalAnimationClip(name: "Still", tracks: [ + try SkeletalAnimationTrack(nodeIndex: 1, path: .translation, interpolation: .linear, times: [0, 1], values: [.zero, [1, 0, 0, 0]]), + ])] + ) + try player.play("Still", transitionDuration: 0) + player.seek(to: 0.5) + #expect(player.morphWeights[0] == [0.2, 0.4]) + } +} diff --git a/Tests/AdaAssetsTests/GLTFMaterialExtensionTests.swift b/Tests/AdaAssetsTests/GLTFMaterialExtensionTests.swift new file mode 100644 index 000000000..6499c3b2c --- /dev/null +++ b/Tests/AdaAssetsTests/GLTFMaterialExtensionTests.swift @@ -0,0 +1,191 @@ +import AdaAssets +import Foundation +import Math +import Testing + +@Suite +struct GLTFMaterialExtensionTests { + private func load(material: [String: Any], used: [String]? = nil, required: [String]? = nil) throws -> GLTFImportResult { + var document: [String: Any] = [ + "asset": ["version": "2.0"], + "images": [["uri": "data:application/octet-stream;base64,AQID"]], + "textures": [["source": 0]], + "materials": [material], + ] + if let used { document["extensionsUsed"] = used } + if let required { document["extensionsRequired"] = required } + return try NativeGLTFLoader().load(data: JSONSerialization.data(withJSONObject: document)) + } + + private func transform(_ values: [String: Any]) -> [String: Any] { + ["extensions": ["KHR_texture_transform": values]] + } + + @Test + func texturesWithoutATransformKeepTheIdentity() throws { + let material = try #require(load(material: ["pbrMetallicRoughness": ["baseColorTexture": ["index": 0]]]).materials.first) + #expect(material.textureTransforms == Array(repeating: .identity, count: 5)) + #expect(material.ior == 1.5 && material.specularFactor == 1 && material.specularColorFactor == .one) + } + + @Test + func importsTextureTransformsPerSlot() throws { + let base = ["index": 0].merging(transform(["offset": [0.25, 0.5], "rotation": 1.5, "scale": [2, 3]])) { $1 } + let normal = ["index": 0].merging(transform(["scale": [4, 4]])) { $1 } + let material = try #require(load(material: [ + "pbrMetallicRoughness": ["baseColorTexture": base], + "normalTexture": normal, + ], used: ["KHR_texture_transform"]).materials.first) + #expect(material.textureTransforms[0] == .init(offset: Vector2(0.25, 0.5), rotation: 1.5, scale: Vector2(2, 3))) + #expect(material.textureTransforms[2] == .init(offset: .zero, rotation: 0, scale: Vector2(4, 4))) + #expect(material.textureTransforms[1] == .identity && material.textureTransforms[4] == .identity) + } + + @Test + func aTransformCanSelectAnotherUVSet() throws { + let texture = ["index": 0, "texCoord": 0].merging(transform(["texCoord": 1])) { $1 } + let material = try #require(load(material: ["emissiveTexture": texture]).materials.first) + #expect(material.textureCoordinates == [0, 0, 0, 0, 1]) + } + + private func object(_ json: String) throws -> [String: Any] { + try #require(JSONSerialization.jsonObject(with: Data(json.utf8)) as? [String: Any]) + } + + @Test(arguments: [#"{"offset":[1]}"#, #"{"scale":[1,2,3]}"#, #"{"texCoord":5}"#]) + func rejectsMalformedTransforms(json: String) throws { + let values = try object(json) + let texture = ["index": 0].merging(transform(values)) { $1 } + #expect(throws: NativeGLTFLoader.GLTFError.invalidMaterial(0)) { + try load(material: ["pbrMetallicRoughness": ["baseColorTexture": texture]]) + } + } + + @Test + func importsIORAndSpecular() throws { + let material = try #require(load(material: ["extensions": [ + "KHR_materials_ior": ["ior": 1.33], + "KHR_materials_specular": ["specularFactor": 0.5, "specularColorFactor": [1, 0.5, 0.25]], + ]], used: ["KHR_materials_ior", "KHR_materials_specular"]).materials.first) + #expect(material.ior == 1.33 && material.specularFactor == 0.5) + #expect(material.specularColorFactor == Vector3(1, 0.5, 0.25)) + } + + @Test(arguments: [ + #"{"KHR_materials_ior":{"ior":0.5}}"#, + #"{"KHR_materials_ior":{"ior":500}}"#, + #"{"KHR_materials_specular":{"specularFactor":-1}}"#, + #"{"KHR_materials_specular":{"specularColorFactor":[1,1]}}"#, + ]) + func rejectsOutOfRangeReflectance(json: String) throws { + let extensions = try object(json) + #expect(throws: NativeGLTFLoader.GLTFError.invalidMaterial(0)) { try load(material: ["extensions": extensions]) } + } + + @Test + func importsClearcoat() throws { + let material = try #require(load(material: ["extensions": [ + "KHR_materials_clearcoat": ["clearcoatFactor": 0.8, "clearcoatRoughnessFactor": 0.1], + ]], used: ["KHR_materials_clearcoat"]).materials.first) + #expect(material.clearcoatFactor == 0.8 && material.clearcoatRoughnessFactor == 0.1) + let plain = try #require(load(material: [:]).materials.first) + #expect(plain.clearcoatFactor == 0 && plain.clearcoatRoughnessFactor == 0) + } + + @Test(arguments: [ + #"{"KHR_materials_clearcoat":{"clearcoatFactor":1.5}}"#, + #"{"KHR_materials_clearcoat":{"clearcoatFactor":-0.1}}"#, + #"{"KHR_materials_clearcoat":{"clearcoatRoughnessFactor":2}}"#, + ]) + func rejectsOutOfRangeClearcoat(json: String) throws { + let extensions = try object(json) + #expect(throws: NativeGLTFLoader.GLTFError.invalidMaterial(0)) { try load(material: ["extensions": extensions]) } + } + + @Test + func importsSheenAndAnisotropy() throws { + let material = try #require(load(material: ["extensions": [ + "KHR_materials_sheen": ["sheenColorFactor": [0.9, 0.5, 0.2], "sheenRoughnessFactor": 0.4], + "KHR_materials_anisotropy": ["anisotropyStrength": 0.6, "anisotropyRotation": 1.2], + ]], used: ["KHR_materials_sheen", "KHR_materials_anisotropy"]).materials.first) + #expect(material.sheenColorFactor == Vector3(0.9, 0.5, 0.2) && material.sheenRoughnessFactor == 0.4) + #expect(material.anisotropyStrength == 0.6 && material.anisotropyRotation == 1.2) + let plain = try #require(load(material: [:]).materials.first) + #expect(plain.sheenColorFactor == .zero && plain.sheenRoughnessFactor == 0) + #expect(plain.anisotropyStrength == 0 && plain.anisotropyRotation == 0) + } + + @Test(arguments: [ + #"{"KHR_materials_sheen":{"sheenColorFactor":[2,0,0]}}"#, + #"{"KHR_materials_sheen":{"sheenColorFactor":[1,1]}}"#, + #"{"KHR_materials_sheen":{"sheenRoughnessFactor":-1}}"#, + #"{"KHR_materials_anisotropy":{"anisotropyStrength":2}}"#, + #"{"KHR_materials_anisotropy":{"anisotropyStrength":-0.5}}"#, + ]) + func rejectsOutOfRangeSheenAndAnisotropy(json: String) throws { + let extensions = try object(json) + #expect(throws: NativeGLTFLoader.GLTFError.invalidMaterial(0)) { try load(material: ["extensions": extensions]) } + } + + @Test + func importsEveryLayerTextureWithItsOwnUVSetAndTransform() throws { + func reference(_ values: [String: Any] = [:]) -> [String: Any] { ["index": 0].merging(values) { $1 } } + let result = try load(material: ["extensions": [ + "KHR_materials_clearcoat": [ + "clearcoatFactor": 1, + "clearcoatTexture": reference(), + "clearcoatRoughnessTexture": reference(["texCoord": 1]), + "clearcoatNormalTexture": reference(["scale": 0.4]), + ], + "KHR_materials_sheen": ["sheenColorTexture": reference(), "sheenRoughnessTexture": reference()], + "KHR_materials_anisotropy": ["anisotropyTexture": reference(transform(["rotation": 0.5]))], + "KHR_materials_specular": [ + "specularTexture": reference(transform(["scale": [2, 3], "texCoord": 1])), + "specularColorTexture": reference(), + ], + ]], used: ["KHR_materials_clearcoat", "KHR_materials_sheen", "KHR_materials_anisotropy", "KHR_materials_specular"]) + let layers = try #require(result.materials.first).layerTextures + #expect(layers.clearcoat == .init(index: 0) && layers.sheenColor == .init(index: 0) && layers.specularColor == .init(index: 0)) + #expect(layers.clearcoatRoughness?.texCoord == 1 && layers.clearcoatNormal?.scale == 0.4) + #expect(layers.anisotropy?.transform.rotation == 0.5) + #expect(layers.specular?.transform.scale == Vector2(2, 3), "A texture transform applies to a layer texture.") + #expect(layers.specular?.texCoord == 1, "A texture transform can pick the UV set for a layer texture.") + #expect(layers.sheenRoughness != nil) + #expect(result.warnings.isEmpty, "Supported layer textures must not be reported as skipped.") + let plain = try #require(load(material: [:]).materials.first) + #expect(plain.layerTextures == .init()) + } + + @Test(arguments: [ + #"{"KHR_materials_clearcoat":{"clearcoatTexture":{"index":9}}}"#, + #"{"KHR_materials_sheen":{"sheenColorTexture":{"index":0,"texCoord":2}}}"#, + #"{"KHR_materials_specular":{"specularTexture":{"index":0,"extensions":{"KHR_texture_transform":{"offset":[1]}}}}}"#, + #"{"KHR_materials_anisotropy":{"anisotropyTexture":{"index":-1}}}"#, + ]) + func rejectsLayerTexturesThatCannotBeRead(json: String) throws { + let extensions = try object(json) + #expect(throws: NativeGLTFLoader.GLTFError.invalidMaterial(0)) { try load(material: ["extensions": extensions]) } + } + + @Test + func unsupportedUsedExtensionsLoadWithAWarning() throws { + let result = try load(material: [:], used: ["KHR_materials_transmission", "KHR_texture_transform"]) + #expect(result.warnings.count == 1) + #expect(result.warnings.first?.contains("KHR_materials_transmission") == true) + } + + @Test + func unsupportedRequiredExtensionsStillFail() { + #expect(throws: NativeGLTFLoader.GLTFError.unsupportedRequiredExtension("KHR_materials_transmission")) { + try load(material: [:], used: ["KHR_materials_transmission"], required: ["KHR_materials_transmission"]) + } + } + + @Test + func theSupportedListNamesEveryImplementedExtension() { + let supported = NativeGLTFLoader.supportedExtensions + for name in ["KHR_texture_transform", "KHR_materials_ior", "KHR_materials_specular", "KHR_materials_clearcoat", "KHR_materials_sheen", "KHR_materials_anisotropy", "KHR_materials_emissive_strength", "KHR_mesh_quantization"] { + #expect(supported.contains(name)) + } + } +} diff --git a/Tests/AdaAssetsTests/GLTFMorphTargetImportTests.swift b/Tests/AdaAssetsTests/GLTFMorphTargetImportTests.swift new file mode 100644 index 000000000..afc6e475f --- /dev/null +++ b/Tests/AdaAssetsTests/GLTFMorphTargetImportTests.swift @@ -0,0 +1,231 @@ +import AdaAssets +import Foundation +import Math +import Testing + +/// A one-triangle glTF with two morph targets, built from a data URI so every case is a few edits of the JSON. +private struct MorphGLTFFixture { + var meshWeights: [Float]? = [0.25, 0.75] + var targetNames: [String]? = ["smile", "blink"] + var nodeWeights: [Float]? + var nodeMatrix: [Float]? + var secondPrimitiveTargetCount: Int? + var targetAttributes: [[String: Int]]? + var sparseFirstTarget = false + var animation: [String: Any]? + + // Layout: 0 positions, 1 target 0 positions, 2 target 1 positions, 3 target 1 normals, 4 times, 5 weights (2 keys x 2 targets), + // 6 sparse indices, 7 sparse values. + static let positions: [Float] = [0, 0, 0, 1, 0, 0, 0, 1, 0] + static let target0: [Float] = [0, 0.5, 0, 0, 0.5, 0, 0, 0.5, 0] + static let target1: [Float] = [1, 0, 0, 1, 0, 0, 1, 0, 0] + static let target1Normals: [Float] = [0, 0, 0.1, 0, 0, 0.1, 0, 0, 0.1] + static let times: [Float] = [0, 1] + static let weightKeys: [Float] = [0, 1, 1, 0] + + func data() throws -> Data { + var bytes = Data() + var views: [[String: Any]] = [] + func add(_ floats: [Float]) { + var chunk = Data() + for value in floats { withUnsafeBytes(of: value.bitPattern.littleEndian) { chunk.append(contentsOf: $0) } } + views.append(["buffer": 0, "byteOffset": bytes.count, "byteLength": chunk.count]) + bytes.append(chunk) + } + add(Self.positions) + add(Self.target0) + add(Self.target1) + add(Self.target1Normals) + add(Self.times) + add(animation?["values"] as? [Float] ?? Self.weightKeys) + // Sparse: vertex 1 of the first target is displaced by (0, 2, 0); the rest stays zero. + var indices = Data() + withUnsafeBytes(of: UInt16(1).littleEndian) { indices.append(contentsOf: $0) } + views.append(["buffer": 0, "byteOffset": bytes.count, "byteLength": 2]) + bytes.append(indices) + while bytes.count % 4 != 0 { bytes.append(0) } + add([0, 2, 0]) + + func accessor(_ view: Int, _ count: Int, _ type: String) -> [String: Any] { + ["bufferView": view, "componentType": 5126, "count": count, "type": type] + } + var accessors: [[String: Any]] = [ + accessor(0, 3, "VEC3"), accessor(1, 3, "VEC3"), accessor(2, 3, "VEC3"), accessor(3, 3, "VEC3"), + accessor(4, 2, "SCALAR"), accessor(5, animation?["count"] as? Int ?? 4, "SCALAR"), + [ + "componentType": 5126, "count": 3, "type": "VEC3", + "sparse": ["count": 1, "indices": ["bufferView": 6, "componentType": 5123], "values": ["bufferView": 7]], + ], + ] + accessors[0]["min"] = [0, 0, 0] + accessors[0]["max"] = [1, 1, 0] + + let targets = targetAttributes ?? [ + ["POSITION": sparseFirstTarget ? 6 : 1], + ["POSITION": 2, "NORMAL": 3], + ] + var primitive: [String: Any] = ["attributes": ["POSITION": 0], "targets": targets] + var primitives: [[String: Any]] = [primitive] + if let count = secondPrimitiveTargetCount { + primitive["targets"] = Array(repeating: ["POSITION": 1], count: count) + primitives.append(primitive) + } + var mesh: [String: Any] = ["primitives": primitives] + if let meshWeights { mesh["weights"] = meshWeights } + if let targetNames { mesh["extras"] = ["targetNames": targetNames] } + var node: [String: Any] = ["mesh": 0] + if let nodeWeights { node["weights"] = nodeWeights } + if let nodeMatrix { node["matrix"] = nodeMatrix } + + var document: [String: Any] = [ + "asset": ["version": "2.0"], + "buffers": [["byteLength": bytes.count, "uri": "data:application/octet-stream;base64," + bytes.base64EncodedString()]], + "bufferViews": views, + "accessors": accessors, + "meshes": [mesh], + "nodes": [node], + "scenes": [["nodes": [0]]], + ] + if let animation { + document["animations"] = [[ + "samplers": [["input": 4, "output": 5, "interpolation": animation["interpolation"] as? String ?? "LINEAR"]], + "channels": [["sampler": 0, "target": ["node": animation["node"] as? Int ?? 0, "path": "weights"]]], + ]] + } + return try JSONSerialization.data(withJSONObject: document) + } +} + +@Suite +struct GLTFMorphTargetImportTests { + @Test + func importsTargetsWeightsAndNames() throws { + let result = try NativeGLTFLoader().load(data: MorphGLTFFixture().data()) + let mesh = try #require(result.meshes.first) + #expect(mesh.morphTargetCount == 2) + #expect(mesh.defaultWeights == [0.25, 0.75] && mesh.targetNames == ["smile", "blink"]) + let targets = try #require(mesh.primitives.first).morphTargets + #expect(targets[0].positionDeltas == [Vector3(0, 0.5, 0), Vector3(0, 0.5, 0), Vector3(0, 0.5, 0)]) + #expect(targets[0].normalDeltas.isEmpty && targets[0].tangentDeltas.isEmpty, "A target without NORMAL leaves normals alone.") + #expect(targets[1].positionDeltas.count == 3 && targets[1].positionDeltas[0] == Vector3(1, 0, 0)) + #expect(targets[1].normalDeltas.first == Vector3(0, 0, 0.1)) + #expect(result.warnings.isEmpty) + } + + @Test + func meshesWithoutTargetsHaveNoMorphData() throws { + var fixture = MorphGLTFFixture() + fixture.targetAttributes = [] + fixture.meshWeights = nil + fixture.targetNames = nil + let result = try NativeGLTFLoader().load(data: fixture.data()) + #expect(result.meshes[0].morphTargetCount == 0 && result.meshes[0].defaultWeights.isEmpty) + #expect(result.meshes[0].primitives[0].morphTargets.isEmpty) + } + + @Test + func missingWeightsDefaultToZero() throws { + var fixture = MorphGLTFFixture() + fixture.meshWeights = nil + fixture.targetNames = nil + let mesh = try #require(NativeGLTFLoader().load(data: fixture.data()).meshes.first) + #expect(mesh.defaultWeights == [0, 0] && mesh.targetNames.isEmpty) + } + + @Test + func sparseTargetAccessorsExpandToDenseDeltas() throws { + var fixture = MorphGLTFFixture() + fixture.sparseFirstTarget = true + let target = try #require(NativeGLTFLoader().load(data: fixture.data()).meshes[0].primitives[0].morphTargets.first) + #expect(target.positionDeltas == [.zero, Vector3(0, 2, 0), .zero]) + } + + @Test + func nodeWeightsOverrideTheMeshWeights() throws { + var fixture = MorphGLTFFixture() + fixture.nodeWeights = [1, 0] + let result = try NativeGLTFLoader().load(data: fixture.data()) + #expect(result.nodes[0].weights == [1, 0] && result.meshes[0].defaultWeights == [0.25, 0.75]) + #expect(try NativeGLTFLoader().load(data: MorphGLTFFixture().data()).nodes[0].weights == nil) + } + + @Test + func rejectsNodeWeightsThatDoNotMatchTheTargets() throws { + var fixture = MorphGLTFFixture() + fixture.nodeWeights = [1] + let data = try fixture.data() + #expect(throws: NativeGLTFLoader.GLTFError.invalidNode(0)) { try NativeGLTFLoader().load(data: data) } + } + + @Test(arguments: ["meshWeights", "names", "primitiveCount", "accessorCount", "withoutPositions"]) + func rejectsInconsistentMorphData(kind: String) throws { + var fixture = MorphGLTFFixture() + switch kind { + case "meshWeights": fixture.meshWeights = [1, 2, 3] + case "names": fixture.targetNames = ["only one"] + case "primitiveCount": fixture.secondPrimitiveTargetCount = 1 + case "accessorCount": fixture.targetAttributes = [["POSITION": 4]] + default: break + } + var data = try fixture.data() + if kind == "withoutPositions" { + var document = try #require(JSONSerialization.jsonObject(with: data) as? [String: Any]) + var meshes = try #require(document["meshes"] as? [[String: Any]]) + var primitives = try #require(meshes[0]["primitives"] as? [[String: Any]]) + primitives[0]["attributes"] = ["NORMAL": 3] + meshes[0]["primitives"] = primitives + document["meshes"] = meshes + data = try JSONSerialization.data(withJSONObject: document) + } + let loaded = data + #expect(throws: NativeGLTFLoader.GLTFError.invalidMorphTargets(0)) { try NativeGLTFLoader().load(data: loaded) } + } + + @Test + func morphedAttributesOtherThanPositionNormalAndTangentAreReported() throws { + var fixture = MorphGLTFFixture() + fixture.targetAttributes = [["POSITION": 1, "TEXCOORD_0": 1], ["POSITION": 2]] + let result = try NativeGLTFLoader().load(data: fixture.data()) + #expect(result.warnings.count == 1 && result.warnings[0].contains("TEXCOORD_0") && result.warnings[0].contains("Mesh 0")) + #expect(result.meshes[0].primitives[0].morphTargets[0].positionDeltas.count == 3) + } + + @Test(arguments: ["LINEAR", "STEP"]) + func importsAnimatedWeights(interpolation: String) throws { + var fixture = MorphGLTFFixture() + fixture.animation = ["interpolation": interpolation] + let result = try NativeGLTFLoader().load(data: fixture.data()) + let animation = try #require(result.animations.first) + #expect(animation.channels[0].path == .weights && animation.channels[0].nodeIndex == 0) + #expect(animation.samplers[0].times == [0, 1]) + #expect(animation.samplers[0].output.values == [0, 1, 1, 0], "Two keys of two target weights.") + #expect(animation.samplers[0].interpolation.rawValue == interpolation) + } + + @Test + func cubicWeightSamplersCarryTangents() throws { + var fixture = MorphGLTFFixture() + let values = [Float](repeating: 0.5, count: 12) + fixture.animation = ["interpolation": "CUBICSPLINE", "values": values, "count": 12] + let sampler = try #require(NativeGLTFLoader().load(data: fixture.data()).animations.first?.samplers.first) + #expect(sampler.output.values.count == 12, "In-tangent, value and out-tangent for each of two targets at each of two keys.") + } + + @Test + func weightChannelsAreValidOnMatrixNodesButNeedTargets() throws { + var fixture = MorphGLTFFixture() + fixture.animation = [:] + fixture.nodeMatrix = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1] + #expect(try NativeGLTFLoader().load(data: fixture.data()).animations.count == 1) + fixture.animation = ["count": 6, "values": [Float](repeating: 0, count: 6)] + let wrongCount = try fixture.data() + #expect(throws: NativeGLTFLoader.GLTFError.invalidAnimation(0)) { try NativeGLTFLoader().load(data: wrongCount) } + var untargeted = MorphGLTFFixture() + untargeted.targetAttributes = [] + untargeted.meshWeights = nil + untargeted.targetNames = nil + untargeted.animation = [:] + let withoutTargets = try untargeted.data() + #expect(throws: NativeGLTFLoader.GLTFError.invalidAnimation(0)) { try NativeGLTFLoader().load(data: withoutTargets) } + } +} diff --git a/Tests/AdaAssetsTests/GLTFSkeletalImportTests.swift b/Tests/AdaAssetsTests/GLTFSkeletalImportTests.swift index da2fe3143..f9da54aeb 100644 --- a/Tests/AdaAssetsTests/GLTFSkeletalImportTests.swift +++ b/Tests/AdaAssetsTests/GLTFSkeletalImportTests.swift @@ -173,11 +173,12 @@ struct GLTFSkeletalImportTests { } @Test - func unsupportedMorphAnimationFailsExplicitly() throws { + func weightsChannelOnANodeWithoutMorphTargetsIsInvalid() throws { + // Weights animation is supported (see GLTFMorphTargetImportTests), but this node's mesh has no targets to weigh. var fixture = SkeletalGLTFFixture() fixture.channels[0]["target"] = ["node": 2, "path": "weights"] let data = try fixture.data() - #expect(throws: NativeGLTFLoader.GLTFError.unsupportedAnimationPath("weights")) { try NativeGLTFLoader().load(data: data) } + #expect(throws: NativeGLTFLoader.GLTFError.invalidAnimation(0)) { try NativeGLTFLoader().load(data: data) } } @Test diff --git a/Tests/AdaEngineTests/GLTFMaterialExtensionMappingTests.swift b/Tests/AdaEngineTests/GLTFMaterialExtensionMappingTests.swift new file mode 100644 index 000000000..cda38991a --- /dev/null +++ b/Tests/AdaEngineTests/GLTFMaterialExtensionMappingTests.swift @@ -0,0 +1,44 @@ +import AdaAssets +@_spi(Internal) @testable import AdaRender +@testable import AdaScene +import Math +import Testing + +@Suite +struct GLTFMaterialExtensionMappingTests { + @Test + func importedExtensionValuesReachThePBRMaterial() throws { + var transforms = Array(repeating: GLTFImportResult.TextureTransform.identity, count: 5) + transforms[0] = .init(offset: Vector2(0.1, 0.2), rotation: 0.5, scale: Vector2(2, 3)) + transforms[2] = .init(scale: Vector2(4, 4)) + let source = GLTFImportResult.Material( + name: nil, + baseColorFactor: .one, + baseColorTextureIndex: nil, + metallicFactor: 0, + roughnessFactor: 0.4, + metallicRoughnessTextureIndex: nil, + normalTextureIndex: nil, + textureTransforms: transforms, + ior: 1.33, + specularFactor: 0.6, + specularColorFactor: Vector3(1, 0.8, 0.6), + clearcoatFactor: 0.7, + clearcoatRoughnessFactor: 0.15, + sheenColorFactor: Vector3(0.5, 0.4, 0.3), + sheenRoughnessFactor: 0.35, + anisotropyStrength: 0.8, + anisotropyRotation: 0.25 + ) + let result = GLTFImportResult(nodes: [], meshes: [], materials: [source], textures: [], images: [], scenes: [], defaultScene: nil) + let material = try #require(ModelAsset3D.makeMaterials(result).first as? PBRMaterial) + #expect(material.textureTransforms[0] == PBRTextureTransform(offset: Vector2(0.1, 0.2), rotation: 0.5, scale: Vector2(2, 3))) + #expect(material.textureTransforms[2] == PBRTextureTransform(scale: Vector2(4, 4))) + #expect(material.textureTransforms[1] == .identity) + #expect(material.ior == 1.33 && material.specularFactor == 0.6) + #expect(material.specularColorFactor == Vector3(1, 0.8, 0.6)) + #expect(material.clearcoatFactor == 0.7 && material.clearcoatRoughnessFactor == 0.15) + #expect(material.sheenColorFactor == Vector3(0.5, 0.4, 0.3) && material.sheenRoughnessFactor == 0.35) + #expect(material.anisotropyStrength == 0.8 && material.anisotropyRotation == 0.25) + } +} diff --git a/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-anisotropy.png b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-anisotropy.png new file mode 100644 index 000000000..403857bf8 Binary files /dev/null and b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-anisotropy.png differ diff --git a/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-clearcoat.png b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-clearcoat.png new file mode 100644 index 000000000..b9141d5fe Binary files /dev/null and b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-clearcoat.png differ diff --git a/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-sheen.png b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-sheen.png new file mode 100644 index 000000000..866fd2ca8 Binary files /dev/null and b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-sphere-sheen.png differ diff --git a/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-fog.png b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-fog.png new file mode 100644 index 000000000..7c60f3f52 Binary files /dev/null and b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-fog.png differ diff --git a/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-light-shafts.png b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-light-shafts.png new file mode 100644 index 000000000..91570ce7d Binary files /dev/null and b/Tests/AdaEngineTests/GoldenFrames/metal/pbr-spheres-light-shafts.png differ diff --git a/Tests/AdaEngineTests/Render3DColorGradingTests.swift b/Tests/AdaEngineTests/Render3DColorGradingTests.swift index 3b9c18009..6fbc544e3 100644 --- a/Tests/AdaEngineTests/Render3DColorGradingTests.swift +++ b/Tests/AdaEngineTests/Render3DColorGradingTests.swift @@ -205,12 +205,21 @@ extension Render3DGPUTests { ) } + // The temporal resolve is not exactly repeatable at a bright highlight: two untouched renders already differ by a + // few steps there, and by a different amount from run to run. Its checks allow that measured drift on top of + // the LUT's own tolerance, so they test the LUT and not the noise floor. + func noise(_ x: Int, _ y: Int, _ channel: Int) -> Float { + guard path == .temporal else { return 0 } + return Self.steps(Self.channels(plain.getPixel(x: x, y: y))[channel], Self.channels(repeated.getPixel(x: x, y: y))[channel]) + } + let identity = AssetHandle(Self.lutTexture()) try scene.configure { $0.postProcess.colorGrading = ColorGrading3D(lut: identity) } let same = try await scene.render(.none, frames: frames) for (x, y) in Self.probes { - for (left, right) in zip(Self.channels(same.getPixel(x: x, y: y)), Self.channels(plain.getPixel(x: x, y: y))) { - #expect(Self.steps(left, right) <= 3, "\(path): identity LUT changed (\(x), \(y)) by \(Self.steps(left, right)) steps.") + for (channel, (left, right)) in zip(Self.channels(same.getPixel(x: x, y: y)), Self.channels(plain.getPixel(x: x, y: y))).enumerated() { + let allowed = 3 + noise(x, y, channel) + #expect(Self.steps(left, right) <= allowed, "\(path): identity LUT changed (\(x), \(y)) channel \(channel) by \(Self.steps(left, right)) steps (allowed \(allowed)).") } } @@ -222,7 +231,7 @@ extension Render3DGPUTests { for (x, y) in Self.probes { let base = Self.channels(plain.getPixel(x: x, y: y)) for (index, value) in Self.channels(inverted.getPixel(x: x, y: y)).enumerated() { - #expect(Self.steps(value, 1 - base[index]) <= 5, "\(path): inverted LUT at (\(x), \(y)) channel \(index): \(value * 255) vs \((1 - base[index]) * 255).") + #expect(Self.steps(value, 1 - base[index]) <= 5 + noise(x, y, index), "\(path): inverted LUT at (\(x), \(y)) channel \(index): \(value * 255) vs \((1 - base[index]) * 255).") } // Halfway between a color and its inverse is mid gray whatever the color. for value in Self.channels(halfway.getPixel(x: x, y: y)) { diff --git a/Tests/AdaEngineTests/Render3DFogTests.swift b/Tests/AdaEngineTests/Render3DFogTests.swift new file mode 100644 index 000000000..33261b72c --- /dev/null +++ b/Tests/AdaEngineTests/Render3DFogTests.swift @@ -0,0 +1,240 @@ +#if os(macOS) +import AdaECS +@_spi(Internal) import AdaRender +import AdaScene +import AdaUtils +import Foundation +import Math +import Testing + +@testable import AdaCorePipelines + +/// Fog settings that need no GPU. +@Suite("3D fog settings") +struct Render3DFogSettingsTests { + @Test func defaultsAreOff() { + let fog = Fog3D() + #expect(!fog.isEnabled && !fog.changesImage) + #expect(Environment3D().fog == fog) + } + + @Test func olderPayloadsDecodeWithoutFog() throws { + let environment = try JSONDecoder().decode(Environment3D.self, from: Data("{}".utf8)) + #expect(environment.fog == Fog3D()) + let partial = try JSONDecoder().decode(Fog3D.self, from: Data(#"{"isEnabled":true,"density":0.1}"#.utf8)) + #expect(partial.isEnabled && partial.density == 0.1 && partial.maximumOpacity == Fog3D().maximumOpacity) + } + + @Test func settingsRoundTripAndFlatAccessors() throws { + var environment = Environment3D() + environment.fogEnabled = true + environment.fogDensity = 0.3 + environment.fog.heightFalloff = 0.5 + let decoded = try JSONDecoder().decode(Environment3D.self, from: JSONEncoder().encode(environment)) + #expect(decoded.fog == environment.fog) + #expect(decoded.fogEnabled && decoded.fogDensity == 0.3) + } + + @Test func wildValuesAreClamped() { + let wild = Fog3D(isEnabled: true, density: .nan, heightFalloff: -4, startDistance: -1, maximumOpacity: 7, skyInfluence: 2, sunExponent: 0).sanitized + #expect(wild.density == Fog3D().density) + #expect(wild.heightFalloff == 0 && wild.startDistance == 0) + #expect(wild.maximumOpacity == 1 && wild.skyInfluence == 1 && wild.sunExponent == 1) + } + + @Test func volumetricDefaultsAreOffAndOlderPayloadsDecode() throws { + #expect(!VolumetricFog3D().isEnabled && !Fog3D(isEnabled: true).volumetricActive) + let decoded = try JSONDecoder().decode(Fog3D.self, from: Data(#"{"isEnabled":true}"#.utf8)) + #expect(decoded.volumetric == VolumetricFog3D()) + let partial = try JSONDecoder().decode(Fog3D.self, from: Data(#"{"volumetric":{"isEnabled":true,"steps":40}}"#.utf8)) + #expect(partial.volumetric.isEnabled && partial.volumetric.steps == 40) + } + + @Test func volumetricRequiresFogAndClampsValues() { + var fog = Fog3D(isEnabled: false, volumetric: VolumetricFog3D(isEnabled: true)) + #expect(!fog.volumetricActive, "Light shafts need the fog medium.") + fog.isEnabled = true + #expect(fog.volumetricActive) + fog.volumetric.intensity = 0 + #expect(!fog.volumetricActive) + let wild = VolumetricFog3D(isEnabled: true, intensity: .nan, anisotropy: 5, steps: 1000, maxDistance: -3).sanitized + #expect(wild.intensity == VolumetricFog3D().intensity && wild.anisotropy == 0.95) + #expect(wild.steps == 64 && wild.maxDistance == 1) + #expect(VolumetricFog3D(steps: -2).sanitized.steps == 4) + } + + @Test func settingsThatCannotChangeTheImageAreNotCountedAsAnEffect() { + #expect(!Fog3D(isEnabled: false).changesImage) + #expect(!Fog3D(isEnabled: true, density: 0).changesImage) + #expect(!Fog3D(isEnabled: true, maximumOpacity: 0).changesImage) + #expect(Fog3D(isEnabled: true).changesImage) + } +} + +extension Render3DGPUTests { + /// Renders fog through the real composite pass. + /// + /// Run alone: `ADAENGINE_GOLDEN_METAL=1 swift test --filter Render3DGPUTests/Fog`. + @Suite("3D fog on GPU") + @MainActor + struct Fog { + private static let fogColor = Color(red: 1, green: 0, blue: 0) + /// A pixel on the first sphere's lit side, 3.4 units from the camera. + private static let sphere = (x: 100, y: 215) + + /// How far the pixel is from the fog color, as the red channel's lead over blue. + private static func redness(_ image: Image) -> Float { + let pixel = image.getPixel(x: sphere.x, y: sphere.y) + return pixel.red - pixel.blue + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func fogOutOfReachChangesNothing() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let baseline = try await scene.render(.none) + let unreachable: [Fog3D] = [ + Fog3D(isEnabled: false, color: Self.fogColor, density: 2), + Fog3D(isEnabled: true, color: Self.fogColor, density: 2, startDistance: 50), + Fog3D(isEnabled: true, color: Self.fogColor, density: 2, heightFalloff: 10, baseHeight: -50), + ] + for fog in unreachable { + try scene.configure { $0.fog = fog } + let image = try await scene.render(.none) + #expect(GoldenFrame.compare(baseline, image, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance)) + } + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func denserFogPullsSurfacesTowardTheFogColor() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let clear = Self.redness(try await scene.render(.none)) + try scene.configure { $0.fog = Fog3D(isEnabled: true, color: Self.fogColor, density: 0.1) } + let light = Self.redness(try await scene.render(.none)) + try scene.configure { $0.fog.density = 1 } + let dense = Self.redness(try await scene.render(.none)) + #expect(light > clear && dense > light, "Fog must tint monotonically with density (\(clear), \(light), \(dense)).") + try scene.configure { $0.fog.maximumOpacity = 0.2 } + let capped = Self.redness(try await scene.render(.none)) + #expect(capped < dense, "A lower maximum opacity must leave more of the surface (\(capped) vs \(dense)).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func heightFogThinsAboveItsBaseHeight() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + // The camera and spheres sit near y = 0.5, so a base at 0.5 gives full density and one far above gives none. + try scene.configure { $0.fog = Fog3D(isEnabled: true, color: Self.fogColor, density: 0.5, heightFalloff: 1, baseHeight: 0.5) } + let inside = Self.redness(try await scene.render(.none)) + try scene.configure { $0.fog.baseHeight = -8 } + let above = Self.redness(try await scene.render(.none)) + #expect(inside > above, "Fog must be thinner when the camera is above its base (\(inside) vs \(above)).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func fogMatchesGoldenFrames() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene(farMarker: true) + try scene.configure { + $0.fog = Fog3D( + isEnabled: true, + color: Color(red: 0.62, green: 0.72, blue: 0.82), + density: 0.12, + heightFalloff: 0.3, + baseHeight: 0, + startDistance: 1, + maximumOpacity: 0.9, + skyInfluence: 0.5, + sunInscattering: 0.4, + sunExponent: 6 + ) + } + switch try GoldenFrame.check(try await scene.render(.none), name: "pbr-spheres-fog") { + case .recorded, .matched: + break + case .missingReference(let actual): + Issue.record("No golden frame for pbr-spheres-fog; record with ADAENGINE_GOLDEN_RECORD=1. Actual: \(actual.path)") + case let .mismatched(comparison, actual, difference): + Issue.record("pbr-spheres-fog differs in \(comparison.differingPixels) pixels. Actual: \(actual.path), difference: \(difference.path)") + } + } + + private static func meanLuminance(_ image: Image) -> Float { + var sum: Float = 0 + for y in stride(from: 0, to: 480, by: 4) { + for x in stride(from: 0, to: 640, by: 4) { + let pixel = image.getPixel(x: x, y: y) + sum += (pixel.red + pixel.green + pixel.blue) / 3 + } + } + return sum / Float(120 * 160) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func lightShaftsAddSunLightAndReleaseTheirTexture() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene(farMarker: true) + let renderWorld = try #require(scene.app.getSubworldBuilder(by: .renderWorld)?.main) + func shafts() throws -> VolumetricFog3DViews { try #require(renderWorld.getResource(VolumetricFog3DViews.self)) } + let fog = Fog3D(isEnabled: true, color: Color(red: 0.2, green: 0.2, blue: 0.2), density: 0.15, maximumOpacity: 0.6) + try scene.configure { $0.fog = fog } + let plain = try await scene.render(.none) + #expect(try shafts().output(view: scene.camera.id) == nil) + try scene.configure { $0.fog.volumetric = VolumetricFog3D(isEnabled: true, intensity: 4, anisotropy: 0, steps: 24, maxDistance: 20) } + let lit = try await scene.render(.none) + #expect(try shafts().output(view: scene.camera.id) != nil) + #expect(Self.meanLuminance(lit) > Self.meanLuminance(plain), "Light shafts must add sun light to the fog.") + try scene.configure { $0.fog.volumetric.isEnabled = false } + let released = try await scene.render(.none) + #expect(try shafts().output(view: scene.camera.id) == nil, "Turning the effect off must release the texture.") + #expect(GoldenFrame.compare(plain, released, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance)) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func shadowsCarveTheLightShafts() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene(farMarker: true) + let fog = Fog3D( + isEnabled: true, + color: Color(red: 0.2, green: 0.2, blue: 0.2), + density: 0.15, + maximumOpacity: 0.6, + volumetric: VolumetricFog3D(isEnabled: true, intensity: 4, anisotropy: 0, steps: 32, maxDistance: 20) + ) + try scene.configure { $0.fog = fog; $0.shadowsEnabled = true } + let shadowed = try await scene.render(.none) + try scene.configure { $0.shadowsEnabled = false } + let unshadowed = try await scene.render(.none) + #expect(Self.meanLuminance(unshadowed) > Self.meanLuminance(shadowed), "Unshadowed shafts must be brighter than shadowed ones.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func lightShaftsMatchGoldenFrames() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene(farMarker: true) + try scene.configure { + $0.fog = Fog3D( + isEnabled: true, + color: Color(red: 0.4, green: 0.45, blue: 0.5), + density: 0.1, + maximumOpacity: 0.7, + volumetric: VolumetricFog3D(isEnabled: true, intensity: 3, anisotropy: 0.3, steps: 32, maxDistance: 25) + ) + } + switch try GoldenFrame.check(try await scene.render(.none), name: "pbr-spheres-light-shafts") { + case .recorded, .matched: + break + case .missingReference(let actual): + Issue.record("No golden frame for pbr-spheres-light-shafts; record with ADAENGINE_GOLDEN_RECORD=1. Actual: \(actual.path)") + case let .mismatched(comparison, actual, difference): + Issue.record("pbr-spheres-light-shafts differs in \(comparison.differingPixels) pixels. Actual: \(actual.path), difference: \(difference.path)") + } + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func skyInfluenceControlsFogOverTheSky() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + func sky(_ image: Image) -> Float { image.getPixel(x: 10, y: 10).red - image.getPixel(x: 10, y: 10).blue } + try scene.configure { $0.fog = Fog3D(isEnabled: true, color: Self.fogColor, density: 0.5) } + let untouched = sky(try await scene.render(.none)) + try scene.configure { $0.fog.skyInfluence = 1 } + let fogged = sky(try await scene.render(.none)) + #expect(fogged > untouched, "Sky influence must tint the sky (\(untouched) vs \(fogged)).") + } + } +} +#endif diff --git a/Tests/AdaEngineTests/Render3DLightTilesTests.swift b/Tests/AdaEngineTests/Render3DLightTilesTests.swift new file mode 100644 index 000000000..e59065017 --- /dev/null +++ b/Tests/AdaEngineTests/Render3DLightTilesTests.swift @@ -0,0 +1,120 @@ +#if os(macOS) +import AdaECS +@_spi(Internal) import AdaRender +import AdaScene +import AdaUtils +import Foundation +import Math +import Testing + +@testable import AdaCorePipelines + +/// Tile bounds for local lights; no GPU needed. +@Suite("3D light tiles") +struct Render3DLightTilesMathTests { + private static let width = 640, height = 480 + private static let layout = LightTiles3DMath.layout(width: width, height: height) + private static let viewport = Rect(x: 0, y: 0, width: Float(width), height: Float(height)) + private static let projection = Transform3D.perspective(fieldOfView: .degrees(60), aspectRatio: Float(width) / Float(height), zNear: 0.1, zFar: 100) + + private static func masks(_ lights: [(center: Vector3, range: Float)]) -> [UInt32] { + LightTiles3DMath.masks(spheres: lights, projection: projection, viewport: viewport, layout: layout) + } + + private static func tile(x: Int, y: Int) -> Int { + (y / layout.tileSize) * layout.columns + x / layout.tileSize + } + + @Test func layoutKeepsTileCountWithinTheMaskBuffer() { + let full = LightTiles3DMath.layout(width: 1920, height: 1080) + #expect(full == .init(tileSize: 32, columns: 60, rows: 34)) + for (width, height) in [(3840, 2160), (7680, 4320), (1, 1), (100_000, 10)] { + let layout = LightTiles3DMath.layout(width: width, height: height) + #expect(layout.count <= LightTiles3DMath.maximumTiles && layout.columns * layout.tileSize >= width && layout.rows * layout.tileSize >= height) + } + } + + @Test func allLightsSetsTheLowBits() { + #expect(LightTiles3DMath.allLights(0) == 0) + #expect(LightTiles3DMath.allLights(3) == 0b111) + #expect(LightTiles3DMath.allLights(32) == UInt32.max) + #expect(LightTiles3DMath.allLights(99) == UInt32.max) + } + + @Test func aSmallLightAheadReachesOnlyTilesAroundItsProjection() { + let masks = Self.masks([(center: [0, 0, 10], range: 1)]) + #expect(masks[Self.tile(x: 320, y: 240)] == 1, "The tile under the light must list it.") + #expect(masks[Self.tile(x: 5, y: 5)] == 0 && masks[Self.tile(x: 630, y: 470)] == 0, "Far corners cannot be reached.") + #expect(masks.filter { $0 != 0 }.count < Self.layout.count / 4) + } + + @Test func lightsBehindTheCameraOrOffToTheSideReachNothing() { + let behind = Self.masks([(center: [0, 0, -10], range: 2)]) + #expect(behind.allSatisfy { $0 == 0 }) + let aside = Self.masks([(center: [100, 0, 10], range: 1)]) + #expect(aside.allSatisfy { $0 == 0 }) + } + + @Test func aLightReachingTheCameraPlaneCoversEveryTile() { + let masks = Self.masks([(center: [0, 0, 1], range: 3)]) + #expect(masks.allSatisfy { $0 == 1 }) + } + + @Test func eachLightGetsItsOwnBit() { + let masks = Self.masks([(center: [-4, 0, 10], range: 1), (center: [4, 0, 10], range: 1)]) + #expect(masks[Self.tile(x: 100, y: 240)] & 1 != 0 && masks[Self.tile(x: 100, y: 240)] & 2 == 0) + #expect(masks[Self.tile(x: 540, y: 240)] & 2 != 0 && masks[Self.tile(x: 540, y: 240)] & 1 == 0) + } + + @Test func lightsBeyondThirtyTwoAreIgnored() { + let lights = (0..<40).map { _ in (center: Vector3(0, 0, 10), range: Float(1)) } + let masks = Self.masks(lights) + #expect(masks[Self.tile(x: 320, y: 240)] == UInt32.max) + } + + @Test func aViewportOffsetMovesTheTiles() { + let offset = Rect(x: 320, y: 0, width: 320, height: 480) + let masks = LightTiles3DMath.masks(spheres: [(center: [0, 0, 10], range: 0.5)], projection: Self.projection, viewport: offset, layout: Self.layout) + #expect(masks[Self.tile(x: 480, y: 240)] == 1 && masks[Self.tile(x: 100, y: 240)] == 0) + } + + @Test func tilingDefaultsOnAndOlderPayloadsDecode() throws { + #expect(Environment3D().tiledLocalLights) + #expect(try JSONDecoder().decode(Environment3D.self, from: Data("{}".utf8)).tiledLocalLights) + var environment = Environment3D() + environment.tiledLocalLights = false + #expect(try !JSONDecoder().decode(Environment3D.self, from: JSONEncoder().encode(environment)).tiledLocalLights) + } +} + +extension Render3DGPUTests { + /// Run alone: `ADAENGINE_GOLDEN_METAL=1 swift test --filter Render3DGPUTests/LightTiles`. + @Suite("3D light tiles on GPU") + @MainActor + struct LightTiles { + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func tiledLightingMatchesTheFlatLoop() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + for index in 0..<10 { + let side: Float = index % 2 == 0 ? -1 : 1 + scene.app.main.spawn("Fill \(index)") { + PointLightComponent(radiance: [0.4 + 0.06 * Float(index), 0.6, 1 - 0.07 * Float(index)], intensity: 6, castShadows: false, range: 2.2) + Transform(position: [side * Float(index) * 0.45, 0.4 + Float(index % 3) * 0.3, Float(index) * 0.2 - 0.6]) + } + } + scene.app.main.spawn("Shadowed") { + PointLightComponent(radiance: [1, 0.8, 0.6], intensity: 8, castShadows: true, range: 4) + Transform(position: [0, 1.5, -1]) + } + try scene.configure { $0.tiledLocalLights = false } + let flat = try await scene.render(.none) + try scene.configure { $0.tiledLocalLights = true } + let tiled = try await scene.render(.none) + #expect( + GoldenFrame.compare(flat, tiled, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance), + "Tile masks are conservative, so they must not change the lit image." + ) + } + } +} +#endif diff --git a/Tests/AdaEngineTests/Render3DMaterialExtensionTests.swift b/Tests/AdaEngineTests/Render3DMaterialExtensionTests.swift new file mode 100644 index 000000000..25ed81625 --- /dev/null +++ b/Tests/AdaEngineTests/Render3DMaterialExtensionTests.swift @@ -0,0 +1,585 @@ +#if os(macOS) +import AdaAssets +import AdaECS +@_spi(Internal) import AdaRender +@testable import AdaScene +import AdaTransform +import AdaUtils +import Foundation +import Math +import Testing + +@testable import AdaCorePipelines + +/// How material extension values become shader constants; no GPU needed. +@Suite("3D material extension constants") +struct Render3DMaterialExtensionConstantTests { + @Test func defaultsGiveTheFourPercentReflectanceAndIdentityTransforms() { + let uniform = PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()) + #expect(abs(uniform.dielectric.x - 0.04) < 0.000001 && uniform.dielectric.x == uniform.dielectric.y) + for transform in [uniform.transform0, uniform.transform1, uniform.transform2, uniform.transform3, uniform.transform4] { + #expect(transform.matrix == Vector4(1, 0, 0, 1) && transform.offset == .zero) + } + let fallback = PBR3DUniform(material: nil, descriptor: VertexDescriptor()) + #expect(abs(fallback.dielectric.x - 0.04) < 0.000001) + } + + @Test func iorAndSpecularScaleTheDielectricReflectance() { + let material = PBRMaterial() + material.ior = 1 + #expect(PBR3DUniform.dielectricReflectance(material) == .zero) + material.ior = 3 + #expect(abs(PBR3DUniform.dielectricReflectance(material).x - 0.25) < 0.000001) + material.specularFactor = 0.5 + #expect(abs(PBR3DUniform.dielectricReflectance(material).x - 0.125) < 0.000001) + material.specularColorFactor = Vector3(1, 0.5, 0) + let tinted = PBR3DUniform.dielectricReflectance(material) + #expect(tinted.x > tinted.y && tinted.y > tinted.z && tinted.z == 0) + material.ior = 100 + material.specularFactor = 1 + material.specularColorFactor = Vector3(50, 50, 50) + #expect(PBR3DUniform.dielectricReflectance(material).x <= 1, "The tint cannot push reflectance beyond total reflection.") + } + + @Test func clearcoatDefaultsToOffAndClampsToTheUnitRange() { + #expect(PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()).coat == .zero) + #expect(PBR3DUniform(material: nil, descriptor: VertexDescriptor()).coat == .zero) + let material = PBRMaterial() + material.clearcoatFactor = 0.6 + material.clearcoatRoughnessFactor = 0.2 + #expect(PBR3DUniform(material: material, descriptor: VertexDescriptor()).coat == Vector4(0.6, 0.2, 0, 0)) + material.clearcoatFactor = 7 + material.clearcoatRoughnessFactor = .nan + let wild = PBR3DUniform(material: material, descriptor: VertexDescriptor()).coat + #expect(wild.x == 1 && wild.y.isFinite) + let plain = PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()) + #expect(plain.bitPatterns != PBR3DUniform(material: material, descriptor: VertexDescriptor()).bitPatterns) + } + + @Test func sheenAndAnisotropyDefaultToOffAndClamp() { + let plain = PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()) + #expect(plain.sheen == .zero && plain.anisotropy == .zero) + let material = PBRMaterial() + material.sheenColorFactor = Vector3(0.9, 0.5, 0.2) + material.sheenRoughnessFactor = 0.4 + material.anisotropyStrength = 0.6 + material.anisotropyRotation = 1.2 + let value = PBR3DUniform(material: material, descriptor: VertexDescriptor()) + #expect(value.sheen == Vector4(0.9, 0.5, 0.2, 0.4) && value.anisotropy == Vector4(0.6, 1.2, 0, 0)) + material.sheenColorFactor = Vector3(5, .nan, -1) + material.anisotropyStrength = .infinity + let wild = PBR3DUniform(material: material, descriptor: VertexDescriptor()) + #expect(wild.sheen.x == 1 && wild.sheen.y.isFinite && wild.sheen.z == 0 && wild.anisotropy.x.isFinite) + #expect(plain.bitPatterns != value.bitPatterns && value.bitPatterns != wild.bitPatterns) + } + + @Test func shaderFeaturesFollowTheLayersInUse() { + let material = PBRMaterial() + #expect(material.shaderFeatures.isEmpty && material.shaderFeatures.enablingMacros.isEmpty) + material.clearcoatFactor = 0.5 + #expect(material.shaderFeatures == [.clearcoat]) + material.sheenColorFactor = Vector3(0, 0.2, 0) + material.anisotropyStrength = 0.3 + #expect(material.shaderFeatures == .all) + #expect(Set(material.shaderFeatures.enablingMacros) == ["ADA_CLEARCOAT", "ADA_SHEEN", "ADA_ANISOTROPY"]) + material.clearcoatFactor = 0 + material.anisotropyStrength = 0 + #expect(material.shaderFeatures == [.sheen] && material.shaderFeatures.enablingMacros == ["ADA_SHEEN"]) + // A roughness or rotation alone does not use a layer. + let inert = PBRMaterial() + inert.clearcoatRoughnessFactor = 0.4 + inert.sheenRoughnessFactor = 0.4 + inert.anisotropyRotation = 1 + #expect(inert.shaderFeatures.isEmpty) + } + + @Test func pipelineKeysSplitByFeaturesAndDefaultToThePlainShader() { + let vertex = VertexDescriptor() + let plain = PBR3DConfiguration(vertex: vertex, blended: false) + #expect(plain.features.isEmpty) + let coated = PBR3DConfiguration(vertex: vertex, blended: false, features: .clearcoat) + #expect(plain != coated && coated == PBR3DConfiguration(vertex: vertex, blended: false, features: .clearcoat)) + #expect(Set([plain, coated, PBR3DConfiguration(vertex: vertex, blended: false, features: .all)]).count == 3) + } + + @Test func nonFiniteValuesFallBackToDefaults() { + let material = PBRMaterial() + material.ior = .nan + material.specularFactor = .infinity + material.specularColorFactor = Vector3(.nan, 1, 1) + let value = PBR3DUniform.dielectricReflectance(material) + #expect(value.x.isFinite && value.y.isFinite && value.z.isFinite) + } + + @Test func textureTransformFollowsTheGlTFMatrix() { + let rotated = PBR3DTextureTransform(PBRTextureTransform(offset: Vector2(0.5, 0.25), rotation: .pi / 2, scale: Vector2(2, 3))) + // uv' = [cos*sx*u + sin*sy*v, -sin*sx*u + cos*sy*v] + offset + #expect(abs(rotated.matrix.x) < 0.00001 && abs(rotated.matrix.y - 3) < 0.00001) + #expect(abs(rotated.matrix.z + 2) < 0.00001 && abs(rotated.matrix.w) < 0.00001) + #expect(rotated.offset == Vector4(0.5, 0.25, 0, 0)) + } + + @Test func materialEditsChangeTheRevisionAndTheSharedBuffer() { + let material = PBRMaterial() + let before = material.uniformRevision + material.textureTransforms[0] = PBRTextureTransform(scale: Vector2(2, 2)) + material.ior = 1.8 + #expect(material.uniformRevision > before) + let plain = PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()) + let edited = PBR3DUniform(material: material, descriptor: VertexDescriptor()) + #expect(plain != edited && plain.bitPatterns != edited.bitPatterns) + #expect(plain.bitPatterns == PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()).bitPatterns) + } +} + +extension Render3DGPUTests { + /// Run alone: `ADAENGINE_GOLDEN_METAL=1 swift test --filter Render3DGPUTests/MaterialExtensions`. + @Suite("3D material extensions on GPU") + @MainActor + struct MaterialExtensions { + private static func checker() -> Texture2D { + var image = Image(width: 8, height: 8, color: .white) + for y in 0..<8 { + for x in 0..<8 where (x / 2 + y / 2) % 2 == 0 { + image.setPixel(in: Point(Float(x), Float(y)), color: Color(red: 0.9, green: 0.1, blue: 0.1)) + } + } + return Texture2D(image: image) + } + + /// A large sphere in front of the camera, in the middle of the frame. + private static func makeScene(polesSideways: Bool = false, _ configure: (PBRMaterial) -> Void) async throws -> (scene: Render3DGPUTests.DebugViews.Scene, material: PBRMaterial) { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let material = PBRMaterial() + material.baseColorFactor = [0.5, 0.5, 0.5, 1] + material.metallicFactor = 0 + material.roughnessFactor = 0.2 + configure(material) + scene.app.main.spawn("Probe") { + Mesh3DComponent(mesh: Mesh.generateSphere(radius: 0.5, renderDevice: scene.device), materials: [material]) + Transform(rotation: polesSideways ? Quat(axis: Vector3(0, 0, 1), angle: .pi / 2) : .identity, position: [0, 0.5, -2.2]) + } + return (scene, material) + } + + private static func peakHighlight(_ image: Image) -> Color { + var peak = Color.black + var best: Float = 0 + for y in stride(from: 130, to: 350, by: 2) { + for x in stride(from: 170, to: 470, by: 2) { + let pixel = image.getPixel(x: x, y: y) + let value = pixel.red + pixel.green + pixel.blue + if value > best { + best = value + peak = pixel + } + } + } + return peak + } + + /// The largest brightness a frame gained over `baseline` anywhere in the middle of the frame. + private static func largestGain(_ image: Image, over baseline: Image) -> Float { + var gain: Float = 0 + for y in stride(from: 130, to: 350, by: 2) { + for x in stride(from: 170, to: 470, by: 2) { + let now = image.getPixel(x: x, y: y), before = baseline.getPixel(x: x, y: y) + gain = max(gain, (now.red + now.green + now.blue) - (before.red + before.green + before.blue)) + } + } + return gain + } + + /// The largest shift toward red (red minus blue) a frame gained over `baseline` in the middle of the frame. + private static func largestRedShift(_ image: Image, over baseline: Image) -> Float { + var shift: Float = 0 + for y in stride(from: 130, to: 350, by: 2) { + for x in stride(from: 170, to: 470, by: 2) { + let now = image.getPixel(x: x, y: y), before = baseline.getPixel(x: x, y: y) + shift = max(shift, (now.red - now.blue) - (before.red - before.blue)) + } + } + return shift + } + + private static func matches(_ left: Image, _ right: Image) -> Bool { + GoldenFrame.compare(left, right, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func textureTransformsMoveScaleAndRotateTheTexture() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorTexture = Self.checker() } + let plain = try await scene.render(.none) + material.textureTransforms[0] = PBRTextureTransform(scale: Vector2(3, 3)) + let scaled = try await scene.render(.none) + material.textureTransforms[0] = PBRTextureTransform(offset: Vector2(0.3, 0.1)) + let shifted = try await scene.render(.none) + material.textureTransforms[0] = PBRTextureTransform(rotation: 0.7) + let rotated = try await scene.render(.none) + for (name, image) in [("scale", scaled), ("offset", shifted), ("rotation", rotated)] { + #expect(!Self.matches(plain, image), "A \\(name) transform must change how the texture appears.") + } + material.textureTransforms[0] = PBRTextureTransform(rotation: 2 * .pi) + #expect(Self.matches(plain, try await scene.render(.none)), "A full turn must leave the texture where it was.") + material.textureTransforms[0] = .identity + #expect(Self.matches(plain, try await scene.render(.none)), "Resetting to the identity must restore the frame.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aHigherIORBrightensTheHighlightAndZeroSpecularRemovesIt() async throws { + let (scene, material) = try await Self.makeScene { _ in } + let standard = Self.peakHighlight(try await scene.render(.none)) + material.ior = 3 + let strong = Self.peakHighlight(try await scene.render(.none)) + material.ior = 1.5 + material.specularFactor = 0 + let none = Self.peakHighlight(try await scene.render(.none)) + func total(_ color: Color) -> Float { color.red + color.green + color.blue } + #expect(total(strong) > total(standard), "An IOR of 3 reflects more than 1.5 (\\(total(strong)) vs \\(total(standard))).") + #expect(total(none) < total(standard), "No specular strength must dim the highlight (\\(total(none)) vs \\(total(standard))).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func specularColorTintsTheHighlight() async throws { + let (scene, material) = try await Self.makeScene { $0.ior = 2.5 } + let neutral = Self.peakHighlight(try await scene.render(.none)) + material.specularColorFactor = Vector3(1, 0, 0) + let red = Self.peakHighlight(try await scene.render(.none)) + #expect(red.red - red.blue > neutral.red - neutral.blue, "A red specular tint must shift the highlight toward red.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func clearcoatAddsASharpHighlightToARoughSurface() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.9 } + let baseline = try await scene.render(.none) + material.clearcoatFactor = 1 + material.clearcoatRoughnessFactor = 0.05 + let glossy = Self.largestGain(try await scene.render(.none), over: baseline) + material.clearcoatRoughnessFactor = 0.6 + let blurred = Self.largestGain(try await scene.render(.none), over: baseline) + #expect(glossy > 0.3, "A smooth coat must add a bright highlight to rough paint (gain \(glossy)).") + #expect(glossy > blurred, "A rougher coat must spread and dim the highlight (\(glossy) vs \(blurred)).") + material.clearcoatFactor = 0 + #expect(Self.matches(baseline, try await scene.render(.none)), "Turning the coat off must restore the frame.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func clearcoatStrengthScalesTheCoatMonotonically() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.9; $0.clearcoatRoughnessFactor = 0.05 } + let baseline = try await scene.render(.none) + var gains: [Float] = [] + for strength: Float in [0.3, 0.7, 1] { + material.clearcoatFactor = strength + gains.append(Self.largestGain(try await scene.render(.none), over: baseline)) + } + #expect(zip(gains, gains.dropFirst()).allSatisfy { $0 < $1 }, "The coat's highlight must grow with its strength: \(gains)") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func sheenAddsSoftLightThatGrowsWithItsColorAndTurnsOff() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.9 } + let baseline = try await scene.render(.none) + material.sheenRoughnessFactor = 0.4 + var gains: [Float] = [] + for strength: Float in [0.3, 0.7, 1] { + material.sheenColorFactor = Vector3(strength, strength, strength) + gains.append(Self.largestGain(try await scene.render(.none), over: baseline)) + } + #expect(gains[0] > 0.05, "Sheen must add light (gain \(gains[0])).") + #expect(zip(gains, gains.dropFirst()).allSatisfy { $0 < $1 }, "Sheen must grow with its color: \(gains)") + material.sheenColorFactor = .zero + #expect(Self.matches(baseline, try await scene.render(.none)), "Turning sheen off must restore the frame.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func sheenColorTintsTheAddedLight() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.9; $0.sheenRoughnessFactor = 0.4 } + material.sheenColorFactor = Vector3(1, 0, 0) + let red = try await scene.render(.none) + material.sheenColorFactor = Vector3(0, 0, 1) + let blue = try await scene.render(.none) + var redExcess: Float = 0 + for y in stride(from: 130, to: 350, by: 2) { + for x in stride(from: 170, to: 470, by: 2) { + let a = red.getPixel(x: x, y: y), b = blue.getPixel(x: x, y: y) + redExcess += (a.red - a.blue) - (b.red - b.blue) + } + } + #expect(redExcess > 0, "A red sheen must leave the surface redder than a blue one (\(redExcess)).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func anisotropyStretchesTheHighlightAndRotationTurnsIt() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.35 } + let round = try await scene.render(.none) + material.anisotropyStrength = 0.8 + let stretched = try await scene.render(.none) + material.anisotropyRotation = .pi / 2 + let turned = try await scene.render(.none) + #expect(!Self.matches(round, stretched), "Anisotropy must change the highlight.") + #expect(!Self.matches(stretched, turned), "Rotating the stretch direction must change the highlight.") + material.anisotropyRotation = .pi + #expect(Self.matches(stretched, try await scene.render(.none)), "Half a turn flips the direction and must look the same.") + material.anisotropyStrength = 0 + material.anisotropyRotation = 0 + #expect(Self.matches(round, try await scene.render(.none)), "Zero strength must restore the round highlight.") + } + + // MARK: Layer textures + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func layerTexturesSelectShaderFeaturesAndReachTheUniform() async throws { + _ = try await Render3DDebugViewGPUTests.makeScene() + let material = PBRMaterial() + // A layer texture scales its layer's factor, so without the layer it selects nothing. + material.layerTextures[.clearcoat] = PBRTextureBinding(texture: Self.solid(1)) + material.layerTextures[.sheenColor] = PBRTextureBinding(texture: Self.solid(1)) + #expect(material.shaderFeatures.isEmpty) + // The specular textures belong to no layer. + material.layerTextures[.specular] = PBRTextureBinding(texture: Self.solid(1)) + #expect(material.shaderFeatures == PBRLayerTexture.specular.feature) + material.clearcoatFactor = 1 + #expect(material.shaderFeatures == [.clearcoat, PBRLayerTexture.clearcoat.feature, PBRLayerTexture.specular.feature]) + #expect(Set(material.shaderFeatures.enablingMacros) == ["ADA_CLEARCOAT", "ADA_CLEARCOAT_TEXTURE", "ADA_SPECULAR_TEXTURE"]) + #expect(Set(PBRLayerTexture.allCases.map(\.feature)).count == 8 && PBRLayerTexture.allCases.map(\.binding) == Array(32...39)) + #expect(Set(PBRLayerTexture.allCases.map(\.feature)).isDisjoint(with: [.clearcoat, .sheen, .anisotropy])) + + material.layerTextures[.clearcoatNormal] = PBRTextureBinding( + texture: Self.solid(1), texCoord: 1, transform: PBRTextureTransform(offset: Vector2(0.5, 0), scale: Vector2(2, 2)), scale: 0.3 + ) + let value = PBR3DUniform(material: material, descriptor: VertexDescriptor()) + #expect(value.layerSets0 == Vector4(0, 0, 1, 0) && value.layerSets1 == .zero) + #expect(value.layerTransform2.matrix == Vector4(2, 0, 0, 2) && value.layerTransform2.offset == Vector4(0.5, 0, 0, 0)) + #expect(value.layerTransform0.matrix == Vector4(1, 0, 0, 1)) + #expect(value.layerParameters.x == 0.3) + let plain = PBR3DUniform(material: PBRMaterial(), descriptor: VertexDescriptor()) + #expect(plain.layerSets0 == .zero && plain.layerParameters.x == 1) + #expect(plain.bitPatterns != value.bitPatterns) + let revision = material.uniformRevision + material.layerTextures[.clearcoat] = nil + #expect(material.uniformRevision > revision, "Adding or removing a layer texture must refresh the shared uniform.") + } + + private static func solid(_ red: Float, _ green: Float = 0, _ blue: Float = 0, _ alpha: Float = 1) -> Texture2D { + Texture2D(image: Image(width: 1, height: 1, color: Color(red: red, green: green, blue: blue, alpha: alpha))) + } + + /// Eight texels that are black on the left half and white on the right half. + private static func halves() -> Texture2D { + var image = Image(width: 8, height: 1, color: .black) + for x in 4..<8 { image.setPixel(in: Point(Float(x), 0), color: .white) } + return Texture2D(image: image) + } + + private static let dark: (PBRMaterial) -> Void = { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.9 } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aClearcoatTextureMasksTheCoat() async throws { + let (scene, material) = try await Self.makeScene(Self.dark) + let baseline = try await scene.render(.none) + material.clearcoatFactor = 1 + material.clearcoatRoughnessFactor = 0.05 + let full = Self.largestGain(try await scene.render(.none), over: baseline) + var gains: [Float] = [] + for mask: Float in [0, 0.4, 0.8] { + material.layerTextures[.clearcoat] = PBRTextureBinding(texture: Self.solid(mask)) + let image = try await scene.render(.none) + gains.append(Self.largestGain(image, over: baseline)) + if mask == 0 { #expect(Self.matches(baseline, image), "A black coat mask must leave the paint uncoated.") } + } + #expect(zip(gains, gains.dropFirst()).allSatisfy { $0 < $1 } && gains[2] < full, "The coat must follow its mask: \(gains) against \(full) unmasked.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aClearcoatRoughnessTextureSetsTheCoatRoughness() async throws { + let (scene, material) = try await Self.makeScene(Self.dark) + let baseline = try await scene.render(.none) + material.clearcoatFactor = 1 + material.clearcoatRoughnessFactor = 0.05 + let glossy = Self.largestGain(try await scene.render(.none), over: baseline) + material.clearcoatRoughnessFactor = 1 + let blurred = Self.largestGain(try await scene.render(.none), over: baseline) + material.layerTextures[.clearcoatRoughness] = PBRTextureBinding(texture: Self.solid(0, 0.05)) + let textured = Self.largestGain(try await scene.render(.none), over: baseline) + #expect(textured > blurred * 1.5 && abs(textured - glossy) < glossy * 0.3, "A green texel of 0.05 must act like a roughness of 0.05 (\(textured) vs \(glossy), rough \(blurred)).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aClearcoatNormalTextureBendsTheCoatOnly() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.2, 0.2, 0.2, 1]; $0.roughnessFactor = 0.6; $0.clearcoatFactor = 1; $0.clearcoatRoughnessFactor = 0.08 } + let smooth = try await scene.render(.none) + material.layerTextures[.clearcoatNormal] = PBRTextureBinding(texture: Self.solid(0.5, 0.5, 1)) + #expect(Self.matches(smooth, try await scene.render(.none)), "A flat normal map must not change the coat.") + material.layerTextures[.clearcoatNormal] = PBRTextureBinding(texture: Self.solid(0.95, 0.5, 0.7)) + #expect(!Self.matches(smooth, try await scene.render(.none)), "A tilted normal map must move the coat's highlight.") + material.layerTextures[.clearcoatNormal] = PBRTextureBinding(texture: Self.solid(0.95, 0.5, 0.7), scale: 0) + #expect(Self.matches(smooth, try await scene.render(.none)), "A normal scale of zero must flatten the map.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func sheenTexturesScaleTheirFactors() async throws { + let (scene, material) = try await Self.makeScene(Self.dark) + let baseline = try await scene.render(.none) + material.sheenColorFactor = Vector3(1, 1, 1) + material.sheenRoughnessFactor = 0.4 + let plain = Self.largestGain(try await scene.render(.none), over: baseline) + material.layerTextures[.sheenColor] = PBRTextureBinding(texture: Self.solid(0, 0, 0)) + #expect(Self.matches(baseline, try await scene.render(.none)), "A black sheen color texture must remove the sheen.") + material.layerTextures[.sheenColor] = PBRTextureBinding(texture: Self.solid(1, 0, 0)) + let red = try await scene.render(.none) + #expect(Self.largestGain(red, over: baseline) > 0.05 && Self.largestGain(red, over: baseline) < plain, "A red sheen color adds less light than white (\(Self.largestGain(red, over: baseline)) vs \(plain)).") + material.layerTextures[.sheenColor] = nil + material.layerTextures[.sheenRoughness] = PBRTextureBinding(texture: Self.solid(0, 0, 0, 0.1)) + let tightened = try await scene.render(.none) + material.layerTextures[.sheenRoughness] = nil + #expect(!Self.matches(tightened, try await scene.render(.none)), "A sheen roughness texture must change the lobe.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func anAnisotropyTextureSuppliesStrengthAndDirection() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.35 } + let round = try await scene.render(.none) + material.anisotropyStrength = 1 + let alongTangent = try await scene.render(.none) + material.anisotropyRotation = .pi / 2 + let alongBitangent = try await scene.render(.none) + material.anisotropyRotation = 0 + // Direction (1, 0) is the tangent itself and (0, 1) the bitangent; blue is the strength. + material.layerTextures[.anisotropy] = PBRTextureBinding(texture: Self.solid(1, 0.5, 1)) + #expect(Self.matches(alongTangent, try await scene.render(.none)), "A texture direction of (1, 0) must equal the tangent direction.") + material.layerTextures[.anisotropy] = PBRTextureBinding(texture: Self.solid(0.5, 1, 1)) + #expect(Self.matches(alongBitangent, try await scene.render(.none)), "A texture direction of (0, 1) must equal a quarter turn.") + material.layerTextures[.anisotropy] = PBRTextureBinding(texture: Self.solid(1, 0.5, 0)) + #expect(Self.matches(round, try await scene.render(.none)), "A blue strength of zero must remove the anisotropy.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func specularTexturesScaleTheDielectricReflection() async throws { + let (scene, material) = try await Self.makeScene { $0.baseColorFactor = [0.03, 0.03, 0.03, 1]; $0.roughnessFactor = 0.2 } + let standard = try await scene.render(.none) + material.specularFactor = 0 + let none = try await scene.render(.none) + material.specularFactor = 1 + material.layerTextures[.specular] = PBRTextureBinding(texture: Self.solid(1, 1, 1, 0)) + #expect(Self.matches(none, try await scene.render(.none)), "A specular texture alpha of zero must equal a specular factor of zero.") + material.layerTextures[.specular] = PBRTextureBinding(texture: Self.solid(1, 1, 1, 1)) + #expect(Self.matches(standard, try await scene.render(.none)), "A specular texture alpha of one must change nothing.") + material.layerTextures[.specular] = nil + material.ior = 2.5 + let neutral = try await scene.render(.none) + material.layerTextures[.specularColor] = PBRTextureBinding(texture: Self.solid(1, 0, 0)) + #expect(Self.largestRedShift(try await scene.render(.none), over: neutral) > 0.1, "A red specular color texture must shift the highlight toward red.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aLayerTextureReadsThroughItsOwnTransform() async throws { + let (scene, material) = try await Self.makeScene(Self.dark) + let baseline = try await scene.render(.none) + material.clearcoatFactor = 1 + material.clearcoatRoughnessFactor = 0.05 + // Stretching the sampled range to a sliver sends every pixel to one half of the mask. + let squeeze = Vector2(0.0001, 1) + material.layerTextures[.clearcoat] = PBRTextureBinding(texture: Self.halves(), transform: PBRTextureTransform(offset: Vector2(0.9, 0), scale: squeeze)) + let white = Self.largestGain(try await scene.render(.none), over: baseline) + material.layerTextures[.clearcoat] = PBRTextureBinding(texture: Self.halves(), transform: PBRTextureTransform(offset: Vector2(0.1, 0), scale: squeeze)) + let black = try await scene.render(.none) + #expect(white > 0.3, "Sampling the white half must coat the sphere (\(white)).") + #expect(Self.matches(baseline, black), "Sampling the black half must leave it uncoated.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func importedLayerTexturesReachTheMaterial() async throws { + _ = try await Render3DDebugViewGPUTests.makeScene() + // A 1x1 PNG. + let png = try #require(Data(base64Encoded: "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==")) + let source = GLTFImportResult.Material( + name: nil, + baseColorFactor: .one, + baseColorTextureIndex: nil, + metallicFactor: 0, + roughnessFactor: 0.5, + metallicRoughnessTextureIndex: nil, + normalTextureIndex: nil, + clearcoatFactor: 1, + layerTextures: GLTFImportResult.LayerTextures( + clearcoat: .init(index: 0), + clearcoatNormal: .init(index: 0, texCoord: 1, transform: .init(scale: Vector2(2, 2)), scale: 0.5), + specularColor: .init(index: 0) + ) + ) + let result = GLTFImportResult( + nodes: [], meshes: [], materials: [source], + textures: [.init(source: 0, sampler: nil)], + images: [.init(uri: nil, data: png, mimeType: "image/png")], + scenes: [], defaultScene: nil + ) + let material = try #require(ModelAsset3D.makeMaterials(result).first as? PBRMaterial) + #expect(Set(material.layerTextures.keys) == [.clearcoat, .clearcoatNormal, .specularColor]) + let normal = try #require(material.layerTextures[.clearcoatNormal]) + #expect(normal.texCoord == 1 && normal.scale == 0.5 && normal.transform.scale == Vector2(2, 2)) + #expect(material.shaderFeatures == [.clearcoat, PBRLayerTexture.clearcoat.feature, PBRLayerTexture.clearcoatNormal.feature, PBRLayerTexture.specularColor.feature]) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func clearcoatMatchesItsGoldenFrame() async throws { + let (scene, _) = try await Self.makeScene { + $0.baseColorFactor = [0.55, 0.04, 0.04, 1] + $0.metallicFactor = 0 + $0.roughnessFactor = 0.5 + $0.clearcoatFactor = 1 + $0.clearcoatRoughnessFactor = 0.04 + } + switch try GoldenFrame.check(try await scene.render(.none), name: "pbr-sphere-clearcoat") { + case .recorded, .matched: + break + case .missingReference(let actual): + Issue.record("No golden frame for pbr-sphere-clearcoat; record with ADAENGINE_GOLDEN_RECORD=1. Actual: \(actual.path)") + case let .mismatched(comparison, actual, difference): + Issue.record("pbr-sphere-clearcoat differs in \(comparison.differingPixels) pixels. Actual: \(actual.path), difference: \(difference.path)") + } + } + + private static func expectGolden(_ image: Image, named name: String) throws { + switch try GoldenFrame.check(image, name: name) { + case .recorded, .matched: + break + case .missingReference(let actual): + Issue.record("No golden frame for \(name); record with ADAENGINE_GOLDEN_RECORD=1. Actual: \(actual.path)") + case let .mismatched(comparison, actual, difference): + Issue.record("\(name) differs in \(comparison.differingPixels) pixels. Actual: \(actual.path), difference: \(difference.path)") + } + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func sheenAndAnisotropyMatchTheirGoldenFrames() async throws { + let (velvet, _) = try await Self.makeScene { + $0.baseColorFactor = [0.25, 0.02, 0.05, 1] + $0.metallicFactor = 0 + $0.roughnessFactor = 0.85 + $0.sheenColorFactor = Vector3(0.9, 0.55, 0.65) + $0.sheenRoughnessFactor = 0.45 + } + try Self.expectGolden(try await velvet.render(.none), named: "pbr-sphere-sheen") + // The sphere's UV poles degenerate, which shows in a derivative-based tangent frame, so keep them at the sides. + let (brushed, _) = try await Self.makeScene(polesSideways: true) { + $0.baseColorFactor = [0.8, 0.8, 0.82, 1] + $0.metallicFactor = 1 + $0.roughnessFactor = 0.35 + $0.anisotropyStrength = 0.85 + } + try Self.expectGolden(try await brushed.render(.none), named: "pbr-sphere-anisotropy") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func defaultExtensionValuesChangeNothing() async throws { + let (scene, material) = try await Self.makeScene { _ in } + let baseline = try await scene.render(.none) + material.ior = 1.5 + material.specularFactor = 1 + material.specularColorFactor = .one + material.textureTransforms = Array(repeating: .identity, count: 5) + #expect(Self.matches(baseline, try await scene.render(.none))) + } + } +} +#endif diff --git a/Tests/AdaEngineTests/Render3DMorphTargetTests.swift b/Tests/AdaEngineTests/Render3DMorphTargetTests.swift new file mode 100644 index 000000000..aa5d38a8a --- /dev/null +++ b/Tests/AdaEngineTests/Render3DMorphTargetTests.swift @@ -0,0 +1,470 @@ +#if os(macOS) +import AdaAssets +import AdaECS +@_spi(Internal) import AdaRender +import AdaScene +import AdaTransform +import AdaUtils +import Foundation +import Math +import Testing + +@testable import AdaCorePipelines + +/// Morph target packing and selection; no GPU needed except where a texture is created. +@Suite("3D morph target selection") +struct Render3DMorphTargetSelectionTests { + @Test func halfPrecisionConversionRoundTrips() { + for value: Float in [0, 1, -1, 0.5, 0.6, -0.333, 100.25, 0.0001, 65504, -65504] { + let half = MorphTargetSet.halfBits(value) + let back = MorphTargetSet.float(fromHalfBits: half) + #expect(abs(back - value) <= max(abs(value) * 0.001, 0.00001), "\(value) came back as \(back).") + } + #expect(MorphTargetSet.halfBits(1) == 0x3C00 && MorphTargetSet.halfBits(-2) == 0xC000 && MorphTargetSet.halfBits(0) == 0) + #expect(MorphTargetSet.halfBits(1e9) == 0x7BFF && MorphTargetSet.halfBits(-1e9) == 0xFBFF, "Out-of-range values saturate to the largest finite half.") + #expect(MorphTargetSet.float(fromHalfBits: MorphTargetSet.halfBits(1e-9)) == 0, "Values below the half range flush to zero.") + } + + @Test func theHeaviestWeightsWin() { + let weights: [Float] = [0.1, -0.9, 0, 0.5, 0.5, 2, .nan, 0.05, 0.3, 0.2, 0.4] + let active = MorphTargetSet.activeTargets(weights: weights) + #expect(active.count == 8) + #expect(active.map(\.index) == [5, 1, 3, 4, 10, 8, 9, 0], "Largest magnitude first, ties by index; zero and NaN weights skipped.") + #expect(active.first?.weight == 2 && active[1].weight == -0.9) + #expect(MorphTargetSet.activeTargets(weights: [0, 0, .infinity]).isEmpty) + #expect(MorphTargetSet.activeTargets(weights: [0.5, 0.6, 0.7], limit: 2).map(\.index) == [2, 1]) + } + + @Test func uniformPackingHoldsCurrentAndPreviousTargets() { + var vectors = [Vector4](repeating: .one, count: Morph3DUniforms.vectorCountForTests) + let current: [(index: Int, weight: Float)] = [(3, 0.5), (1, 0.25)] + let previous: [(index: Int, weight: Float)] = [(3, 0.4)] + // Only the counts, flags and entries matter, so a set is not needed to check the layout. + Morph3DUniforms.packForTests(current: current, previous: previous, vertexCount: 120, attributes: 2, flags: 1, into: &vectors) + #expect(vectors[0] == Vector4(2, 1, 120, 2) && vectors[1] == Vector4(1, 0, 0, 0)) + #expect(vectors[2] == Vector4(3, 0.5, 0, 0) && vectors[3] == Vector4(1, 0.25, 0, 0) && vectors[4] == .zero) + #expect(vectors[10] == Vector4(3, 0.4, 0, 0) && vectors[11] == .zero) + } + + @Test func morphKeysIdentifyModelAndPart() { + #expect(Mesh3DRenderSource.morphKey(model: 0, part: 0) != Mesh3DRenderSource.morphKey(model: 1, part: 0)) + #expect(Mesh3DRenderSource.morphKey(model: 0, part: 1) != Mesh3DRenderSource.morphKey(model: 1, part: 0)) + #expect(Set((0..<4).flatMap { model in (0..<4).map { Mesh3DRenderSource.morphKey(model: model, part: $0) } }).count == 16) + } + + @Test func componentRoundTripsThroughCodable() throws { + let component = MorphWeights3DComponent(weights: [0.25, 1], targetNames: ["smile", "blink"]) + let decoded = try JSONDecoder().decode(MorphWeights3DComponent.self, from: JSONEncoder().encode(component)) + #expect(decoded.weights == [0.25, 1] && decoded.targetNames == ["smile", "blink"]) + } + + @Test func scriptsAndTheInspectorSteerOneTargetAtATime() throws { + let descriptor = MorphWeights3DComponent.componentDescriptor + func field(_ key: String) -> ReflectedComponentField? { descriptor.fields.first { $0.key == key } } + #expect(field("selectedTarget")?.kind == .int && field("selectedTarget")?.isWritable == true) + #expect(field("selectedTargetName")?.kind == .string && field("selectedTargetName")?.isWritable == true) + #expect(field("selectedTargetWeight")?.kind == .float && field("selectedTargetWeight")?.isWritable == true) + #expect(field("weights")?.isWritable == false, "Arrays are not reflected as editable fields.") + + var component = MorphWeights3DComponent(weights: [0, 0.5, 0], targetNames: ["smile", "blink", "frown"]) + component.selectedTargetName = "blink" + #expect(component.selectedTarget == 1 && component.selectedTargetWeight == 0.5) + component.selectedTargetWeight = 0.9 + #expect(component.weights == [0, 0.9, 0]) + component.selectedTargetWeight = .nan + component.selectedTargetName = "unknown" + #expect(component.weights == [0, 0.9, 0] && component.selectedTarget == 1, "Bad input leaves the component alone.") + component.selectedTarget = 7 + component.selectedTargetWeight = 1 + #expect(component.weights == [0, 0.9, 0] && component.selectedTargetWeight == 0 && component.selectedTargetName.isEmpty) + #expect(component.targetCount == 3) + + // Written back through reflection, as the inspector and AdaScript do. + let written = try #require(field("selectedTargetWeight")?.write(MorphWeights3DComponent(weights: [0, 0], selectedTarget: 1), .double(0.75)) as? MorphWeights3DComponent) + #expect(written.weights == [0, 0.75]) + } + + @Test func modelSourceKeepsAuthoredMorphWeightsAndDecodesOlderScenes() throws { + let source = Model3DSource(source: "model.glb", morphWeights: ["smile": 0.5, "blink": 1]) + let decoded = try JSONDecoder().decode(Model3DSource.self, from: JSONEncoder().encode(source)) + #expect(decoded == source && decoded.morphWeights == ["smile": 0.5, "blink": 1]) + let older = try JSONDecoder().decode(Model3DSource.self, from: Data(#"{"source":"a.glb","animation":"","autoplay":false,"repeats":true}"#.utf8)) + #expect(older.morphWeights.isEmpty && older.source == "a.glb") + #expect(Model3DSource(source: "a.glb") != source && Model3DSource(source: "model.glb", morphWeights: ["smile": 0.6]) != source) + } + + @Test func componentFindsTargetsByName() { + var component = MorphWeights3DComponent(weights: [0, 0.5], targetNames: ["smile", "blink"]) + #expect(component.weight(named: "blink") == 0.5 && component.weight(named: "frown") == nil) + #expect(component.setWeight(1, named: "smile") && component.weights == [1, 0.5]) + #expect(!component.setWeight(1, named: "frown") && component.weights == [1, 0.5]) + } +} + +extension Render3DGPUTests { + /// Run alone: `ADAENGINE_GOLDEN_METAL=1 swift test --filter Render3DGPUTests/MorphTargets`. + @Suite("3D morph targets on GPU") + @MainActor + struct MorphTargets { + /// A sphere with one target per entry of `shifts`, each moving every vertex by that vector at weight 1. + private static func morphedSphere(device: RenderDevice, shifts: [Vector3], withNormalDeltas: Bool = false) throws -> Mesh { + let base = Mesh.generateSphere(radius: 0.5, renderDevice: device) + var part = try #require(base.models.first?.parts.first) + let positions = part.meshDescriptor.positions.elements + let normals = part.meshDescriptor.normals?.elements ?? [] + part.morphTargets = try #require(MorphTargetSet( + positionDeltas: shifts.map { shift in positions.map { _ in shift } }, + // A normal delta that leans every normal toward +X. + normalDeltas: shifts.map { _ in withNormalDeltas ? normals.map { _ in Vector3(0.8, 0, 0) } : [] }, + tangentDeltas: shifts.map { _ in [] }, + vertexCount: positions.count, + targetNames: shifts.indices.map { "target \($0)" } + )) + return Mesh(models: [Mesh.Model(name: "Morphed", parts: [part])]) + } + + private struct Fixture { + let scene: Render3DGPUTests.DebugViews.Scene + let probe: Entity + + func place(at position: Vector3) { + probe.components[Transform.self] = Transform(position: position) + } + + func setWeights(_ weights: [Float]) { + var component = probe.components[MorphWeights3DComponent.self] + component?.weights = weights + probe.components[MorphWeights3DComponent.self] = component + } + } + + private static let home: Vector3 = [-0.3, 0.5, -2.2] + + private static func makeFixture(shifts: [Vector3], withNormalDeltas: Bool = false, localLight: Bool = false) async throws -> Fixture { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let material = PBRMaterial() + material.baseColorFactor = [0.7, 0.35, 0.3, 1] + material.metallicFactor = 0 + material.roughnessFactor = 0.45 + let mesh = try morphedSphere(device: scene.device, shifts: shifts, withNormalDeltas: withNormalDeltas) + let probe = scene.app.main.spawn("Probe") { + Mesh3DComponent(mesh: mesh, materials: [material]) + MorphWeights3DComponent(weights: shifts.map { _ in 0 }, targetNames: mesh.morphTargetNames) + Transform(position: home) + } + if localLight { + scene.app.main.spawn("Shadow Point") { + PointLightComponent(radiance: [1, 0.9, 0.8], intensity: 10, castShadows: true, range: 6) + Transform(position: [1.4, 2.2, -1.4]) + } + } + return Fixture(scene: scene, probe: probe) + } + + private static func matches(_ left: Image, _ right: Image) -> Bool { + GoldenFrame.compare(left, right, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func zeroWeightsDrawTheRestShape() async throws { + let fixture = try await Self.makeFixture(shifts: [[0.6, 0, 0]], localLight: true) + let morphedMesh = try await fixture.scene.render(.none) + // The same sphere without any morph data. + var component = try #require(fixture.probe.components[Mesh3DComponent.self]) + let plain = Mesh.generateSphere(radius: 0.5, renderDevice: fixture.scene.device) + component.mesh = plain + fixture.probe.components[Mesh3DComponent.self] = component + fixture.probe.components[MorphWeights3DComponent.self] = nil + #expect(Self.matches(morphedMesh, try await fixture.scene.render(.none)), "Weight zero must look exactly like the unmorphed mesh.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled), arguments: [0.25 as Float, 0.5, 1]) + func aWeightedShiftMatchesMovingTheEntity(weight: Float) async throws { + let fixture = try await Self.makeFixture(shifts: [[0.6, 0, 0]], localLight: true) + fixture.setWeights([weight]) + let morphed = try await fixture.scene.render(.none) + fixture.setWeights([0]) + fixture.place(at: Self.home + Vector3(0.6 * weight, 0, 0)) + let moved = try await fixture.scene.render(.none) + #expect(Self.matches(morphed, moved), "A morph shift of \(0.6 * weight) must equal moving the entity by it, shadows from the sun and a local light included.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func weightsChangingEveryFrameUpdateTheShadows() async throws { + let fixture = try await Self.makeFixture(shifts: [[0.6, 0, 0]], localLight: true) + // Several settled frames, then a change: the local shadow faces must not be reused from before the change. + fixture.setWeights([0.1]) + _ = try await fixture.scene.render(.none, frames: 4) + fixture.setWeights([0.9]) + let morphed = try await fixture.scene.render(.none, frames: 1) + fixture.setWeights([0]) + fixture.place(at: Self.home + Vector3(0.54, 0, 0)) + #expect(Self.matches(morphed, try await fixture.scene.render(.none, frames: 1)), "The shadows must follow the new weight on the first frame.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func onlyTheHeaviestEightTargetsApply() async throws { + let shifts = (0..<10).map { Vector3(0.04 * Float($0 + 1), 0, 0) } + let fixture = try await Self.makeFixture(shifts: shifts) + let weights = (0..<10).map { 1 - 0.05 * Float($0) } + fixture.setWeights(weights) + let morphed = try await fixture.scene.render(.none) + var expected: Float = 0 + for index in 0..<8 { expected += weights[index] * 0.04 * Float(index + 1) } + fixture.setWeights([Float](repeating: 0, count: 10)) + fixture.place(at: Self.home + Vector3(expected, 0, 0)) + #expect(Self.matches(morphed, try await fixture.scene.render(.none)), "Targets nine and ten have the smallest weights and must be ignored.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func morphingBringsAnOffscreenMeshIntoView() async throws { + // Resting far to the left, outside the frustum; the shift brings it to the middle. Culling must use the morphed bounds. + let fixture = try await Self.makeFixture(shifts: [[8, 0, 0]]) + fixture.place(at: Vector3(-8.3, 0.5, -2.2)) + fixture.setWeights([1]) + let morphed = try await fixture.scene.render(.none) + fixture.setWeights([0]) + fixture.place(at: Self.home) + #expect(Self.matches(morphed, try await fixture.scene.render(.none)), "The morphed mesh must not be culled by its rest bounds.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func normalDeltasChangeTheShadingButNotTheShape() async throws { + let fixture = try await Self.makeFixture(shifts: [[0, 0, 0]], withNormalDeltas: true) + let rest = try await fixture.scene.render(.none) + fixture.setWeights([1]) + let leaned = try await fixture.scene.render(.none) + #expect(!Self.matches(rest, leaned), "Leaning the normals must change the lighting.") + // The silhouette along the middle row is where the sphere meets the background, and it must not move. + func edges(_ image: Image) -> [Int] { + let background = image.getPixel(x: 2, y: 240) + var result: [Int] = [] + var previousInside = false + for x in 0..<640 { + let pixel = image.getPixel(x: x, y: 240) + let inside = abs(pixel.red - background.red) + abs(pixel.green - background.green) + abs(pixel.blue - background.blue) > 0.35 + if inside != previousInside { result.append(x) } + previousInside = inside + } + return result + } + let restEdges = edges(rest) + #expect(!restEdges.isEmpty && restEdges == edges(leaned), "Normal deltas move no vertex.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func morphedMeshesWorkWithGPUOcclusion() async throws { + let fixture = try await Self.makeFixture(shifts: [[0.6, 0, 0]]) + try fixture.scene.configure { $0.gpuOcclusionEnabled = true } + fixture.setWeights([0.7]) + let morphed = try await fixture.scene.render(.none) + fixture.setWeights([0]) + fixture.place(at: Self.home + Vector3(0.42, 0, 0)) + #expect(Self.matches(morphed, try await fixture.scene.render(.none)), "The visibility prepass must draw the morphed shape.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func morphedMeshesDrawThroughTheTemporalMotionPass() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene(temporalRenderScale: 1) + let device = scene.device + let material = PBRMaterial() + material.baseColorFactor = [0.7, 0.35, 0.3, 1] + material.metallicFactor = 0 + let mesh = try Self.morphedSphere(device: device, shifts: [[0.6, 0, 0]]) + let probe = scene.app.main.spawn("Probe") { + Mesh3DComponent(mesh: mesh, materials: [material]) + MorphWeights3DComponent(weights: [0.5], targetNames: mesh.morphTargetNames) + Transform(position: Self.home) + } + let fixture = Fixture(scene: scene, probe: probe) + // Temporal accumulation is not exactly repeatable, so compare by which reference the frame is closer to. + func difference(_ left: Image, _ right: Image) -> Int { + GoldenFrame.compare(left, right, tolerance: GoldenFrame.activeTolerance).differingPixels + } + let morphed = try await scene.render(.none, frames: 16) + fixture.setWeights([0]) + fixture.place(at: Self.home + Vector3(0.3, 0, 0)) + let moved = try await scene.render(.none, frames: 16) + fixture.place(at: Self.home) + let rest = try await scene.render(.none, frames: 16) + let toMoved = difference(morphed, moved), toRest = difference(morphed, rest) + #expect(toMoved * 4 < toRest, "Through the motion pass the morphed mesh must still land where its weights put it (\(toMoved) pixels from the moved mesh, \(toRest) from the rest shape).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func previousWeightsAreKeptForMotionVectors() async throws { + let fixture = try await Self.makeFixture(shifts: [[0.6, 0, 0], [0, 0.6, 0]]) + let renderWorld = try #require(fixture.scene.app.getSubworldBuilder(by: .renderWorld)?.main) + func header() throws -> (current: Vector4, firstCurrent: Vector4, firstPrevious: Vector4) { + let uniforms = try #require(renderWorld.getResource(Morph3DUniforms.self)) + let buffer = try #require(uniforms.debugBuffers().first) + let vectors = unsafe (0..<18).map { buffer.contents().load(fromByteOffset: $0 * 16, as: Vector4.self) } + return (vectors[0], vectors[2], vectors[10]) + } + fixture.setWeights([0.2, 0]) + _ = try await fixture.scene.render(.none, frames: 3) + fixture.setWeights([0.8, 0]) + _ = try await fixture.scene.render(.none, frames: 1) + let moved = try header() + #expect(moved.current.x == 1 && moved.current.y == 1) + #expect(moved.firstCurrent == Vector4(0, 0.8, 0, 0) && moved.firstPrevious == Vector4(0, 0.2, 0, 0), "The previous frame's weight must sit next to the current one.") + _ = try await fixture.scene.render(.none, frames: 1) + let settled = try header() + #expect(settled.firstCurrent == Vector4(0, 0.8, 0, 0) && settled.firstPrevious == Vector4(0, 0.8, 0, 0), "With nothing changing, previous equals current.") + } + + /// A one-quad glTF skinned to a single joint that is turned a quarter turn about Z, with one morph target that moves + /// every vertex by +0.6 on X (mesh weight 1 by default). Written to a temporary file and loaded like any asset. + private static func skinnedMorphModel(animated: Bool = false) throws -> ModelAsset3D { + var bytes = Data() + var views: [[String: Any]] = [] + func add(_ data: Data) { + views.append(["buffer": 0, "byteOffset": bytes.count, "byteLength": data.count]) + bytes.append(data) + while bytes.count % 4 != 0 { bytes.append(0) } + } + func floats(_ values: [Float]) -> Data { + var data = Data() + for value in values { withUnsafeBytes(of: value.bitPattern.littleEndian) { data.append(contentsOf: $0) } } + return data + } + add(floats([-0.5, 0, 0, 0.5, 0, 0, -0.5, 1, 0, 0.5, 1, 0])) + add(floats([0, 0, -1, 0, 0, -1, 0, 0, -1, 0, 0, -1])) + var indices = Data() + for index in [0, 1, 2, 2, 1, 3] as [UInt16] { withUnsafeBytes(of: index.littleEndian) { indices.append(contentsOf: $0) } } + add(indices) + add(Data([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])) + add(floats([1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0])) + add(floats([0.6, 0, 0, 0.6, 0, 0, 0.6, 0, 0, 0.6, 0, 0])) + add(floats([1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1])) + // Animation keys: the weight goes from 0 to 1 over one second. + add(floats([0, 1])) + add(floats([0, 1])) + func accessor(_ view: Int, _ type: String, _ count: Int, _ component: Int = 5126) -> [String: Any] { + ["bufferView": view, "componentType": component, "count": count, "type": type] + } + var positionAccessor = accessor(0, "VEC3", 4) + positionAccessor["min"] = [-0.5, 0, 0] + positionAccessor["max"] = [0.5, 1, 0] + var document: [String: Any] = [ + "asset": ["version": "2.0"], + "buffers": [["byteLength": bytes.count, "uri": "data:application/octet-stream;base64," + bytes.base64EncodedString()]], + "bufferViews": views, + "accessors": [ + positionAccessor, accessor(1, "VEC3", 4), accessor(2, "SCALAR", 6, 5123), accessor(3, "VEC4", 4, 5121), + accessor(4, "VEC4", 4), accessor(5, "VEC3", 4), accessor(6, "MAT4", 1), + accessor(7, "SCALAR", 2), accessor(8, "SCALAR", 2), + ], + "materials": [["doubleSided": true, "pbrMetallicRoughness": ["baseColorFactor": [0.8, 0.4, 0.3, 1], "metallicFactor": 0]]], + "meshes": [[ + "weights": [1], + "extras": ["targetNames": ["shift"]], + "primitives": [[ + "attributes": ["POSITION": 0, "NORMAL": 1, "JOINTS_0": 3, "WEIGHTS_0": 4], + "indices": 2, "material": 0, "targets": [["POSITION": 5]], + ]], + ]], + "skins": [["joints": [1], "inverseBindMatrices": 6, "skeleton": 1]], + "nodes": [["mesh": 0, "skin": 0], ["rotation": [0, 0, 0.7071068, 0.7071068]]], + "scenes": [["nodes": [0, 1]]], + "scene": 0, + ] + if animated { + document["animations"] = [[ + "name": "Open", + "samplers": [["input": 7, "output": 8, "interpolation": "LINEAR"]], + "channels": [["sampler": 0, "target": ["node": 0, "path": "weights"]]], + ]] + } + let url = FileManager.default.temporaryDirectory.appendingPathComponent("morph-skin-\(UUID().uuidString).gltf") + try JSONSerialization.data(withJSONObject: document).write(to: url) + return try #require(AssetsManager.loadSync(ModelAsset3D.self, at: url.path).asset) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func animatedWeightsDriveTheMorphedMesh() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let model = try Self.skinnedMorphModel(animated: true) + #expect(model.animationClips.map(\.name) == ["Open"] && model.animationClips[0].morphTracks.count == 1) + let root = model.instantiate(in: scene.app.main) + root.components[Transform.self] = Transform(position: [0, 0, -1.6]) + let mesh = try #require(Self.morphedEntity(under: root)) + var animation = try #require(root.components[ModelAnimation3DComponent.self]) + try animation.player.play("Open", transitionDuration: 0) + animation.player.repeats = false + animation.player.isPlaying = false + animation.player.seek(to: 0.5) + root.components[ModelAnimation3DComponent.self] = animation + let animated = try await scene.render(.none, frames: 4) + #expect(mesh.components[MorphWeights3DComponent.self]?.weights == [0.5], "The clip's weight at half a second must reach the entity.") + // The same weight set by hand, with the animation out of the way. + root.components[ModelAnimation3DComponent.self] = nil + var component = try #require(mesh.components[MorphWeights3DComponent.self]) + component.weights = [0.5] + mesh.components[MorphWeights3DComponent.self] = component + #expect(Self.matches(animated, try await scene.render(.none, frames: 4)), "Animated weights must draw like the same weights set directly.") + // And the end of the clip applies the full shift. + var finished = animation + finished.player.seek(to: 1) + root.components[ModelAnimation3DComponent.self] = finished + _ = try await scene.render(.none, frames: 2) + #expect(mesh.components[MorphWeights3DComponent.self]?.weights == [1]) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func authoredWeightsStartTheInstanceAndYieldToAnimation() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + // A rigged model whose clips do not animate weights: the authored weight must survive the animation system. + let still = try Self.skinnedMorphModel() + // The owner carries the same reference, as in a scene; otherwise the source system treats the instance as stale. + let reference = Model3DSource(source: "x", morphWeights: ["shift": 0.4, "unknown": 1]) + let owner = scene.app.main.spawn("Owner") + owner.components[Model3DSource.self] = reference + let root = try reference.instantiate(still, in: scene.app.main, under: owner) + let mesh = try #require(Self.morphedEntity(under: root)) + #expect(mesh.components[MorphWeights3DComponent.self]?.weights == [0.4], "A name the model does not have is ignored.") + _ = try await scene.render(.none, frames: 4) + #expect(mesh.components[MorphWeights3DComponent.self]?.weights == [0.4], "Rest weights from the player must not overwrite authored ones.") + // A model whose clip animates the weights: the clip takes over on the first pose. + let animated = try Self.skinnedMorphModel(animated: true) + let animatedReference = Model3DSource(source: "y", animation: "Open", autoplay: false, morphWeights: ["shift": 0.4]) + let animatedOwner = scene.app.main.spawn("Animated owner") + animatedOwner.components[Model3DSource.self] = animatedReference + let animatedRoot = try animatedReference.instantiate(animated, in: scene.app.main, under: animatedOwner) + let animatedMesh = try #require(Self.morphedEntity(under: animatedRoot)) + _ = try await scene.render(.none, frames: 4) + #expect(animatedMesh.components[MorphWeights3DComponent.self]?.weights == [0], "A clip that animates the weights owns them.") + } + + private static func morphedEntity(under root: Entity) -> Entity? { + if root.components[MorphWeights3DComponent.self] != nil { return root } + for child in root.children { + if let found = morphedEntity(under: child) { return found } + } + return nil + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func importedMorphTargetsApplyBeforeSkinning() async throws { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let model = try Self.skinnedMorphModel() + #expect(model.meshes[0].hasMorphTargets && model.meshes[0].morphTargetNames == ["shift"] && model.meshMorphWeights == [[1]]) + let root = model.instantiate(in: scene.app.main) + root.components[Transform.self] = Transform(position: [0, 0, -1.6]) + let mesh = try #require(Self.morphedEntity(under: root)) + #expect(mesh.components[MorphWeights3DComponent.self]?.weights == [1], "The file's default weight must reach the entity.") + let morphed = try await scene.render(.none, frames: 4) + // Morph first, then skin: the quarter turn carries the +0.6 X shift to +0.6 on Y. Skinning first would leave it on X. + var component = try #require(mesh.components[MorphWeights3DComponent.self]) + component.weights = [0] + mesh.components[MorphWeights3DComponent.self] = component + root.components[Transform.self] = Transform(position: [0, 0.6, -1.6]) + let shiftedUp = try await scene.render(.none, frames: 4) + root.components[Transform.self] = Transform(position: [0.6, 0, -1.6]) + let shiftedAcross = try await scene.render(.none, frames: 4) + #expect(Self.matches(morphed, shiftedUp), "A morph shift must be turned by the skin (morph first, then skin).") + #expect(!Self.matches(morphed, shiftedAcross)) + } + } +} +#endif diff --git a/Tests/AdaEngineTests/Render3DPerformanceBenchmarkTests.swift b/Tests/AdaEngineTests/Render3DPerformanceBenchmarkTests.swift index 8e604d60b..f0f85e4be 100644 --- a/Tests/AdaEngineTests/Render3DPerformanceBenchmarkTests.swift +++ b/Tests/AdaEngineTests/Render3DPerformanceBenchmarkTests.swift @@ -33,6 +33,8 @@ extension Render3DGPUTests { var apply: (inout Environment3D) -> Void var temporalScale: Float? var extraLights = true + /// Additional unshadowed point lights spread over the grid, on top of the default lights. + var manyLights = 0 } /// Spheres per side of the grid (400 spheres by default). `ADAENGINE_PERF_GRID=70` stresses the CPU with 4,900. @@ -44,7 +46,7 @@ extension Render3DGPUTests { } /// A grid of spheres with eight materials, a ground, a sun with cascaded shadows and a ring of local lights. - private static func makeScene(size: SizeInt, temporalScale: Float?, localLights: Bool) async throws -> Render3DGPUTests.DebugViews.Scene { + private static func makeScene(size: SizeInt, temporalScale: Float?, localLights: Bool, manyLights: Int = 0) async throws -> Render3DGPUTests.DebugViews.Scene { unsafe RenderEngine.configurations.preferredBackend = .metal try RenderEngine.setupRenderEngine() try #require(unsafe RenderEngine.shared.type == .metal) @@ -106,6 +108,13 @@ extension Render3DGPUTests { Mesh3DComponent(mesh: Mesh.generateCube(size: [2 * Float(grid), 0.1, 2 * Float(grid)], renderDevice: device), materials: [ground]) Transform(position: [0, -0.05, Float(grid) * 0.4]) } + for index in 0.. Result { - let scene = try await makeScene(size: size, temporalScale: configuration.temporalScale, localLights: configuration.extraLights) + let scene = try await makeScene(size: size, temporalScale: configuration.temporalScale, localLights: configuration.extraLights, manyLights: configuration.manyLights) try scene.configure(configuration.apply) let metrics = Render3DPerformanceMetrics(warmupSamples: 0, synchronizesPasses: true) let renderWorld = try #require(scene.app.getSubworldBuilder(by: .renderWorld)?.main) diff --git a/Tests/AdaEngineTests/Render3DShadowFaceCacheTests.swift b/Tests/AdaEngineTests/Render3DShadowFaceCacheTests.swift new file mode 100644 index 000000000..f9f807b54 --- /dev/null +++ b/Tests/AdaEngineTests/Render3DShadowFaceCacheTests.swift @@ -0,0 +1,158 @@ +#if os(macOS) +import AdaECS +@_spi(Internal) import AdaRender +import AdaScene +import AdaTransform +import AdaUtils +import Foundation +import Math +import Testing + +@testable import AdaCorePipelines + +@Suite("3D local shadow face cache settings") +struct Render3DShadowFaceCacheSettingsTests { + @Test func reuseDefaultsOnAndOlderPayloadsDecode() throws { + #expect(LocalShadowSettings3D().reusesStaticFaces) + let decoded = try JSONDecoder().decode(LocalShadowSettings3D.self, from: Data(#"{"isEnabled":true,"maximumLights":2}"#.utf8)) + #expect(decoded.reusesStaticFaces && decoded.maximumLights == 2) + let off = LocalShadowSettings3D(reusesStaticFaces: false) + #expect(try !JSONDecoder().decode(LocalShadowSettings3D.self, from: JSONEncoder().encode(off)).reusesStaticFaces) + } + + @Test func digestsDependOnEveryWord() { + var first = FrameDigest3D() + var second = FrameDigest3D() + first.combine(Transform3D.identity) + second.combine(Transform3D.identity) + #expect(first.value == second.value) + second.combine(1 as Float) + #expect(first.value != second.value) + var moved = FrameDigest3D() + moved.combine(Vector4(0, 0, 0.001, 1)) + #expect(moved.value != FrameDigest3D().value) + } +} + +extension Render3DGPUTests { + /// Run alone: `ADAENGINE_GOLDEN_METAL=1 swift test --filter Render3DGPUTests/ShadowFaceCache`. + @Suite("3D local shadow face cache on GPU") + @MainActor + struct ShadowFaceCache { + struct Fixture { + let scene: Render3DGPUTests.DebugViews.Scene + let caster: Entity + let light: Entity + let renderWorld: World + + var snapshot: Render3DLightStatistics.Snapshot? { + renderWorld.getResource(Render3DLightStatistics.self)?.snapshots.values.first + } + + func moveCaster(to position: Vector3) { + caster.components[Transform.self] = Transform(position: position) + } + } + + private static func makeFixture() async throws -> Fixture { + let scene = try await Render3DDebugViewGPUTests.makeScene() + let caster = scene.app.main.spawn("Caster") { + Mesh3DComponent(mesh: Mesh.generateSphere(radius: 0.25, renderDevice: scene.device), materials: [PBRMaterial()]) + Transform(position: [0.2, 1.1, -0.4]) + } + let light = scene.app.main.spawn("Shadow Point") { + PointLightComponent(radiance: [1, 0.85, 0.7], intensity: 9, castShadows: true, range: 5) + Transform(position: [0, 1.6, -1.2]) + } + let renderWorld = try #require(scene.app.getSubworldBuilder(by: .renderWorld)?.main) + return Fixture(scene: scene, caster: caster, light: light, renderWorld: renderWorld) + } + + private static func matches(_ left: Image, _ right: Image) -> Bool { + GoldenFrame.compare(left, right, tolerance: GoldenFrame.activeTolerance).passes(GoldenFrame.activeTolerance) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aStaticSceneReusesEveryFaceAndLooksTheSame() async throws { + let fixture = try await Self.makeFixture() + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = false } + let uncached = try await fixture.scene.render(.none) + #expect(fixture.snapshot?.reusedShadowFaces == 0) + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = true } + _ = try await fixture.scene.render(.none, frames: 2) + let faces = try #require(fixture.snapshot).shadowFaces + #expect(faces == 6) + let cached = try await fixture.scene.render(.none) + #expect(fixture.snapshot?.reusedShadowFaces == faces, "Nothing changed, so every face must be kept.") + #expect(Self.matches(uncached, cached), "Reused faces must show exactly what a redraw shows.") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func movingACasterRedrawsOnlyTheFacesItTouches() async throws { + let fixture = try await Self.makeFixture() + _ = try await fixture.scene.render(.none, frames: 3) + fixture.moveCaster(to: [0.2, 1.1, 0.6]) + // One frame: only the first frame after a change redraws, later frames reuse the new faces again. + let cached = try await fixture.scene.render(.none, frames: 1) + let snapshot = try #require(fixture.snapshot) + #expect(snapshot.reusedShadowFaces < snapshot.shadowFaces, "A moved caster must dirty a face.") + #expect(snapshot.reusedShadowFaces > 0, "Faces the caster never reaches must be kept, so the load path is exercised.") + #expect(snapshot.reusedShadowFaces > 0, "Faces the caster never touches must be kept, which exercises the partial redraw.") + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = false } + let redrawn = try await fixture.scene.render(.none) + #expect(Self.matches(redrawn, cached), "A partly reused atlas must equal a full redraw after the caster moved.") + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = true } + fixture.moveCaster(to: [0.2, 1.1, -0.4]) + let back = try await fixture.scene.render(.none, frames: 1) + fixture.moveCaster(to: [0.2, 1.1, -0.4]) + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = false } + let backRedrawn = try await fixture.scene.render(.none) + #expect(Self.matches(back, backRedrawn)) + } + + private func preparedPasses(_ fixture: Fixture) throws -> Int { + try #require(fixture.renderWorld.getResource(VisibleMesh3DLists.self)).preparedPassCount + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func anUnchangedSceneDoesNotRebuildFaceLists() async throws { + let fixture = try await Self.makeFixture() + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = false } + _ = try await fixture.scene.render(.none, frames: 3) + let before = try preparedPasses(fixture) + _ = try await fixture.scene.render(.none, frames: 1) + let rebuilding = try preparedPasses(fixture) - before + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = true } + _ = try await fixture.scene.render(.none, frames: 3) + let kept = try preparedPasses(fixture) + _ = try await fixture.scene.render(.none, frames: 1) + let reusing = try preparedPasses(fixture) - kept + #expect(rebuilding - reusing == 6, "The six local faces must not be prepared again (\(rebuilding) without reuse, \(reusing) with).") + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func aNewCasterIsNoticedWithoutRebuildingEverything() async throws { + let fixture = try await Self.makeFixture() + _ = try await fixture.scene.render(.none, frames: 3) + fixture.scene.app.main.spawn("Late Caster") { + Mesh3DComponent(mesh: Mesh.generateSphere(radius: 0.3, renderDevice: fixture.scene.device), materials: [PBRMaterial()]) + Transform(position: [-0.4, 0.9, -0.6]) + } + let cached = try await fixture.scene.render(.none, frames: 1) + #expect(try #require(fixture.snapshot).reusedShadowFaces < 6, "A caster entering the light's range must dirty a face.") + try fixture.scene.configure { $0.localShadows.reusesStaticFaces = false } + let redrawn = try await fixture.scene.render(.none) + #expect(Self.matches(cached, redrawn)) + } + + @Test(.enabled(if: Render3DDebugViewGPUTests.enabled)) + func movingTheLightRedrawsEveryFace() async throws { + let fixture = try await Self.makeFixture() + _ = try await fixture.scene.render(.none, frames: 3) + fixture.light.components[Transform.self] = Transform(position: [0.3, 1.6, -1.2]) + _ = try await fixture.scene.render(.none, frames: 1) + #expect(fixture.snapshot?.reusedShadowFaces == 0) + } + } +} +#endif