iris-hostgl: SGI_color_matrix's state on a host without a colour matrix (Linux)

[atomchild411]

iris-hostgl: SGI_color_matrix's state on a host without a colour matrix (Linux)

On a Linux host, SGI's own IRIS GL running on host GL crashed in gconfig() for any window with a stencil buffer. irisgltest's stencil window hits it, and so does any IRIS GL program that calls stensize().

Cause

• SGI's IRIS GL (IGLOO, on IMPACT) asks for GL_MAX_COLOR_MATRIX_STACK_DEPTH when it sets up such a window, and sizes a stack from the answer.
• The host GL's answer passes straight through:
  • macOS's GL has a colour matrix and answers 10;
  • Mesa's compatibility profile has no ARB_imaging, so it refuses the query with GL_INVALID_ENUM and the answer is left at 0.
• IGLOO crashed on 0. A call trace of the same program on both hosts shows that query as the last one before the crash, and the only answer that differs in a way that matters.
• SGI's own libGL on IMPACT answers 2.

The change

On a host whose GL cannot look up glColorTable (no ARB_imaging), the executor answers three queries itself, as IMPACT's libGL does, and doesn't ask the host:

• GL_COLOR_MATRIX: the identity;
• GL_COLOR_MATRIX_STACK_DEPTH: 1;
• GL_MAX_COLOR_MATRIX_STACK_DEPTH: 2.

So no error is raised. Hosts with a colour matrix are asked as before. It is generated by tools/glshim.py (GUARDED, for the four glGet*v); calls.rs is regenerated.

The colour matrix itself is still not there on such a host. The extension list advertises GL_SGI_color_matrix (and convolution and histogram) whatever the host has; I'd suggest that list follow the host as a separate change.

Tests

• Unit test: color_matrix_limits_are_answered_and_raise_no_error asks for both depths and then glGetError.
  • On Linux it fails without the change (GL_INVALID_ENUM, 1280) and passes with it.
  • On macOS it passes either way, because the host answers.
• cargo test --manifest-path iris-hostgl/Cargo.toml: 33 passed on macOS and on Linux (Debian 12 arm64, Mesa 22.3.6, virgl).
• On the Linux VM (IRIX 6.5.22m, Max IMPACT, guest tools v20261017-proto2, SGI's IRIS GL through hostgl):
  • irisgltest crashed at the stencil window before the change. After it, irisgltest runs to the end: stencil checks pass, and the remaining failures are the same 12 as under SGI's own libraries on the emulated board.
  • flight runs.

🤖 Generated with Claude Code