Fix LiveCodeBench stdin tests for solutions that read sys.stdin.buffer

[iziz]

Problem

run_test_std in llm_evaluation/livecodebench_util.py replaces sys.stdin with StringIO(test_input). A StringIO has no buffer attribute, so a solution that reads its input through sys.stdin.buffer (the common fast-input idiom data = sys.stdin.buffer.read().split()) raises AttributeError before printing anything, and every stdin test fails even when the solution is correct. The failure is recorded as an ordinary wrong answer, so it does not stand out.

LiveCodeBench's own harness handles this case: since LiveCodeBench/LiveCodeBench@16de2a1 (May 2025), lcb_runner/evaluation/testing_util.py replaces stdin with a mock that has a buffer attribute (MockStdinWithBuffer).

Reproduction

from livecodebench_util import run_test_std

code = "import sys\ndata = sys.stdin.buffer.read().split()\nprint(int(data[0]) + int(data[1]))\n"
print(run_test_std(code, "2 3\n", "5"))
# before: AttributeError: '_io.StringIO' object has no attribute 'buffer'
# after:  (True, '5')

(run_test_std replaces sys.stdout; restore it before printing the result.) sys.stdin.buffer.readline() fails the same way. input(), sys.stdin.read(), sys.stdin.readline() and for line in sys.stdin pass before and after the change.

Impact

LiveCodeBench stdin rows whose last code block reads sys.stdin.buffer, re-scored inside the evaluation image with no network, with the current code and with this patch:

prediction file	rows	pass now	pass with this patch
AsiaInfo-Router	53	0	51
BARouter	50	0	44
lynkr	32	0	30
weave-router	25	0	4
sqwish-fvcta8	11	0	10
PSAI-Auto-Router	9	0	9
cross-router	8	0	3
vllm-sr	8	0	1
agentforge-router	4	0	0
sqwish-router	4	0	0
llm-router	3	0	3
orcarouter-adaptive	3	0	3
Paix2-router	2	0	0
total (main, 04c99fd)	212	0	158
KT-ModelRouter and the other prediction files	0

Open submissions: #228 (dalpha-llm-router) has 42 such rows (0 pass now, 40 with this patch), #232 (divyam-router) has 34 (0 now, 30 with this patch), and #229 (BizRouter) has none.

Regression check: every LiveCodeBench stdin row of KT-ModelRouter (202 rows, none of which reads sys.stdin.buffer) scores the same with and without the patch (178 pass in both), and no row that passes with the current code fails with the patch.

Disclosure: we submitted #229 and #230. #229 has no rows that read sys.stdin.buffer, so this change does not affect its score; it raises the scores of other submissions, including open ones. We found the bug while evaluating a model that reads stdin this way in most of its stdin answers. As with #230, whether and how to re-evaluate existing entries is up to the maintainers.

Fix

Give run_test_std a text stream over a bytes buffer (io.TextIOWrapper(io.BytesIO(...), encoding="utf-8", newline="\n")) instead of a StringIO. Text reads behave as before (newline="\n" keeps StringIO's newline handling), and sys.stdin.buffer reads the same input.

This is independent of #230: the two changes touch different lines of run_test_std and merge cleanly in either order.

Checked with the pinned pre-commit hooks (ruff v0.11.7, ruff-format, codespell v2.4.1, mypy).