Python: [Bug]: `LocalEvaluator` with zero checks reports items as passed

[CTWalk]

Description

LocalEvaluator() currently marks each evaluated item as passing when no
checks are configured. The result contains zero scores, but it reports
{"passed": 1, "failed": 0, "errored": 0}, all_passed is True, and
raise_for_status() does not raise.

I expected an item with zero evaluated checks to fail closed. This also matches
the .NET LocalEvaluator contract in this repository: ItemPassed ends with
return result.Metrics.Count > 0;, and a focused test requires an item with
zero metrics to count as failed.

Steps to reproduce:

1. Construct one ordinary EvalItem.
2. Run await LocalEvaluator().evaluate([item]).
3. Inspect the score count, aggregate counts, and all_passed.

Code Sample

import asyncio

from agent_framework import EvalItem, LocalEvaluator, Message


async def main():
    item = EvalItem(
        conversation=[
            Message("user", ["What is the weather?"]),
            Message("assistant", ["It is sunny."]),
        ]
    )
    result = await LocalEvaluator().evaluate([item])

    print(result.result_counts)
    print(result.all_passed)
    print(result.items[0].scores)
    result.raise_for_status()


asyncio.run(main())

Observed:

{'passed': 1, 'failed': 0, 'errored': 0}
True
[]

Expected: the item counts as failed, all_passed is False, and
raise_for_status() raises.

Error Messages / Stack Traces

No exception is raised. That is the problem: raise_for_status() returns
normally, so a caller using it as a quality gate sees a pass with no evidence.

Package Versions

agent-framework-core 1.12.1; also reproduced on
main@5f84917f15249518a5575e679fbac41678294e2c.

Python Version

Python 3.11.14 on macOS arm64.

Additional Context

Why this matters. An empty check sequence is realistic when checks are
assembled from configuration, optional plugins, or a filter that matches
nothing. A false pass is materially worse than an error, because all_passed
and raise_for_status() are exactly the surfaces a caller would wire into CI.

Root cause. LocalEvaluator.evaluate initializes item_passed = True
before iterating over check results and only ever clears the flag inside the
loop. With no checks, asyncio.gather() returns an empty list, the loop never
runs, and vacuous truth is recorded as an explicit pass. EvalResults.all_passed
then sees a nonzero passed total and accepts the run — note that all_passed
already guards self.total > 0, so it is fail-closed for an empty item list;
this case slips through because the zero-check item is counted as passed.

Cross-language inconsistency. .NET already decided the opposite way, in
this same repository:

• AgentEvaluationResults.ItemPassed
  returns result.Metrics.Count > 0.
• LocalEvaluator_WithZeroChecks_ItemsHaveZeroMetricsAndFailAsync
  asserts Total == 1, Passed == 0, Failed == 1, and an empty metric
  collection.

Suggested fix. Initializing item_passed = bool(check_results) is a
one-line change that aligns Python with the .NET guard. A focused regression
plus the existing test_local_eval.py file (79 tests) pass with that change,
and the new test fails without it.

On compatibility. This does change observable behavior for anyone calling
LocalEvaluator() with no checks — they would go from a silent pass to a
failure. That population is by definition receiving a verdict with no evidence
behind it, and the evals surface is marked
@experimental(feature_id=ExperimentalFeature.EVALS), so I read this as a bug
fix rather than a breaking change. Happy to follow your call on labeling.

I intend to take this on. I have a focused two-file patch and regression test
ready and will link a draft PR for concrete review. Please assign this issue to
me if that works for you.