Add extensible chat client routing

[joshuajyue]

Resolves #7647

Summary

Adds experimental chat-routing primitives for selecting configured IChatClient instances without imposing a particular routing policy.

• RoutingChatClient supports one-shot client selection.
• FailoverChatClient retries uncanceled failures that occur before output is exposed.
• OrderedFailoverChatClient provides ordered fallback.
• SemanticRoutingChatClient selects clients using cached profile embeddings.
• RoutingContext exposes mutable request messages and options.
• FailoverChatClientAttempt reports invocation outcome, duration, time to first update, completion, and output commitment.

API shape

Routing policies have two seams:

protected override ValueTask<IChatClient> SelectClientAsync(
    RoutingContext context,
    CancellationToken cancellationToken);

protected override ValueTask OnRoutingUpdateAsync(
    RoutingContext context,
    FailoverChatClientAttempt? attempt,
    bool isTerminal,
    CancellationToken cancellationToken);

SelectClientAsync runs before every invocation. OnRoutingUpdateAsync reports each attempt so policy state can influence the next selection.

isTerminal means the base will not select another client after the callback completes. A null attempt represents selection terminating before a client was invoked.

Failover behavior

• Cancellation never triggers reselection.
• Streaming fallback occurs only before the first update is exposed.
• Mid-stream failures and caller abandonment are terminal.
• Enumerator disposal failures are included in the attempt.
• Attempt limits are request-local.
• Null selector results are treated as terminal selection failures.
• Ordered failover retains request state only between a nonterminal update and the immediately following selection; it holds no state while a provider invocation or stream is active.
• Streaming callers must dispose active enumerators for terminal observation and inner-resource cleanup.

Semantic routing

SemanticRoutingChatClient:

• Lazily embeds and caches profile utterances.
• Embeds the final user message for selection.
• Supports global top-K profile matching.
• Aggregates matches per client using mean or sum.
• Preserves the existing single-best-match behavior with topK: 1.
• Uses a required default client when no profile clears the threshold.
• Supports caller-controlled ownership through leaveOpen.

Tests

Coverage includes selection failures, retries, cancellation, attempt limits, streaming commitment, early disposal, enumerator failures, concurrent requests, state cleanup, semantic thresholds, top-K aggregation, caching, and ownership.

• 753 tests passed on net10.0
• net472 test project builds successfully

Microsoft Reviewers: Open in CodeFlow

[joshuajyue]

Addressed in commit 0abbfd1, along with some cancellation races

[jozkee]

Looking good.

[joshuajyue]

Thank you David for the feedback, addressed in 8b79ae4