Add BizRouter submission (balanced Smart Routing)

[iziz]

Router submission: BizRouter

BizRouter (https://bizrouter.ai) is an LLM gateway with a built-in Smart Router. This submission is the production balanced routing policy as served by the API on 2026-10-13: every prompt was sent once through the production endpoint (bizrouter/route/ra-balanced, one user message, no system prompt, max_tokens 4096; the three answers cut at that limit were resent at 16384), and generated_result carries the answer the selected model returned.

How it routes. Each request is labeled by an LLM analyzer into a task taxonomy (type × difficulty); candidates are ranked from measured capability evidence per taxonomy cell (our own benchmark measurements, not RouterArena data) together with list prices, within a quality-loss band. The router was not trained, fitted or tuned on RouterArena data or labels; RouterArena was used only for external evaluation.

Files

• router_inference/config/BizRouter.json – 26-model pool (public model names)
• router_inference/predictions/BizRouter.json – 8,400 rows with generated_result
• router_inference/predictions/BizRouter-robustness.json – 420 rows, selections only
• model_cost/model_cost.json – 17 pool models added with the providers' list prices (USD per 1M tokens, base tier); existing entries untouched
• universal_model_names.py – the same 17 names appended to universal_names

Local check (check_config_prediction_files.py full + robustness): all checks passed. Local llm_evaluation/run.py on the 2026-09-28 checkout: accuracy 0.8095, $1.0023 / 1K queries, score 76.50, robustness 0.9738. Five prompts whose answer was blocked by the provider's content filter are kept as failed generations (success: false).

Closed-source router. Contact: zen.isis@gmail.com.

[iziz]

/evaluate

[github-actions]

Router Evaluation Results

Router: BizRouter
Dataset Split: full

RouterArena Metrics

Metric	Value
RouterArena Score	0.7648
Accuracy	80.92%
Total Cost	$8.419195
Avg Cost per Query	$0.001002
Avg Cost per 1K Queries	$1.0023
Number of Queries	8400
Abnormal Entries	5
Robustness Score	0.9738

⚠️ 5 of 8400 queries (0.1%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

[iziz]

Thanks for the evaluation. On the 5 abnormal entries (NarrativeQA_5816, 9276, 4374, 5807, 5831):

These are not inference failures on our side. For each of them the frozen policy selected google/gemini-3.7-flash, and Gemini returned an empty completion with finish_reason: content_filter (the prompts are long narrative passages). We resent the five prompts once through the same policy; the router selected the same model and the provider blocked them again. Regenerating them with a different model would mean overriding the router's selection, so we kept them as failed generations (success: false), scored as incorrect under the leaderboard rule.

The submission therefore reflects exactly what the production policy returns, including the provider's safety filter. Happy to adjust if you prefer a different handling.