Skip to content

feat(groups): weights on load-balance #289

Closed

Description

@svipm

Summary

Strategy groups cover ordering (fallback), manual pick (select), round robin
(load-balance), latency (url-test) and price (cheapest). Priority is therefore
already covered — fallback is "first healthy one wins, cache keeps hitting", which is
declaration order as priority.

What is missing is weight: there is no way to send 70% of traffic to one upstream
and 30% to another. load-balance is equal by construction, and fallback is
all-or-nothing.

Current shape

Group.providers is Vec<String> (crates/tw-engine/src/engine.rs:93) — a membership
list, with no per-member number:

pub struct Group {
    pub name: String,
    #[serde(default, rename = "type")]
    pub kind: GroupType,
    pub providers: Vec<String>,
    pub selected: Option<String>,
}

and the distribution itself (engine.rs:127):

fn rotate(members: &[String], f: &Facts) -> Vec<String> {
    let start = (f.seq % members.len() as u64) as usize;
    ...
}

Equal weight, by seq.

Proposal

groups:
  - name: mixed
    type: load-balance
    providers:
      - { name: cheap-relay, weight: 7 }
      - { name: official, weight: 3 }

Selection becomes "walk the weights against f.seq % total" instead of
seq % len. Keeping it a function of Facts preserves the existing constraint — the
engine is pure so the dry-run page predicts the data plane exactly and cannot disagree
with it.

Open questions I would rather not guess at:

  • Keep Vec<String> accepted as shorthand for all-weights-1, or require the object
    form everywhere? The former avoids breaking every existing config; the field is
    deny_unknown_fields, so a mixed list either works or fails loudly, which is at
    least not silent.
  • Does weight belong only on load-balance, or also on url-test / cheapest as a
    tiebreak and a floor? Only load-balance seems load-bearing.
  • Same place as session affinity. The existing note says a conversation that has
    already been answered stays pinned to the upstream that answered it ("同一段对话不
    在这里粘"), so rotate only covers requests nobody has answered yet. Weights should
    apply to exactly that set and not fight the pinning.

Possible this already works

expand_group clones g.providers verbatim (engine.rs:825) and nothing dedupes it,
so providers: [a, a, a, b] with load-balance may already give 3:1 — and validation
does not appear to reject a repeated name inside a group. If that is the intended
workaround, then this is a docs request rather than a feature request, and it should be
written down next to the group types; relying on a reader to notice that duplicates are
meaningful is the kind of thing that gets "cleaned up" later by someone tidying the
list.

Version

ThinkWatch-Core 291ef53.

Activity

  1. fylorn commented on Oct 5, 2026

    @fylorn
    Contributor

    Weights are out in core v0.63.0 and ThinkWatch Lite 2026.10.6, in the shape you proposed:

    groups:
      - name: mixed
        type: load-balance
        providers:
          - { name: cheap-relay, weight: 7 }
          - official            # a plain name is weight 1

    On your open questions:

    • Shorthand: a plain name stays valid and means weight 1, and a member with weight 1 is written back as a plain name, so existing configurations don't change. A weight is a whole number from 1 to 100.
    • Which types: only load-balance. A weight on any other group type is refused when the configuration loads, rather than ignored.
    • Stickiness: a conversation that has been answered still stays on its upstream, and that request is charged to the upstream that serves it. So the weight is the long-run share of requests, and new conversations make up the difference. Distribution is smooth weighted round-robin (7 : 3 interleaves instead of arriving in runs). Its state is passed into the engine, which stays pure, so the dry-run still predicts the next upstream exactly. Paused and full members sit out a round.

    There is also balance_by: latency | health | latency-health, which multiplies the weights by a factor from each upstream's time to first content and its recent success rate. An upstream without enough measurements counts as average.

    You were right about duplicates: providers: [a, a, a, b] was accepted and would have made failover try a again. A group that lists an upstream twice, or a member that isn't an upstream, is now refused when the configuration loads (#291).

    Details are in the configuration reference under groups. Thanks for the clear write-up!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions