All pull requests

Pull requests

• perf(models): raise the Hy-MT2-1.8B KV budget to 65536 tokens

  #839·ShaiShmuel opened Oct 20, 2026Contributor·

  ·
  1

• fix(server): tokenize the Qwen3 reranker row as the model card does

  #838·svonava opened Oct 20, 2026Contributor·

  ·
  1

• feat: run an SGLang LLM and encoders on one GPU in one pool (composite bundles)

  #836·svonava opened Oct 20, 2026Contributor·

  ·
  9

• fix: restart a worker with a wedged CUDA context in seconds

  #835·svonava opened Oct 20, 2026Contributor·

  ·
  4

• feat(server): keep evicted models in host RAM and wake them instead of reloading

  #834·svonava opened Oct 20, 2026Contributor·

  ·
  13

• feat(sglang): size more engines automatically and add shared-GPU LLM profiles

  #832·svonava opened Oct 20, 2026Contributor·

  ·
  5

• feat(gateway)!: let API pools live up to 30 days or never expire

  #831·svonava opened Oct 20, 2026Contributor·

  ·
  6

• feat(server): support runtime LoRA for native decision models

  #829·svonava opened Oct 20, 2026Contributor·

  ·
  15

• chore(deps): bump the uv group across 34 directories with 1 update

  dependenciespython:uv
  #824·dependabot[bot] opened Oct 19, 2026Bot·

  ·
  1

• perf(server): avoid masked hidden-state copy in CPU fp32 mean pooling

  #822·phiomegaruntime-rgb opened Oct 19, 2026·

  ·
  1

• fix(models): undo Moses entity escaping in MADLAD-400 output

  #819·ShaiShmuel opened Oct 19, 2026Contributor·

  ·
  1

• fix(helm): stop rendering the GKE node label on dcgm-exporter outside GKE

  #810·caohhung opened Oct 19, 2026·

  ·
  8

• feat(helm): add extraInitContainers and extraVolumes to worker pods

  #809·caohhung opened Oct 19, 2026·

  ·
  5

• feat(examples): compare trained routers by answer quality and cost

  #795·svonava opened Oct 18, 2026Contributor·

  ·
  1

• fix(examples): support single-arm rerank URL replay

  #707·svonava opened Oct 12, 2026Contributor·

  ·
  9

• fix(server): preserve Qwen reranker semantics without Flash Attention

  #693·svonava opened Oct 11, 2026Contributor·

  ·
  5

• feat(server): add opt-in exclude_labels and require_uppercase to GLiNER extraction

  #664·svonava opened Oct 9, 2026Contributor·

  ·
  1

• feat(server): add pinned OntoNotes entity extraction

  #661·svonava opened Oct 9, 2026Contributor·

  ·
  1

• fix(server): dispatch a lone idle request without the coalesce window

  #658·YeonwooSung opened Oct 9, 2026Contributor·

  ·
  21

• fix(server): pack over-budget turns in arrival order

  #656·YeonwooSung opened Oct 9, 2026Contributor·

  ·
  20

• fix(server): read long documents whole in GLiNER2 extraction, not only their first window

  #649·entity-0-0-2 opened Oct 9, 2026·

  ·
  1

  4

• feat(server): serve fixed-head text classifiers through extract

  #644·svonava opened Oct 9, 2026Contributor·

  ·
  1

• feat(server): add Qwen3-ASR transcription support

  #612·svonava opened Oct 7, 2026Contributor·

  ·
  3

• chore(deps): bump the uv group across 3 directories with 3 updates

  dependenciespython:uv
  #538·dependabot[bot] opened Oct 3, 2026Bot·

  ·
  2

• docs(examples): add AWS Comprehend to the redact example and price LLMs at real-time list

  #477·svonava opened Sep 30, 2026Contributor·

  ·
  6