Proposal: an exo-node worker hive
Summary
A worker hive whose bees raise exo / tinygrad inference nodes as cells. This is the heterogeneous data plane: it lets a mix of Macs, AMD, NVIDIA, and single-node devices pool into one logical model, which is the direction for running frontier-scale MoEs across mixed hardware.
Motivation
hum's worker hives today are single-process cells. exo (built on tinygrad) runs frontier LLMs across any heterogeneous set of devices by splitting layers and tensors across them. Nesting it as a worker hive gives hum a cell type whose compute spans whatever devices are available, which is exactly the pooled-inference gap we identified for models like GLM-5.2 / Kimi K2.5 (~250GB) that no single machine fits.
Hive stencil
- Typology: worker (accepts
chi:"prompt", emits chunk then finish).
- Bee binary:
exo-node, launches an exo/tinygrad instance as its cell, optionally coordinating with peer exo nodes.
- Propensity:
StatelessPerCall.
- Richness: medium (system, content, tools).
- Wire: as a worker it speaks thrum; exo's OpenAI-ish surface can be the optional forager contract.
Egg spec (candidate)
- model id and quant, device list (which local devices to pool)
- tensor/layer sharding plan, network addresses for peer nodes
- dtype and batch knobs
Cell shape
One exo process. feed is the prompt JSON, mmm is the chunk stream, emerged is the node exit, silence cancels. Standard WorkerBee::raise(egg) -> Cell, tended by lifecycle.rs.
Integration with humd / Nest
Handshakes a humd, announces bee:["worker"], becomes nestable. The Nest pool enforces max_procs and idle eviction. hum's ensemble gossip is a natural fit for exo device/topology discovery across humds.
Risks
- exo/tinygrad is research-grade; throughput varies across devices.
- Device discovery across a LAN needs trust and topology reporting.
- A model spanning several exo nodes could be one cell or several; needs a clear representation.
Open questions
- Should a multi-device model be one cell or N cells in the Nest?
- Can ensemble gossip drive exo node discovery, or does exo need its own discovery?
- How does humd express which devices a node may pool (privacy/isolation)?
Relationship to other hives
Overlaps ds4-worker on layer slicing; exo adds heterogeneous-device pooling that ds4 does not have. The two should be reconciled, not duplicated.
Proposal: an
exo-nodeworker hiveSummary
A worker hive whose bees raise exo / tinygrad inference nodes as cells. This is the heterogeneous data plane: it lets a mix of Macs, AMD, NVIDIA, and single-node devices pool into one logical model, which is the direction for running frontier-scale MoEs across mixed hardware.
Motivation
hum's worker hives today are single-process cells. exo (built on tinygrad) runs frontier LLMs across any heterogeneous set of devices by splitting layers and tensors across them. Nesting it as a worker hive gives hum a cell type whose compute spans whatever devices are available, which is exactly the pooled-inference gap we identified for models like GLM-5.2 / Kimi K2.5 (~250GB) that no single machine fits.
Hive stencil
chi:"prompt", emitschunkthenfinish).exo-node, launches an exo/tinygrad instance as its cell, optionally coordinating with peer exo nodes.StatelessPerCall.Egg spec (candidate)
Cell shape
One exo process.
feedis the prompt JSON,mmmis the chunk stream,emergedis the node exit,silencecancels. StandardWorkerBee::raise(egg) -> Cell, tended bylifecycle.rs.Integration with humd / Nest
Handshakes a humd, announces
bee:["worker"], becomes nestable. The Nest pool enforcesmax_procsand idle eviction. hum's ensemble gossip is a natural fit for exo device/topology discovery across humds.Risks
Open questions
Relationship to other hives
Overlaps
ds4-workeron layer slicing; exo adds heterogeneous-device pooling that ds4 does not have. The two should be reconciled, not duplicated.