Skip to content

[Bug]: Setting a column on an empty AgentSet fails with polars 2.x (ShapeError); CI on main is red #207

Open

Description

@Jay-Naik2526

📝 Description

With polars 2.x, setting a column on an empty AgentSet raises polars.exceptions.ShapeError: can't broadcast Series 'wealth' of length 4 to length 0. It works with polars 1.36.1, which is the version pinned in uv.lock.

The cause is a behavior change in polars itself: pl.DataFrame().with_columns(pl.Series("wealth", [1, 2, 3, 4])) returns a (4, 1) DataFrame on polars 1.36.1 and raises the same ShapeError on polars 2.1.0.

This affects users of the library (creating an AgentSet and setting its first column) and it also breaks the test suite: the shared fixtures such as ExampleAgentSet / ExampleAgentSetA do exactly this in __init__. The CI install is not locked, so it pulls polars 2.1.0 and CI Checks fail on every OS and Python version. The scheduled runs on main have been failing the same way since at least Oct 19 (for example https://github.com/mesa/mesa-frames/actions/runs/41457455719).

🔄 Steps to Reproduce

import polars as pl
from mesa_frames import AgentSet, Model

# Plain polars
print(pl.DataFrame().with_columns(pl.Series("wealth", [1, 2, 3, 4])).shape)
# polars 1.36.1 -> (4, 1);  polars 2.1.0 -> ShapeError

# mesa-frames
class A(AgentSet):
    def step(self):
        pass

a = A(Model())
a["wealth"] = pl.Series("wealth", [1, 2, 3, 4])
# polars 1.36.1 -> works (len(a) == 4);  polars 2.1.0 -> ShapeError

🎯 Expected Behavior

Setting the first column on a new AgentSet works with the supported polars versions (pyproject.toml currently allows polars>=1.30.0, with no upper bound).

🚨 Actual Behavior

polars.exceptions.ShapeError: can't broadcast Series 'wealth' of length 4 to length 0

With polars 2.1.0 and the project's tests (MESA_FRAMES_RUNTIME_TYPECHECKING=1), running only tests/test_agentset.py and tests/test_agentsetregistry.py gives 52 errors (in fixture setup) and 9 failures. With the locked polars 1.36.1 the only failure in the full suite is test_datacollector.py::test_postgress, which needs a PostgreSQL database.

🖥️ Python Version

Python 3.12.14

Operating System

macOS (Apple Silicon); the same failure shows in CI on Ubuntu (3.11, 3.12, 3.13, 3.13 --pre), macOS and Windows.

Relevant Packages

polars 1.36.1 (works) vs polars 2.1.0 (fails)

📊 Relevant Log Output

ERROR at setup of Test_AgentSet.test_add
E       polars.exceptions.ShapeError: can't broadcast Series 'wealth' of length 4 to length 0

➕ Additional Context

I have not yet checked whether anything besides this pattern breaks with polars 2.x, because the setup error hides later failures. Two possible directions, both up to the maintainers: temporarily constrain polars<2 in pyproject.toml to get CI green again, and/or make setting a column on an empty AgentSet work with polars 2.x. I am happy to work on a fix if that would be useful.

Related: #204 / #206 (the tests added there avoid these fixtures, so they pass on both polars versions).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions