Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation

Weijie Wang^1,*   Haomin Zhang^1,*   Ye Li^1,*   Akide Liu²   Zeyu Zhang³
Haoyu Zhao¹   Yefei He¹   Donny Y. Chen²   Feng Chen⁴   Bohan Zhuang^1,†

¹Zhejiang University     ²Monash University     ³BAIR, UC Berkeley     ⁴Adelaide University

^*Equal contribution.   ^†Corresponding author.

ATAS keeps one persistent world behind a whole film.
Every shot is generated by projecting the world state into the requested view, and every generated shot is reconstructed, registered and validated before it updates that world.

Highlights

🧭 World-State Compiler A screenplay becomes bounded shot requests with explicit scene, camera, time and event.	🏠 Scene Memory Static geometry of every place in one shared frame, rendered for any revisit from any viewpoint.
⏱️ Event Memory Dynamic observations indexed by scene, event and time, so a past moment can be re-filmed from a new angle.	✅ Validated Updates The world changes only when film review, registration and the state check all pass.

Method

A world-state compiler turns the screenplay into shot requests; each request queries the scene and event memory, the rendered context conditions the video generator, and the new clip is reconstructed and registered before it can enter the world.

Results

_{Human evaluation against multi-shot baselines (1–5).}

_{Spatial revisit and event recapture.}

_{Runtime, candidate lifecycle and delivery efficiency across memory configurations.}

Installation

The client needs no GPU; the geometry service needs an NVIDIA GPU (about 20 GB free memory).

# geometry service (GPU machine, Python 3.12 with torch + CUDA)
pip install -r geometry/environment.snapshot.txt && pip install -e geometry
cp geometry/.env.example geometry/.env      # set GEOMETRY_PYTHON, GEOMETRY_MODEL_ROOT, GEOMETRY_GPU_IDS
cd geometry && ./run_geometry_service.sh    # check: curl http://127.0.0.1:18300/readyz

# client (any machine, Python 3.12, ffmpeg on PATH)
pip install -e client
cp client/.env.example client/.env          # set DASHSCOPE_API_KEY, ATAS_LLM_*, GEOMETRY_SERVICE_URL

GEOMETRY_MODEL_ROOT contains pi3x/ (Pi3X) and sam3/sam3.pt (SAM3).

Usage

atas run --screenplay examples/bookstore/screenplay.txt --characters examples/bookstore/characters.json --story-id bookstore

Any plain-text screenplay works (--characters is optional). Other options: --memory none (no world state), --plan-only, --max-shots N, --output DIR. Re-running the same command resumes without repeating paid generations. Results go to client/runs/<run-id>/story_<story-id>/: final.mp4, a per-shot manifest.json and report.md.

Examples in examples/: bookstore, clockshop, record. Design notes: docs/design.md. Tests: cd client && python -m pytest, cd geometry && python -m pytest.

Citation

@article{wang2026atas,
  title     = {Any Time, Any Space: A World-State-Centric Framework for Agentic Video Generation},
  author    = {Wang, Weijie and Zhang, Haomin and Li, Ye and Liu, Akide and Zhang, Zeyu and Zhao, Haoyu and
               He, Yefei and Chen, Donny Y. and Chen, Feng and Zhuang, Bohan},
  journal   = {arXiv preprint arXiv:2610.20571},
  year      = {2026},
  url       = {https://arxiv.org/abs/2610.20571}
}

Acknowledgements

ATAS builds on Pi3 and SAM 3. We thank the authors for releasing their work.

License

This project is released under the MIT License. Third-party code under geometry/backbones/ keeps its own license files.