Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
edge; `performer_count` reports how many perform in a span as a high percentile of the
per-frame counts (the wide shots), with median and maximum beside it. Exact for a soloist,
a duo and a five-piece band from an operated concert camera; a choir is under-counted.
- `performer_count(..., stat="typical")` takes the median over framings instead of the widest, for
lecture halls where the widest framings are the hall and the slides; `detect_people` and
`make_proxy` / `camera_motion` take `ffmpeg_input_args` (e.g. `["-hwaccel", "cuda"]`).
- **Camera cuts and PTZ** — new `_camera` module. `camera_motion` labels each sample of a video
`still`, `moving` or `cut` from ORB matches and a partial-affine RANSAC fit between consecutive
frames of a small 2 fps proxy (`make_proxy`), and returns cuts, shots and the share of time in
Expand Down
13 changes: 13 additions & 0 deletions docs/user-guide/people-and-camera.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,18 @@ Two uses follow:
where the plain percentile counted the audience in the band's wide shot; a choir stays
under-counted because singers occlude each other.

## Lecture halls and long files

A lecture or defence recording differs in two ways. There is no raised stage, so pass
`head_below=1.0, cut_head_below=1.0` to switch the audience filter off; and the widest framings
show the hall and the projected slides, whose figures are not the speakers, so count the *typical*
framing: `performer_count(det, a, b, camera=cam, stat="typical")` takes the median over framings
instead of the maximum. On a defence this gave 1, 1, 2, 2 for the lecture, the introduction and
the two opponent discussions.

A 36 GB 1080p50 file decodes on the GPU by passing ffmpeg input options through:
`detect_people(video, ffmpeg_input_args=["-hwaccel", "cuda"])` and
`camera_motion(video, ffmpeg_input_args=["-hwaccel", "cuda"])`.

Both analyses cache well: keep the proxy and the detections next to the recording and the
counts for any span are instant.
7 changes: 4 additions & 3 deletions musicalgestures/_performers.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ def people_track(detections: dict, **filter_kw) -> tuple[np.ndarray, np.ndarray]

def performer_count(detections: dict, start_s: float = 0.0, end_s: float | None = None,
percentile: float = 90.0, camera: dict | None = None, min_framing_s: float = 10.0,
**filter_kw) -> dict:
stat: str = "widest", **filter_kw) -> dict:
"""How many performers a span shows.

Without `camera`: ``estimate`` is the `percentile` of the per-frame counts (the wide shots),
Expand All @@ -133,7 +133,8 @@ def performer_count(detections: dict, start_s: float = 0.0, end_s: float | None
if m.sum() >= 5:
vals.append(int(np.percentile(c[m], 75)))
if vals:
return {"estimate": max(vals), "low": min(vals), "median": int(np.median(c)), "max": int(c.max()),
"frames": int(c.size), "framings": len(vals), "method": "framings"}
est = max(vals) if stat == "widest" else int(np.median(vals))
return {"estimate": est, "low": min(vals), "high": max(vals), "median": int(np.median(c)), "max": int(c.max()),
"frames": int(c.size), "framings": len(vals), "method": f"framings-{stat}"}
return {"estimate": int(round(float(np.percentile(c, percentile)))), "low": int(np.median(c)),
"median": int(np.median(c)), "max": int(c.max()), "frames": int(c.size), "method": "percentile"}
2 changes: 1 addition & 1 deletion tests/test_camera.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,4 @@ def test_performer_count_per_framing_ignores_the_moving_camera():
n = 1 if t < 15 else (4 if t < 20 else 3) # close shot, then the camera swings past a crowd, then wide
frames.append({"t": float(t), "boxes": [[0.1 * k, 0.2, 0.1 * k + 0.08, 0.7, 0.9] for k in range(n)]})
c = performer_count({"fps": 1.0, "frames": frames}, 0, 40, camera=cam)
assert c["method"] == "framings" and c["estimate"] == 3 and c["low"] == 1
assert c["method"] == "framings-widest" and c["estimate"] == 3 and c["low"] == 1
Loading