Eval revisited

[Josef-Haupt]

No description provided.

[Copilot]

Pull request overview

This PR revisits the evaluation pipeline to make matching/scoring behavior more robust and transparent: recording keys no longer collide on dotted names, recordings with predictions but no annotations are handled explicitly (drop vs score-as-empty), and aggregate metrics can exclude empty classes and support multiple averaging methods (macro/micro/weighted), with corresponding GUI and localization updates.

Changes:

• Introduces a canonical recording_key() and updates preprocessing/tests to preserve dots and align filename extraction behavior.
• Updates scoring/aggregation: optional scoring of unannotated recordings as empty, exclusion of empty classes from overall aggregates, support counts, and selectable averaging methods.
• Refactors the evaluation GUI to compute per-class + overall metrics separately, add averaging/“score un-annotated” controls, and update i18n strings; adds new end-to-end tests.

Reviewed changes

Copilot reviewed 20 out of 20 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/evaluation/test_process_data.py	Adds end-to-end tests covering unmatched recordings, dotted names, empty-class exclusion, support, and averaging.
tests/evaluation/preprocessing/test_utils.py	Updates expectations for filename/key extraction with dots and leading-dot cases.
tests/evaluation/preprocessing/test_data_processor.py	Adjusts tests around matching predictions/annotations and patcher teardown order.
tests/evaluation/assessment/test_performance_assessor.py	Adds tests for empty-class handling, support, and averaging differences.
birdnet_analyzer/lang/en.json	Adds new evaluation UI strings for averaging, notes, per-class/overall labels, and unannotated scoring toggle; removes old unmatched warning strings.
birdnet_analyzer/lang/de.json	Same as above (German).
birdnet_analyzer/lang/fr.json	Same as above (French).
birdnet_analyzer/lang/fi.json	Same as above (Finnish).
birdnet_analyzer/lang/id.json	Same as above (Indonesian).
birdnet_analyzer/lang/pt-br.json	Same as above (Portuguese - Brazil).
birdnet_analyzer/lang/ru.json	Same as above (Russian).
birdnet_analyzer/lang/se.json	Same as above (Swedish).
birdnet_analyzer/lang/tlh.json	Same as above (Klingon).
birdnet_analyzer/lang/zh_TW.json	Same as above (Traditional Chinese).
birdnet_analyzer/gui/evaluation.py	Refactors UI to build processor from selected directories, adds averaging + “score un-annotated” controls, and separates per-class vs overall metric tables with notes.
birdnet_analyzer/evaluation/preprocessing/utils.py	Adds recording_key() and uses it for recording-name extraction, preserving dots and stripping BirdNET table suffixes/audio extensions.
birdnet_analyzer/evaluation/preprocessing/data_processor.py	Adds score_unannotated_as_empty option and tracks unmatched prediction recordings; drops or keeps unmatched predictions based on the option.
birdnet_analyzer/evaluation/assessment/performance_assessor.py	Adds metric display mapping, support/empty-class helpers, averaging controls, and improved threshold-sweep restoration.
birdnet_analyzer/evaluation/assessment/metrics.py	Centralizes metric input validation via _check_inputs() and reuses it across metrics.
birdnet_analyzer/evaluation/__init__.py	Changes process_data() to return an EvaluationResult (metrics + tensors + context), adds averaging and unannotated scoring options, and updates CLI wiring.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.