Observed behavior
When running the netrid_early_integration_assessment configuration in qual-partners with 3 NetRID Service Providers, the run produces a broken HTML report and fails with:
ValueError: 3 service providers were specified, but data for only 2 test flights were provided
This appears to come from the scenario using the Kentland RID KML flight data:
monitoring/uss_qualifier/test_data/usa/kentland/rid.kml
That KML currently appears to define only 2 nominal flights:
flight: south u
flight: dairy circle
The scenario documentation for sp_operator_notify_slow_update says flights_data must contain 1 nominal flight per SP under test, so a 3-SP run needs 3 nominal flights.
Downstream issue with attached report artifact: https://github.com/utmimplementationus/tech_committee/issues/202
Test check
This fails before reaching a specific standards check. The relevant scenario appears to be:
monitoring/uss_qualifier/scenarios/astm/netrid/v22a/sp_operator_notify_slow_update.md
The mismatch is between:
- 3 configured
service_providers
- 2 flights available in the
flights_data source
Difference from expected behavior
A baseline qual configuration intended to run against 3 SPs should provide at least 3 nominal flights, or use a flight-data source that can generate enough flights for the configured SP count.
The ValueError itself appears correct, but the current baseline/data combination makes the qual run unusable for 3-SP testing. Also, the HTML report should ideally surface this as a clean configuration/setup failure rather than ending up broken.
Additional context
Possible fixes:
- Add at least one more nominal flight to the Kentland RID KML, with distinct metadata and enough telemetry duration for the slow-update scenario.
- Change the relevant baseline to use
adjacent_circular_flights_simulation_source instead of the fixed Kentland KML source, so enough flights can be generated for the SP count.
- Improve report generation for this configuration mismatch so the report remains readable even when the scenario cannot start.
Observed behavior
When running the
netrid_early_integration_assessmentconfiguration inqual-partnerswith 3 NetRID Service Providers, the run produces a broken HTML report and fails with:This appears to come from the scenario using the Kentland RID KML flight data:
monitoring/uss_qualifier/test_data/usa/kentland/rid.kmlThat KML currently appears to define only 2 nominal flights:
flight: south uflight: dairy circleThe scenario documentation for
sp_operator_notify_slow_updatesaysflights_datamust contain 1 nominal flight per SP under test, so a 3-SP run needs 3 nominal flights.Downstream issue with attached report artifact: https://github.com/utmimplementationus/tech_committee/issues/202
Test check
This fails before reaching a specific standards check. The relevant scenario appears to be:
monitoring/uss_qualifier/scenarios/astm/netrid/v22a/sp_operator_notify_slow_update.mdThe mismatch is between:
service_providersflights_datasourceDifference from expected behavior
A baseline qual configuration intended to run against 3 SPs should provide at least 3 nominal flights, or use a flight-data source that can generate enough flights for the configured SP count.
The
ValueErroritself appears correct, but the current baseline/data combination makes the qual run unusable for 3-SP testing. Also, the HTML report should ideally surface this as a clean configuration/setup failure rather than ending up broken.Additional context
Possible fixes:
adjacent_circular_flights_simulation_sourceinstead of the fixed Kentland KML source, so enough flights can be generated for the SP count.