Skip to content

Fix bcp 008#504

Merged
lo-simon merged 2 commits into
sony:masterfrom
lo-simon:fix-bcp-008
Jul 20, 2026
Merged

Fix bcp 008#504
lo-simon merged 2 commits into
sony:masterfrom
lo-simon:fix-bcp-008

Conversation

@lo-simon

Copy link
Copy Markdown
Collaborator

The following bug has been reported by Niitsu, Keita-san from Sony.

When a transition to a healthier status is pending and a new event reconfirms that the actual status level has not improved and remains the same as the current status, the pending improvement is not cancelled.
As a result, when the pending timer eventually expires, the system incorrectly applies the healthier status, even though the system is still not actually healthy.

  1. New status is worse than the current status, outside the activation period

    • set_monitor_status is called immediately.
    • pending_received_time is reset to 0.
    • The pending transition is cancelled.
    • This behaviour is correct.
  2. New status is worse than the current status, inside the activation period

    • The delay branch overwrites pending_status with the worse value.
    • The timer will apply the worse status.
    • This behaviour is correct.
  3. New status is equal to the current status

    • Only the status message is updated.
    • pending_received_time is not reset.
    • The old pending improvement may later be applied incorrectly.
    • This behaviour appears to be incorrect.

Reproduction steps:

Configuration:

  • There is one network interface: NIC1
  • statusReportingDelay = 3 seconds
  • Initial state: NIC1 = LinkDown
  • Therefore, linkStatus = unhealthy (3)

Sequence:

  1. Bring NIC1 UP.
    The synthesised link status becomes healthy (1).
    set_monitor_status_with_delay(status=1) is called.
    Since 1 < current_status(3), the function enters the delay branch.

    The following pending state is stored:
    pending_status = 1
    pending_received_time = T

  2. Before the 3-second delay expires, bring NIC1 DOWN again.
    The synthesised link status returns to unhealthy (3).
    set_monitor_status_with_delay(status=3, message="NIC1 is down") is called.
    Since 3 == current_status(3), the function takes the early return path.
    Only the status message is updated to "NIC1 is down".
    pending_received_time is not reset.

  3. After 3 seconds, the behaviour thread fires.
    pending_received_time = T > 0
    The delay threshold has been reached.
    The behaviour thread applies pending_status = 1.

As a result, monitor status is registered with an incorrect status: healthy (1), and the message, Previously: NIC1 is down.

Expected final state:
linkStatus = unhealthy (3)
message = "NIC1 is down"

Actual final state:
linkStatus = healthy (1)
message = "Previously: NIC1 is down"

Proposed fix:

Reset status_pending_received_time_field_name to 0 at the beginning of the status == current_status branch, before any early return.

if (status == current_status.as_integer())
{
    // Cancel any pending improvement: the current status level is being re-confirmed by a new event,
    // so a previously pending transition to a healthier state should not be applied.
    set_property(resources, oid, status_pending_received_time_field_name, web::json::value::number(0), gate);
 
    // has status message changed?
    const auto& current_status_message = get_property(resources, oid, status_message_property_id, get_control_protocol_class_descriptor, gate);
 
    ...
}

Why this fix is safe:

This change should be safe because pending_received_time becomes non-zero only when there is a pending improvement.

When status == current_status, any existing pending status must represent a healthier state, i.e. a smaller status value.

The new event is re-confirming that the system has not actually reached that healthier state, so the pending transition should be discarded.

If there is no pending transition, meaning pending_received_time is already 0, calling set_property with 0 should effectively be a no-op.

lo-simon added 2 commits July 17, 2026 16:31
…tatus when the current status is reconfirmed
…e from bad to good, then back to bad.

Add edge case test to Tests status transitions from unhealthy (3) -> healthy (1) -> unhealthy (3) within the 3 second report delay window.

@jonathan-r-thorpe jonathan-r-thorpe left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lo-simon
lo-simon merged commit d1c8da7 into sony:master Jul 20, 2026
10 checks passed
@lo-simon
lo-simon deleted the fix-bcp-008 branch July 20, 2026 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants