Skip to content

Finish closing usage sharing within the 5 seconds the specification allows - #180

Draft
jwrosewell wants to merge 8 commits into
rcfrom
remote/java-06
Draft

jwrosewell wants to merge 8 commits into
rcfrom
remote/java-06

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

The problem

Closing the usage sharing element waited the whole 5,000 ms of SHARE_USAGE_DEFAULT_HTTP_POST_TIMEOUT for the data in the queue to be sent, and only then discarded what was left, logged the warning and freed its resources. So when the remote service stopped answering, closing took longer than the 5 seconds the specification allows for the whole of cleanup. Nothing measured how long closing took.

An earlier version of this pull request set the test's limit to 4,800 ms, a figure taken from the runners' measurements. The limit belongs to the specification, so the test now asserts the specification's 5,000 ms, and the measurements below show that every runner closes well inside it.

The specification

usage-sharing-element.md lines 387 to 390 at 6570727d, in the Cleanup section, follows the three steps of cleanup at lines 382 to 385 (wait while the data in the queue is sent, send a final message, free the resources).

Cleanup SHOULD complete within 5 seconds, so that closing the element does not hold up the process that closes it. If the data waiting in the queue has not all been sent after 5 seconds, the element SHOULD stop sending, discard the data remaining in the queue and log a warning.

6570727d is the newest commit to that file on main.

The change

  • bbef6e4, the test, which replaces its first version in 1ba2d24. ShareUsageCloseTimeoutTests.ShareUsageElement_ServiceStopsAnswering_CloseWithin5Seconds starts a send to a service that does not answer, closes the element, and fails when close() takes too long measured from its own start, while the send is still waiting for its answer. A 30 second guard stops the test if closing waits for the service. The element setup the two tests in the class share moved into one method.
  • bcdb06b, the fix. ShareUsageBase.managedResourcesCleanup notes when closing starts and waits for the sending thread only until 4,500 ms after that, being the close limit less the reserve. The reserve is kept for discarding the rest, logging the warning and freeing the resources. The two values are the new package-private constants SHARE_USAGE_CLOSE_LIMIT_MILLIS (5000) and SHARE_USAGE_CLOSE_RESERVE_MILLIS (500) in Constants. The warning now gives the time waited. SHARE_USAGE_DEFAULT_HTTP_POST_TIMEOUT is unchanged and is still the timeout of a send.
  • 8a48c33 set the test's limit to 4,800 ms from the measurements below, and 77a0e5e set it back to the specification's 5,000 ms with a strict <, so a close that takes exactly 5,000 ms fails.
  • 3276aab quotes the specification sentence and its commit beside CLOSE_LIMIT_MILLIS in the test, and says that the element waits 4,500 ms and then does the rest, so it closes in a little over 4,500 ms. This commit changes a comment only.
  • f957fff and 79de9f2 merge rc into the branch.

The code in ShareUsageBase and Constants has not changed on this branch since bcdb06b. The later commits change the test file only. No public name is removed or renamed, and no package gains a dependency.

The runs

Red. Run 43856688032 built 8a48c33 with the fix taken out of ShareUsageBase, so that closing waits the whole 5 seconds again. The new test failed on all five runners, with "Closing took 5000 ms" on ubuntu-22.04, ubuntu-22.04-arm and windows-latest, 5078 ms on macos-15 and 5074 ms on macos-15-intel. The other test in the class passed in every job. That run asserted the earlier 4,800 ms limit.

Red at the 5,000 ms limit. Run 44009239287, failure as expected. The run is the pull request build, without macOS, of a temporary branch whose commit, 31af4b1, is rc 46115d7 with only ShareUsageCloseTimeoutTests.java as at the head 3276aab added, so it checks the same 5,000 ms limit. In all nine jobs pipeline.engines.fiftyone gave "Tests run: 193, Failures: 1, Errors: 0, Skipped: 1", the one failure being ShareUsageElement_ServiceStopsAnswering_CloseWithin5Seconds at line 195 with Closing took 5000 ms, not within the 5000 ms the specification puts on cleanup. ==> expected: <true> but was: <false>. Closing took 5000 ms on Ubuntu x64 with Java 8, 11, 17 and 21 and on Ubuntu arm64 with Java 17, and 5002, 5011, 5013 and 5003 ms on Windows with Java 8, 11, 17 and 21. The other test in the class passed in every job.

Each runner's own measurement. Run 43854244981 measured close() five times on each of the five runners, at the fix bcdb06b and at the test commit bbef6e4 without it. The test's limit was set to 1 ms in the checkout so that its failure message printed the time taken in every round. Milliseconds, in the order measured:

Runner With the fix Without the fix
ubuntu-22.04 4500, 4500, 4500, 4500, 4500 5000, 5000, 5000, 5000, 5000
ubuntu-22.04-arm 4500, 4500, 4500, 4500, 4500 5000, 5000, 5000, 5000, 5000
windows-latest 4501, 4500, 4502, 4501, 4504 5012, 5007, 5010, 5005, 5002
macos-15 4520, 4535, 4567, 4573, 4574 5050, 5076, 5074, 5019, 5076
macos-15-intel 4549, 4596, 4548, 4642, 4565 5115, 5143, 5114, 5137, 5021

The slowest of the 25 rounds with the fix is 4,642 ms, on macos-15-intel, which is 358 ms inside the 5,000 ms limit. The fastest of the 25 rounds without the fix is 5,000 ms, on the Ubuntu runners, which the strict < fails.

On a macOS Intel machine outside CI, 30 closes at 8a48c33 took 4,500 to 4,510 ms (median 4,505 ms), being the 4,500 ms wait plus 0 to 10 ms of work. The log of that measurement is not published.

Full build. Run 43948904885 built 3276aab merged with rc on nine configurations, Ubuntu x64 with Java 8, 11, 17 and 21, Ubuntu arm64 with Java 17, and Windows with Java 8, 11, 17 and 21. Every job passed, and ShareUsageCloseTimeoutTests ran 2 tests with 0 failures in each. Each job's PullRequestSha line names 3276aab. Its Configure job printed "Left out 6 macOS entries", so macOS is not in this build.

What remains

  • The close test has not run on macOS at 3276aab, because the full build left macOS out. The macOS figures above come from the measurement run at bcdb06b, which has the same ShareUsageBase and Constants code.

Before you upgrade

When usage sharing is closed while the remote service is slow to answer, closing now stops waiting after 4.5 seconds rather than 5, so a little less of the queued data may be sent before the rest is discarded. The warning now gives the time waited, 4,500 ms, rather than the HTTP timeout.

…tops answering

The usage sharing element page, at 51Degrees/specifications 706de1f0,
says in pipeline-specification/pipeline-elements/usage-sharing-element.md
line 387:

  "Cleanup SHOULD complete within 5 seconds, so that closing the element
  does not hold up the process that closes it."

https://github.com/51Degrees/specifications/blob/706de1f0cb3edd560eb7fd7ef3bb19b93a4883dc/pipeline-specification/pipeline-elements/usage-sharing-element.md#L387

Closing already waits at most SHARE_USAGE_DEFAULT_HTTP_POST_TIMEOUT, 5000
milliseconds, and ShareUsageCloseTimeoutTests checked what is sent and
logged after that wait, not how long closing takes. The new test starts a
send to a service that does not answer, then closes the element, and
checks that closing returns within the 5 second wait and a 5 second margin
for discarding the rest, while the send is still waiting for its answer.
The element setup the two tests share moves into one helper. The code is
unchanged.

Specification conformance item US-30.
@jwrosewell jwrosewell added the remote-conformance Specification conformance work from the remote packs label Nov 15, 2026
The usage sharing element page, at 51Degrees/specifications dbaf099c,
lists cleanup as three steps in
pipeline-specification/pipeline-elements/usage-sharing-element.md lines
418 to 421 (wait while the consumer sends the data in the queue, send a
final message, free the resources), and says at line 423:

  "Cleanup SHOULD complete within 5 seconds, so that closing the element
  does not hold up the process that closes it."

https://github.com/51Degrees/specifications/blob/dbaf099c/pipeline-specification/pipeline-elements/usage-sharing-element.md#L413-L426

So the 5 seconds bound the whole of closing. The test of a service that
stops answering now measures close from its start and fails when it takes
more than 5000 milliseconds, the bound the specification sets, rather than
allowing 5 seconds of margin beyond the wait. A guard of 30 seconds still
stops the test if closing waits for the service.

Specification conformance item US-30.
Closing waited the full 5000 milliseconds for the data in the queue to be
sent, then discarded the rest, logged the warning and stopped the sending
thread, so closing took just over 5 seconds.

Closing now measures from its start and stops waiting at 4500
milliseconds, keeping 500 milliseconds of the 5 second limit for
discarding the rest, the warning and freeing the resources. No second
wait is added. The two values are named constants,
SHARE_USAGE_CLOSE_LIMIT_MILLIS (5000) and SHARE_USAGE_CLOSE_RESERVE_MILLIS
(500), and the warning gives the time waited, 4500 milliseconds. The HTTP
timeout, SHARE_USAGE_DEFAULT_HTTP_POST_TIMEOUT, which closing used for its
wait before, is unchanged.

Measured on an Apple M1 (macOS 14.4.1, OpenJDK 17.0.1), the only runner
measured: before this change closing took 5010 milliseconds, so the work
after the wait took 10 milliseconds. After it the test of a service that
stops answering took 4.675, 4.667 and 4.665 seconds over three runs,
setup of about 0.17 seconds included, so closing took about 4.5 seconds.
The 500 millisecond reserve is 50 times what the work after the wait took
there. The CI runners (Ubuntu x64 and arm64, Windows, macOS) were not
measured.

Answers pipeline-specification/pipeline-elements/usage-sharing-element.md
line 423 at 51Degrees/specifications dbaf099c, "Cleanup SHOULD complete
within 5 seconds, so that closing the element does not hold up the
process that closes it."

Specification conformance item US-30.
@jwrosewell jwrosewell changed the title Specification conformance, 51degrees-java, section 6 Finish closing usage sharing within the 5 seconds the specification allows Nov 16, 2026
The close test asserts the 5,000 ms the usage sharing element specification puts on cleanup. Quote its sentence, at the fixed commit 6570727d, beside the limit, and say why the element closes in a little over 4,500 ms. Comment only, so there is no failing test to add.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

remote-conformance Specification conformance work from the remote packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant