Skip to content

Name the elements of each pipeline in its usage sharing records - #202

Merged
jwrosewell merged 2 commits into
rcfrom
fix/usage-sharing-elements-per-pipeline
Nov 15, 2026
Merged

jwrosewell merged 2 commits into
rcfrom
fix/usage-sharing-elements-per-pipeline

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

One usage sharing element in several pipelines writes, in each record, the flow elements of the pipeline the request came from. It replaces #199, which was stacked on #193, so that it does not wait for #193's Package element.

Problem

The record's constant part, which names the flow elements of the pipeline, was made from the element's one pipeline (share_usage.py at cdd53eef, get_constant_xml and get_flow_elements). In more than one pipeline the element logged a warning that it was "registered to too many pipelines" and sent records naming no flow elements at all.

What the specification says

multi-pipeline-elements.md lines 16 to 19 at 706de1f0:

If this feature is enabled, then all Flow Elements that maintain any such state information MUST ensure that they have the means to keep track of isolated instances of this state for each Pipeline to which they are added and use the correct state information when processing.

What changed

  1. 847ed666, the test, a new test_share_usage_pipelines.py. One usage sharing element in two pipelines, each with an element of its own, processes a request in each. The record of each request names the flow elements of its own pipeline.
  2. 2a42c979, the change. get_data_from_evidence keeps the request's pipeline on its ShareUsageData, and add_to_share_usage asks get_constant_xml for that pipeline's constant part. It is made once for each pipeline, once the server address has been looked up, and kept. get_constant_xml and get_flow_elements take the pipeline as an argument that may be left out, and with none they do what they did before, so constant_xml and the warning for more than one pipeline still work for a caller that gives none.

How it was verified

  • Red, run 43742450411 on f2b6488, being 847ed66 (the test on rc cdd53ee) plus a throwaway workflow. On Ubuntu and Windows the new test failed, as both records named no flow elements, and the package's tests gave "1 failed, 109 passed".
  • Green, run 43742461275 on 5156e19, being the head 2a42c97 plus the same workflow, "1 passed" and "110 passed" on both.
  • Full build, run 43742497721 on tmp/pr-matrix-202, the head 2a42c97 merged with rc cdd53ee, green in all ten Build and Test jobs, with 11 of 11 Setting 'PullRequestSha' lines on 2a42c97 and Configure printing "Left out 5 macOS entries". macOS was left out of this draft's full build under the 9 November decision. The rc head is built with macOS before it merges into main.

What it releases

fiftyone_pipeline_engines_fiftyone and its dependants, which Monday's release plan releases already.

What remains

#199 is for closing, as this replaces it.

@jwrosewell
jwrosewell marked this pull request as ready for review November 15, 2026 18:31
@jwrosewell
jwrosewell merged commit f35e269 into rc Nov 15, 2026
3 checks passed
@jwrosewell
jwrosewell deleted the fix/usage-sharing-elements-per-pipeline branch November 15, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant