Conversation
Two agent-facing docs pointed at an external repo a reader of this repo cannot open. Route cross-cutting fixes to the maintainer and phrase defaults without an external attribution. Docs only, no behavior change.
…tics.Metrics (#865) Add an always-on System.Diagnostics.Metrics meter (PlexCleaner.Process) so a long-running process or monitor pass can be inspected live with dotnet-counters. Overall progress is weighted by input bytes, not file count. Includes per-outcome counters, in-flight and thread gauges, byte-weighted progress and ETA, and file and per-tool duration histograms, all with bounded tags. Ships a Docker counters wrapper. Zero new packages.
* Add per-file partial-credit progress to the runtime metrics Fold in-flight files' partial progress into the byte-weighted progress.ratio and eta.seconds gauges so a long scan or re-encode advances the aggregate continuously instead of only at completion. - Metrics: per-file FileSink carrying byte weight and an atomic permyriad fraction, a lock-free in-flight registry, and a ThreadLocal current-file sink captured at each tool call site. - MediaTool.ExecuteStreamStdOut streams stdout for progress while capturing stderr for failure logging. - Instrument the long ops only: ffprobe full-file scan (pts over duration), ffmpeg re-encode (-progress pipe:1), HandBrake re-encode (--json Working.Progress). Fast remux/copy/setts stay at zero until completion. - Tests for the fold math and the three tool progress parsers. - Convert all README and HISTORY inline URIs to reference-style links, grouped shields/internal/external and alphabetized. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Track max pts so the scan fraction only moves forward Packets arrive in demux order with interleaved audio/video and reordered B-frame timestamps, so raw per-packet PtsTime is non-monotonic. Report the running max pts over duration so the in-flight scan fraction advances without backward jitter. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Model one-file-per-thread in the observable metrics test Complete one file and start a second that stays in flight, instead of starting two files on one thread, so the test matches the ThreadLocal current-file semantics, and clear the in-flight sink at the end. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Read the sink duration via Volatile across threads DurationUs is written on the worker thread and read per progress line on the tool pipe thread, so back it with a Volatile.Read getter and a SetDurationUs writer, matching the permyriad fraction pattern. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Switch runtime metrics to operation-weighted progress Replace the in-tool partial-credit progress with a simpler operation-counting model: each heavy full-file operation adds the file's size to the work total when it starts and to the completed total when it ends, so progress reflects the actual, non-deterministic work with no per-file fraction weighting and no parsing of tool progress output. - Metrics: work.total/work.completed byte counters incremented at operation boundaries via an ambient thread-local file size; drop the FileSink partial-credit machinery. bytes.total stays the input-size reference. - Bracket the heavy full-file operations (closed-caption scan, idet, ffmpeg re-encode, HandBrake deinterlace, packet analysis, verify) with OpStarted/OpCompleted. - Keep the tool progress-parsing capability (ParseProgressFraction, ExecuteStreamStdOut, the Progress()/Json() builders) and its unit tests unused by this feature, for the media-tools library. - NativeAOT: enable EventSourceSupport when PublishAot is set so dotnet-counters and dotnet-monitor can read the meter from an AOT build; default builds are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Call the work instruments gauges not counters in the docs work.total and work.completed are run-scoped observable gauges, not Counter instruments, so name them gauges to avoid confusing dotnet-counters and OpenTelemetry consumers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Count operation completion only when the tool actually ran Move OpCompleted after the executed guard so a cancelled or failed-to-start tool is not credited as completed work, which would skew progress and ETA on interrupted runs. A non-zero exit still counts, the tool did the full-file work. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Count the packet scan when it ran but exited non-zero GetPackets returns false on a non-zero exit where ffprobe still ran the scan, so gate OpCompleted on cancellation instead of the return value, otherwise a ran-but-non-zero scan strands its work in the total and skews progress for the rest of the run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Credit the packet scan only when ffprobe actually ran Gate OpCompleted on the scan having run, evidenced by exit 0 or captured stderr, instead of a cancellation check, so a cancellation or a failure to start (empty error) is excluded while a ran-but-non-zero scan is still counted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Roll back discovered work when an operation never runs OpStarted adds the file size to the work total up front, so an operation that is cancelled or fails to start would leave the total permanently inflated and stop progress converging. Add OpAborted to roll that size back out, and call it on every abort path so each started operation resolves to exactly one of completed or aborted. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
) Bumps the actions-deps group with 1 update: [DavidAnson/markdownlint-cli2-action](https://github.com/davidanson/markdownlint-cli2-action). Updates `DavidAnson/markdownlint-cli2-action` from 24.0.0 to 24.1.0 - [Release notes](https://github.com/davidanson/markdownlint-cli2-action/releases) - [Commits](DavidAnson/markdownlint-cli2-action@8de2aa0...6bf21b0) --- updated-dependencies: - dependency-name: DavidAnson/markdownlint-cli2-action dependency-version: 24.1.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps AwesomeAssertions from 9.4.0 to 9.5.0 Bumps ptr727.LanguageTags from 1.5.48 to 1.5.53 --- updated-dependencies: - dependency-name: AwesomeAssertions dependency-version: 9.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: nuget-deps - dependency-name: ptr727.LanguageTags dependency-version: 1.5.53 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps ptr727.LanguageTags from 1.5.53 to 1.5.56 Bumps ptr727.Utilities from 4.0.18 to 4.0.28 --- updated-dependencies: - dependency-name: ptr727.LanguageTags dependency-version: 1.5.56 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps - dependency-name: ptr727.Utilities dependency-version: 4.0.28 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…874) Bumps the actions-deps group with 1 update: [actions/checkout](https://github.com/actions/checkout). Updates `actions/checkout` from 7.0.0 to 7.0.1 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@9c091bb...3d3c42e) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 7.0.1 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: actions-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#878) Bumps the actions-deps group with 1 update: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv). Updates `astral-sh/setup-uv` from 8.3.2 to 9.0.0 - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@11f9893...c771a70) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 9.0.0 dependency-type: direct:production update-type: version-update:semver-major dependency-group: actions-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Vendor the Repository Boundaries and Write Safety and Verification Discipline sections, refresh the canonical Git and Commit Rules bullets, and re-vendor .markdownlint-cli2.jsonc and repo-config/configure.sh. Convert Docker/README.md to reference-style links, default the Dockerfile BUILD_CONFIGURATION to Release, standardize set -Eeuo pipefail in every non github-release run: block, and pin the publisher ref to github.sha. The github-release job is untouched pending the release-pipeline reconciliation that carries the prerelease backstop.
Add the canonical validate-release job: it asserts branch<->version classification once, before the builds, covering both arms (main must be public, every other branch prerelease) where the old in-job backstop checked only main and only after building. Take the canonical github-release job, which adds the release-asset-* cleanup at its point of consumption, gated to the release-create step and best-effort. publish-release grants the actions: write it needs. Adopt the Docker atomicity gate so the image is never pushed on a partial run, and add expect_release_assets, which the canonical github-release job reads. Sweep WORKFLOW.md for the moved gate: D2.2, D5.4, the publish diagram, the 5A assertions, and the scenario table. Correct two stale style claims found in the sweep - the shell prologue is set -Eeuo pipefail, and workflow YAML is pinned LF, not CRLF.
Re-vendor the two verbatim sections the hub batch changed: Git and Commit Rules (the history-rewrite bullet's grammar) and Verification Discipline (the lint-gate bullet is now self-contained, so it no longer points at a section this repo does not carry). Repository Boundaries and Write Safety is unchanged upstream and verified byte-identical. Add the structural sections this file was missing: Foundational Principles, Repository Onboarding and Conformance, Supported Development Platforms, Editor and Tasks, and Repository Details. Rename Project Structure to Repository Layout, its canonical name, keeping this repo's own content. Sections whose canonical body is not about this repo are adapted rather than copied: Foundational Principles drops the operational-repo cross-reference and names this repo's two pull channels, and Repository Onboarding states the audit obligation without the machinery links that do not resolve here. Devcontainer is left out - this repo ships none.
There was a problem hiding this comment.
Pull request overview
Promotes develop to main, bringing the 3.22 runtime-metrics feature, the #880 hub convergence workflow/governance updates, documentation refreshes, and Dependabot bumps into the stable branch and next scheduled/dispatch release train.
Changes:
- Add always-on runtime metrics (
System.Diagnostics.Metrics) with operation-weighted progress and per-tool/per-outcome instrumentation. - Converge CI/CD release orchestration on the updated entry gate + artifact-cleanup model (pin publish to
github.sha, validate branch/version classification, avoid partial Docker pushes). - Refresh governance/config/docs (AGENTS/WORKFLOW/CODESTYLE/markdownlint) and bump dependency versions.
Reviewed changes
Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| WORKFLOW.md | Updates workflow spec (shell strictness, LF note, publish ref pinning, validate-release gate, artifact cleanup). |
| version.json | Bumps version floor from 3.21 to 3.22. |
| repo-config/README.md | Documents new configure.sh apply / check modes and added security feature management. |
| repo-config/configure.sh | Adds read-only check mode, safer ruleset lookup, and applies Dependabot security settings. |
| README.md | Updates release summary to 3.22 and adds a Runtime Metrics section + ref-style link normalization. |
| HISTORY.md | Adds 3.22 release notes covering runtime metrics behavior and instruments. |
| ARCHITECTURE.md | Documents runtime metrics architecture, hooks, and progress semantics. |
| AGENTS.md | Adds foundational principles, write-safety + verification discipline sections, and refreshes governance structure. |
| CODESTYLE.md | Clarifies markdownlint expectations (MD033 enabled) and cspell CI scope wording. |
| .markdownlint-cli2.jsonc | Enables MD033 again (keeps MD013 disabled) and updates commentary. |
| Directory.Packages.props | Bumps AwesomeAssertions, ptr727.LanguageTags, and ptr727.Utilities versions. |
| PlexCleaner/Metrics.cs | Introduces the process-wide meter and run-scoped gauges/counters for runtime metrics. |
| PlexCleaner/ProcessDriver.cs | Adds per-run file/byte aggregation and per-file metrics accounting around task execution. |
| PlexCleaner/Process.cs | Records per-outcome state/modified/verifyfailed metrics at the point state is known. |
| PlexCleaner/MediaTool.cs | Adds tool-duration recording and a new stdout-streaming execution path. |
| PlexCleaner/FfProbeTool.cs | Adds tool-duration recording and brackets heavy ops with OpStarted/OpCompleted/OpAborted. |
| PlexCleaner/FfMpegTool.cs | Adds ffmpeg progress parsing helper and brackets heavy ops for operation-weighted progress. |
| PlexCleaner/FfMpegBuilder.cs | Adds -progress pipe:1 builder helper for future progress parsing use. |
| PlexCleaner/HandBrakeTool.cs | Adds HandBrake JSON progress parsing helper and brackets heavy ops for progress weighting. |
| PlexCleaner/HandBrakeBuilder.cs | Adds --json builder helper for future progress parsing use. |
| PlexCleaner/PlexCleaner.csproj | Enables EventSourceSupport for NativeAOT builds to allow metrics consumption. |
| PlexCleaner/Program.cs | Disposes Metrics on program exit. |
| PlexCleaner/ProcessFile.cs | Minor formatting-only change (blank line). |
| PlexCleanerTests/MetricsTests.cs | Adds sequential tests for progress/ETA math, flag enumeration, and meter observability. |
| PlexCleanerTests/ToolProgressParsingTests.cs | Adds tests for ffmpeg/HandBrake progress-line parsing helpers. |
| Docker/Dockerfile | Switches default BUILD_CONFIGURATION to Release and installs counters wrapper. |
| Docker/counters.sh | Adds counters wrapper to read runtime metrics inside the container. |
| Docker/README.md | Converts to reference-style links + adds organized link definitions. |
| .github/workflows/publish-release.yml | Pins publish ref to github.sha and grants actions: write for artifact cleanup. |
| .github/workflows/build-release-task.yml | Adds validate-release gate, improves docker push gating, and adds release-asset cleanup. |
| .github/workflows/get-version-task.yml | Updates comments and bumps checkout pin. |
| .github/workflows/validate-task.yml | Bumps action pins, standardizes set -Eeuo pipefail, and updates uv setup action. |
| .github/workflows/test-pull-request.yml | Standardizes set -Eeuo pipefail. |
| .github/workflows/merge-bot-pull-request.yml | Standardizes set -Eeuo pipefail. |
| .github/workflows/build-executable-task.yml | Bumps checkout pin and adds set -Eeuo pipefail for publish step. |
| .github/workflows/build-docker-task.yml | Bumps checkout pin. |
| .github/copilot-instructions.md | Removes external-repo pointer; routes shared-convention discrepancies to maintainer. |
Comment on lines
+220
to
+223
| for id in $ids; do | ||
| gh api --method DELETE "repos/$GITHUB_REPOSITORY/actions/artifacts/$id" \ | ||
| || echo "::warning::Failed to delete artifact $id; retention-days backstop will reap it." | ||
| done |
#885) Bumps the actions-deps group with 1 update: [docker/login-action](https://github.com/docker/login-action). Updates `docker/login-action` from 4.4.0 to 4.5.0 - [Release notes](https://github.com/docker/login-action/releases) - [Commits](docker/login-action@af1e73f...06fb636) --- updated-dependencies: - dependency-name: docker/login-action dependency-version: 4.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: actions-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps ptr727.LanguageTags from 1.5.56 to 1.5.58 --- updated-dependencies: - dependency-name: ptr727.LanguageTags dependency-version: 1.5.58 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: nuget-deps ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Comment on lines
+82
to
+83
| - Hooks: `ProcessDriver.ProcessFiles` (the choke point every command and monitor cycle funnels through) drives the file/byte/in-flight instruments and the byte-weighted `progress.ratio`. `Process.ProcessFiles` records the per-`SidecarFile.StatesType` outcomes. `MediaTool` execution paths record `tool.duration`. | ||
| - Progress is weighted by input bytes (summed once up front and credited at completion from the same map), not file count. |
Re-vendor all 15 applicable verbatim AGENTS.md sections byte-for-byte from hub main per the per-section fidelity model (spec/section-model.md), add a Devcontainer intent section (this repo ships none, so it documents the native toolchain), and keep Repository Layout as this repo's own. Operational Repositories is operational-only, so N/A here. Remove two sections the model does not permit, without losing content: Shared Configuration and Tooling was a near-duplicate of the now-verbatim canonical sections plus CODESTYLE, so its unique C# format-gate note moves to CODESTYLE and its release-notes rule to ARCHITECTURE; Logging Conventions is PlexCleaner-specific app behavior, so it moves whole to ARCHITECTURE. Drop the Git repo-specific subsection, which broke the section byte-match and is already covered by the verbatim Merge Gate and Branching Model.
| # docker exec <container> counters # live monitor of the PlexCleaner.Process meter | ||
| # docker exec <container> counters collect ... # any other dotnet-counters verb/args, passed through | ||
|
|
||
| set -euo pipefail |
The hub genericized the verbatim sections (ProjectTemplate#422): the hub-only 404 links inside them are de-linked to prose, and Operational Repositories is reclassified appliesTo:* so its cross-references resolve everywhere. Re-vendor all 16 verbatim sections byte-for-byte from the current hub main. Carry Operational Repositories as fleet context, and drop Repository Onboarding and Conformance, which the hub reclassified as hub-only audit machinery. The verbatim text now carries zero relative links to hub-only paths. Devcontainer and Repository Layout stay as this repo's own intent sections. Description consistency (ProjectTemplate#423) already holds.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 37 out of 37 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
Docker/counters.sh:9
- This script starts with
set -euo pipefail, but the repo's bash-safety convention isset -Eeuo pipefail(include-Eso an ERR trap would inherit into functions/subshells). Update this to match the documented standard used across workflows/scripts.
set -euo pipefail
ARCHITECTURE.md:83
- The new Runtime Metrics section describes
progress.ratioas byte-weighted and credited at completion, but the implementation inMetrics.csis operation-weighted (work.total/work.completed grows as operations are discovered). This documentation should match the actual metrics model to avoid misleading readers.
- Hooks: `ProcessDriver.ProcessFiles` (the choke point every command and monitor cycle funnels through) drives the file/byte/in-flight instruments and the byte-weighted `progress.ratio`. `Process.ProcessFiles` records the per-`SidecarFile.StatesType` outcomes. `MediaTool` execution paths record `tool.duration`.
- Progress is weighted by input bytes (summed once up front and credited at completion from the same map), not file count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotion of
developtomain. Twelve commits since the last promotion: the runtime-metrics feature, the #880 convergence work, a docs change, and five Dependabot bumps.Referencing #880 rather than closing it - the hub still has open dispositions there and it is the live coordination channel for them. It gets closed by hand once those land.
What is in it
Runtime metrics (the functional change, and why the floor moved to 3.22)
System.Diagnostics.Metricsunder thePlexCleaner.Processmeter, readable withdotnet-countersand no extra infrastructure. Inert until a listener observes them, so there is no configuration flag and negligible idle cost.state/tooltags and no filename tags.counterswrapper, so reading them isdocker exec <container> counters.Hub convergence (#880) -
086cb8b,ba3b01a,f91af07Repository Boundaries and Write SafetyandVerification Disciplinevendored,Git and Commit Rulesrefreshed, and the AGENTS.md structure re-derived against the canonical.validate-releaseentry gate now asserts branch/version consistency in both directions before anything builds, the canonicalgithub-releasejob brings therelease-asset-*cleanup, and the Docker job can no longer push on a partial run..markdownlint-cli2.jsoncandrepo-config/configure.shre-vendored (the latter gaining the read-onlycheckmode),Docker/README.mdon reference-style links,BUILD_CONFIGURATIONdefaulting toRelease,set -Eeuo pipefailstandardized, and the publisher pinningref: github.sha.Other - #866 (self-contained agent docs), plus Dependabot: #870, #872, #874, #876, #878.
Release impact
version.jsonis already at 3.22 (raised in #865 for the metrics feature), and README + HISTORY both carry the 3.22 notes, so no version edit is needed here.maincurrently releases 3.21.x; the next dispatch or scheduled run after this merges publishes 3.22.x as stable and refresheslatest.Note this promotion is the first to run under the re-vendored release task, so the publish that follows it exercises
validate-releaseand the new artifact cleanup againstmainfor the first time. Themainarm of the gate (a public release must carry no prerelease suffix) has not run outside smoke yet, where it self-skips by design.Verification
Every commit here came through CI on
developwith a Copilot review on its head SHA. The full lint gate (markdownlint-cli2, actionlint, editorconfig-checker, cspell) and the husky pre-commit gate are clean on the tip. Thevalidate-releasesmoke self-skip was confirmed from its job log rather than inferred from a green check.Merge-commit only, and never with
--delete-branch- the head of a promotion PR isdevelopitself.