Skip to content

[8.0] "InvalidOperationException: Category does not exist" failure in net48 win x86 in System.Diagnostics.Tests.PerformanceCounterTests.PerformanceCounter_IncrementBy_IncrementBy2 #103435

Open
Open
[8.0] "InvalidOperationException: Category does not exist" failure in net48 win x86 in System.Diagnostics.Tests.PerformanceCounterTests.PerformanceCounter_IncrementBy_IncrementBy2#103435
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabarch-x86area-System.Diagnostics.PerformanceCounterneeds-further-triageIssue has been initially triaged, but needs deeper consideration or reconsiderationos-windowstest-run-desktopTest failures in .NET Framework "Desktop" test runs (running CoreFX test assets)
Milestone
Future

Description

@carlossanlop
carlossanlop
opened on Jun 13, 2024

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=706881
Build error leg or test failing: System.Diagnostics.Tests.PerformanceCounterTests.PerformanceCounter_IncrementBy_IncrementBy2

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "System.InvalidOperationException : Category does not exist.",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

PR: #103426
Job: https://dev.azure.com/dnceng-public/public/_build/results?buildId=706881&view=logs&j=885df564-30f3-5dad-0bdc-caf444244bdf&t=71aa97be-149e-50ab-5d1c-74d5540a091f&l=50
Job name: windows-x86 Release Libraries_NET48
Log: https://helixre107v0xd1eu3ibi6ka.blob.core.windows.net/dotnet-runtime-refs-pull-103426-merge-01018e12059a47ae96/System.Diagnostics.PerformanceCounter.Tests/1/console.bc703876.log?helixlogtype=result
Output:

    System.Diagnostics.Tests.PerformanceCounterTests.PerformanceCounter_IncrementBy_IncrementBy2 [FAIL]
      System.AggregateException : One or more errors occurred.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      ---- System.InvalidOperationException : Category does not exist.
      Stack Trace:
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(54,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(78,0): at System.Diagnostics.Tests.Helpers.RetryOnAllPlatforms[T](Func`1 func)
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(237,0): at System.Diagnostics.Tests.PerformanceCounterTests.PerformanceCounter_IncrementBy_IncrementBy2()
        ----- Inner Stack Trace #1 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #2 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #3 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #4 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #5 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #6 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #7 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #8 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #9 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)
        ----- Inner Stack Trace #10 (System.InvalidOperationException) -----
           at System.Diagnostics.PerformanceCounterLib.GetCategorySample(String machine, String category)
           at System.Diagnostics.PerformanceCounter.NextSample()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/PerformanceCounterTests.cs(243,0): at System.Diagnostics.Tests.PerformanceCounterTests.<>c__DisplayClass15_0.<PerformanceCounter_IncrementBy_IncrementBy2>b__1()
        /_/src/libraries/System.Diagnostics.PerformanceCounter/tests/Helpers.cs(75,0): at System.Diagnostics.Tests.Helpers.<>c__DisplayClass12_0`1.<RetryOnAllPlatforms>b__0()
        /_/src/libraries/Common/tests/TestUtilities/System/RetryHelper.cs(45,0): at System.RetryHelper.Execute(Action test, Int32 maxAttempts, Func`2 backoffFunc, Predicate`1 retryWhen, String testName)

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=706881
Error message validated: [System.InvalidOperationException : Category does not exist.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/13/2024 6:20:23 PM UTC

Report

Build Repository Test Pull Request
1646603 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #136298
1645938 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #130050
1641792 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #136072
1640849 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters
1638654 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters
1638420 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #135911
1633235 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #135668
1631732 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #135618
1623618 dotnet/runtime System.Diagnostics.Tests.InstanceDataTests.InstanceDataCollectionCollection_CopyTo #134958
1621972 dotnet/runtime MonoTests.System.Runtime.Caching.CountersTest.Basic_Counters #134562
1613872 dotnet/runtime System.Diagnostics.Tests.InstanceDataTests.InstanceDataCollectionCollection_CopyTo #134738

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 4 11

Activity

  1. carlossanlop
    added
    arch-x86
    os-windows
    test-run-desktopTest failures in .NET Framework "Desktop" test runs (running CoreFX test assets)
    area-System.Diagnostics.PerformanceCounter
    Known Build ErrorUse this to report build issues in the .NET Helix tab
    on Jun 13, 2024
  2. dotnet-policy-service
    added
    untriagedNew issue has not been triaged by the area owner
    on Jun 13, 2024

  3. dotnet-policy-service commented on Jun 13, 2024

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Jun 13, 2024 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @dotnet/area-system-diagnostics-performancecounter
    See info in area-owners.md if you want to be subscribed.

  4. ericstj commented on Jun 17, 2024

    @ericstj
    ericstj
    on Jun 17, 2024
    Member

    There's a number of perfcounter issues with failure mode System.InvalidOperationException : Category does not exist. This is a common mode of flakiness for these tests. IIRC there's some loss-tolerance in the actual platform APIs that can cause this. I'm not sure the RetryHelper in use here helps, because I think the problem causes the static counter list in the process to be missing the counter category. https://github.com/search?q=repo%3Adotnet%2Fruntime+%22Category+does+not+exist%22&type=issues

  5. ericstj
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jun 17, 2024
  6. ericstj
    added this to the Future milestone on Jun 17, 2024
  7. ericstj
    added
    needs-further-triageIssue has been initially triaged, but needs deeper consideration or reconsideration
    on Jun 17, 2024

  8. danmoseley commented on Mar 22, 2026

    @danmoseley
    danmoseley
    on Mar 22, 2026
    Contributor

    Note

    This analysis was generated with the assistance of GitHub Copilot.

    This appears to be effectively intractable, as @ericstj's comment suggests. A few observations:

    Why retries don't help: The root cause is in the Windows platform perf counter APIs themselves — there's inherent race/loss-tolerance that can cause the category to go missing. Once the static counter list is cached in-process without the category, RetryHelper just hits the same poisoned cache on every attempt (all 10 inner exceptions are identical).

    Why a real fix is disproportionate:

    1. Platform-level issue — the Win32 perf counter infrastructure has inherent flakiness
    2. Cache poisoning — the category list is cached statically per-process, so once it's wrong, the process is stuck
    3. net48 constraint — can't change Framework code, only test infrastructure

    Possible mitigations:

    • The test could use reflection to clear the static cache on PerformanceCounterLib between retries, so RetryHelper would actually get a fresh lookup. Ugly, but it would make retries meaningful rather than repeatedly hitting the same stale state.
    • Mark the test [OuterLoop] for net48 if the noise becomes bothersome.
    • Running each retry in a fresh process, though that's a heavier lift.

    Current approach seems right: The "Known Build Error" with BuildRetry: true is the pragmatic answer given the very low hit rate (~1/month per the report).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Known Build ErrorUse this to report build issues in the .NET Helix tabarch-x86area-System.Diagnostics.PerformanceCounterneeds-further-triageIssue has been initially triaged, but needs deeper consideration or reconsiderationos-windowstest-run-desktopTest failures in .NET Framework "Desktop" test runs (running CoreFX test assets)

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions