Skip to content

CI fails when other jobs are running because of time out #832

Description

@n-eiling

See https://git.rwth-aachen.de/acs/public/villas/node/-/jobs/5510111 for details.

Relevant output:

[TIME]  pipe-loopback-zeromq (ran for more then 1m)
[SKIP]  pipe-python-protobuf: Test is broken
[SKIP]  relay: Test is broken
[PASS]  signal (ran for .100993659s)
[PASS]  test-config (ran for 7.858488891s)
Summary: 1 of 61 tests failed.
   Timedout: 1
   Skipped: 18

Maybe we can remove the time out?

Activity

  1. stv0g commented on Nov 29, 2024

    @stv0g
    Contributor

    I guess the timeout is probably more of an indicator for a broken test?

    Shouldn't we attempt to fix the test, or disable it?

  2. n-eiling commented on Dec 2, 2024

    @n-eiling
    MemberAuthor

    The test does work if it I rerun it. So it seems to work, but is just too slow if run in parallel with everything else.
    I haven't investigated thoroughly though and I don't know if a similar functionality is tested in another test.

  3. stv0g commented on Feb 10, 2025

    @stv0g
    Contributor

    Is this still happening? I've seen it not myself yet..

  4. n-eiling commented on Feb 13, 2025

    @n-eiling
    MemberAuthor

    Haven't seen it in a while - but that doesn't mean it fixed itself. Don't you think it's bad practice to rely on timing in the CI? The runner might be really slow sometimes.

  5. stv0g commented on Feb 13, 2025

    @stv0g
    Contributor

    Yeah, its a valid consern.

    I have added those timeout checks to avoid clogging the CI with jobs which are dead-locked.
    Maybe we should raise the timeout to something like 1h?

  6. n-eiling commented on Feb 13, 2025

    @n-eiling
    MemberAuthor

    sounds good to me

  7. IgnoreWarnings commented on Feb 21, 2025

    @IgnoreWarnings
    Collaborator

    Haven't seen it in a while - but that doesn't mean it fixed itself. Don't you think it's bad practice to rely on timing in the CI? The runner might be really slow sometimes.

    Is this still happening? I've seen it not myself yet..

    I got the timeout yesterday, it is still there.

  8. IgnoreWarnings commented on Feb 24, 2025

    @IgnoreWarnings
    Collaborator

    I get it consistently now on every single commit.

  9. added a commit that references this issue on Feb 24, 2025
    0cf5db0
  10. IgnoreWarnings commented on Feb 24, 2025

    @IgnoreWarnings
    Collaborator

    Increasing the timeout didnt help, the integrationtest seems to get stuck somehow.

    Image

  11. IgnoreWarnings commented on Feb 24, 2025

    @IgnoreWarnings
    Collaborator

    I guess the timeout is probably more of an indicator for a broken test?

    Shouldn't we attempt to fix the test, or disable it?

    The test does work if it I rerun it. So it seems to work, but is just too slow if run in parallel with everything else. I haven't investigated thoroughly though and I don't know if a similar functionality is tested in another test.

    It looks like @stv0g is right that the test is broken. It looks like the Test breaks under some external condition paralell Test/Ports in use or something else.
    I increased the timeout and the CI hangs on integration test.

  12. IgnoreWarnings commented on Feb 24, 2025

    @IgnoreWarnings
    Collaborator

    "[TIME] pipe-loopback-nanomsg (ran for more then 15m)"

  13. IgnoreWarnings commented on Feb 24, 2025

    @IgnoreWarnings
    Collaborator

    But it looks like before it was on another test: [TIME] pipe-loopback-zeromq (ran for more then 1m)

  14. stv0g commented on Feb 25, 2025

    @stv0g
    Contributor

    Okay, if the tests are broken, we need to figure out what broke them as they used to work in previous commits.

    Maybe git bisect could help..

  15. n-eiling commented on Feb 25, 2025

    @n-eiling
    MemberAuthor

    I suspect this is more due to docker / strange interactions between the tests.

  16. n-eiling commented on May 23, 2025

    @n-eiling
    MemberAuthor

    This is still happening...

  17. IgnoreWarnings commented on May 23, 2025

    @IgnoreWarnings
    Collaborator

    I can look into this with my current task (broken stats hook on arm).

  18. changed the title [-]CI fails when other jobs are running because of time out.[/-] [+]CI fails when other jobs are running because of time out[/+] on Jun 20, 2025
  19. linked a pull request that will close this issueMigrate CI from GitLab to GitHub #1022on Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingci

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions