Skip to content

Rework the assert_return() macro#375

Open
evelikov wants to merge 5 commits into
kmod-project:masterfrom
evelikov:assert-return
Open

Rework the assert_return() macro#375
evelikov wants to merge 5 commits into
kmod-project:masterfrom
evelikov:assert-return

Conversation

@evelikov

@evelikov evelikov commented Jun 16, 2025

Copy link
Copy Markdown
Collaborator

From the commit introducing it:

    The current assert_return() macro has a few short comings:
     - the name implies assert()
     - all users unnecessarily provide EXIT_FAILURE
     - the users cannot provide a message - "Failed assertion:" isn't great

    Tweak it up and in the process bump it to ALL_CAPS, to better indicate
    that it's a macro and not a function.

... and the follow-up commit:

    Swap the manual checks for the new OK() macro. In the process, we remove
    the cleanup in the error path.

    Many other tests don't bother, plus in the places that do try they don't
    always get it right/fully.

Before going forward and porting more tests, would be great to hear if the direction is right.

If* it is, a couple of questions:

  • would people prefer different name - OK one is based on wine, but I'm not married to it ;-)
  • how to handle the conversion - one test at a time, bulk (all assert_return -> OK, all manual -> OK), other?

@evelikov evelikov marked this pull request as draft June 16, 2025 18:32
@codecov

codecov Bot commented Jun 16, 2025

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 16.08696% with 193 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
testsuite/test-array.c 0.00% 0 Missing and 35 partials ⚠️
testsuite/test-list.c 0.00% 0 Missing and 29 partials ⚠️
testsuite/test-strbuf.c 9.67% 0 Missing and 28 partials ⚠️
testsuite/test-hash.c 0.00% 0 Missing and 24 partials ⚠️
testsuite/test-util.c 54.28% 0 Missing and 16 partials ⚠️
testsuite/test-init.c 26.66% 0 Missing and 11 partials ⚠️
testsuite/test-testsuite.c 10.00% 0 Missing and 9 partials ⚠️
testsuite/test-initstate.c 20.00% 0 Missing and 8 partials ⚠️
testsuite/test-blacklist.c 14.28% 0 Missing and 6 partials ⚠️
testsuite/test-multi-softdep.c 14.28% 2 Missing and 4 partials ⚠️
... and 7 more
Files with missing lines Coverage Δ
testsuite/test-modprobe.c 95.65% <0.00%> (+3.98%) ⬆️
testsuite/test-loaded.c 84.37% <33.33%> (+7.23%) ⬆️
testsuite/test-dependencies.c 86.20% <25.00%> (+5.56%) ⬆️
testsuite/test-weakdep.c 87.09% <25.00%> (+12.09%) ⬆️
testsuite/testsuite.c 56.52% <0.00%> (+1.23%) ⬆️
testsuite/test-new-module.c 85.29% <33.33%> (+2.43%) ⬆️
testsuite/test-remove.c 68.42% <16.66%> (+20.27%) ⬆️
testsuite/test-blacklist.c 75.86% <14.28%> (+22.20%) ⬆️
testsuite/test-multi-softdep.c 63.82% <14.28%> (-2.84%) ⬇️
testsuite/test-initstate.c 73.52% <20.00%> (+19.18%) ⬆️
... and 7 more

... and 13 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@evelikov evelikov mentioned this pull request Jun 26, 2025
@lucasdemarchi

Copy link
Copy Markdown
Contributor
The current assert_return() macro has a few short comings:
- the name implies assert()
  • all users unnecessarily provide EXIT_FAILURE
  • the users cannot provide a message - "Failed assertion:" isn't great

I think not needing to provide the same EXIT_FAILURE and being able to provide a message is good. The rename I think is not really needed. What's the problem with the term assert? IMO it's clearer than "ok". igt for example uses a bunch of igt_assert_*. ok() IMO is odd, but it maybe because I've never seen it used like this. Rust also has both Ok() and assert!(), which I think will add to the confusion.

how to handle the conversion - one test at a time, bulk (all assert_return -> OK, all manual -> OK), other?

I think a bulk one. If the message is optional, then the first patch is automated. Additional patches may follow later (maybe per compile unit) to add some good messages.

@evelikov

evelikov commented Jul 5, 2025

Copy link
Copy Markdown
Collaborator Author

What's the problem with the term assert?

There is nothing wrong with the term in itself. The snafu happens due to assert(3) and the associated NDEBUG, et al... As previously #239 (review)

Other naming options:

  • add a prefix - ts_ (short for testsuite, after TS_EXPORT), other
  • other names - check(), test()
  • combination - eg. like igt ts_assert_ok or otherwise

Don't have a strong preference though.

I think a bulk one. If the message is optional, then the first patch is automated.

Awesome, will do.

Additional patches may follow later (maybe per compile unit) to add some good messages.

Do the example commits reach that bar, or shall I just drop them?

@evelikov

Copy link
Copy Markdown
Collaborator Author

@lucasdemarchi humble poke

@lucasdemarchi

Copy link
Copy Markdown
Contributor

ts_expect() or ts_assert() would be fine IMO. Other places that use the "expect" terminology: GoogleTest and kunit... Both of them use semantics like

  • KUNIT_EXPECT: Marks the test as failed but continues executing the rest of the test case
  • KUNIT_ASSERT: Marks the test as failed and immediately terminates the test case.

So for our uses I think it matches more the "assert" semantics

@evelikov evelikov marked this pull request as ready for review July 3, 2026 12:22
@evelikov evelikov requested a review from lucasdemarchi July 3, 2026 12:22
@evelikov

evelikov commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

Changes:

  • open-code assert_return(... -EINVAL) instance
  • OK -> TS_ASSERT
    • based on KUNIT_ASSERT
    • slightly different formatting/reporting - "inspired" by KUNIT_ASSERT
    • no user-provided messages - can add TS_ASSERT_MSG if needed
  • different split - introduce macro, sed the whole tree
  • convert all tests to TS_ASSERT
  • s/return EXIT_SUCCESS/return 0/ - obligatory OCD bikeshed 😅

@evelikov

evelikov commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

Will check the CI failures later today/tomorrow - the PR should be good for review though.

@evelikov evelikov changed the title RFC: rework assert_return() macro Rework the assert_return() macro Jul 3, 2026
@evelikov

evelikov commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

From the failures:

@lucasdemarchi lucasdemarchi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nit on commit message. LGTM.

Comment thread testsuite/testsuite.h
__LINE__, __PRETTY_FUNCTION__); \
return (r); \
ERR("ASSERTION FAILED at %s:%d\n", __FILE__, __LINE__); \
ERR("\t" #expr "\n"); \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From commit message:

  • embeds the expression into the error string, growing the file size

which is still true here, right? Also, for the testsuite the file size due to this would not be of great concern to point out.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It meant that the expression string is separate from the boilerplate "foobar failed" error message, allowing the latter to be de-duplicated. Nevertheless, dropped since it's a very meh point.

evelikov added 5 commits July 6, 2026 22:29
With follow-up commit we'll rework/remove the macro, where the new
version does not allow for providing the return value.

Swap it for a manual ERR, printing a more useful error message.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
The existing macro has a few shortcomings:
 - lowercase, non-prefixed so it can be confused with assert(3), which
   has varying behaviour depending on -DNDEBUG
 - the return value is constant across the project

Introduce TS_ASSERT which is modelled after KUNIT_ASSERT, addressing the
above and tweaking the output format.

Existing instances of assert_return will be updated with a follow-up
commit.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Mechanical change, done with:

sed -i  "s/assert_return/TS_ASSERT/;s/, EXIT_FAILURE//" testsuite/*.c

In addition, manually fixup the two multiline instances (test-strbuf.c
and test-util.c) and remove the no longer used macro.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Currently we have some inconsistencies across the tests:
 - using TS_ASSERT vs not
 - (non TS_ASSERT tests) leaking on error vs not

In practical terms, we are not too worried about the leaks, since the
test failure comes first. As such, convert all the tests to TS_ASSERT().

This means we loose the useful error messages in a few instances, which
could be re-introduced at a later point alongside a TS_ASSERT_MSG()
macro.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
With all the EXIT_FAILURE instances done from the tests (only ones in
tools and the test runner remain), lets drop the EXIT_SUCCESS ones.

Functionally identical and arguably slightly neater.

Signed-off-by: Emil Velikov <emil.l.velikov@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants