Pytest

[fmeynadier]

This is a proposal to put all the existing tests under the "pytest" framework.

This way a simple hatch test or python -m pytest command from the root directory will run all the tests and show which are failing. I tried to adapt the actions in github to run the full test suite, but 1) it is difficult to test it other than doing this PR, so let's see... and 2) it can be tuned later. Maybe we don't want to test everything everytime (but pytest allows to select certain tests and leave other optional)

To make this work I added a conftest.py file that triggers automatically a regeneration of the TTL files prior to the tests. It also meant I had to introduce a non-interactive call to the TTL generation function.

Tests remain individually launchable, even outside the pytest framework if a "if name=="main":" test is present.

[fmeynadier]

@mgrub as you wrote the initial tests (and I know you added several in add_unit_tests, I can help merge that into the current one) I would be interested to know if this new arrangement suits you... I have the feeling that pytest is a convenient way to go, but I really don't want to add too much of an entry barrier to the writing of tests !

[mgrub]

@fmeynadier : Thank you very much for your work of migrating the tests to pytest! I gave this PR a in-depth review. The actual port to the pytest framework simply works 👍 Just two small observations:

• Do we want to add test outputs/output-directories in .gitignore?
• The coverage option in the github action could be changed as follows:

- python -m pytest --junitxml=junit/test-results.xml --cov=com --cov-report=xml --cov-report=html
+ python -m pytest --junitxml=junit/test-results.xml --cov=src/si_ref_point --cov-report=xml --cov-report=html

In addition, I made a few observations that are concerned with what the tests test, not how they are implemented using pytest:

• test_rb.py
  • resbod_shacl.ttl queries for si:definitions
  • cgpm.ttl does not seem to contain any si:definitions
  • hence, the test - although passing - seems not to be meaningful?
• test_units.py
  • test works
  • it mainly checks the definition of the units, so maybe "test_unit_definitions.py" (and "unit_definition_shacl.ttl") might be more suitable
• warnings that show up while running the tests:
  • SyntaxWarning: invalid escape sequence '-' in rb_utils.py (L.15)
  • DeprecationWarning: Dataset.default_context is deprecated, use Dataset.default_graph instead. --> seems to come from pyshacl and is already reported: pySHACL RDFLib 7.3+ Compatibility: Deprecation of default_context and Dataset.identifier RDFLib/pySHACL#316

I can take care of all these points, but I just wanted to see what you think about this before I implement the changes.

I could also include the additional tests that I proposed in #68.

[fmeynadier]

Great, thanks @mgrub !
Yes if you are in the mood / momentum to fix the issues you have spotted please feel free to, I stopped at a "working" state and felt your opinion was needed before refining things further, but that was also my plan !

I think TTL/ and JSONLD/ entries in the .gitignore already ignore tests/TTL and tests/JSONLD (at least it is the behavior I see on my system), but if you see additional things to add, please do.

Yes test_rb.py is only a stub for now, but I guessed it could host real tests later ?

One question I had in mind was how the tests should be splitted : is it more convenient to have 1 pytest loading a large number of shapes in one big shacl file, or split it over several tests loading only one (or few...) shape ? (I think it depends on how useful is the output of "validate" in case of failure... So far it seems OK but would it scale with a large number of tests ?).

[mgrub]

I will check, whether loading one "big" shacl-file still allows for the error-tracing needed in the tests. If this is not the case, I will change it to multiple "small" shacl-shapes in different files, to keep the tests specific.

I will try to add some commits to this PR in the next two days.