Most lit/help tests fail with new python-filecheck

[rathann]

In binaryen lit tests, filecheck is used to test the --help output of various wasm* tools.

With filecheck-0.0.24, the following test passes:

$ wasm-shell --help | filecheck help/wasm-shell.test
$ 

However, with filecheck-1.0.3 from the new upstream repo, it fails:

$ wasm-shell --help | filecheck help/wasm-shell.test
help/wasm-shell.test:4:9 found empty check string with prefix 'NEXT:'
;; CHECK-NEXT:
        ^
$

New upstream says the new behaviour is closer to the original filecheck and recommends using CHECK-EMPTY instead of CHECK-NEXT.

[tlively]

I'd be happy to update the tests (and auto-update scripts) to use CHECK-EMPTY in this case as long as our current pinned version of filecheck supports it as well. (Or alternatively if someone wants to do the work to migrate us entirely to the latest filecheck.)

Out of curiosity, what's the reason for wanting the tests to pass with the latest filecheck release?

[rathann]

[...]

Out of curiosity, what's the reason for wanting the tests to pass with the latest filecheck release?

The reason is that I've updated python-filecheck in Fedora to the latest and binaryen tests started failing. I've disabled the failing tests for now, but it doesn't feel right. By the way, the old python-filecheck is no longer developed and the original owner transferred the pypi name to the new upstream as well.

[tlively]

I've updated us to use the latest filecheck as of #9054. I've also made two PRs in upstream filecheck to fix things that we had to hack around: AntonLydike/filecheck#50 and AntonLydike/filecheck#51.