Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# 0.7.1

* Stream stdout/stderr live while capturing JUnit output without losing shared
test state. Write provisional active-function reports for interrupted runs,
drain capture workers before final results, and fail on capture/report errors.

* Report source locations for failed expectations, including user helpers and
subshells. Print golden-file diffs on stderr by default and capture them in JUnit
failure reports, without duplicating diffs in verbose mode.
Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ for filter precedence, shard assignment, and teardown behavior.

Failed expectations include their source file and line. Golden-file failures
print unified diffs on stderr and include them in JUnit reports without requiring
`--verbose`; see [failure diagnostics](bashtest/README.md#failure-diagnostics).
`--verbose`. Output streams live while reports capture it, and an interrupted run
leaves a provisional report identifying the last active function; see [failure diagnostics](bashtest/README.md#failure-diagnostics).

### Example

Expand Down Expand Up @@ -109,7 +110,8 @@ hand-rolled pipelines.
## Installation and requirements

Linux and macOS require Bash and Unix utilities (`coreutils`, `diff`, `sed`, and
`grep`); PCRE assertions need a supported PCRE tool.
`grep`); JUnit reporting also uses `head`, `tail`, `tee`, and `wc` (plus the system
`ps` on macOS). PCRE assertions need a supported PCRE tool.

On Windows x86-64, `bashtest` automatically downloads a pinned MSYS2 runtime,
including Bash, coreutils, diffutils, grep with PCRE support, sed, and their DLLs.
Expand Down
38 changes: 30 additions & 8 deletions bashtest/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ A failed expectation remains a failure even if its return status is handled or
followed by a successful command. This includes expectations in command
substitutions, `(subshells)`, and background jobs that the test waits for.
`test_has_error` sees these failures in the parent shell. Tests must wait for
background work before returning. Each test and lifecycle hook starts with a
background assertion work before returning; long-lived services may instead
be stopped in teardown. Each test and lifecycle hook starts with a
fresh error state; failures do not leak into subsequent cases.

By default, ordinary commands follow Bash's final-return-status semantics.
Expand Down Expand Up @@ -207,10 +208,19 @@ keep the location independent of Bazel's working directory. An empty
`GTEST_OUTPUT` or `--gtest_output=` disables reporting. JSON output is unsupported;
an unsupported format or failure to write a report makes the test fail.

With reporting enabled, function output is buffered and replayed after each
function, while functions continue to run in the same shell. Setup and teardown
state remains visible to the tests. Failed `test_init` and `test_done` hooks appear
as additional error cases; successful hooks do not count as test cases. Filters
With reporting enabled, function output streams to its original stdout/stderr
while a copy is captured for the report. Standard `head`, `tail`, `tee`, and `wc` tools
must be available on `PATH`; they are included in the managed Windows runtime.
macOS also uses its system `ps` to detect an exited, unreaped coordinator.
Functions still run in the same shell, and setup/teardown state remains visible
to the tests. Capture workers do not
join a test's `wait` or `wait -n`. Background services may keep writing across
setup, cases, and teardown; captured output is attributed to the function active
when it was written. Output between functions is still streamed. Tests should
wait for background assertions before returning and stop services in teardown.
A capture failure makes the test fail as a reporting error rather than silently
losing output. Failed `test_init` and `test_done` hooks appear as additional error
cases; successful hooks do not count as test cases. Filters
produce skipped cases, and `--no-skip` turns runtime skip requests into failures.
An assertion failure or nonzero function return always takes precedence over a
skip request, even if the skip happens after the failure. Invalid extended
Expand All @@ -219,9 +229,21 @@ syntax-error diagnostic.
The existing exit code 2 for no passing or failing cases is retained and recorded
as a `test_runner` error. An early exit or a catchable signal during a function
records that function as an error and retains a nonzero exit status; `exit 0`
inside a function is also an error because it aborts the runner. SIGKILL and
machine termination cannot be handled; Bazel supplies its fallback reporting when
no custom report exists. Reporting begins when `test_runner` is called.
inside a function is also an error because it aborts the runner.

Before each setup, test, or teardown function, the runner atomically writes a
provisional report containing completed cases and an error for the function
about to run. Its message explicitly labels the report provisional. A completed
run replaces that checkpoint with final results. Termination without cleanup,
such as SIGKILL, leaves the last written checkpoint identifying the function
whose completion was not yet recorded. Readers detect coordinator death, drain
a finite snapshot, and exit even without a stop request. Output already streamed
remains in Bazel's test log; use
`--test_output=streamed` to see output while Bazel runs. Catchable signals finalize
the report with available output. A failed progress-report write prevents a new
setup/test body from running and fails the target. Teardown still runs so a
reporting failure cannot prevent cleanup. Reporting begins at `test_runner`;
if termination happens before any report exists, Bazel supplies its fallback.

Nested runner/verifier scripts should select their own output file, or disable
reporting with `GTEST_OUTPUT=`, so an expected failure does not become the parent
Expand Down
Loading
Loading