Skip to content

Add test/[ alphabet matrix (mix bash_fixtures.gen test) - #102

Merged
davydog187 merged 2 commits into
mainfrom
cursor/test-alphabet-matrix-c59e
Aug 23, 2026
Merged

Add test/[ alphabet matrix (mix bash_fixtures.gen test)#102
davydog187 merged 2 commits into
mainfrom
cursor/test-alphabet-matrix-c59e

Conversation

@davydog187

@davydog187 davydog187 commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Implements #70 item 2 for test / [ only: a generated test_matrix suite in the same generate / record / test path as date_matrix and printf_matrix.

This is not the item-3 filesystem-shape × command cube (~480 cells). Operators are crossed with revealing operand shapes for that operator so padding/boundaries/false-vs-missing stay visible — the date lesson — without enumerating every FS command.

What was enumerated

mix bash_fixtures.gen test writes 246 cases:

  • Full POSIX/[ operator list on test: -b -c -d -e -f -g -h -k -L -n -p -r -S -s -t -u -w -x -z, = == != < >, -eq -ne -lt -le -gt -ge, -nt -ot -ef, -a -o, !, plus bash extras -G -O -N
  • File operators × file / empty / dir / missing / symlink / dangling, plus chmod'd, fifo, /dev/null, trailing-slash and ENOTDIR boundaries
  • -z/-n and one-arg forms at empty / hi / 0 / space (and operator-as-operand)
  • String and integer compares at two or more pairs per family so one value cannot hide the bug (hi/hi vs a/b, 0/1 vs 10/9)
  • Both test and [ on a representative subset (-e/-f/-d/-L/-s, -z/-n, =/!=, -eq/-lt, !, one-arg) so the two spellings cannot drift
  • Syntax cells: no args, extra args, unknown operators, missing ], grouping

mix bash_fixtures.gen test --dry-run reports test_matrix: 246 cases. mix bash_fixtures.gen now generates date, printf, and test.

Recording

Docker was not available on the agent VM. Oracle output was recorded with the same test/fixtures/runner.sh the Docker task runs, under GNU bash 5.2.21 / Ubuntu 24.04 (uid ubuntu, not root — so chmod 000 permission cells are meaningful), then pretty-printed through Mix.Tasks.BashFixtures.write_json!/2:

mix bash_fixtures.gen test
bash test/fixtures/runner.sh < test/fixtures/bash_cases/test_matrix.json > /tmp/test_matrix_raw.json
# then Jason-pretty-print + Fixtures.validate, as `mix bash_fixtures` does

To re-record through Docker when it is available:

mix bash_fixtures test_matrix

Known gaps (44 of 246)

No cell was omitted. Divergences are opts.known_gap (excluded from the digest). The fixture runner still executes them with the assertion inverted.

Count Reason
6 -r/-w/-x are existence checks; bash tests the permission bit (-x on a 644 file, -r/-w on chmod 000)
6 Leading-space integer operands ( 7) — Integer.parse/1 rejects them
6 Non-numeric -eq family: JustBash exits 2 with no diagnostic; bash writes stderr
4 Cannot construct fifo / setuid / sticky / dated-mtime shapes, and/or the operator is unimplemented
4 -nt/-ot/-ef unimplemented (revealing true side: newer-than, older-than, same-file, hardlink)
3 [ without a closing ] still evaluates; bash exits 2
3 ( ) grouping unimplemented
2 /dev/null typed as a regular file; bash reports a character device (-f, -c)
2 Extra / too-many arguments: exit 1 vs bash exit 2 + diagnostic
2 Unknown operator (-Z, -xx): exit 1 vs bash exit 2 + diagnostic
2 -O/-G unimplemented (true on a file we own)
1 Directory size 0 so test -s dir is false
1 Trailing slash on a regular file treated as the file; bash requires a directory
1 Unary -a is not a synonym of -e
1 4-arg -f file -a -d dir is not expression AND/OR

JustBash.exec/2 did not raise on any cell. No test/[ implementation bugs were fixed in this PR — it is the enumeration + harness.

Cheap follow-ups the matrix now makes un-skippable

  • -x/-r/-w should consult mode bits, not existence
  • -c / character-device type for /dev/null (and stop reporting it as -f)
  • -nt/-ot/-ef
  • [ missing-] and unknown-operator diagnostics (exit 2)
  • Integer operands: leading spaces, and a stderr diagnostic on non-numeric
  • Unary -a as -e; 4-arg -a/-o as expression AND/OR
  • Directory size so test -s dir is true

Not in this PR: the item-3 FS-shape × command cube, Oils activation, CI topology changes.

Tests

All local quality gates passed:

  • mix compile --warnings-as-errors
  • mix format --check-formatted
  • mix credo --strict (only the existing intentional banned_fixture_apply finding)
  • mix dialyzer (zero new errors)
  • mix test — 6346 tests, 0 failures
  • mix test --only suite:test_matrix — 246 tests, 0 failures
  • mix bash_fixtures.gen test --dry-runtest_matrix: 246 cases

Related Issues

Related to #70 (item 2, test/[ only; not closing the issue)

Type of Change

  • New feature (non-breaking change which adds functionality)

Testing

  • Added new tests
  • All existing tests pass
  • Tested manually (mix bash_fixtures.gen test --dry-run, local runner.sh recording, JustBash vs oracle classification)

Checklist

  • My code follows the style guidelines of this project
  • I have run mix format
  • I have run mix credo and addressed any issues
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing tests pass locally with my changes
  • I have updated the documentation accordingly
  • I have updated the CHANGELOG.md (for non-trivial changes)
Open in Web Open in Cursor 

cursoragent and others added 2 commits August 23, 2026 14:50
Enumerate every POSIX test/[ operator across revealing operand
shapes, the same way date_matrix and printf_matrix already work.
Both test and [ are asked on a representative subset so they cannot
drift. Gaps are marked with reasons rather than omitted; recordings
were taken from GNU bash 5.2 via runner.sh.

Co-authored-by: Dave Lucia <davelucianyc@gmail.com>
Split the post-recording gap classifier into file-shape, operator,
and syntax helpers so cyclomatic complexity stays under the limit.

Co-authored-by: Dave Lucia <davelucianyc@gmail.com>
@davydog187
davydog187 marked this pull request as ready for review August 23, 2026 14:55
@davydog187
davydog187 merged commit c237485 into main Aug 23, 2026
4 checks passed
@davydog187
davydog187 deleted the cursor/test-alphabet-matrix-c59e branch August 23, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants