The suites acceptance test is flaky on Bash 3.0 under load

[Chemaclass]

Problem

test_repeated_suites_run_the_union_of_their_paths in
tests/acceptance/bashunit_suites_test.sh has failed twice today on unrelated
PRs, both times only on Bash 3.0 jobs:

• fix(coverage): write hit records straight to disk instead of buffering them #1103, "Bash 3.0 - Sequential" and "Bash 3.0 - Simple":
  ✗ Error: Source — Failed to source 'tests/acceptance/b_test.sh' (exit 1)
• fix(provider): a value ending in a backslash no longer vanishes #1135, "Bash 3.0 - Strict": same test, Expected 'Running tests/unit/a_test.sh…'

Both re-ran green, so each occurrence costs a cycle and a human deciding the
red tick is not about the change.

What I could and could not establish

• Not reproducible in isolation: 5/5 clean running just that file under a
  real Bash 3.0 build (--parallel --simple --strict).
• Only ever seen inside the full CI suite, which suggests contention rather
  than anything in the fixture: the test writes a small project into a per-test
  temp dir and runs nested bashunit invocations against it.
• The union case is the one that fails, and it is the one that merges two
  suites whose options conflict (parallel = true from one, no-parallel = true from the other), so which mode the nested run ends up in depends on
  argv order.

That last point is the thread worth pulling: if the nested run can end up
parallel, two workers sourcing the same fixture concurrently on Bash 3.0 would
explain an intermittent "failed to source" that never appears when the file is
run on its own.

[Chemaclass]

Pulled the thread from the last comment, and the mechanism checks out.

When several suites are named, a conflicting option is resolved by argv
order — the one named later wins:

--suite unit --suite acc  ->  BASHUNIT_PARALLEL_RUN: false
--suite acc --suite unit  ->  BASHUNIT_PARALLEL_RUN: true

(unit sets parallel = true, acc sets no-parallel = true.)

The fixture in bashunit_suites_test.sh defines exactly that conflict, and the
failing case is the one that names both suites. So the nested run's mode is
decided by the order the test passes them, and whichever mode that is, it is
the same on every platform — which weakens my "two workers sourcing
concurrently" guess, since the failure only ever appears on Bash 3.0.

Still worth knowing, and it was undocumented; I am documenting the rule
separately. For this issue the open question is narrower than I first framed
it: what is different about sourcing that fixture on Bash 3.0 specifically,
under CI load, that never reproduces standalone (5/5 clean on a real 3.0
build).

[Chemaclass]

Second reproduction attempt, also negative — recording it so nobody repeats it.

• 5 sequential runs of the file alone under a real Bash 3.0 build
  (--parallel --simple --strict): 10/10 passed each time.
• 6 concurrent copies of the same run in one container: 10/10 passed each
  time.

So neither the file on its own nor contention between copies of it triggers the
failure. What is left that CI has and this does not: the other ~2200 tests
running alongside it, many of them acceptance tests spawning their own nested
bashunit processes, on a slower and more constrained runner.

That points at something environmental — process or file-descriptor pressure,
or mktemp collisions under a filled temp dir — rather than anything in the
fixture or the suite merge. Next step for whoever picks this up would be to run
the full acceptance directory under Bash 3.0 in a container with constrained
CPU, rather than the single file.

[Chemaclass]

Reproduced locally, and the cause is not what this issue assumed

Recipe (4th attempt; the three earlier ones failed because they ran the file in
isolation):

# three full suites at once in the real Bash 3.0 image, twice over
for flags in "" "--parallel" "--simple --parallel --strict"; do
  docker run --rm -v "$(pwd)":/bashunit -w /bashunit bashunit-bash3 \
    /opt/bash-3.0/bin/bash ./bashunit $flags tests/ >"$out/$tag.out" 2>&1 &
done; wait

1 of 6 runs failed, in the --parallel one, on exactly the reported test.

What the captured output shows

The diagnostic added in #1144 does its job:

✗ Error: Source
    Failed to source 'tests/acceptance/b_test.sh' (exit 1, 52 bytes, no stderr)

52 bytes — the fixture is fully written, not truncated. So the
"two workers sourcing a half-written file" hypothesis in the issue body is
wrong, and so is anything about set_up racing: each test gets its own
mktemp -d directory.

The real fault is two lines above it in the same capture:

src/runner/discovery.sh: line 71: /tmp/bashunit/run/Linux/Kpg56NvL/source_err: No such file or directory
src/runner/result.sh:    line 114: /tmp/bashunit/run/Linux/Kpg56NvL/failures:    No such file or directory

Line 71 is source "$test_file" 2>"$source_err_file". The redirect fails, not
the source — the run-output directory is gone. That explains every part of
the symptom: exit 1 with no stderr (the capture file could not be created),
a perfectly valid 52-byte fixture, and a message that blames the test file.

The second message, from a different module, confirms the whole run directory
has vanished rather than one file.

Still open

What removes it. Ruled out so far:

• Token collision — random_str 8 is reseeded per process; 40 concurrent
  Bash 3.0 processes produced 40 distinct tokens.
• Inheritance from the outer run — env.sh assigns
  _BASHUNIT_RUN_OUTPUT_DIR unconditionally, so a nested run cannot adopt its
  parent's.
• Another run's cleanup — env::cleanup_run_output_dir only removes its
  own $target, and parallel::cleanup works in /bashunit/parallel/.
• The leak-probe test — it points its nested runs at a private TMPDIR.

Each docker run has its own /tmp, so the deletion happened within a single
suite run.

Worth fixing regardless

discovery.sh cannot tell "the test file failed to source" from "I could not
create the file I redirect stderr into". The second is reported as the first,
which is what sent three investigations at the fixture.

[Chemaclass]

Correction to my reproduction harness

The recipe I posted mounts one working copy into three concurrent
containers, so they share the repo-relative ./tmp that build.sh writes
into. Re-running it after a fix produced a different failure entirely:

✗ Error: Set up
    ./build.sh: line 78: tmp/temp.51733.sh: No such file or directory
    rm: cannot remove 'tmp/temp.54349.sh': No such file or directory
    at tests/acceptance/bashunit_upgrade_test.sh:43

That is my harness, not bashunit: real CI gives each job its own checkout. Any
run of that recipe needs a separate copy per container.

What this does not invalidate: _BASHUNIT_RUN_OUTPUT_DIR lives under
${TMPDIR:-/tmp}, which is private to each container, so the original
observation — the run scratch directory vanishing within a single run — still
stands on its own evidence.

I am re-verifying with one copy per container.

[Chemaclass]

Retracting the reproduction claim

I ran the control I should have run first: the same isolated harness (one repo
copy per container) with the candidate fix reverted.

harness	runs	Failed to source
shared working copy (flawed)	6	1
isolated copies, fix applied	6	0
isolated copies, fix reverted	6	0

The control is as clean as the treatment, so my single reproduction is best
explained by the shared-copy confound rather than by this flake. The fix is
not demonstrated to address #1137, and I am not closing it.

What survives, because it rests on the captured artifact rather than on the
harness:

• The fixture was 52 bytes and valid, so nothing was truncated.
• The run scratch directory was genuinely gone —
  discovery.sh:71 .../source_err: No such file or directory, and a second
  message from result.sh:114 for .../failures.
• source "$file" 2>"$dir/source_err" therefore failed on the redirect,
  which is why the exit was 1 with no stderr and the message blamed the file.

Whether the same mechanism is what hits Bash 3.0 CI is now unproven either way.

What I am proposing anyway

A run should survive a missing scratch directory instead of reporting a valid
test file as unsourceable. That is worth having on its own, with its own test,
independent of this issue — so it goes up as a robustness fix referencing
#1137, not as a fix for it.