Reduce the flakiness of TestPeach with some random jitter.

There was a recent failure of the test that checks peach is
nondeterministic: https://github.com/elves/elvish/runs/2926941131

The failure can be reproduced reliably when setting GOMAXPROCS=1.
However, GitHub Action's Windows runner has two CPU cores, which means
GOMAXPROCS should be 2. Presumably, during that particular one, one of
the cores is occupied by another process, so Go had to run the
goroutines in order on one CPU core.

Go's time.Sleep yields the current goroutine and allow other scheduled
goroutines to run on the same thread. As a result, adding a random
jitter guarantees nondeterminism in execution order, even if only one
CPU core is available to execute threads.

Also run the nondeterminism check in an infinite loop.
This commit is contained in:
Qi Xiao 2021-06-28 00:38:43 +01:00
parent 8c97ed2184
commit 9981b2a777

View File

@ -36,16 +36,20 @@ func TestPeach(t *testing.T) {
Test(t,
// Verify the output has the expected values when sorted.
That(`range 5 | peach [x]{ * 2 $x } | order`).Puts(0, 2, 4, 6, 8),
// Verify that successive runs produce the output in different order. This test can
// theoretically suffer false positives but the vast majority of the time this will produce
// the expected output in the first iteration. The probability it will produce the same
// order of output in 100 iterations is effectively zero.
// Test that the order of output does not necessarily match the order of
// input.
//
// Most of the time this effect can be observed without the need of any
// jitter, but if the system only has one CPU core to execute goroutines
// (which can happen even when GOMAXPROCS > 1), the scheduling of
// goroutines can become deterministic. The random jitter fixes that by
// forcing goroutines to yield the thread and allow other goroutines to
// execute.
That(`
fn f { range 100 | peach $put~ | put [(all)] }
var x = (f)
for _ [(range 100)] {
var y = (f)
if (not-eq $x $y) {
var @in = (range 100)
while $true {
var @out = (all $in | peach [x]{ sleep (* (rand) 0.01); put $x })
if (not-eq $in $out) {
put $true
break
}