flaky: a BlazorAdapter test fails under parallel load and passes in isolation #83

New issue

Open

opened 2026-07-27 19:48:07 +02:00 by scarlet · 0 comments

scarlet commented

2026-07-27 19:48:07 +02:00

Member

What's known

While reviewing PR #82, Jibril reported:

one BlazorAdapter test flaked under parallel load, passed in isolation — pre-existing pumpAndSettle timing, not this PR

That is the entire evidence. The test was not named, and the failure output was not captured.

What I could not establish

I tried to reproduce before writing this, and failed:

5× dotnet test tests/Orihon.BlazorAdapter.Tests in isolation — 193/193 green each time
3× full dotnet test (all four projects, which run in parallel) with all 8 cores saturated by busy loops — green each time

So the first job on this issue is catching it, not fixing it. Anyone picking this up should assume the test name is unknown.

How to catch it

The suite prints the failing test name to stdout, but a passing run tells you nothing, so it needs a loop that preserves the failure:

for i in $(seq 1 50); do
  dotnet test --nologo --logger "trx;LogFileName=run-$i.trx" \
    || echo "FAILED on run $i" >> flakes.log
done

Run it with the machine loaded (the reported condition), and keep the .trx — it carries the assertion message and the render counts, which is what distinguishes a genuine timing bug from a too-short bUnit timeout.

Where to look

bUnit WaitForAssertion / WaitForState sites. The reported symptom — passes alone, fails under load — is the signature of a wait whose default timeout (1s) is too short when the renderer is competing for CPU, rather than of a wrong assertion. SettingsPageTests, RunMonitorTests and PageWorkspacePageTests are the heaviest users.
Anything asserting on a debounced field. Debounce plus a real timer is the other classic source; see the verify-editors-with-real-input lesson.
Related history: PR #71 fixed a UseCases hang under load and established the 60s hard test timeout. Worth reading before touching timing here — the conclusion there was to baseline main before blaming a branch, which applies to this too.

Why it matters

A flake that nobody can name is worse than a red test: every future PR review has to decide whether an unexplained BlazorAdapter failure is "the known flake" or a real regression, and the honest answer is currently "nobody can tell". That judgement call is the actual cost here, not the seconds of CI time.

Not urgent

main is green, and this does not block anything. It wants doing before someone starts trusting a red BlazorAdapter run as noise.

🤖 Generated with Claude Code

## What's known While reviewing PR #82, Jibril reported: > one BlazorAdapter test flaked under parallel load, passed in isolation — pre-existing pumpAndSettle timing, not this PR That is the **entire** evidence. The test was not named, and the failure output was not captured. ## What I could not establish I tried to reproduce before writing this, and failed: - 5× `dotnet test tests/Orihon.BlazorAdapter.Tests` in isolation — 193/193 green each time - 3× full `dotnet test` (all four projects, which run in parallel) with all 8 cores saturated by busy loops — green each time So **the first job on this issue is catching it**, not fixing it. Anyone picking this up should assume the test name is unknown. ## How to catch it The suite prints the failing test name to stdout, but a passing run tells you nothing, so it needs a loop that preserves the failure: ```bash for i in $(seq 1 50); do dotnet test --nologo --logger "trx;LogFileName=run-$i.trx" \ || echo "FAILED on run $i" >> flakes.log done ``` Run it with the machine loaded (the reported condition), and keep the `.trx` — it carries the assertion message and the render counts, which is what distinguishes a genuine timing bug from a too-short bUnit timeout. ## Where to look - **bUnit `WaitForAssertion` / `WaitForState` sites.** The reported symptom — passes alone, fails under load — is the signature of a wait whose default timeout (1s) is too short when the renderer is competing for CPU, rather than of a wrong assertion. `SettingsPageTests`, `RunMonitorTests` and `PageWorkspacePageTests` are the heaviest users. - **Anything asserting on a debounced field.** `Debounce` plus a real timer is the other classic source; see the `verify-editors-with-real-input` lesson. - Related history: PR #71 fixed a UseCases hang under load and established the **60s hard test timeout**. Worth reading before touching timing here — the conclusion there was to baseline `main` before blaming a branch, which applies to this too. ## Why it matters A flake that nobody can name is worse than a red test: every future PR review has to decide whether an unexplained BlazorAdapter failure is "the known flake" or a real regression, and the honest answer is currently "nobody can tell". That judgement call is the actual cost here, not the seconds of CI time. ## Not urgent `main` is green, and this does not block anything. It wants doing before someone starts trusting a red BlazorAdapter run as noise. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

TeamAI/Orihon#83

No description provided.

Rows
Columns