Skip to content

[Repo Assist] fix: improve test infrastructure resilience (flaky diagnostics + cleanup failures)#1520

Draft
github-actions[bot] wants to merge 2 commits intomainfrom
repo-assist/fix-flaky-diagnostics-test-20260405-13b2c35310660f7a
Draft

[Repo Assist] fix: improve test infrastructure resilience (flaky diagnostics + cleanup failures)#1520
github-actions[bot] wants to merge 2 commits intomainfrom
repo-assist/fix-flaky-diagnostics-test-20260405-13b2c35310660f7a

Conversation

@github-actions
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot commented Apr 5, 2026

🤖 This PR was created by Repo Assist, an automated AI assistant.

Summary

Fixes two pre-existing flaky test infrastructure issues that cause intermittent CI failures.


Fix 1: waitForLatestDiagnostics throws on empty diagnostics buffer

File: test/FsAutoComplete.Tests.Lsp/Utils/Server.fs

Problem: waitForLatestDiagnostics called Seq.last on the buffered diagnostic notifications. When the language server produces no diagnostic notifications (e.g., for error-free code), Observable.bufferSpan completes with an empty buffer and Seq.last throws:

The input sequence was empty. (Parameter 'source')
  at Utils.Server.DocumentModule.waitForLatestDiagnostics@301-9
```

This caused the `InlayHint.voption.show: for custom function` test to fail intermittently on ubuntu-latest/net9.0.

**Fix:** Replace `Seq.last` with `Seq.tryLast |> Option.defaultValue [||]` to return an empty diagnostics array when no notifications are received.

---

### Fix 2: `dotnetCleanup` throws when directory deletion fails

**File:** `test/FsAutoComplete.Tests.Lsp/Helpers.fs`

**Problem:** `dotnetCleanup` called `Directory.Delete(path, true)` directly. On macOS with .NET 10, this throws `"Directory not empty"` because background SDK processes (hot reload daemon, etc.) still hold file locks on some artifacts:

```
System.IO.FileSystem.RemoveDirectoryRecursive(String fullPath)
at Helpers.dotnetCleanup(String baseDir)

This caused 15 test failures in Go to definition tests.GoTo Tests on the macos-15 / net10.0 / TransparentCompiler CI matrix (pre-existing failures visible on main branch, run #2908).

Fix: Wrap the deletion in a try-catch, retry once after 200ms (for transient locks), and then silently continue if cleanup still fails. Stale build artifacts don't prevent tests from running correctly.


Test Status

Build:dotnet build test/FsAutoComplete.Tests.Lsp -f net8.0 — 0 errors, 0 new warnings.

Formatting:dotnet fantomas --check — both files pass.

These are test-infrastructure-only changes (no production code modified). The fixes remove sources of intermittent CI failure without changing any test assertions or test logic.

Closes #700 (partially — the dotnetCleanup fix addresses one source of test flakiness)

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@7ee2b60744abf71b985bead4599640f165edcd93

github-actions bot and others added 2 commits April 5, 2026 04:46
When the language server does not publish any diagnostics (e.g. for
error-free code), the Observable.bufferSpan window completes with an
empty IList. The previous Seq.last call threw 'The input sequence was
empty', causing intermittent test failures in tests like the InlayHint
voption 'show: for custom function' test.

Change Seq.last to Seq.tryLast |> Option.defaultValue [||] to return
an empty diagnostics array instead of throwing when no diagnostic
notifications are received.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On some platforms (notably macOS with .NET 10), Directory.Delete with
recursive=true can fail with 'Directory not empty' when background SDK
processes (e.g., hot reload daemon) still hold file locks. This caused
15 test failures in Go-to-definition tests on macOS+net10.0 CI matrix.

Change dotnetCleanup to retry once after 200ms and then silently ignore
cleanup failures. Stale build artifacts from a previous run are not
critical -- tests still produce correct results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The Linter Diagnostics test is very flaky, it needs to be looked at

0 participants