Skip to content

Tests | Attempt to fix enable file stream failures + fix flaky tests#4220

Open
cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/failing-tests
Open

Tests | Attempt to fix enable file stream failures + fix flaky tests#4220
cheenamalhotra wants to merge 3 commits into
mainfrom
dev/cheena/failing-tests

Conversation

@cheenamalhotra
Copy link
Copy Markdown
Member

@cheenamalhotra cheenamalhotra commented Apr 22, 2026

Problem

The CI pipeline's Windows SQL Server configuration step intermittently fails in two places:

  1. "Create SQL user" step — After enabling TCP/NP protocols, SQL Server may not accept connections immediately, causing user creation to fail after 5 retries and silently proceeding (the old code used break instead of throw).
  2. "Enable FileStream" step — After the WMI EnableFilestream(3, ...) call, the sp_configure filestream_access_level T-SQL command fails because SQL Server needs a restart for the WMI change to take effect.

Example failure: https://sqlclientdrivers.visualstudio.com/public/_build/results?buildId=148483&view=logs&j=0d02c018-afa8-5f77-e1de-32c6d598a678&t=5b158d7f-5532-5d37-977c-ecb257065e86&s=da54b7cd-c327-58c0-3741-799c147adcfe

Fix

Pipeline (configure-sql-server-win-step.yml):

  • "Create SQL user" step: If connections fail after 5 attempts, restart SQL Server once (protocol changes may require it), then retry up to 10 more attempts. Changed breakthrow so failure is no longer silently ignored. Added -ConnectionTimeout 5 to avoid hanging.
  • "Enable FileStream" step: Simplified to only do the WMI EnableFilestream call. Moved the sp_configure filestream_access_level T-SQL to a new step that runs after the existing "Restart SQL Server [Win]" step, eliminating the need for a double restart.
  • Added numbered step comments throughout the template for maintainability.

Flaky test annotations:

Marked 3 intermittently failing tests with [Trait("Category", "flaky")]:

  • SqlVariantParameterTests.SqlType_BulkCopyFromDataRow_RoundTripsCorrectly
  • MetricsTest.TransactedConnectionPool_VerifyActiveConnectionCounters
  • ConnectionFailoverTests.TransientFault_IgnoreServerProvidedFailoverPartner_ShouldConnectToUserProvidedPartner

Checklist

  • Pipeline changes tested against known failure scenarios
  • No breaking changes to test execution (flaky trait is additive)
  • No source code changes to the driver itself
  • Step ordering preserved — FileStream WMI still runs before the restart, sp_configure runs after

Co-authored-by: Copilot <copilot@github.com>
@cheenamalhotra cheenamalhotra added this to the 7.1.0-preview1 milestone Apr 22, 2026
@cheenamalhotra cheenamalhotra added the Area\Tests Issues that are targeted to tests or test projects label Apr 22, 2026
Copilot AI review requested due to automatic review settings April 22, 2026 03:14
@cheenamalhotra cheenamalhotra requested a review from a team as a code owner April 22, 2026 03:14
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Apr 22, 2026
@cheenamalhotra cheenamalhotra moved this from To triage to In review in SqlClient Board Apr 22, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to reduce intermittent CI failures by making Windows SQL Server FileStream enablement more reliable during pipeline setup, and by quarantining a small set of intermittently failing tests.

Changes:

  • Restart SQL Server after the WMI EnableFilestream(3, ...) call and increase the retry budget for the follow-up sp_configure step.
  • Mark 3 intermittently failing tests as [Trait("Category", "flaky")] so they are quarantined from default test runs.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
eng/pipelines/common/templates/steps/configure-sql-server-win-step.yml Adds SQL Server restart + expands retry loop to reduce FileStream setup flakiness on Windows agents.
src/Microsoft.Data.SqlClient/tests/UnitTests/SimulatedServerTests/ConnectionFailoverTests.cs Quarantines a simulated failover unit test as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/TracingTests/MetricsTest.cs Quarantines a metrics/manual test as flaky.
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/SqlVariantParameterTests.cs Quarantines a sql_variant bulk copy manual test as flaky.

Comment thread eng/pipelines/common/templates/steps/configure-sql-server-win-step.yml Outdated
Comment thread eng/pipelines/common/templates/steps/configure-sql-server-win-step.yml Outdated
@github-project-automation github-project-automation Bot moved this from In review to In progress in SqlClient Board Apr 22, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.41%. Comparing base (eacf112) to head (5f8e721).
⚠️ Report is 40 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4220      +/-   ##
==========================================
- Coverage   65.94%   64.41%   -1.53%     
==========================================
  Files         277      270       -7     
  Lines       42949    65777   +22828     
==========================================
+ Hits        28321    42369   +14048     
- Misses      14628    23408    +8780     
Flag Coverage Δ
CI-SqlClient ?
PR-SqlClient-Project 64.41% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cheenamalhotra cheenamalhotra added the Author attention needed PRs that require author to respond or make updates to PR. label May 8, 2026
Copilot AI review requested due to automatic review settings May 11, 2026 20:06
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comment on lines +187 to +193
#Enable FileStream via WMI.
# The sp_configure call is deferred to after the "Restart SQL Server [Win]"
# step so we don't need a separate restart here.
$instance = "${{parameters.instanceName }}"
$wmi = Get-WmiObject -Namespace "${{parameters.SQLRootPath }}" -Class FilestreamSettings | where {$_.InstanceName -eq $instance}
$wmi.EnableFilestream(3, $instance)

$machineName = $env:COMPUTERNAME

if ("${{parameters.instanceName }}" -ne "MSSQLSERVER"){
$machineName += "\${{parameters.instanceName }}"
}

#Change the access level for FileStream for SQLServer
Set-ExecutionPolicy Unrestricted
Import-Module "sqlps"

# Retry loop: SQL Server may be temporarily unavailable after enabling FileStream via WMI.
# Worst-case budget: 10 attempts x (5s connection timeout + 5s sleep) = ~100s
$tries = 0
while ($true) {
$tries++
try {
Invoke-Sqlcmd -ServerInstance "$machineName" -ConnectionTimeout 5 @"
EXEC sp_configure filestream_access_level, 2;
RECONFIGURE;
"@
Write-Host "FileStream access level configured successfully."
break
} catch {
if ($tries -ge 10) {
Write-Host "##[error]Failed to configure FileStream access level after $tries tries."
throw
}
Write-Host "Failed to connect to SQL Server (attempt $tries/10). Retrying in 5 seconds..."
Start-Sleep -Seconds 5
}
}
Write-Host "FileStream enabled via WMI for instance '$instance'."
@cheenamalhotra cheenamalhotra removed the Author attention needed PRs that require author to respond or make updates to PR. label May 11, 2026
@cheenamalhotra cheenamalhotra moved this from In progress to In review in SqlClient Board May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area\Tests Issues that are targeted to tests or test projects

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

6 participants