Sort host device_mapping emails deterministically

[mostlikelee]

Related issue: NA

The device_mapping GROUP_CONCAT in the host list (hosts.go) and label host list (labels.go) queries had no ORDER BY, so a host's emails came back in an unspecified order that varied between query executions. This produced inconsistent API output and flaky integration test failures (TestHostsReportDownload, TestListHostsByLabel). Emails are now sorted by email, source, matching the single-host ListHostDeviceMapping query.

Checklist for submitter

• Input data is properly validated, SELECT * is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters.

Testing

• Added/updated automated tests
• Where appropriate, automated tests simulate multiple hosts and test for host isolation (updates to one hosts's records do not affect another)
• QA'd all new/changed functionality manually

Summary by CodeRabbit

• Bug Fixes
  • Device mapping emails now appear in a consistent, sorted order in host and label host responses.
  • Host report CSV output now lists device mapping emails deterministically.
  • Updated validation coverage to reflect the stable ordering behavior.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.97%. Comparing base (80b883a) to head (5eb4ae3).
⚠️ Report is 10 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #48602      +/-   ##
==========================================
- Coverage   68.01%   67.97%   -0.04%     
==========================================
  Files        3678     3678              
  Lines      233760   233760              
  Branches    12267    12267              
==========================================
- Hits       158986   158908      -78     
- Misses      60475    60535      +60     
- Partials    14299    14317      +18     

Flag	Coverage Δ
backend	69.62% <100.00%> (-0.04%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[mostlikelee]

@coderabbitai full review

[coderabbitai]

✅ Action performed

Full review finished.

[coderabbitai]

Walkthrough

This PR makes device_mapping email ordering deterministic in host and label host listing responses. It adds ORDER BY email, source clauses inside the GROUP_CONCAT/JSON_OBJECT aggregations in ListHosts (hosts.go) and label host queries (labels.go). Integration tests are updated to insert device mappings out of order to verify sorted output in host listing, label host listing, and host report CSV downloads. A changelog entry documents the fix.

Possibly related PRs

• fleetdm/fleet#48488: Both PRs modify the device_mapping aggregation in ListHosts within server/datastore/mysql/hosts.go.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: deterministic sorting of host device_mapping emails.
Description check	✅ Passed	The description covers the issue, affected areas, and testing, and it matches the template well enough despite some optional sections being omitted.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-device-mapping-order

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ast-grep (0.44.0)

server/datastore/mysql/hosts.go

[{"text":"deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109813,"end":109967},"start":{"line":2997,"column":7},"end":{"line":2997,"column":161}},"file":"server/datastore/mysql/hosts.go","lines":"\tconst deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND) //nolint:gosec // G101 false positive, this is a SQL query","charCount":{"leading":7,"trailing":59},"language":"Go","metaVariables":{"single":{},"multi":{"secondary":[{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}}},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end"

... [truncated 1339 characters] ...

olumn":7},"end":{"line":2997,"column":161}},"style":"primary"},{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"}]}

server/datastore/mysql/labels.go

[{"text":"deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109813,"end":109967},"start":{"line":2997,"column":7},"end":{"line":2997,"column":161}},"file":"server/datastore/mysql/hosts.go","lines":"\tconst deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND) //nolint:gosec // G101 false positive, this is a SQL query","charCount":{"leading":7,"trailing":59},"language":"Go","metaVariables":{"single":{},"multi":{"secondary":[{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}}},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end"

... [truncated 1339 characters] ...

olumn":7},"end":{"line":2997,"column":161}},"style":"primary"},{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"}]}

server/service/integration_core_test.go

[{"text":"deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109813,"end":109967},"start":{"line":2997,"column":7},"end":{"line":2997,"column":161}},"file":"server/datastore/mysql/hosts.go","lines":"\tconst deviceAuthTokenQuery = SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND) //nolint:gosec // G101 false positive, this is a SQL query","charCount":{"leading":7,"trailing":59},"language":"Go","metaVariables":{"single":{},"multi":{"secondary":[{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}}},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end"

... [truncated 1339 characters] ...

olumn":7},"end":{"line":2997,"column":161}},"style":"primary"},{"text":"deviceAuthTokenQuery","range":{"byteOffset":{"start":109813,"end":109833},"start":{"line":2997,"column":7},"end":{"line":2997,"column":27}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"},{"text":"SELECT host_id FROM host_device_auth WHERE (token = ? OR previous_token = ?) AND updated_at >= DATE_SUB(NOW(), INTERVAL ? SECOND)","range":{"byteOffset":{"start":109836,"end":109967},"start":{"line":2997,"column":30},"end":{"line":2997,"column":161}},"style":"secondary"}]}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

server/datastore/mysql/hosts.go (1)

1170-1170: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Sort key uses raw source, but the emitted value is translated.

ORDER BY he.email, he.source sorts on the raw source column, while JSON_OBJECT emits the translated value from deviceMappingTranslateSourceColumn("he"). If a single host has multiple emails with the same address but different sources, the tie-break order could diverge from the displayed (translated) source values. This is a narrow edge case (same email, multiple sources) but worth confirming is acceptable.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@server/datastore/mysql/hosts.go` at line 1170, The host JSON aggregation
query sorts by the raw he.source value while emitting the translated source from
deviceMappingTranslateSourceColumn("he"), so the ordering can differ from what
is shown. Update the GROUP_CONCAT ORDER BY in the hosts query to use the same
translated source expression used in JSON_OBJECT, or otherwise make the sort key
explicitly match the emitted value in the host aggregation logic so the result
ordering stays consistent.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@server/datastore/mysql/hosts.go`:
- Line 1170: The host JSON aggregation query sorts by the raw he.source value
while emitting the translated source from
deviceMappingTranslateSourceColumn("he"), so the ordering can differ from what
is shown. Update the GROUP_CONCAT ORDER BY in the hosts query to use the same
translated source expression used in JSON_OBJECT, or otherwise make the sort key
explicitly match the emitted value in the host aggregation logic so the result
ordering stays consistent.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0fa609a1-5cd9-4889-8bf1-6d8c335478c0

📥 Commits

Reviewing files that changed from the base of the PR and between 80b883a and 5eb4ae3.

📒 Files selected for processing (4)

• changes/device-mapping-order
• server/datastore/mysql/hosts.go
• server/datastore/mysql/labels.go
• server/service/integration_core_test.go