fix(gs): exclude self-emitted audit lines from TRACES_BY_MESSAGE results#3756
Closed
TaprootFreak wants to merge 5 commits into
Closed
fix(gs): exclude self-emitted audit lines from TRACES_BY_MESSAGE results#3756TaprootFreak wants to merge 5 commits into
TaprootFreak wants to merge 5 commits into
Conversation
/gs/debug/logs writes a verbose audit entry on every call that includes the messageFilter verbatim. With a high-frequency caller (the RealUnit tracing dashboard polling at 5 s), these audit entries recursively match the same messageFilter and crowd real traces out of the 200-row response — the dashboard sees its own queries instead of the data it asked for. Filtering out severityLevel == 0 (Verbose) at the template level keeps Info+ traces, which is what every consumer of this template actually wants.
Reverts the earlier blanket severityLevel filter in TRACES_BY_MESSAGE (which suppressed ~100 unrelated logger.verbose call sites from the /gs/debug/logs results) and instead fixes the root cause: the audit log emitted by gs.service stringified the full DTO, including messageFilter and eventName verbatim — which a same-filter query then matched against itself. Redacting those two free-text fields makes self-match structurally impossible and keeps the verbose-level visibility for legitimate verbose producers (Sift, Exchange, liquidity-management, KYC, etc.). Adds a regression spec that asserts the audit log argument never contains a sentinel messageFilter value.
TaprootFreak
changed the title
Moves the messageFilter/eventName redaction out of the inline DTO spread into a named helper method with JSDoc that flags the self-match-recursion risk and reminds future contributors to redact any new free-text DTO fields here too. Adds spec coverage for eventName redaction and a positive assertion that legitimate non-free-text fields (template, hours, user) remain visible in the audit log.
Replaces the in-code redactAuditDto approach with a KQL-level filter
directly in the TRACES_BY_MESSAGE template:
| where not(message startswith "[GsService] Log query by ")
Both the verbose success-audit and the info failure-audit emitted by
gs.service share that prefix, so a single line covers both. The audit
log itself stays intact and forensically searchable via other means.
Reverts the redactAuditDto helper, its JSDoc, the inline call site,
and the three audit-redaction spec tests introduced earlier on this
branch. gs.service.ts and the spec are byte-identical to develop;
the only remaining change in this PR is one line in gs.dto.ts plus
a comment that explains the self-match concern.
TaprootFreak
changed the title
Collaborator
Author
|
Verworfen: bei 60s-Polling (statt 5s) im Dashboard reduziert sich die Audit-Last um Faktor 12 auf ~60/h, was bei normalem RealUnit-Traffic (~700 Traces/h) eine echte/audit-Ratio von ~85-90% in den 200 neuesten Rows ergibt. Der Frontend-Parser filtert |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
The RealUnit tracing dashboard (polls
/gs/logswithtemplate=traces-by-messageevery ~5 s) saw 0 real traces. Reason: the endpoint emits its own audit line at verbose severity ([GsService] Log query by <user>: template=..., params={...}). With the dashboard's frequent polling, those self-emitted lines monopolised the 200-row default response, evicting actual traces.Approaches considered and discarded
redactAuditDto) — previous attempt on this branch. Rejected by senior-architect review:[redacted-len=N]Partial<LogQueryDto>)catch-block failure-audit line, which also matches the same queryWhat remains
A single-line KQL post-filter in the
TRACES_BY_MESSAGEtemplate ings.dto.ts:TRACES_BY_OPERATION, App Insights direct).gs.service.tsandgs.service.spec.tsare byte-identical todevelop. The only diff in this PR is one KQL line plus an inline comment explaining the self-match concern.Test plan
npx prettier --check "src/subdomains/generic/gs/**/*.ts"— cleannpx eslint "src/subdomains/generic/gs/**/*.ts" --no-fix— cleannpx jest --testPathPattern 'gs.service'— 29/29 greennpm run build— cleangit diff develop..HEAD -- src/subdomains/generic/gs/gs.service.ts— emptygit diff develop..HEAD -- src/subdomains/generic/gs/__tests__/gs.service.spec.ts— emptyTRACES_BY_MESSAGEfrom RealUnit dashboard, confirm real traces visible againRelated
DFXswiss/services#1113