Fix CI test failure: increase test coverage above 95% threshold

[Copilot]

The GitHub Actions "test" job was failing because cargo tarpaulin --fail-under 95 measured 92.41% coverage (1728/1870 lines), below the required 95% threshold.

Root Cause

Code coverage was insufficient across several modules due to untested code paths in the correlation engine, matcher, pipeline, and parser.

Changes Made

Added 35 new test functions across 4 source files to cover previously untested code paths:

• src/processor.rs: with_correlation_rules() builder method, product/service log-source mismatch, event window eviction, rule matching by rule ID, all eval_simple_condition branches (gt/lt/lte/eq/neq), value_percentile correlation type (full coverage), value_avg with missing field, temporal correlation with extended boolean condition
• src/matcher.rs: BTreeMap-backed events (contains_key, get, values), Value::as_str() and Display for non-String variants (Integer, Float, Boolean, Time, Ip), bool pattern string-representation matching, null pattern with non-string value, unparseable numeric strings for int/float patterns, cached invalid regex path
• src/pipeline.rs: Invalid regex in replace_string transformation (error path), RuleCondition::matches product mismatch, RuleCondition::matches service mismatch
• src/parser.rs: parse_aliases field-value-not-string error case, temporal_ordered with extended condition (valid path), correlation rule custom fields

Results

File	Before	After
matcher.rs	367/406	383/406
parser.rs	486/516	487/516
pipeline.rs	376/380	379/380
processor.rs	325/392	358/392
Total	1728/1870 (92.41%)	1781/1870 (95.24%)

cargo tarpaulin --fail-under 95 now passes. All 294 tests pass.

Original prompt

Start implementation

The user has attached the following file paths as relevant context:

• .github\copilot-instructions.md

[Chronological Review: The conversation began with the user requesting the implementation of matching Sigma correlation rules based on a specified GitHub document. The user provided a detailed description of the requirements, including the need for timestamps in events, modifications to the existing `#sym:Event` trait, and the use of a message passing system for correlation matching. The user also emphasized the need for efficient data structures and handling of late-arriving events. The conversation progressed with the user actively working on the `c:\Users\thoma\OneDrive\Devel\sigma_engine\src\processor.rs` file, focusing on implementing these features.]

[Intent Mapping: The user explicitly requested the following:

1. "Implement matching of Sigma correlation rules as defined in the specification..."
2. "Events must now contain a timestamp."
3. "Add a get_time function to the #sym:Event trait."
4. "Use the already existing message passing system in #sym:impl LogProcessor for correlation matching."
5. "Create efficient data structures for maintenance of the correlation rule state."]

[Technical Inventory: The conversation involved the following technical concepts:

• #sym:SigmaCorrelationRule: A struct representing a fully-parsed Sigma correlation rule.
• #sym:Event trait: Required modification to include a timestamp.
• Message passing system: Existing system in #sym:impl LogProcessor to be utilized for correlation matching.
• UUIDs: To uniquely identify detections.
• Sliding time window: For correlation logic.
• Handling late-arriving events: With a configurable threshold.]

[Code Archaeology: The user is currently working on the processor.rs file, which is likely to contain the implementation logic for the correlation rules and event processing. The SigmaCorrelationRule struct was previously defined in types.rs, which includes various fields relevant to correlation rules.]

[Progress Assessment: The user has outlined the requirements for the correlation rule matching but has not yet implemented the changes. The focus is on defining the logic and data structures needed for the correlation matching process.]

[Context Validation: All critical information for continuation is captured, including the user's goals, the specific requirements for the implementation, and the current file being worked on.]

[Recent Commands Analysis: The last agent command was the user's request to implement the correlation rule matching, which included multiple specifications for functionality and structure. The immediate pre-summarization state involved the user actively working on the processor.rs file, detailing how to implement the correlation logic and event handling.]

1. Conversation Overview: - Primary Objectives: The user requested to "Implement matching of Sigma correlation rules as defined in the specification..." and specified various requirements for timestamps, event handling, and correlation logic. - Session Context: The conversation focused on implementing features in the Sigma engine, particularly in the `processor.rs` file, to support correlation rules. - User Intent Evolution: The user's needs have remained consistent, focusing on implementing a robust correlation matching system with specific requirements for event handling and data structures.

2. Technical Foundation:

• #sym:SigmaCorrelationRule: A struct for Sigma correlation rules, containing fields like title, id, description, and correlation details.
• #sym:Event trait: Needs modification to include a get_time function for timestamps.
• Message passing system: To be utilized for correlation matching in #sym:impl LogProcessor.
• UUIDs: To uniquely identify detections in the correlation process.

3. Codebase Status:

• processor.rs:
  • Purpose: Contains the implementation logic for processing events and correlation rules.
  • Current State: The user is actively working on implementing the correlation matching logic.
  • Key Code Segments: Not specified yet, as implementation is pending.
  • Dependencies: Relies on the #sym:SigmaCorrelationRule struct and the message passing system.

4. Problem Resolution:

• Issues Encountered: None explicitly mentioned, but challenges related to implementing the correlation logic and handling late events are anticipated.
• Solutions Implemented: No solutions have been implemented yet; the user is in the planning phase.
• Debugging Context: Ongoing development with no known issues reported.
• Lessons Learned: Insights will emerge as the implementation progresses.

5. Progress Tracking:

• Completed Tasks: None yet; the user has outlined requirements but not implemented them.
• Partially Complete Work: The user is in the process of defining the correlation matching logic.
• Validated Outcomes: None yet, as testing will follow implementatio...

Created from VS Code.

---

⚡ Quickly spin up Copilot coding agent tasks from anywhere on your macOS or Windows machine with Raycast.

[thomaspatzke]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved in commit 9563c28. The conflict was in src/processor.rs due to origin/main changing LogEvent.log_source from LogSource to Option<LogSource>. Resolved by:

• Keeping our added doc comment on from_fields while adopting the impl Into<Option<LogSource>> signature from main
• Using event.log_source.as_ref() in worker_thread
• Updating log_source_matches to accept Option<&LogSource> with an early None => return true bypass
• Fixing the merged test test_processor_none_log_source_matches_all_rules to pattern-match DetectionResult::Rule (since start() now returns Receiver<DetectionResult>)

All 259 tests pass.