Motivation
The YARA scanner currently runs Scan() synchronously on the caller's goroutine, which means every scan, including the underlying libyara call executes inline on the ETW event processing path. This couples scan latency directly to event throughput: a slow or backed-up scan stalls the event loop rather than just delaying the scan result, which is a problem under high event volume or when scanning larger payloads (e.g. ADS reads).
Expectation
Scanning should no longer block event processing.
Scan() returns immediately after enqueuing the work; scan results reach consumers only through the existing alert-sender pipeline
- Under load, once the pending-scan queue is full, new scan requests are dropped (not blocked), and the drop is observable via a metric
- The number of concurrent scan workers is bounded and configurable
- A
Wait()/Flush() mechanism allows draining in-flight scans deterministically (used by tests, and by shutdown)
Proposal
- Introduce a bounded queue (buffered, size controlled by a new AsyncQueueSize config option, default 1024).
Scan() does a non-blocking send. A full channel drops the event and increments a yara.dropped.scans metric
- Add a semaphore (
sem chan struct{}, capacity MaxWorkers, default 32, also configurable), gating how many scan goroutines run concurrently; a single dispatch() goroutine acquires a slot before launching each worker
- Track every launched goroutine with a drainWg sync.WaitGroup so Close() can drain all in-flight work before destroying the YARA compiler, avoiding calls into libyara after
Destroy()
- Extract the eligibility/setup logic currently inline in
Scan() into a function that runs synchronously on the caller's goroutine (fast path); only the actual yara.Scanner invocation moves into the worker pool. This includes making the ADS sys.ReadFile read synchronous (capped at 1MB with a 1s timeout) so scan bytes are captured before the event may be recycled
- Protect the
rwxs/mmaps maps with a mutex, since worker goroutines now write to them concurrently
- Update the
Scan() interface contract: it currently returns (bool, error) with the bool indicating a match, which is incompatible with async execution. It should always return (false, nil), with match results delivered exclusively via the alert-sender pipeline
- Add new config fields for
AsyncQueueSize and MaxWorkers
Here is the proposed architecture diagram:

Motivation
The YARA scanner currently runs
Scan()synchronously on the caller's goroutine, which means every scan, including the underlyinglibyaracall executes inline on the ETW event processing path. This couples scan latency directly to event throughput: a slow or backed-up scan stalls the event loop rather than just delaying the scan result, which is a problem under high event volume or when scanning larger payloads (e.g. ADS reads).Expectation
Scanning should no longer block event processing.
Scan()returns immediately after enqueuing the work; scan results reach consumers only through the existing alert-sender pipelineWait()/Flush()mechanism allows draining in-flight scans deterministically (used by tests, and by shutdown)Proposal
Scan()does a non-blocking send. A full channel drops the event and increments ayara.dropped.scansmetricsem chan struct{}, capacity MaxWorkers, default 32, also configurable), gating how many scan goroutines run concurrently; a single dispatch() goroutine acquires a slot before launching each workerDestroy()Scan()into a function that runs synchronously on the caller's goroutine (fast path); only the actual yara.Scanner invocation moves into the worker pool. This includes making the ADSsys.ReadFileread synchronous (capped at 1MB with a 1s timeout) so scan bytes are captured before the event may be recycledrwxs/mmapsmaps with a mutex, since worker goroutines now write to them concurrentlyScan()interface contract: it currently returns (bool, error) with the bool indicating a match, which is incompatible with async execution. It should always return (false, nil), with match results delivered exclusively via the alert-sender pipelineAsyncQueueSizeandMaxWorkersHere is the proposed architecture diagram: