|
| 1 | +# pxSentinel — Development & Testing Log |
| 2 | + |
| 3 | +This document describes how pxSentinel was built, how its architecture evolved, and how each design decision was validated against real backdoor activity observed on a live FiveM server. |
| 4 | + |
| 5 | +--- |
| 6 | + |
| 7 | +## Origin |
| 8 | + |
| 9 | +pxSentinel began as `pxSkidDetector` — a minimal single-file resource containing a hardcoded signature list and a basic `onResourceStart` handler. While functional in concept, it had several problems that made it unsuitable for production use: |
| 10 | + |
| 11 | +- Discord webhook URL was hardcoded directly in source code |
| 12 | +- `string.find` calls lacked the plain-text flag, meaning any Lua metacharacter in a signature string would be silently misinterpreted as a pattern |
| 13 | +- Scanning happened only at self-start with no coverage of post-startup dynamic resource loads |
| 14 | +- No threading yield, which caused server thread hitches on large resource lists |
| 15 | +- No allowlist, so system resources and legitimate frameworks produced false positives |
| 16 | + |
| 17 | +The resource was fully rewritten under the name `pxSentinel`, retaining none of the original code. |
| 18 | + |
| 19 | +--- |
| 20 | + |
| 21 | +## Architecture Decisions |
| 22 | + |
| 23 | +### Config split across four files |
| 24 | + |
| 25 | +All configuration, signatures, and allowlist entries were separated into dedicated files (`config.lua`, `blocked.lua`, `allowed.lua`) rather than inlined in `server.lua`. This makes signature maintenance and allowlist management possible without touching the core logic file. Files are listed in `fxmanifest.lua` in load order so `Config` is fully populated before `server.lua` executes. |
| 26 | + |
| 27 | +### Plain-text signature matching |
| 28 | + |
| 29 | +Every `string.find` call uses the fourth argument `true` to force plain-text matching: |
| 30 | + |
| 31 | +```lua |
| 32 | +string.find(content, signature, 1, true) |
| 33 | +``` |
| 34 | + |
| 35 | +Without this flag, any signature containing a Lua pattern metacharacter (`.`, `*`, `+`, `(`, etc.) would not match what it was written to match. Domain names like `vac.sv` contain dots that Lua treats as "any character" in pattern mode, meaning the signature would silently match more than intended. |
| 36 | + |
| 37 | +### Convar-based webhook |
| 38 | + |
| 39 | +The Discord webhook is read from a server convar (`pxSentinel:webhook`) rather than stored in source. This prevents the webhook URL from appearing in git history or leaked source archives. |
| 40 | + |
| 41 | +``` |
| 42 | +set pxSentinel:webhook "https://discord.com/api/webhooks/..." |
| 43 | +``` |
| 44 | + |
| 45 | +### Scan timing and thread yield |
| 46 | + |
| 47 | +The initial scan runs after a configurable settle delay (`Config.ScanDelay`, default 5000ms) to allow all resources that start before pxSentinel to fully register their file metadata with the runtime. Without this delay, `GetNumResourceMetadata` would return zero for resources still in the process of starting. |
| 48 | + |
| 49 | +Because `LoadResourceFile` reads from disk and a server may have hundreds of resources, the scan loop yields with `Wait(0)` between iterations to release the main thread: |
| 50 | + |
| 51 | +```lua |
| 52 | +for i = 0, GetNumResources() - 1 do |
| 53 | + -- ... scan ... |
| 54 | + Wait(0) |
| 55 | +end |
| 56 | +``` |
| 57 | + |
| 58 | +Omitting this caused server thread hitch warnings of up to 4–5 seconds in testing. |
| 59 | + |
| 60 | +### O(1) allowlist lookup |
| 61 | + |
| 62 | +The `Config.SafeResources` list is converted to a hash set at startup: |
| 63 | + |
| 64 | +```lua |
| 65 | +local safeResourceSet = {} |
| 66 | +for _, name in ipairs(Config.SafeResources) do |
| 67 | + safeResourceSet[name] = true |
| 68 | +end |
| 69 | +``` |
| 70 | + |
| 71 | +This makes the `isSafe()` check O(1) regardless of list size, compared to O(n) for an `ipairs` scan on every resource check. |
| 72 | + |
| 73 | +### Coverage of all manifest script types |
| 74 | + |
| 75 | +Early versions only scanned `server_script` metadata entries. This missed: |
| 76 | + |
| 77 | +- Resources using `server_only_script` (files explicitly excluded from client delivery) |
| 78 | +- Resources using `shared_script` (files compiled for both client and server) |
| 79 | +- Node.js resources whose declared entry point is a thin loader that `require()`s a larger payload file not listed in the manifest at all |
| 80 | + |
| 81 | +The scanner now iterates all three metadata keys and additionally probes a list of common secondary filenames (`index.js`, `server.js`, `main.js`, etc.) for any resource that declares at least one `.js` file. |
| 82 | + |
| 83 | +### `onResourceFsPermissionViolation` handler |
| 84 | + |
| 85 | +FiveM's Node.js runtime fires this event when a resource attempts a filesystem write outside its permitted scope. pxSentinel subscribes to it and treats any violation as a detection event — logging to console, sending a Discord alert, and optionally stopping the resource. |
| 86 | + |
| 87 | +Importantly, this handler only covers filesystem writes. In-memory RCE delivered via HTTPS fetch and `eval()` executes without any filesystem interaction and will not trigger it. That class of payload must be caught at the static scan stage. |
| 88 | + |
| 89 | +--- |
| 90 | + |
| 91 | +## Testing Against a Live Backdoor |
| 92 | + |
| 93 | +Development occurred alongside active server operation. During this process, `redutzu-mdt` — a resource distributed through unofficial FiveM marketplace channels was discovered to contain a server-side backdoor. The following events were observed and used to harden pxSentinel iteratively. |
| 94 | + |
| 95 | +### Initial filesystem infiltration attempt |
| 96 | + |
| 97 | +Server logs showed `redutzu-mdt` triggering `onResourceFsPermissionViolation` with write attempts targeting: |
| 98 | + |
| 99 | +- `txData/` — the txAdmin data directory, used to plant a persistent Node.js monitoring agent |
| 100 | +- Core txAdmin internal files including `cl_playerlist.lua` — likely an attempt to replace a trusted file with a trojanised version |
| 101 | + |
| 102 | +This informed the first set of filesystem-specific signatures added to `blocked.lua`: |
| 103 | + |
| 104 | +```lua |
| 105 | +'monitoring-agent', |
| 106 | +'data-processor', |
| 107 | +'system_resources', |
| 108 | +'cl_playerlist.lua', |
| 109 | +'cfx-server/citizen', |
| 110 | +``` |
| 111 | + |
| 112 | +It also confirmed that `onResourceFsPermissionViolation` works as a real-time backstop for this class of attack. |
| 113 | + |
| 114 | +### Static scan detection of `server/scanner.js` |
| 115 | + |
| 116 | +After the filesystem signatures were added, pxSentinel's initial scan flagged `redutzu-mdt` a second time — this time for a file called `server/scanner.js` that was not the resource's declared entry point. The file contained: |
| 117 | + |
| 118 | +1. The obfuscator.io anti-tamper loop: `while(!![]){try{` |
| 119 | +2. An `eval()` call operating on an `_0x`-prefixed obfuscated identifier: `eval(_0x...)` |
| 120 | + |
| 121 | +Both are signatures that are mechanically emitted by [obfuscator.io](https://obfuscator.io) and have no equivalent in any legitimate FiveM resource. They were added to `blocked.lua`: |
| 122 | + |
| 123 | +```lua |
| 124 | +'while(!![]){try{', |
| 125 | +'eval(_0x', |
| 126 | +``` |
| 127 | + |
| 128 | +The file was reached because pxSentinel probes common secondary `.js` filenames on any Node.js resource, not just declared manifest entries. `server/scanner.js` was not listed in the resource's `fxmanifest.lua` — it was injected alongside the declared files. |
| 129 | + |
| 130 | +### Kill-switch causing server restart |
| 131 | + |
| 132 | +When pxSentinel called `StopResource('redutzu-mdt')` on detection, the server process exited and txAdmin restarted it automatically. Investigation confirmed that the backdoor hooked the `onResourceStop` event and called `os.exit()` as a self-defence mechanism — a pattern designed to bait administrators into a restart loop. |
| 133 | + |
| 134 | +This is a known technique: a backdoor that can resist being stopped forces administrators to either tolerate its presence or take the server fully offline to remove it. Calling `StopResource()` on an infected resource is therefore unsafe as a default response. |
| 135 | + |
| 136 | +As a result, `Config.StopResources` was changed to default to `false`. The correct remediation procedure when pxSentinel fires is: |
| 137 | + |
| 138 | +1. Note the infected resource from the console output or Discord alert. |
| 139 | +2. Use txAdmin to **stop** the server entirely (not restart — a restart also fires `onResourceStop`). |
| 140 | +3. Delete the infected resource from disk. |
| 141 | +4. Start the server. |
| 142 | + |
| 143 | +A console warning is printed when `StopResources = true` is set, advising of this risk. |
| 144 | + |
| 145 | +--- |
| 146 | + |
| 147 | +## False Positives Encountered and Resolved |
| 148 | + |
| 149 | +| Resource | Trigger | Resolution | |
| 150 | +|---|---|---| |
| 151 | +| `monitor` (txAdmin) | Contained `GetPlayerTokens` | Added to allowlist; removed `GetPlayerTokens` from signatures (legitimate FiveM native) | |
| 152 | +| `webpack` / `yarn` | Contained `fs.writeFile` | Added to allowlist; removed `fs.writeFile` from signatures (standard Node.js build API) | |
| 153 | +| All resources | Server thread hitch (up to 4.8s) | Added `Wait(0)` yield per resource in scan loop | |
| 154 | + |
| 155 | +Each false positive was identified from live server logs and resolved by either adding the resource to `Config.SafeResources` or removing the overly-broad signature and replacing it with more specific alternatives. |
| 156 | + |
| 157 | +--- |
| 158 | + |
| 159 | +## Signature Evaluation Criteria |
| 160 | + |
| 161 | +Before any signature is added to `blocked.lua`, it is evaluated against two questions: |
| 162 | + |
| 163 | +1. **Does it appear in any legitimate FiveM resource?** If yes, it is excluded or replaced with a more specific alternative. |
| 164 | +2. **Is it unique enough to be meaningful?** A signature that matches common patterns in normal code produces noise and erodes trust in detections. |
| 165 | + |
| 166 | +Signatures that were considered and rejected: |
| 167 | + |
| 168 | +- `GetPlayerTokens` — legitimate FiveM native used by txAdmin |
| 169 | +- `fs.writeFile` — standard Node.js API used by webpack and yarn |
| 170 | +- `on('data',` — standard Node.js HTTP streaming callback, present in many legitimate resources |
| 171 | +- `require('https')` — standard Node.js stdlib import, far too broad |
| 172 | + |
| 173 | +--- |
| 174 | + |
| 175 | +## Versioning |
| 176 | + |
| 177 | +pxSentinel follows semantic versioning. The resource is currently at `1.0.0-beta.1`, reflecting that the signature list and scanner behaviour are stable but subject to further refinement as new threats emerge. A `1.0.0` stable release will be cut once the signature list has been validated over a longer operational period. |
0 commit comments