Add web app and Node.js service technology detection#4
Merged
Conversation
Non-native app and service components were previously skipped with "Skip because this component is not native". Instead, detect and report their technology and check per-firmware compatibility, mirroring the native library check. For web/hosted apps, scan the bundled HTML/JS to identify the frontend framework (Enact, React, Vue, Angular, jQuery, webOSTV.js) and its version, and determine the minimum ECMAScript level the shipped bundle requires. For JS/Node services, read the bundled package.json for the declared engines.node requirement and dependencies. Compatibility is checked against each firmware's actual runtime, read from the packages.json already committed under common/data: the Node.js version and the web engine. The web engine resolves across generations and both families -- Chromium (lib32-webruntime / webruntime / chromiumNN / chromium-webos) and the legacy LG WebKit port (webkit-starfish). A definitive incompatibility fails the component and the process exit code; unknown firmware data yields UNKNOWN and never fails. New webdetect-lib crate holds the pure text/JSON detectors; fw-lib gains node_version()/web_engine() accessors; verify-lib gains Package::verify_for_firmware and DetectionResult/CompatVerdict; the ipk-verify report replaces the skip line with a per-firmware summary in all three output formats. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
The JS scan misses two compat-relevant signals that live only in
index.html.
Detect <script type="module">: ES modules require Chromium 61, so a
bundle that reads as ES5 but is loaded as a module still needs a modern
engine. Modeled as a new EsFeature::EsModule (ES2018 bucket, which is
verdict-identical to Chromium 61 for the real firmware set), it flows
through the existing ES-level compatibility path with no verify-layer
change.
Detect remote/hosted resources: src/href references to http(s):// or
protocol-relative URLs in index.html are collected as an informational
note ("loads N remote resources", listed under --details). This does not
affect the compatibility verdict or exit code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
A .ipk is attacker-controlled. Several path fragments from its metadata were joined onto the extraction directory and then opened without a containment check: the app id and service ids from packageinfo.json, and the main/executable entry from appinfo.json/services.json. A value like "../../../../etc/passwd" or "/dev/zero" would make the verifier open a file outside the package (a device read is an easy DoS; tar's unpack_in only guards the extraction writes, not these later reads). Add a lexical path-containment guard (ensure_within) in ipk-lib and apply it at every join site. Traversal is now rejected before the file is opened, with a clear error, so the malicious path is never read. Also cap the index.html read in webdetect to MAX_FILE_BYTES so a pathological entry can't exhaust memory. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
Regex scanning could not tell code from strings/comments, forcing conservative patterns and still misfiring (e.g. a JSDoc /** banner read as the ** operator). Replace it with lightweight parsing for the two places accuracy matters: - JS ES-feature detection now runs over a ress token stream, so keywords and operators inside strings, comments and regex literals are ignored. ?. and ?? (not single tokens in this lexer) are reconstructed from adjacent ?/./? with span adjacency; async is confirmed as a contextual keyword (identifier followed by function/(/ident) rather than a variable named "async". - HTML signals (script type=module, remote src/href, Angular ng-version) now come from a tl DOM instead of attribute regexes, so body text or comments can't false-match. Framework identity stays on banner/marker scanning (license comments are unambiguous as substrings). Both crates are light: tl has zero transitive deps and ress adds only log + unicode-xid. The ES-level result remains a conservative engine floor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
Enact was detected but Enyo -- the legacy LG/webOS framework Enact
descends from, still used by many older webOS TV apps -- was not. Add
FrameworkKind::Enyo, matched from the enyo.* API and its bundle/file
markers (enyo.kind/Control/version/..., @enyo/, enyo.js), with a
best-effort version from `enyo.version = { core: "x.y.z" }`. Enyo ranks
just below Enact in precedence so an Enact bundle (which shares lineage)
is still reported as Enact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
Two fixes surfaced by testing against real webosbrew packages (the Homebrew Channel and its updater): - Many webOS apps (Enyo/Enact single-file builds) inline all JS into index.html instead of shipping separate .js files, so ES-level detection returned UNKNOWN. Collect inline <script> bodies from the parsed DOM (skipping application/json / importmap) and feed them to the token scanner. The Homebrew Channel (an Enyo app) now correctly reads as ES5 instead of UNKNOWN. - Drop all trust in package.json engines.node. webOS services don't declare it reliably, so a Node compatibility verdict derived from it is meaningless. Services no longer carry a compat verdict or affect the exit code; the report shows the service's dependencies plus each firmware's bundled Node.js version as information only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds comprehensive technology detection for non-native webOS components (web apps and Node.js services), enabling compatibility verification against firmware runtimes. Web apps are analyzed for frontend frameworks and JavaScript syntax levels; services are scanned for Node.js dependencies.
Key Changes
New Detection Library (
common/webdetect)web.rs): Parses HTML DOM (not regex) to extract<script type="module">flags, Angularng-versionattributes, and remote resource URLs. Tokenizes JavaScript (viaresslexer) to detect ES syntax features (let/const, arrow functions, async/await, optional chaining, etc.) while ignoring false matches in strings/comments. Identifies frameworks via license banners in JS text (React, Vue, Angular, Enact, Enyo, jQuery).service.rs): Extracts Node.js dependencies and entry point frompackage.json.eslevel.rs): Maps detected syntax features to ES levels (ES5 through ES2021+) and correlates them to minimum Chromium major versions for compatibility checking.Firmware Runtime Resolution (
common/fw/src/runtime.rs)packages.jsonwebruntime,lib32-webruntime, versionedchromium<NN>)Integration with Verification (
common/verify/src/ipk)VerifyForFirmwaretrait extends package verification to include non-native compatibilityDetectionResultenum wraps web app and service detection with per-firmware verdictsCompatVerdictenum (Ok/Fail/Unknown) represents compatibility outcomesPath Traversal Protection (
common/ipk/src/path.rs)Output Formatting (
packages/ipk-verify)Notable Implementation Details
resslexer to avoid false positives from keywords/operators in strings and comments (e.g.,/**as**).upstreamversion strings (e.g.,120.0.6099.270-137.paparoa.1) by extracting leading three numeric components.<script>bodies for ES detection when apps inline all JS intoindex.html.https://claude.ai/code/session_01WNiK747P4W8C2M9RRhRFS3