Skip to content

system/nxpkg: network sync, install hardening, and CLI completion#3642

Draft
aviralgarg05 wants to merge 1 commit into
apache:masterfrom
aviralgarg05:gsoc/nxpkg-sync-lifecycle-hardening-pr5
Draft

system/nxpkg: network sync, install hardening, and CLI completion#3642
aviralgarg05 wants to merge 1 commit into
apache:masterfrom
aviralgarg05:gsoc/nxpkg-sync-lifecycle-hardening-pr5

Conversation

@aviralgarg05

@aviralgarg05 aviralgarg05 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Note: Please adhere to Contributing Guidelines.

Summary

This PR completes the package lifecycle work introduced in #3474. The initial
implementation supported local metadata and basic install/list operations;
this change adds the network and lifecycle operations needed to use nxpkg as
a practical on-device package manager.

The main changes are:

  • add repository synchronization and package downloads over HTTP, with SHA-256
    verification of downloaded artifacts;
  • implement the sync, update, remove, rollback, and available CLI
    paths, and allow install to use either local or network sources;
  • recover cleanly from interrupted downloads and installations;
  • validate manifest paths and bound allocations and input sizes before using
    repository-provided data;
  • harden metadata storage against truncated files and serialize updates to the
    installed-package database so concurrent installs cannot overwrite one
    another;
  • support an optional package icon used by graphical frontends;
  • handle custom boards through CONFIG_ARCH_BOARD_CUSTOM_NAME when
    CONFIG_ARCH_BOARD is not defined;
  • route package diagnostics through syslog, which also works when no console
    is attached; and
  • document the repository layout and local server setup in
    system/nxpkg/README.txt.

This PR depends on the package lifecycle foundation merged in #3474. #3643 is
a graphical frontend that depends on the APIs added here; #3644 is independent.

Impact

When CONFIG_SYSTEM_NXPKG is enabled, users gain a complete package lifecycle:
repository sync, local or remote installation, update, removal, rollback,
listing, and discovery of available packages.

Network operations additionally require CONFIG_NETUTILS_WEBCLIENT; the
existing CONFIG_NETUTILS_CJSON dependency remains. Local-only installations
do not require network access.

The install path now performs network I/O when given a remote source and uses a
global lock while updating installed-package state. Repository metadata is
treated as untrusted input, including path components and declared sizes.

Existing applications are unaffected unless CONFIG_SYSTEM_NXPKG is enabled.
The on-disk development format has gained fields for icon and transaction
state; no migration is provided because the format introduced by #3474 has not
appeared in a release.

Testing

Build host:

  • macOS 26.5, arm64
  • xtensa-esp-elf-gcc 14.2.0 (esp-14.2.0_20241119)

Target:

  • Xtensa / ESP32-S3
  • esp32s3-touch-lcd-7:nsh on a Waveshare ESP32-S3-Touch-LCD-7
    (custom, not-yet-upstream board configuration)

The complete image was built and the package lifecycle was exercised repeatedly
on the physical board. The verified flows included repository sync from a local
HTTP server, remote install, update, removal, rollback, listing, and application
launch. Interrupted install state was recreated and successfully reclaimed.
Two different packages were also installed concurrently; both remained present
in the installed-package database after completion.

The custom-board compatibility fix was checked against the generated
include/nuttx/config.h, where CONFIG_ARCH_BOARD is absent and
CONFIG_ARCH_BOARD_CUSTOM_NAME is defined:

$ xtensa-esp32s3-elf-gcc -c -fsyntax-only \
    -I <nuttx>/include -isystem <nuttx>/include -D__NuttX__ -D__KERNEL__ \
    -I <apps>/include -I system/nxpkg \
    system/nxpkg/pkg_compat.c system/nxpkg/pkg_log.c
# completed successfully

Fill in most of what the initial nxpkg slice deferred: network sync
and artifact acquisition over plain HTTP (verified via SHA-256), an
install rewrite that supports network sources and reclaims state left
by an interrupted previous install, and wiring update/remove/rollback/
available into the CLI.

Harden the package path against untrusted input along the way: bounded
memory-safety helpers and explicit size limits throughout, storage
read/write hardened against partially-written files, path-traversal
rejection in manifest name/version before they reach the filesystem,
and a fix for a lost-update race on the shared installed-packages
database (two concurrent installs of different packages could
otherwise silently clobber each other's recorded state).

Add an optional manifest icon field for the nxstore GUI frontend to
consume, raise PKG_INDEX_MAX now that the in-memory index is
heap-allocated, and document the repository layout and local server
setup in system/nxpkg/README.txt.

Also fixes a real build break: pkg_runtime_compat() unconditionally
referenced CONFIG_ARCH_BOARD, a Kconfig string symbol with no default
clause under ARCH_BOARD_CUSTOM, so it is left entirely undefined
rather than defined-but-empty on a custom board - falls back to
CONFIG_ARCH_BOARD_CUSTOM_NAME instead. Routes pkg_error()/pkg_info()
through syslog rather than stdio, since neither is visible to a
supervisor with no attached console.

Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05
aviralgarg05 force-pushed the gsoc/nxpkg-sync-lifecycle-hardening-pr5 branch from d33ae3b to c4f639c Compare July 17, 2026 19:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant