Separated plugin packaging system structure

[aaidee]

Pinetime saves info like alarm time and settings to NRF's FDS (flash data storage) library now, which is Key-Value file system. FDS is also called as NVS (Non Volatile Storage). It's size is limited.
#2389

Pebble, Winamp, Rockbox, foobar, Roblox, Minecraft, Doom... A rich ecosystem of third-party plugins, like watchfaces and add-ons, encourages active participation from end-users, boosting the overall popularity of the software project. This makes it a must-have feature.

Pebble OS handles slow tasks like package copying on the watch itself, while offloading complex computations to the phone via a companion app. In essence, it uses the smartphone as a sort of co-processor.
Given PineTime's limited storage, we could store apps on the phone via BLEfs and dynamically load them over Bluetooth only when needed. This essentially allows us to run them on demand.

ChatGPT says:

FreeRTOS does not natively support dynamically loading and executing programs from a filesystem like Linux or Windows.

This is because FreeRTOS is a real-time operating system (RTOS) rather than a full-featured operating system. It provides task scheduling and synchronization primitives, but it does not include features such as:

• Process management
• Virtual memory
• Dynamic linking
• An executable loader (ELF/PE)
• User/kernel separation

However, there are several approaches depending on what you want to achieve.

1. Plugin-style modules (Recommended)

The most common approach is to load a binary containing functions rather than an entire application.

Flash
│
├── Main Application
│   ├── Task A
│   ├── Task B
│   └── Plugin API
│
Filesystem
    ├── plugin1.bin
    ├── plugin2.bin
    └── config.dat

The application:

1. Reads the plugin binary from the filesystem.
2. Copies it into executable RAM.
3. Calls its entry function through a function pointer.

Example:

typedef struct {
    void (*init)(void);
    void (*run)(void);
} plugin_api_t;

The plugin usually needs to be compiled as position-independent code (PIC) or designed to avoid absolute addresses.

---

2. Copy code into RAM and execute

If the MCU allows execution from SRAM (many ARM Cortex-M devices do), the process is:

Open file
    ↓
Read binary
    ↓
Allocate RAM
    ↓
Copy code
    ↓
Call entry function

Example:

void (*entry)(void);

entry = (void (*)(void))ram_buffer;

entry();

Requirements:

• SRAM must be executable.
• The code must not rely on fixed addresses.
• Interrupt vectors may require special handling.

---

3. Bootloader-based firmware update

This is the most common solution in embedded systems.

filesystem
    firmware.bin
        ↓
Verify CRC/signature
        ↓
Program Flash
        ↓
Reboot
        ↓
Run new firmware

This is how most OTA (Over-The-Air) firmware updates work.

---

4. Implement an ELF loader

It is possible but significantly more complex.

A complete loader typically needs to implement:

• ELF parsing
• Section loading
• Relocation
• Symbol resolution
• Memory allocation
• Global Offset Table (GOT)
• Procedure Linkage Table (PLT)

Essentially, you would be implementing part of what Linux already provides.

Some open-source projects provide lightweight ELF loaders for embedded systems.

---

5. Use an interpreter (Lua, Python, WebAssembly)

Many embedded systems use scripting instead of native dynamic loading.

Example:

FreeRTOS
    ↓
Lua Interpreter
    ↓
script.lua

or

FreeRTOS
    ↓
MicroPython
    ↓
script.py

This allows new functionality to be added without replacing the firmware.

---

6. Position-Independent Code (PIC)

GCC options:

-fPIC

or

-fPIE

However, support for PIC on bare-metal ARM Cortex-M targets is often limited, and the generated code may not behave like PIC on desktop operating systems.

---

7. RTOSes with dynamic module support

Some embedded RTOSes include built-in support for loading executable modules.

• Apache NuttX — Supports loading ELF executables and shared modules.
• Zephyr Project — Provides Loadable Extensions (LLEXT).
• RT-Thread — Includes the dlmodule mechanism for dynamically loaded modules.

These RTOSes provide more operating-system functionality than FreeRTOS, making dynamic loading much easier.

---

Which approach should you choose?

Goal	Recommended approach
Firmware update	Bootloader + firmware.bin
Add optional native functionality	Load a PIC module into executable RAM
User scripting	Lua or MicroPython
Run ELF executables like Linux	Implement an ELF loader or use an RTOS such as NuttX or Zephyr

In practice, bootloader-based firmware updates and plugin-style modules are by far the most common approaches in FreeRTOS-based embedded systems. Full executable loading is possible but requires substantial additional infrastructure beyond what FreeRTOS itself provides.

Gemini says:

In FreeRTOS, dynamically loading and executing executable files from a file system is not natively supported out-of-the-box in the same way it is in full-fledged operating systems like Linux (which uses ELF loaders and the execve system call). Because FreeRTOS is a real-time operating system (RTOS) designed for deeply embedded systems, it typically runs as a single, monolithic binary where all tasks and functions are linked statically at compile time.

However, you can achieve dynamic loading in FreeRTOS. Here is how it is typically done, the formats used, and the challenges involved.

---

1. How It Works (The Core Mechanism)

To load and run an executable dynamically from a file system (such as an SD card using FATFS) in FreeRTOS, you need three main components:

1. A File System: To store and read the executable file (e.g., FatFS, LittleFS).
2. A Dynamic Linker/Loader: A software component running on FreeRTOS that can parse the executable file format, copy the code/data sections into RAM, and resolve symbols.
3. An Executable Format: A file format designed for relocation (usually ELF or a simplified custom format).

---

2. Common Approaches and Formats

A. ELF (Executable and Linkable Format) Loader

The standard way to do this is by using relocatable ELF files (usually compiled with the -r or -shared flag in GCC).

• The Process: 1. The compiler generates an ELF file containing the compiled task code.

2. In FreeRTOS, your loader reads the ELF header from the file system.
3. The loader allocates RAM (using pvPortMalloc or dedicated fast RAM) for the .text (code), .rodata (read-only data), and .data/.bss (variables) sections.
4. Relocation: The loader resolves any external symbols (like FreeRTOS API functions like vTaskDelay or xQueueSend) by mapping the ELF's import table to the actual function addresses in your main FreeRTOS firmware.
5. The loader jumps to the entry point of the loaded code, often spawning it as a new FreeRTOS task:

xTaskCreate(loaded_entry_point, "DynamicTask", stack_size, NULL, priority, NULL);

B. PIC (Position Independent Code) Binaries

Another approach is compiling your dynamic module as Position Independent Code using the -fPIC compiler flag.

• Since the code can run from any address in memory, the loader simply needs to copy the binary into a chunk of allocated RAM and jump to the entry address.
• However, you still need a way to pass an "API jump table" (a list of pointers to FreeRTOS and standard library functions) from the main application to the loaded module, as PIC alone doesn't resolve external symbols automatically.

C. WebAssembly (Wasm) or MicroPython Engines

Instead of loading native machine code, many modern FreeRTOS systems load bytecode.

• Wasm: You can integrate an interpreter like WAMR (WebAssembly Micro Runtime). You store .wasm files on your file system, load them into memory, and run them securely in a virtual machine environment on FreeRTOS.
• MicroPython/Lua: Similarly, scripting engines can dynamically read and execute script files from the file system.

---

3. Open-Source Libraries for FreeRTOS ELF Loading

If you want to implement native dynamic loading, you don't have to write an ELF loader from scratch. You can use existing lightweight loaders:

• elfloader: A popular, lightweight C library designed specifically for loading ELF files on microcontrollers (ARM Cortex-M).
• tcc (Tiny C Compiler) or custom backports: Some developers port minimalist loaders designed for embedded systems (like Contiki-OS's ELF loader) to FreeRTOS.

---

4. Key Challenges & Considerations

Before implementing dynamic loading in FreeRTOS, keep these critical hardware constraints in mind:

• MPU (Memory Protection Unit): If your MCU uses an MPU (like FreeRTOS-MPU), the RAM region where you load the executable must be explicitly configured to allow Execution (NX/Executable attribute). By default, many systems disable execution from RAM for security (to prevent buffer overflow exploits).
• RAM Consumption: Dynamic loading requires copying code from the file system into RAM. Microcontrollers usually have very limited RAM compared to Flash memory.
• Instruction Cache (I-Cache) Coherency: On higher-end MCUs (like ARM Cortex-M7 or Cortex-A), when you write code to RAM and try to execute it, you must flush the Data Cache (D-Cache) and invalidate the Instruction Cache (I-Cache) so the CPU doesn't execute stale instructions.
• Symbol Exporting: Your main FreeRTOS binary must keep a "symbol table" intact during compilation (not stripped) so the loader can link functions like printf or vTaskDelay to the dynamic module.

DeepSeek says:

Dynamic Loading of Executable Files from a File System in FreeRTOS

1. Fundamental Understanding: FreeRTOS as a Static Environment

It's important to first understand that FreeRTOS is fundamentally a real-time kernel (a scheduler with synchronization primitives) that is typically statically linked with your application into a single monolithic binary . Unlike general-purpose operating systems (GPOS) such as Linux or Windows, it does not natively support the concept of dynamically loading and executing arbitrary user programs as separate processes . A FreeRTOS "task" is more analogous to a thread than a GPOS process—it shares the same memory space and does not inherently have memory protection from other tasks .

However, implementing dynamic loading is possible, but it requires significant custom development. This is often done for advanced use cases like plugin systems, firmware updates, or running large libraries that cannot fit in internal flash memory .

2. The Required Architecture and Key Components

To achieve dynamic loading, you need to build an architecture with several key components.

2.1. File System Integration

First, you need a file system that can read the executable file from storage (e.g., an SD card, external QSPI flash, or a RAM disk) .

• Implementation: FreeRTOS can be integrated with file systems like FreeRTOS+FAT or the popular open-source FatFs. This involves writing low-level drivers for your storage medium (e.g., a disk I/O interface for an SD card or a NOR flash) and then integrating it with the file system's API .
• Source of Executable: Your executable file (e.g., a .bin, .axf, or .elf file) will be stored on this medium .

2.2. Dynamic Memory Manager

The executable's code and data segments need to be loaded into RAM for execution. You will need a heap memory manager to allocate memory for these segments at runtime . Many FreeRTOS ports include a heap manager (heap_1.c to heap_5.c), but you may need a more sophisticated one that can handle arbitrary allocation and deallocation of blocks of memory .

2.3. Executable File Format and Loader

You must choose a file format and implement a loader to parse and load it into memory.

• File Format: The ELF (Executable and Linkable Format) is the most common and well-documented choice . It supports the metadata necessary for relocation, linking, and debugging. You can find reference implementations of ELF loaders online .
• Loader Tasks: The loader's primary job is to :
  1. Parse the file's header and program headers.
  2. Allocate memory on the heap for the .text (code) and .data (initialized data) segments.
  3. Copy the relevant data from the file into the allocated memory regions.
  4. Perform relocation to resolve symbolic addresses.

2.4. Dynamic Linking (Relocation)

This is the most complex part. Your loader must resolve symbols (function and variable references) that the loaded program needs. This requires :

• A Symbol Table: A table in the base (host) firmware that lists the names and addresses of functions that the loaded module is permitted to call (e.g., RTOS API functions like xQueueSend, driver functions like ST7735_Message).
• Relocation Process: The loader parses the relocation tables in the executable. For each entry, it looks up the symbol name in its symbol table. If found, it patches the memory address of the loaded code with the correct absolute address of that function .

3. Execution and Integration with FreeRTOS

Once the code is loaded and linked, it's not automatically a task. You can follow a few strategies to integrate it:

1. Single Entry Point: The simplest approach is to define a single entry point function (e.g., module_main()). After loading, your firmware calls this function directly or creates a single task with it as the entry point .
2. Export/Import Tables: For a more advanced plugin system, define two tables :
   • Export Table: A table of functions that the plugin provides to the host (e.g., plugin_init, plugin_process). The host can retrieve these function pointers by name.
   • Import Table: A table of functions the plugin needs from the host. The loader should populate this table during the linking phase.

4. Additional Considerations and Constraints

• Memory Protection: Without an MMU (Memory Management Unit), you have no hardware protection to prevent a misbehaving loaded module from corrupting the kernel or other tasks. This poses a significant risk for system stability and safety. Some specialized RTOS add-ons can use an MPU (Memory Protection Unit) for this purpose, but FreeRTOS itself does not manage it out-of-the-box .
• Real-time Determinism: Dynamic loading involves file I/O and complex memory operations, which are non-deterministic. This process should be done during initialization or at a time when real-time deadlines are not critical.
• Alternative Approach (Overlays): For systems with severe memory constraints where the entire program cannot fit, consider using overlays. This is a technique where different segments of code are loaded into the same region of memory as needed. This is primarily a linker feature and can be implemented with or without a file system .
• Alternative Approach (Scripting): If absolute performance isn't the core requirement, consider integrating an embedded scripting language like Lua. This allows you to load and run scripts from a file system without the complexity of dynamic linking and native code relocation .

What are some lightweight RTOSes that support dynamic loading of executable modules or executable files?

---

ChatGPT Answer:

Very few lightweight RTOSes provide official support for dynamically loading executable files (such as ELF modules). Most lightweight RTOSes use static linking, where all code is linked into a single firmware image at build time.

RTOS	Dynamic loading	Lightweight	Notes
Apache NuttX	✔ ELF executables, dlopen()	★★★☆☆	Most complete support
Zephyr	✔ Loadable Extensions (LLEXT)	★★★★☆	Plugin/module-oriented
RT-Thread	✔ dlmodule	★★★★☆	Supports ELF modules
Apache Mynewt	✖	★★★★★	Firmware image replacement only
RIOT	✖	★★★★★	No built-in dynamic loader
FreeRTOS	✖	★★★★★	Must implement your own loader

Apache NuttX

NuttX is the closest to a traditional Unix-like operating system.

It supports:

• ELF executables
• Shared libraries
• dlopen()
• dlsym()
• POSIX APIs

It is the most mature option if you need to execute ELF files directly from a filesystem.

Zephyr

Zephyr provides Loadable Extensions (LLEXT).

Features include:

• ELF-based modules
• Relocation support
• Access to symbols exported by the main firmware
• Plugin-style architecture

LLEXT is intended for extending an application rather than running independent user programs.

RT-Thread

RT-Thread includes dlmodule, which supports:

• ELF modules
• Loading modules from a filesystem
• Symbol resolution

It is simpler than NuttX but suitable for many embedded applications.

Other RTOSes

Most lightweight RTOSes do not support dynamic loading, including:

• FreeRTOS
• RIOT
• Apache Mynewt
• Azure RTOS (ThreadX)
• ChibiOS
• embOS

These systems typically follow this workflow:

Compile
    ↓
Link
    ↓
Flash firmware
    ↓
Run

Lightweight ranking (among RTOSes that support dynamic loading)

1. Zephyr
2. RT-Thread
3. Apache NuttX

For resource-constrained hardware such as the PineTime (nRF52832, Cortex-M4F, 64 KB RAM):

• Zephyr is generally the most practical choice.
• RT-Thread can also work with a minimal configuration.
• Apache NuttX provides the richest functionality but has higher memory requirements, especially when using its ELF loader.

Gemini:

While many microcontrollers natively run as a single, statically-linked binary, a few lightweight, open-source RTOSes provide native, structured subsystems designed explicitly to parse, load, relocate, and run compiled executable files (usually ELF files) directly from a file system.

The most prominent lightweight RTOSes that natively support dynamic loading of modules include:

---

1. Zephyr RTOS (LLEXT Subsystem)

Zephyr includes a highly polished framework specifically for this purpose called LLEXT (Linkable Loadable Extensions).

• How it handles files: It allows the application to load standard, relocatable ELF files directly from the device's file system (llext_fs_loader) or from a memory buffer (llext_buf_loader).
• Features: It takes care of parsing the ELF headers, allocating heap memory for the code, and performing runtime symbol relocation so the dynamic module can call core Zephyr APIs (like threads, semaphores, and drivers). It even manages initialization arrays (.init_array) inside the binary.

2. RT-Thread (dlmodule)

RT-Thread (a very popular open-source RTOS in IoT devices) features a dedicated kernel-space component called dlmodule (formerly known as the Application Module).

• How it handles files: It compiles modules independently of the main kernel into custom executable extensions:

• .mo files: Executable dynamic modules. When loaded, the RT-Thread kernel automatically provisions a main thread to execute the module's code and can even pass command-line arguments (argc, argv).

• .so files: Shared libraries that can reside in RAM to be called by other modules.

• Features: It integrates seamlessly with RT-Thread’s virtual file system (DFS) and POSIX layer, enabling standard dlopen(), dlsym(), and dlclose() functionality on resource-constrained microcontrollers.

3. NuttX (BINFMT & ELF/NXFLAT)

Apache NuttX is a deeply POSIX-compliant RTOS designed to mimic Linux but run on microcontrollers. Because of its POSIX nature, dynamic loading is one of its core architectural features.

• How it handles files: NuttX uses a BINFMT (Binary Format) subsystem that registers loaders for different file types. It supports standard ELF binaries and NXFLAT—a highly optimized, lightweight, position-independent binary format unique to NuttX that allows executing code directly from Flash (XIP) to save RAM.
• Features: You can store compiled programs on a FAT or LittleFS file system, type the application name into the NuttX Shell (NSH), and the OS will parse the file, link it against the kernel symbols, and run it as an independent task.

4. Contiki-NG (ELF Loader)

If your hardware is incredibly constrained (e.g., old 8-bit or 16-bit MCUs, or ultra-low-power wireless sensor nodes), Contiki-NG is a lightweight OS designed for the "Internet of Things."

• How it handles files: It features an internal, highly optimized ELF loader module.
• Features: Contiki-NG utilizes a specialized "cle" (Compact Linker for Embedded systems) approach. It reads standard ELF object files over a file system or network, minimizes RAM overhead during relocation, and ties the loaded functions back into Contiki's cooperative protothreads architecture.

---

Comparison Summary

RTOS	Primary Binary Format	Best Suited For	Key Strength
Zephyr	ELF (.elf)	Modern ARM Cortex-M/RISC-V	Excellent ecosystem, modular LLEXT architecture
RT-Thread	Custom ELF (.mo, .so)	IoT & Multimedia Microcontrollers	Offers full dlopen functionality, boots an automatic main thread
NuttX	ELF / NXFLAT	Systems requiring strict Linux/POSIX parity	True process/task segregation, NXFLAT allows execution from Flash
Contiki-NG	ELF (.co)	Ultra-low power, deeply constrained 8/16/32-bit MCUs	Incredibly tiny footprint, built for wireless nodes

A note on FreeRTOS: While FreeRTOS does not have an official dynamic loader built into the main source tree, the community frequently accomplishes this by importing third-party open-source libraries (like the standalone elfloader library) to achieve identical results on top of FreeRTOS tasks.

DeepSeek:

Several lightweight RTOSes support dynamic loading of executable modules or files. Here are the most notable options:

RT-Thread

RT-Thread provides a robust Dynamic Module (dlmodule) feature that supports loading two types of files:

• .mo files: Executable modules that can be loaded and run, automatically creating a main thread that executes the main() function.
• .so files: Dynamic libraries that can be loaded into memory and used by other programs or modules.

The kernel exports functions to dynamic modules using the RTM_EXPORT() macro, and you can list all exported symbols with the list_symbols command. Dynamic modules are compiled with Position-Independent Code (PIC) flags (-fPIC) to support runtime loading.

Apache NuttX

NuttX offers two approaches for dynamic loading:

NXFLAT Binary Format: A customized and simplified binary format designed for deeply embedded environments without MMU. Key features include:

• Dynamically links separately linked programs to the base NuttX code
• Supports executing programs in place (XIP) from flash
• Only copies the .data section to RAM when using ROMFS

ELF Loading: NuttX also supports loading standard ELF binaries dynamically, making it very Unix/Linux-like for microcontrollers.

Zephyr RTOS

Zephyr introduced Linkable Loadable Extensions (LLEXT) starting from v3.5.0. This feature allows you to:

• Load binaries containing function implementations at runtime (like dynamically loading a library)
• Replace models and switch between inference runtimes without reflashing devices
• Load ELF files encoded in hex format directly from the shell

The LLEXT feature is particularly useful for AI/ML applications where models and runtimes need frequent updates during development. Currently, LLEXT primarily supports ARMv7 architecture.

Azure RTOS ThreadX

ThreadX provides a Module feature that enables dynamic loading and unloading of application modules at runtime. Notable characteristics:

• Supports memory protection using the Memory Protection Unit (MPU)
• Loaded modules run in a protected environment with restricted memory access
• Includes demonstration of trapping unauthorized memory writes by modules

PX5 RTOS

PX5 offers a MODULES technology that natively supports dynamic loading of separately built application modules. Key advantages:

• Extremely small footprint: min 3.5KB Flash and 2.5KB RAM
• Supports execution isolation via MPU/MMU memory protection
• Leverages Pointer/Data Verification (PDV) for runtime safety verification

Other Options

Part Only RTOS: A research project designed for 8-bit ATmega2560 microcontrollers that can dynamically load .bin modules from an SD card, with a kernel occupying less than 2.5KB of Flash.

eRTOS: Supports Real-Time Dynamic Link Libraries (RTDLLs) that can be loaded either implicitly (through export libraries) or explicitly (using LoadLibrary calls).

---

When choosing an RTOS, consider factors like memory footprint, architecture support (ARM, RISC-V, Xtensa, etc.), MMU/MPU availability for memory protection, and the complexity of your dynamic loading requirements.

ChatGPT:

How does PebbleOS implement package installation and removal?

PebbleOS supports installing and removing application packages by managing each application as an independent package, separate from the operating system itself. Unlike a typical MCU firmware, where all functionality is linked into a single binary, PebbleOS includes an Application Manager responsible for package management.

The architecture is roughly as follows:

Flash
├── PebbleOS (fixed)
│
├── System Apps
│
├── App Database
│     ├── App A
│     │    ├── Executable
│     │    └── Resources
│     │
│     ├── App B
│     │    ├── Executable
│     │    └── Resources
│     │
│     └── Watchface
│
└── Free Space

1. Application Manager

PebbleOS contains components such as an Application Manager, Application Installation Manager, and Process Manager.

Their responsibilities include:

• Managing the list of installed applications
• Allocating and reclaiming flash storage
• Managing application UUIDs
• Launching applications
• Installing applications
• Removing applications
• Updating applications

---

2. PBW Package Format

Applications are distributed as .pbw packages.

PBW Package
├── Manifest
├── Executable
└── Resource Pack

When installed, the package is stored in flash as an independent application:

Flash

App A
├── Code
├── Manifest
└── Resources

Each application contains its own executable and resources.

---

3. Independent Application Loading

Applications are separate from the operating system.

PebbleOS
    ↓
Application Manager
    ↓
Load Application
    ↓
Start Process

When launching an application, PebbleOS sets up:

• Code location
• Resource location
• Heap
• Stack

before transferring execution to the application.

This allows applications to be installed or replaced without reflashing the operating system.

---

4. Flash Storage Management

PebbleOS manages flash storage as a collection of application packages.

Initially:

PebbleOS

App1

App2

App3

Free Space

After removing an application:

PebbleOS

App1

Free Space

App3

When a new application is installed:

PebbleOS

App1

App4

App3

The Application Manager reuses or reorganizes available flash space as needed.

---

5. The Phone Acts as the Package Manager

One interesting aspect of PebbleOS is that much of the package management logic resides on the paired smartphone.

App Store
    ↓
Phone
    ↓
PBW Package
    ↓
Bluetooth
    ↓
Watch

The watch itself mainly performs:

• Installation
• Removal
• Application launching

If storage becomes full, the phone decides which applications remain installed and can remove existing packages before installing new ones.

---

Comparison with FreeRTOS

A typical FreeRTOS-based system follows this model:

Bootloader
    ↓
firmware.bin
    ↓
Replace entire firmware

PebbleOS instead uses:

PebbleOS (fixed)
    ↓
Application Manager
    ↓
Application Package
    ↓
Install / Remove / Launch

This architecture keeps the operating system fixed while allowing applications and watchfaces to be independently installed, removed, updated, and launched without reflashing the entire firmware.

Gemini:

PebbleOS (the FreeRTOS-based operating system that powered Pebble smartwatches) implemented a unique, resource-constrained model for package installation and removal. Because the hardware had highly limited RAM (often under 128KB) and a relatively small amount of external Flash storage, it relied heavily on a companion smartphone app to act as its heavy-lifter, installer, and storage locker.

Here is a breakdown of how PebbleOS handled the installation, execution, and removal of packages (applications and watchfaces).

---

1. The Package Format: .pbw (Pebble Bundle Wrapper)

Before an app can be installed, it exists as a .pbw file. A .pbw file is essentially a renamed ZIP archive that contains:

• appinfo.json: Metadata including the app's UUID, name, company, version, and required capabilities.
• pebble-app.bin: The compiled ARM Cortex-M machine code (binary) for the watch app.
• pebble-js-app.js (Optional): JavaScript code meant to run on the phone (via PebbleKit JS) to handle internet requests, GPS, or heavy computations.
• resources.pbpack: A binary bundle of localized strings, images, and fonts compiled specifically for the watch's display architecture.

---

2. The Installation Process

PebbleOS itself did not have the storage or processing power to parse complex archives, manage a large marketplace, or store dozens of apps simultaneously. Instead, the installation pipeline was split between the phone and the watch.

Step A: The Phone App Handler

1. When a user downloads a .pbw file (from the app store or via sideloading), the Pebble Mobile App interceptively handles the file.
2. The phone app unzips the bundle, reads appinfo.json, and registers the app's unique UUID into the phone's local database ("App Locker").
3. If the app includes a JavaScript file (pebble-js-app.js), the phone isolates this code to run within the mobile app's background environment.

Step B: Transmission via Pebble Protocol (PPo)

1. The phone establishes a Bluetooth Serial Port Profile (SPP) or BLE connection with the watch.
2. Using a proprietary multiplexed framing protocol (Pebble Protocol), the phone alerts the watch that an app installation is starting.
3. The phone streams the raw bytes of pebble-app.bin and resources.pbpack over Bluetooth.

Step C: Watch-Side Allocation and Storage

1. Flash Slot Assignment: The PebbleOS storage manager assigns the incoming app a dedicated, sequential structural block (a "slot") in the external SPI Flash memory.
2. Writing to Flash: As the bytes arrive over Bluetooth, PebbleOS writes the raw machine binary and the resource pack directly to the assigned flash slot.
3. Registry Update: The OS updates its internal app registry table (stored in a persistent region of flash), linking the app's UUID to its physical memory address bounds inside the flash.

---

3. Dynamic Loading & Execution (The "App Locker" System)

Early versions of PebbleOS could only hold 8 apps at a time. Later versions introduced a dynamic paging system that treated the watch's flash storage as a cache and the smartphone's App Locker as the primary hard drive.

• Launching an App: When you select an app, PebbleOS checks if the binary is present in the watch's internal flash storage.
• On-Demand Fetching: If the app is in your "App Locker" but not currently on the watch's flash, the watch issues an asynchronous request over Bluetooth: "Send binary for UUID X." The phone streams it over, PebbleOS flashes it to an available slot, and the app launches.
• Execution in RAM: When launched, PebbleOS allocates a tiny, static chunk of RAM (usually around 24KB–64KB depending on the watch model) for the app's heap and stack. It loads the .text (code) and .data sections into RAM. Because the ARM Cortex-M architecture allows it, some variations allowed executing code directly from flash via XIP (Execute-In-Place) to conserve RAM, though resource assets were kept separate.

---

4. The Removal Process

Because apps are self-contained within specific, isolated flash slots and managed via a central UUID lookup registry, removal is incredibly lightweight and fast.

Triggered by the Phone or Cache eviction

When a user deletes an app from their phone, or if the watch runs out of flash slots and needs to overwrite an old app to make room for a new one:

1. Registry Invalidation: PebbleOS locates the app's entry in the internal registry table using its UUID. It marks that entry as invalid or deleted.
2. Flash Zeroing/Freeing: The blocks associated with that app's slot are marked as "free." PebbleOS does not necessarily spend CPU cycles erasing the entire sector immediately; it simply marks it as overwritable. The next time a .pbw is streamed to the watch, the storage manager will cleanly overwrite those blocks.
3. RAM Purging: If the app is currently running, the OS terminates the task, completely clears and zeroes out the allocated application RAM region, and reclaims it for the system or the next application.

How does Rockbox OS implement its internal filesystem, dynamic executable loading, and dynamic user data storage?

---

ChatGPT Answer

Rockbox differs from a typical RTOS. It is a single firmware image that provides a plugin system and makes extensive use of the device's filesystem.

The three features can be summarized as follows:

Feature	Supported	Implementation
Internal filesystem	✔	Uses the device's FAT filesystem directly
Dynamic executable loading	✔	Loads Rockbox plugins (.rock files)
Dynamic user data storage	✔	Stores configuration and user data as regular files

---

1. Internal Filesystem

Rockbox does not implement its own filesystem.

Instead, it uses the storage device's existing filesystem, typically:

• FAT16
• FAT32

A typical directory layout looks like this:

/
    Music/
    Podcasts/
    Pictures/
    .rockbox/

The .rockbox directory contains system resources such as:

.rockbox/
    fonts/
    icons/
    themes/
    codecs/
    plugins/
    rocks/
    langs/

---

2. Dynamic Executable Loading

Yes, Rockbox supports dynamic loading of executable modules.

However, unlike Linux, it does not load ELF executables.

Instead, it loads Rockbox-specific plugin (.rock) files.

The execution flow is:

Menu
    ↓
plugin_load()
    ↓
plugin.rock
    ↓
Load into RAM
    ↓
plugin_start()

Every plugin exports an entry point:

enum plugin_status plugin_start(const void *parameter);

The operating system exposes its services through a plugin API table, allowing plugins to access filesystem, display, audio, input, and other services without directly linking against the firmware.

For example:

rb->lcd_puts(...);
rb->open(...);
rb->read(...);

This architecture is conceptually similar to a simplified dlopen() mechanism.

---

3. Plugin Architecture

The overall structure is:

Rockbox
    ↓
Plugin Loader
    ↓
plugin.rock
    ↓
Plugin API
    ↓
LCD
Audio
Filesystem
Buttons

Rather than calling internal firmware functions directly, plugins communicate through the exported API table.

This provides ABI compatibility while allowing plugins to be loaded independently.

---

4. Dynamic User Data Storage

Rockbox fully supports dynamic file creation and modification.

Examples include:

settings.cfg
playlist.m3u
bookmark.bmark
database.tcd
theme.cfg

Plugins are also free to create their own files, for example:

plugin/
    savegame.dat
    highscore.dat
    config.cfg

These files are stored directly in the filesystem.

---

5. Dynamic Resources

Rockbox also treats resources as regular files.

Examples include:

.rockbox/fonts/
.rockbox/themes/
.rockbox/icons/
.rockbox/langs/

New fonts, themes, icons, and language packs can be added or removed simply by copying or deleting files—no firmware recompilation is required.

---

Overall Architecture

Flash

Rockbox Firmware
        │
        ▼
Filesystem (FAT)
        │
 ├── Music
 ├── Themes
 ├── Fonts
 ├── Plugins (.rock)
 ├── Config (.cfg)
 └── User Data
        │
        ▼
Plugin Loader
        │
        ▼
RAM
        │
        ▼
plugin_start()

Summary

Although Rockbox is not an RTOS, it provides several advanced features that are uncommon in embedded firmware:

• ✔ Filesystem-based resource management
• ✔ Dynamic loading of .rock plugin modules
• ✔ Plugin API for operating system services
• ✔ User configuration and data stored as regular files
• ✔ Fonts, themes, icons, and language packs that can be added or removed without rebuilding the firmware

Unlike PebbleOS, Rockbox does not use a package management system. Instead, it relies on a combination of filesystem-based resources and a plugin loader to provide dynamic extensibility.

[gamelaster]