Skip to content

Commit a4f71f3

Browse files
lrgirdwolgirdwood
authored andcommitted
module_adapter: add README for library and iadk subsystems
Adds documentation detailing the ZEPHYR user space and Intel ADSP third party integration mechanisms via the Module Adapter architecture. Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
1 parent 0b5aaa3 commit a4f71f3

2 files changed

Lines changed: 165 additions & 0 deletions

File tree

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# Intel Audio Development Kit (`module_adapter/iadk`)
2+
3+
The `iadk` directory provides the Module Adapter implementation for external 3rd-party audio algorithms developed using the **Intel Audio Development Kit (IADK)**.
4+
5+
Unlike the native SOF `module_interface` API (written primarily in C), the IADK modules are object-oriented C++ classes that derive from `intel_adsp::ProcessingModuleInterface`. The SOF `IadkModuleAdapter` acts as a C++ to C "glue layer" that wraps these IADK methods so they can be natively plugged into the SOF `module_adapter` pipeline without modification to the module's pre-compiled binary.
6+
7+
## Architecture and Class Hierarchy
8+
9+
The system defines an `IadkModuleAdapter` class which internally holds an instance of the 3rd-party `ProcessingModuleInterface`.
10+
11+
```mermaid
12+
classDiagram
13+
class SOF_ModuleAdapter {
14+
+init
15+
+process
16+
+bind
17+
}
18+
19+
class iadk_wrapper {
20+
iadk_wrapper_process
21+
iadk_wrapper_init
22+
}
23+
24+
class IadkModuleAdapter {
25+
-processing_module_
26+
+IadkModuleAdapter_Init
27+
+IadkModuleAdapter_Process
28+
+IadkModuleAdapter_SetConfiguration
29+
}
30+
31+
class ProcessingModuleInterface {
32+
+Init
33+
+Process
34+
+SetConfiguration
35+
+Reset
36+
}
37+
38+
class IADK_3rdParty_Algorithm {
39+
+Init
40+
+Process
41+
}
42+
43+
SOF_ModuleAdapter --> iadk_wrapper : C Function Pointers
44+
iadk_wrapper --> IadkModuleAdapter : Instantiates and Wraps
45+
IadkModuleAdapter --> ProcessingModuleInterface : Polymorphic Interface
46+
ProcessingModuleInterface <|-- IADK_3rdParty_Algorithm : Inherits
47+
```
48+
49+
## System Agent and Instantiation Flow
50+
51+
Because the actual module resides in an external binary, it requires a "System Agent" to correctly instantiate the C++ objects during the component's `init` phase.
52+
53+
1. The OS host driver sends an IPC `INIT_INSTANCE` command for the module.
54+
2. The `system_agent_start()` function intercepts this, invokes the dynamic module's `create_instance` entry point (which invokes a `ModuleFactory`).
55+
3. The `SystemAgent` deduces the pin count (interfaces) and initial pipeline configurations using `ModuleInitialSettingsConcrete`.
56+
4. The factory allocates the concrete algorithm and checks it back into SOF through `SystemAgent::CheckIn`.
57+
58+
```mermaid
59+
sequenceDiagram
60+
participant IPC
61+
participant SA
62+
participant Fac
63+
participant Mod
64+
65+
IPC->>SA: Trigger Mod Creation
66+
SA->>Fac: CI invokes create_instance
67+
Fac->>Fac: Deduce BaseModuleCfgExt
68+
Fac->>Mod: operator new instantiate
69+
70+
Fac->>SA: SystemAgent CheckIn Module
71+
SA->>Adp: Create new IadkModuleAdapter Module
72+
73+
SA-->>IPC: Return CPP adapter to C Pipeline
74+
```
75+
76+
## Data Buffer Translation
77+
78+
A significant task of `IadkModuleAdapter_Process` is converting SOF's underlying buffer formats to IADK's `InputStreamBuffer` and `OutputStreamBuffer` structures.
79+
80+
Instead of letting the module directly touch the SOF `comp_buffer` (which could change with SOF version updates), the adapter uses the abstraction APIs (`source_get_data` / `sink_get_buffer`) and wraps them:
81+
82+
1. Request raw continuous memory pointers from `source_get_data()`.
83+
2. Construct an `intel_adsp::InputStreamBuffer` pointing to that continuous memory chunk.
84+
3. Call the IADK `processing_module_.Process()`.
85+
4. Release precisely the amount of consumed data using `source_release_data()`.
Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Loadable Library & Userspace Proxy (`module_adapter/library`)
2+
3+
The `library` directory within the module adapter manages the lifecycle and execution isolation for dynamically loaded algorithms, often referred to as "LLEXT modules" or "Userspace Modules".
4+
5+
It acts as a secure intermediary layer between the native SOF/Zephyr kernel execution mode (supervisor mode) and the 3rd-party module running in an isolated user-mode context. By relying on Zephyr's Userspace mechanisms (`k_mem_domain`, `K_USER` threads), a faulting or misbehaving loadable extension cannot crash the entire firmware.
6+
7+
## Architecture & Userspace Sandbox
8+
9+
The Userspace Proxy architecture involves:
10+
11+
- **`struct userspace_context`**: This encapsulates the memory domain (`k_mem_domain`) mapped exclusively for this module (code, rodata, BSS, and private heap memory).
12+
- **`user_work_item`**: A Zephyr `k_work_user` mechanism. IPC configuration commands and data processing calls are packaged into a work item and executed safely inside the memory boundaries via `userspace_proxy_worker_handler`.
13+
14+
```mermaid
15+
graph TD
16+
subgraph SOF Supervisor Domain
17+
P["Pipeline DP Scheduler"]
18+
MADP["Module Adapter Context"]
19+
PROXY["Userspace Proxy (userspace_proxy_invoke)"]
20+
end
21+
22+
subgraph Zephyr Userspace Domain
23+
SYSAGENT["LLEXT System Agent"]
24+
HANDLER["Worker Handler (userspace_proxy_handle_request)"]
25+
MOD["External Loadable Module (struct module_interface)"]
26+
end
27+
28+
P -->|Triggers process| MADP
29+
MADP -->|Invokes Proxy| PROXY
30+
31+
PROXY -->|Packages params & k_work| HANDLER
32+
33+
HANDLER -->|Safe Call Context| MOD
34+
SYSAGENT -.->|Bootstraps| MOD
35+
36+
style SOF Supervisor Domain fill:#1d2951,stroke:#333
37+
style Zephyr Userspace Domain fill:#2e0b1a,stroke:#333
38+
```
39+
40+
## State Transitions & IPC Handling
41+
42+
IPC messages arriving from the host (e.g. `SET_CONF`, `GET_CONF`, `MODULE_INIT`, `MODULE_BIND`) first hit the Module Adapter running in Supervisor Mode. The adapter checks its configuration and invokes the Userspace Proxy. The proxy performs the following context switch:
43+
44+
```mermaid
45+
sequenceDiagram
46+
participant IPC as SOF IPC Task
47+
participant MA as Module Adapter
48+
participant Proxy as Userspace Proxy
49+
participant Worker as Zephyr k_work_user Thread
50+
participant UserMod as LLEXT Module
51+
52+
IPC->>MA: IPC SET_CONF config_id
53+
MA->>Proxy: userspace_proxy_set_configuration
54+
55+
note over Proxy: 1. Package proxy params
56+
note over Proxy: 2. Add Mailbox memory to `k_mem_domain`
57+
note over Proxy: 3. Post `k_work_user` / event
58+
59+
Proxy->>Worker: Context Switch -> User Mode
60+
61+
Worker->>UserMod: module_interface set_configuration
62+
63+
note over UserMod: Parse config payload safely
64+
65+
UserMod-->>Worker: Return status
66+
Worker-->>Proxy: Signal Task Done
67+
68+
note over Proxy: Remove Mailbox memory from domain
69+
70+
Proxy-->>MA: Return execution
71+
MA-->>IPC: Send Reply/Status to Host
72+
```
73+
74+
### Memory Domain Adjustments
75+
76+
A crucial aspect of the userspace proxy is dynamic memory permission elevation:
77+
78+
- Normally, the userspace module can only access its own `heap`, `data`, and `bss` partitions.
79+
- When an IPC like `GET_CONF` requests large IPC buffers, the proxy temporarily adds the hardware `MAILBOX_HOSTBOX` into the module's `k_mem_domain` using `k_mem_domain_add_partition()`.
80+
- Once the userspace thread returns, that hardware window is immediately removed from the memory domain to minimize the vulnerability window.

0 commit comments

Comments
 (0)