how-it-works.md

layout	default
title	How ZigXLL Works

How ZigXLL Works

Some technical rambling about how this all works.

Overview

ZigXLL uses Zig's comptime to automatically generate Excel-compatible wrapper functions. The framework is a Zig package that mushes with user code at build time to produce a complete XLL, ready to run in Excel.

The user never writes any Excel boilerplate. The framework's buildXll() build helper creates a complete XLL by:

1. Taking the user's module containing function_modules tuple
2. Using the framework's xll_builder.zig as the root source file
3. The builder exports all Excel entry points (xlAutoOpen, etc.)
4. The builder discovers and registers user functions at compile time

Core

1. ExcelFunction Wrapper (src/excel_function.zig)

The ExcelFunction() function is a compile-time code generator that takes function metadata and produces a struct containing:

• Excel registration metadata (name, description, type string)
• An impl function with the exact signature Excel expects
• Type conversion logic between Zig types and XLOPER12

Arity

The switch statement on params.len generates the exact number of parameters needed:

• 0 params: fn impl() callconv(.c) *xl.XLOPER12
• 1 param: fn impl(a1: *xl.XLOPER12) callconv(.c) *xl.XLOPER12
• 2 params: fn impl(a1: *xl.XLOPER12, a2: *xl.XLOPER12) callconv(.c) *xl.XLOPER12

Up to 8 are permitted.

Type conversion

extractArg() and wrapResult() handle conversion:

• f64 ↔ xltypeNum (numbers)
• bool ↔ xltypeBool (TRUE/FALSE)
• []const u8 ↔ xltypeStr (strings with UTF-8 conversion)
• [][]const f64 ↔ xltypeMulti (2D arrays/ranges of numbers, empty cells → 0.0)
• *XLOPER12 ↔ raw passthrough (advanced usage)

Optional parameters: Optional types (?f64, ?bool, etc.) detect xltypeMissing and return null.

All conversions use the XLValue wrapper for safety.

2. Function discovery (src/function_discovery.zig)

getAllFunctions() uses comptime reflection to find Excel functions in a module. It scans declarations looking for structs with the is_excel_function marker and builds a comptime array of them.

3. Hook up to Excel (src/xll_builder.zig)

The XLL builder is the root source file for all generated XLLs. It:

1. Imports the framework as "xll_framework"
2. Imports the user module as "user_module"
3. Exports all Excel entry points
4. Exposes user_functions for framework discovery

The user never sees or edits this file. It just hooks up the framework to Excel's machinery.

4. Framework Entry (src/framework_entry.zig)

Function discovery

This runs at compile time to build the complete list of functions. It accesses @import("root") which is the xll_builder.zig, which exposes the user's modules.

This takes the zig defined metadata and calls Excel's xlfRegister function for n functions. Again this is mostly code generated at compile time. A dummy trailing empty string is appended to the registration args to prevent Excel from truncating the argument list.

5. Build helper (build.zig)

buildXll() is called from user's build.zig and handles all the wiring. This is how user build files stay minimal.

6. XLValue Wrapper (src/xlvalue.zig)

XLValue wraps XLOPER12 with type safety.

Memory Management

XLValue tracks whether it owns memory via m_owns_memory. When Excel calls xlAutoFree12(), the framework deallocates any owned memory. Excel memory ownership rules are strict - time will tell if this is right.

UTF-8 Conversion

Excel uses UTF-16 wide strings. XLValue handles conversion:

• fromUtf8String(): UTF-8 → UTF-16 (allocates)
• as_utf8str(): UTF-16 → UTF-8 (allocates)

Flow

Execution of the ADD function in Excel

1. User enters =ADD(1, 2) in Excel
2. Excel looks up the registered exported function add_impl in the XLL DLL (dots in names are replaced with underscores by sanitizeExportName() to avoid Windows GetProcAddress issues)
3. Excel calls add_impl(XLOPER12*, XLOPER12*) with C calling convention
4. The generated impl function:
   • Wraps each XLOPER12 in XLValue
   • Calls extractArg() to convert to f64
   • Calls user's add(f64, f64) function
   • Converts f64 result to XLOPER12 via wrapResult()
   • Returns pointer to XLOPER12
5. Excel displays the result
6. Later, Excel calls xlAutoFree12() to free the returned memory (triggered by the xlbitDLLFree flag set on the returned XLOPER12 by heapXloper())

Comptime function registration

1. User defines function in their module (e.g., my_functions.zig):

   pub const add = ExcelFunction(.{
       .name = "add",
       .func = addImpl,
       // ...
   });

2. User lists that module in main.zig:

   pub const function_modules = .{
       @import("my_functions.zig"),
   };

3. User calls buildXll() in their build.zig, passing their module

4. Framework uses xll_builder.zig as the XLL root, which imports user module

5. ExcelFunction() runs at compile time, generating wrapper structs with @export()

6. getAllFunctions() scans modules and finds all wrappers

7. At runtime, xlAutoOpen() registers each discovered function with Excel

Async Functions

Async functions use Excel's RTD (Real-Time Data) mechanism as a completion notifier — the same pattern used by Excel-DNA. The user writes a normal function with .is_async = true; the framework generates all the RTD and threading machinery at compile time.

Architecture

All async functions share a single built-in RTD server (zigxll.async) and a shared thread pool (4 workers via std.Thread.Pool). Results are stored in a thread-safe cache keyed by "FuncName|arg1|arg2|...".

The key components:

• src/async_infra.zig — Thread pool, topic key building, argument duplication/cleanup
• src/async_handler.zig — Built-in RTD handler implementing onConnect/onDisconnect/onRefreshValue
• src/async_cache.zig — Thread-safe HashMap mapping topic keys to XLOPER12 results with completion flags

Execution flow

When Excel calls an async function:

1. Cache miss (first call)

   • impl builds a topic key from function name + serialized args
   • Checks the cache — miss
   • Duplicates arguments (they must outlive the Excel call)
   • Spawns a worker on the thread pool
   • Sets a pending topic key, then calls xlfRtd to subscribe the cell
   • ConnectData in the RTD handler picks up the pending key (this is synchronous — xlfRtd triggers it before returning)
   • Returns #N/A to the cell via RTD

2. Worker completes

   • Stores the result in the cache (marked as completed)
   • Calls UpdateNotify() on the RTD callback — this tells Excel to recalculate

3. Recalc (cache hit)

   • impl checks cache — hit, completed
   • Returns the value directly (bypassing RTD entirely)
   • Excel drops the RTD subscription since the cell is no longer an RTD formula

4. Subsequent calls with same args

   • Instant cache hit — no thread pool, no RTD, just a direct return

Intermediate values (yield)

If the user function takes *AsyncContext as its last parameter, it can call ctx.yield() to push intermediate values to the cell before the final return. Each yield:

1. Stores the intermediate value in the cache (not marked as completed)
2. Calls UpdateNotify() → Excel recalcs → RefreshData returns the intermediate value
3. Cell updates immediately while computation continues

On the final return, the value is stored as completed. The next recalc returns it directly and the RTD subscription drops.

Comptime wiring

The framework detects async functions automatically:

• ExcelFunction() checks .is_async = true and generates the cache-check / thread-pool / RTD subscription logic in impl
• framework_entry.zig scans all registered functions at comptime; if any are async, it auto-registers the built-in async RTD server during xlAutoOpen
• xll_builder.zig generates combined DllGetClassObject/DllCanUnloadNow exports that dispatch to both user-defined RTD servers and the async RTD server

The user doesn't declare or configure anything beyond .is_async = true.

Pending key mechanism

The tricky part of the RTD integration is mapping topic keys to topic IDs. Excel assigns topic IDs in ConnectData, but the UDF needs to set the topic key before calling xlfRtd. The solution:

1. UDF sets a "pending key" on the handler (a module-level variable)
2. UDF calls xlfRtd, which synchronously triggers ConnectData
3. ConnectData reads the pending key and associates it with the topic ID Excel provides
4. The pending key is cleared

This works because xlfRtd calls ConnectData synchronously on the same thread before returning, and async functions are always registered as non-thread-safe (so only one runs at a time on Excel's main thread).

Performance

Compile time

• Function discovery
• Wrapper generation
• All metadata computed at compile time (no runtime overhead)

Runtime

• Function call: Direct C call, no reflection or indirection
• Type conversion: Minimal overhead (pointer deref and type check)
• Memory allocation: Only for strings and returned values
• Registration: One-time cost at XLL load (although you can safely load the XLL again without restarting Excel)

Caveats

• Maximum 8 parameters per function (Excel limitation is 255, but framework currently supports 0-8)
• Supported types: f64, bool, []const u8, [][]const f64, *XLOPER12
• Windows x86_64 only, which makes sense as XLLs can only run on Windows
• Requires Zig 0.15.1 or later

TODO

Adding new parameter types:

1. Update extractArg() in excel_function.zig
2. Add conversion logic using XLValue methods

Adding new return types:

1. Update wrapResult() in excel_function.zig
2. Add conversion logic using XLValue constructors

Supporting more parameters:

Add more cases to the switch statement in ExcelFunction().