Skip to content

ffi: add LibFFI-based foreign function interface module#398

Open
jow- wants to merge 6 commits into
masterfrom
lib-introduce-ffi
Open

ffi: add LibFFI-based foreign function interface module#398
jow- wants to merge 6 commits into
masterfrom
lib-introduce-ffi

Conversation

@jow-

@jow- jow- commented Apr 12, 2026

Copy link
Copy Markdown
Owner

Introduce the ffi module, providing seamless integration with C libraries through a LuaJIT-inspired API. The module combines a C declaration parser with libffi-based function calling for ucode/C interop.

Features:

  • C declaration parsing via ffi.cdef()
  • C data object creation (structs, arrays, primitives) via ffi.ctype()
  • LibFFI-based function calling with full ABI handling
  • Function wrapping via ffi.C.wrap() for callable C functions
  • Dynamic library loading via ffi.dlopen() and ffi.C.dlsym()
  • Type information queries (sizeof, alignof, offsetof)
  • Pointer operations (ptr, deref, get, set, slice)
  • String conversion helpers (ffi.string, cdata.slice)

Implementation:

  • C parser adapted from LuaJIT FFI (lj_cparse.c -> uc_cparse.c)
  • Type system and conversions ported to ucode VM conventions
  • LibFFI used exclusively for all call/closure operations
  • Resource types: "ffi.ctype" and "ffi.clib"
  • Full JSDoc documentation for API surface

Tests:

  • Full unit tests in tests/ffi/ covering all API functions
  • Callback support tested with qsort (int/string/struct arrays)
  • Complex nested structs (sockaddr_in, in_addr) tested
  • Path-based access and dict initialization tested
  • Unified test runner (run_tests.sh) for all tests

Usage:
import * as ffi from 'ffi';
ffi.cdef('size_t strlen(const char *)');
let strlen = ffi.C.wrap('size_t strlen(const char *)');
let len = strlen('hello').get(); // => 5

@systemcrash

Copy link
Copy Markdown
Contributor

I get the following after brew install libffi:

/Volumes/Dev/git/ucode/lib/ffi/uc_ctype.h:27:3: error: redefinition of typedef 'GCcdata' is a C11 feature [-Werror,-Wtypedef-redefinition]
   27 | } GCcdata;
      |   ^
/Volumes/Dev/git/ucode/lib/ffi/uc_def.h:611:24: note: previous definition is here
  611 | typedef struct GCcdata GCcdata;
      |                        ^
/Volumes/Dev/git/ucode/lib/ffi/uc_cparse.c:190:19: error: format string is not a string literal [-Werror,-Wformat-nonliteral]
  190 |         xvasprintf(&msg, em, argp);
      |                          ^~
2 errors generated.

Without libffi it's the same as CI:

-- Configuring done (0.6s)
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
ffi_include_dir
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
ucode_include_dir
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi
   used as include directory in directory /Volumes/Dev/git/ucode/lib/ffi

CMake Error in lib/ffi/CMakeLists.txt:
  Found relative path while evaluating include directories of "ffi":

    "ffi_include_dir-NOTFOUND"

@jow- jow- force-pushed the lib-introduce-ffi branch 17 times, most recently from fc5d64b to 7ab02ac Compare April 15, 2026 11:39
@jow-

jow- commented Apr 15, 2026

Copy link
Copy Markdown
Owner Author

After some further iterations it should work now, at least CI is green and a couple of smoke tests work as well.

@systemcrash

Copy link
Copy Markdown
Contributor

Builds OK now on macos - not done any testing of ffi, however.

-- Checking for module 'libffi'
--   Found libffi, version 3.4-rc1
-- Performing Test HAVE_FFI_BAD_ARGTYPE
-- Performing Test HAVE_FFI_BAD_ARGTYPE - Success
-- Configuring done (1.6s)
-- Generating done (0.1s)
...
[ 66%] Building C object CMakeFiles/ffi_lib.dir/lib/ffi/uc_cparse.c.o
[ 68%] Building C object CMakeFiles/ffi_lib.dir/lib/ffi/uc_ctype.c.o
[ 70%] Building C object CMakeFiles/ffi_lib.dir/lib/ffi/uc_cconv.c.o
[ 72%] Building C object CMakeFiles/ffi_lib.dir/lib/ffi/uc_cdata.c.o
[ 74%] Building C object CMakeFiles/ffi_lib.dir/lib/ffi/ffi.c.o
[ 76%] Linking C shared module ffi.so
[ 76%] Built target ffi_lib

@jow- jow- force-pushed the lib-introduce-ffi branch 7 times, most recently from 5ee6a08 to d280697 Compare April 21, 2026 09:09
jow- added 3 commits April 21, 2026 11:09
Add an `unused` macro expanding to `__attribute__((unused))` to
simplify marking unused variables or function parameters.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
In order to ensure reliable freeing of resource type prototypes in complex
prototype chains with per-resource prototypes, register the created type
prototype object with the VM value pool.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
jow- added 3 commits April 21, 2026 11:09
Add support for storing an optional per-instance prototype slot in
extended resource values. This allows individual resource instances
to override the type-level prototype with their own prototype object.

Changes:
- Add hasproto flag to uc_resource_ext_t to track presence of instance proto
- Add ucv_resource_new_prototyped() constructor for resources with instance proto
- Add ucv_resource_hasproto(), ucv_resource_proto_get(), ucv_resource_proto_set()
  helpers for managing instance prototypes
- Add ucv_resource_create_prototyped() convenience wrapper
- Update GC mark and cleanup to handle instance prototype slot
- Update ucv_prototype_get/set() to check for instance prototype before
  falling back to type-level prototype

When an instance prototype is provided, it is chained to the type's
prototype, ensuring proper prototype chain lookup.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Jo-Philipp Wich <jo@mein.io>
Introduce the ffi module, providing seamless integration with C libraries
through a LuaJIT-inspired API. The module combines a C declaration parser
with libffi-based function calling for ucode/C interop.

Features:
- C declaration parsing via ffi.cdef()
- C data object creation (structs, arrays, primitives) via ffi.ctype()
- LibFFI-based function calling with full ABI handling
- Function wrapping via ffi.C.wrap() for callable C functions
- Dynamic library loading via ffi.dlopen() and ffi.C.dlsym()
- Type information queries (sizeof, alignof, offsetof)
- Pointer operations (ptr, deref, get, set, slice)
- String conversion helpers (ffi.string, cdata.slice)

Implementation:
- C parser adapted from LuaJIT FFI (lj_cparse.c -> uc_cparse.c)
- Type system and conversions ported to ucode VM conventions
- LibFFI used exclusively for all call/closure operations
- Resource types: "ffi.ctype" and "ffi.clib"
- Full JSDoc documentation for API surface

Tests:
- Full unit tests in tests/ffi/ covering all API functions
- Callback support tested with qsort (int/string/struct arrays)
- Complex nested structs (sockaddr_in, in_addr) tested
- Path-based access and dict initialization tested
- Unified test runner (run_tests.sh) for all tests

Usage:
    import * as ffi from 'ffi';
    ffi.cdef('size_t strlen(const char *)');
    let strlen = ffi.C.wrap('size_t strlen(const char *)');
    let len = strlen('hello').get();  // => 5

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
@jow- jow- force-pushed the lib-introduce-ffi branch from d280697 to 848d8a0 Compare April 21, 2026 09:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants