Skip to content

PoC: Use Firebird embedded mode for CI on all platforms#39

Closed
fdcastel wants to merge 2 commits into
asfernandes:mainfrom
fdcastel:issue-33
Closed

PoC: Use Firebird embedded mode for CI on all platforms#39
fdcastel wants to merge 2 commits into
asfernandes:mainfrom
fdcastel:issue-33

Conversation

@fdcastel
Copy link
Copy Markdown
Contributor

@fdcastel fdcastel commented Feb 24, 2026

Summary

Replaces Firebird server installation and service management with embedded mode using cached Firebird distributions on all three platforms (Linux, Windows, macOS).

This eliminates the need for sudo installer, install_service.bat, launchctl, isql password setup, and running Firebird as a system service during CI. Tests use local filesystem database paths instead of TCP connections.

What changed

Linux (build-linux)

Before After
wget + tar + install.sh -silent wget + tar (extract buildroot.tar.gz only)
systemctl stop/start + isql password setup Not needed — embedded auth
No Firebird caching actions/cache for /tmp/firebird
Template: CMakeUserPresets.json.posix.template Template: CMakeUserPresets.json.embedded-posix.template
FBCPP_TEST_SERVER=localhost (TCP) No FBCPP_TEST_SERVER (embedded — local filesystem paths)
LD_LIBRARY_PATH=/tmp/firebird/lib + FIREBIRD=/tmp/firebird at test time

Windows (build-windows)

Before After
Invoke-WebRequest + 7z + install_service.bat Invoke-WebRequest + Expand-Archive
isql password setup + install_service.bat Not needed — embedded auth
No Firebird caching actions/cache for C:\Firebird
Template: CMakeUserPresets.json.windows.template Template: CMakeUserPresets.json.embedded-windows.template
FBCPP_TEST_SERVER=localhost (TCP) No FBCPP_TEST_SERVER (embedded)
Providers = Engine13 in firebird.conf (force embedded engine)
Remove vcpkg's client-only fbclient.dll from build tree (force PATH resolution to full Firebird distribution)

macOS (build-macos)

Before After
wget + sudo installer -pkg wget + pkgutil --expand + tar (no sudo needed)
launchctl unload/load + isql password setup Not needed — embedded auth
No Firebird caching actions/cache for /tmp/firebird
Template: CMakeUserPresets.json.posix.template Template: CMakeUserPresets.json.embedded-posix.template
FBCPP_TEST_SERVER=localhost (TCP) No FBCPP_TEST_SERVER (embedded)
Replace vcpkg's client-only libfbclient.dylib with full distribution version
DYLD_LIBRARY_PATH + FIREBIRD + RPATH configuration for dependencies

New files

  • CMakeUserPresets.json.embedded-posix.template — same as posix template, without FBCPP_TEST_SERVER (shared by Linux and macOS)
  • CMakeUserPresets.json.embedded-windows.template — same as windows template, without FBCPP_TEST_SERVER

How it works

The existing test code in TestUtil.cpp already supports embedded mode — when FBCPP_TEST_SERVER is unset, testServerPrefix stays empty and database URIs become plain filesystem paths, triggering the embedded engine provider.

At runtime, LD_LIBRARY_PATH (Linux) / DYLD_LIBRARY_PATH (macOS) / PATH (Windows) is configured so the process loads the full Firebird distribution's client library (with embedded engine support) instead of vcpkg's client-only version. The FIREBIRD env var tells the engine where to find plugins/, firebird.conf, and security5.fdb.

On Windows and macOS, an extra step is needed because vcpkg installs its client-only library into the build tree, which takes priority in library search order. The workflow removes/replaces the vcpkg copy so the full Firebird distribution library is loaded instead.

Build time comparison

Timing data from issue-33 branch runs on fdcastel/fb-cpp:

Job Before (server mode) After (embedded + cached) Difference
build-linux 10m 21s 11m 3s +42s
build-windows 5m 14s 3m 53s -1m 21s
build-macos 15m 51s 9m 57s -5m 54s

On cache hit, the Firebird download step is skipped entirely. The Install Firebird, Configure Firebird, and Start Firebird Server steps are all eliminated across all platforms.

Linux shows a slight increase due to runner variance — the Firebird setup was already fast there. Windows and macOS see significant improvements from eliminating service setup.

Notes

  • No C++ code changes — the test infrastructure already supported embedded mode via the FBCPP_TEST_SERVER env var
  • The pull-request.yml workflow has the same changes applied

Workflow runs

@fdcastel fdcastel changed the title PoC: Use Firebird embedded mode for Linux and Windows CI Use Firebird embedded mode for CI on all platforms Feb 24, 2026
@fdcastel
Copy link
Copy Markdown
Contributor Author

@asfernandes This is just a proof of concept and not priority. Just to demonstrate my ideas from #33.

Whenever you have some time to review it, I’d really appreciate your thoughts.

I asked Claude to measure the time difference twice, and in both runs the Linux times were actually higher when using cache. I’m having trouble understanding why 😟

@fdcastel fdcastel changed the title Use Firebird embedded mode for CI on all platforms PoC: Use Firebird embedded mode for CI on all platforms Feb 24, 2026
@fdcastel
Copy link
Copy Markdown
Contributor Author

image

On the bright side, I’ve had time to reflect on all my life choices.

@fdcastel
Copy link
Copy Markdown
Contributor Author

Ugh!

Both Linux and macOS are caching build/Release/vcpkg_installed, but the preset in use is posix-ninja-debug, which generates output in Debug.

As a result, the cache is targeting the wrong directory (Release), so vcpkg ends up rebuilding everything (~9 minutes) on every run.

Working on it.

@fdcastel
Copy link
Copy Markdown
Contributor Author

Here's the summary of the investigation and fix:

Root cause: The vcpkg cache paths in main.yml for Linux and macOS were set to build/Release/vcpkg_installed, but CMakePresets.json explicitly sets VCPKG_INSTALLED_DIR to vcpkg_installed (shared across all presets). This path mismatch meant the cache was saving/restoring the wrong directory, causing vcpkg to rebuild all ~60 packages from source on every run (~9 min).

Windows worked correctly because its cache path was already vcpkg_installed.

Fix (commit e336cf4):

  1. Changed Linux/macOS cache paths from build/Release/vcpkg_installed to vcpkg_installed in main.yml
  2. Added missing vcpkg cache steps to pull-request.yml for all platforms
  3. Bumped cache key version to v3 to invalidate stale entries

"Configure CMake" step timing improvement (with cache hit):

Platform Before After Improvement
Linux 532s (8m52s) 17s -8m35s
macOS 520s (8m40s) 18s -8m22s
Windows 42s 51s (already working)

@fdcastel
Copy link
Copy Markdown
Contributor Author

fdcastel commented Mar 9, 2026

When you have a moment, please share your thoughts on this PR. It should significantly speed up the build and testing process.

@asfernandes
Copy link
Copy Markdown
Owner

My GitHub account is with a problem (wrong billing) and no actions are working.
Anyway, looks like the decreased time is to correction of the cache path (and not use of embedded), isn't it?

fdcastel added 2 commits March 9, 2026 21:57
Replace Firebird server installation and service management with
embedded mode using cached Firebird distributions on all platforms.

Linux:
- Download Firebird tarball, extract buildroot.tar.gz to /tmp/firebird
- Set LD_LIBRARY_PATH and FIREBIRD env vars at test time
- Cache /tmp/firebird with actions/cache

Windows:
- Download Firebird zip, extract to C:\Firebird
- Configure Providers=Engine13 in firebird.conf (force embedded mode)
- Remove vcpkg client-only fbclient.dll from build tree so PATH
  resolves to full Firebird distribution DLL
- Cache C:\Firebird with actions/cache

macOS:
- Extract .pkg payload with pkgutil + tar (no sudo installer needed)
- Replace vcpkg client-only libfbclient.dylib with full distribution
  version and configure RPATH for dependencies
- Set FIREBIRD and DYLD_LIBRARY_PATH at test time
- Cache /tmp/firebird with actions/cache

New CMakeUserPresets templates for embedded mode omit FBCPP_TEST_SERVER
so tests use local filesystem paths instead of TCP connections.

Closes asfernandes#33.
The vcpkg cache was configured with build/Release/vcpkg_installed but
CMakePresets.json sets VCPKG_INSTALLED_DIR to build/vcpkg_installed
(shared across all presets). This caused a cache miss on every run,
rebuilding all vcpkg packages from source (~9 min).

Fixed:
- main.yml: use build/vcpkg_installed for Linux and macOS (matching
  VCPKG_INSTALLED_DIR in CMakePresets.json)
- pull-request.yml: add missing vcpkg cache steps for all platforms
- Bump cache key version to invalidate stale entries
@fdcastel
Copy link
Copy Markdown
Contributor Author

Anyway, looks like the decreased time is to correction of the cache path (and not use of embedded), isn't it?

Hmm. I have to admit I didn’t measure the timings without the embedded version. The cache path issue was only discovered later.

Anyway, is there any reason NOT to use the embedded version instead of the installed/service version? (Sorry if you already mentioned it -- it’s been a few weeks since I worked on this.)

@fdcastel
Copy link
Copy Markdown
Contributor Author

P.S.: Rebased onto latest main.

@asfernandes
Copy link
Copy Markdown
Owner

Anyway, is there any reason NOT to use the embedded version instead of the installed/service version? (Sorry if you already mentioned it -- it’s been a few weeks since I worked on this.)

It's clear seeing lines added and removed in this PR that it make things more complex.

@fdcastel
Copy link
Copy Markdown
Contributor Author

Anyway, is there any reason NOT to use the embedded version instead of the installed/service version? (Sorry if you already mentioned it -- it’s been a few weeks since I worked on this.)

It's clear seeing lines added and removed in this PR that it make things more complex.

Got it. I’ll just fix the cache paths then, in a separate PR and close this one. Give me a few hours.

@fdcastel
Copy link
Copy Markdown
Contributor Author

Moved to #43.

@fdcastel fdcastel closed this Mar 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants