Automate tests against latest libLcm on nuget

.github/workflows/test-windows.yml

@@ -0,0 +1,99 @@
+name: Windows tests
+
+on:
+  push:
+    branches: [main, "ci/**"]
+  pull_request:
+  workflow_dispatch:
+
+concurrency:
+  group: windows-tests-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  FW_VERSION: "9.3.9.1"
+  FW_BUILD: "1439"
+  FW_CODE_DIR: "C:\\Program Files\\FieldWorks9"
+  FW_PROJECTS_DIR: "C:\\ProgramData\\SIL\\FieldWorks\\Projects"
+  FW_INSTALLER_DIR: "C:\\fw-ci\\installer"
+  FW_SAMPLE_DIR: "C:\\fw-ci\\samples"
+
+jobs:
+  test-nuget-overlay:
+    name: FieldWorks + NuGet overlay
+    runs-on: windows-latest
+    timeout-minutes: 120
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v7
+
+      - name: Set up Python
+        uses: actions/setup-python@v7
+        with:
+          python-version: "3.12"
+          architecture: x64
+
+      - name: Set up .NET SDK
+        uses: actions/setup-dotnet@v6
+        with:
+          dotnet-version: "8.0.x"
+
+      - name: Cache FieldWorks install + installer + sample
+        id: cache-fw
+        uses: actions/cache@v6
+        with:
+          path: |
+            ${{ env.FW_CODE_DIR }}
+            ${{ env.FW_PROJECTS_DIR }}
+            ${{ env.FW_INSTALLER_DIR }}
+            ${{ env.FW_SAMPLE_DIR }}
+            C:\ProgramData\SIL\FieldWorks
+            C:\fw-ci\fw-code-dir.txt
+          key: fw-${{ env.FW_VERSION }}-${{ env.FW_BUILD }}-win-x64-v3
+
+      - name: Install FieldWorks (or refresh registry after cache restore)
+        shell: pwsh
+        run: |
+          ./scripts/ci/Install-FieldWorks.ps1 `
+            -Version $env:FW_VERSION `
+            -Build $env:FW_BUILD `
+            -CodeDir $env:FW_CODE_DIR `
+            -ProjectsDir $env:FW_PROJECTS_DIR `
+            -DownloadDir $env:FW_INSTALLER_DIR
+
+      - name: Upload FieldWorks install logs
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: fw-install-logs
+          path: |
+            C:\fw-ci\fw-install*.log
+            C:\fw-ci\fw-code-dir.txt
+          if-no-files-found: ignore
+
+      - name: Set up sample FieldWorks project
+        shell: pwsh
+        run: |
+          ./scripts/ci/Setup-SampleProject.ps1 `
+            -ProjectsDir $env:FW_PROJECTS_DIR `
+            -DownloadDir $env:FW_SAMPLE_DIR
+
+      - name: Install Python test dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -r requirements.txt
+          python -m pip install -e .
+
+      - name: Restore NuGet overlay and run tests
+        shell: cmd
+        env:
+          PYTHON: python
+          FLEXLIBS_FW_CODE_DIR: ${{ env.FW_CODE_DIR }}
+          FLEXLIBS_FW_PROJECTS_DIR: ${{ env.FW_PROJECTS_DIR }}
+        run: |
+          if exist C:\fw-ci\fw-code-dir.txt (
+            set /p FLEXLIBS_FW_CODE_DIR=<C:\fw-ci\fw-code-dir.txt
+          )
+          echo FLEXLIBS_FW_CODE_DIR=%FLEXLIBS_FW_CODE_DIR%
+          call make.bat test-nuget -k "not CustomFields" -q

.gitignore

@@ -54,6 +54,11 @@ coverage.xml
 .hypothesis/
 .pytest_cache/
 
+# NuGet overlay restore output (make test-nuget)
+.fw-nuget-overlay/
+tests/nuget-overlay/bin/
+tests/nuget-overlay/obj/
+
 # Translations
 *.mo
 *.pot

README.rst

@@ -33,6 +33,54 @@ Installation
 Run:
 ``pip install flexlibs``
 
+Testing against latest NuGet packages
+-------------------------------------
+
+By default flexlibs loads FieldWorks assemblies from an installed FieldWorks
+9.x tree (discovered via the Windows registry).
+
+For integration testing against newer SIL libraries from nuget.org, restore
+an overlay and run the existing tests::
+
+  make test-nuget
+
+This publishes the latest prerelease ``SIL.LCModel``, ``SIL.Core``,
+``SIL.WritingSystems`` (and related) packages into ``.fw-nuget-overlay`` and
+sets ``FLEXLIBS_ASSEMBLY_DIR`` so those DLLs shadow the copies from the
+FieldWorks install. App assemblies such as ``FwUtils`` still come from
+FieldWorks.
+
+Environment overrides (useful for CI without a Windows installer)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+- ``FLEXLIBS_FW_CODE_DIR`` — directory containing ``FieldWorks.exe`` and app DLLs
+- ``FLEXLIBS_FW_PROJECTS_DIR`` — FieldWorks projects directory (required when
+  ``FLEXLIBS_FW_CODE_DIR`` is set)
+- ``FLEXLIBS_ASSEMBLY_DIR`` — optional folder of DLLs prepended on the assembly
+  search path (set automatically by ``make test-nuget``)
+
+When the code/projects env vars are set, registry discovery is skipped. You
+still need a complete FieldWorks binary tree (install or unzipped build) plus
+matching Python bitness; NuGet alone does not supply ``FwUtils`` /
+``FieldWorks.exe``.
+
+Requires the .NET SDK (for ``dotnet publish``) in addition to the normal
+test prerequisites.
+
+The custom-field write test needs a FieldWorks project with an entry-level
+custom text field. Defaults are project ``__flexlibs_testing`` and field
+``EntryFlags``; override with::
+
+  set FLEXLIBS_TEST_PROJECT=MyProject
+  set FLEXLIBS_TEST_CUSTOM_FIELD=MyCustomField
+
+The other tests use any project already present in the projects directory.
+
+Skip the custom-field test (or pass any pytest args)::
+
+  .\make.bat test-nuget -k "not CustomFields"
+
+
 Usage
 -----
 

docs/future-testing.md

@@ -0,0 +1,76 @@
+# Future testing ideas
+
+Notes for expanding beyond the current smoke tests now that Windows CI can
+install FieldWorks, seed a sample project (Sena 2), and run the NuGet overlay.
+
+## Current coverage
+
+- `test_FLExInit` — initialize / cleanup
+- `test_FLExProject` — list projects, open first project, walk headwords
+- `test_CustomFields` — write/read a named entry custom field (skipped in CI;
+  needs a fixture project with that field)
+
+CI runs `make test-nuget -k "not CustomFields"` against the Sena 2 sample.
+
+## Suggested next steps (priority order)
+
+### 1. Project-shape read-only test
+
+Promote the checks in `flexlibs/examples/demo_openproject.py` into pytest:
+
+- parts of speech (non-empty list of strings)
+- writing systems and default vernacular / analysis WS
+- custom field discovery (entry / sense lists are iterable)
+- lexicon entry count and text count are non-negative integers
+
+Use the sample project (or `FLEXLIBS_TEST_PROJECT`). Fully CI-friendly; no writes.
+
+### 2. Deeper lexicon field access
+
+Extend beyond “headword is a string”. For the first N entries (or a few known
+ones), call and type-check:
+
+- lexeme / citation form
+- sense gloss and POS
+- semantic domains (may be empty)
+- example text when present
+
+Still read-only. Catches writing-system and LCM regressions that the headword
+smoke test misses.
+
+### 3. Write round-trip on a project copy
+
+Do not mutate the cached Sena 2 tree in place. In CI:
+
+1. Copy the project folder to a temp projects path
+2. Open write-enabled
+3. Set a gloss (or lexeme), read it back, clear it
+4. Close and discard the copy
+
+Exercises the real write path without requiring `__flexlibs_testing` /
+`EntryFlags`. Keep `test_CustomFields` as an optional local fixture.
+
+### 4. NuGet overlay load assertion
+
+When `FLEXLIBS_ASSEMBLY_DIR` is set, assert that `SIL.LCModel` (and optionally
+`SIL.Core`) loaded from a path under that directory. Turns “overlay actually
+applied” into a failing test instead of a log-only check.
+
+### 5. Optional matrices (later)
+
+- Python 3.9 and 3.12
+- Job without overlay (install DLLs only) vs with NuGet overlay
+- Pinned LCM version vs floating `*-*` latest
+
+Useful for separating “new SIL beta broke us” from “our wrapper is wrong.”
+
+## Lower priority / skip for now
+
+- Deep custom-field and list-field coverage in CI (fixture-heavy)
+- Text corpus, reversal indexes, publications (interesting later, lower ROI)
+- Mocking LCM — wrong layer; this library needs real interop tests
+
+## Practical first PR
+
+Implement (1), (2), and (4) together: still fully read-only on Sena 2, then
+add (3) once that suite is stable.