How to deal with pip in venv attempting and failing to build cffi?

[przemoc]

I would appreciate some recommendations regarding:
How to deal with pip in venv attempting and failing to build cffi?

AFAICT mingw-w64-python-cffi has extra patches that are not applied when cffi is being build for venv, hence we get errors.

Failure example

przemoc@NUCPHKi7C UCRT64 ~
$ python3.14 -m venv venvs/crypto

przemoc@NUCPHKi7C UCRT64 ~
$ . ./venvs/crypto/bin/activate
(crypto)
przemoc@NUCPHKi7C UCRT64 ~
$ python3.14 -m pip install --upgrade pip
Requirement already satisfied: pip in d:/msys64/home/przemoc/venvs/crypto/lib/python3.14/site-packages (25.3)
Collecting pip
  Using cached pip-26.0.1-py3-none-any.whl.metadata (4.7 kB)
Using cached pip-26.0.1-py3-none-any.whl (1.8 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 25.3
    Uninstalling pip-25.3:
      Successfully uninstalled pip-25.3
Successfully installed pip-26.0.1
(crypto)
przemoc@NUCPHKi7C UCRT64 ~
$ python3.14 -m pip install cryptography
Collecting cryptography
  Using cached cryptography-46.0.5.tar.gz (750 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × installing build dependencies for cryptography did not run successfully.
  │ exit code: 1
  ╰─> [169 lines of output]
      Ignoring cffi: markers 'platform_python_implementation != "PyPy" and python_version == "3.8"' don't match your environment
      Collecting maturin<2,>=1.9.4
        Using cached maturin-1.12.6-cp314-cp314-mingw_x86_64_ucrt_gnu.whl
      Collecting cffi>=2.0.0
        Using cached cffi-2.0.0.tar.gz (523 kB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting setuptools!=74.0.0,!=74.1.0,!=74.1.1,!=74.1.2
        Using cached setuptools-82.0.0-py3-none-any.whl.metadata (6.6 kB)
      Collecting pycparser (from cffi>=2.0.0)
        Using cached pycparser-3.0-py3-none-any.whl.metadata (8.2 kB)
      Using cached setuptools-82.0.0-py3-none-any.whl (1.0 MB)
      Using cached pycparser-3.0-py3-none-any.whl (48 kB)
      Building wheels for collected packages: cffi
        Building wheel for cffi (pyproject.toml): started
        Building wheel for cffi (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error

        Building wheel for cffi (pyproject.toml) did not run successfully.
        exit code: 1

        [133 lines of output]
        _configtest.c:2:2: error: #error "not MSVC"
            2 | #error "not MSVC"
              |  ^~~~~
        Note: will not use '__thread' in the C code
        ***** The above error message can be safely ignored.

        running bdist_wheel
        running build
        running build_py
        creating build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/api.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/backend_ctypes.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/cffi_opcode.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/commontypes.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/cparser.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/error.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/ffiplatform.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/lock.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/model.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/pkgconfig.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/recompiler.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/setuptools_ext.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/vengine_cpy.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/vengine_gen.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/verifier.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/_imp_emulation.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/_shimmed_dist_utils.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/__init__.py -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        running egg_info
        writing src/cffi.egg-info/PKG-INFO
        writing dependency_links to src/cffi.egg-info/dependency_links.txt
        writing entry points to src/cffi.egg-info/entry_points.txt
        writing requirements to src/cffi.egg-info/requires.txt
        writing top-level names to src/cffi.egg-info/top_level.txt
        reading manifest file 'src/cffi.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        adding license file 'LICENSE'
        adding license file 'AUTHORS'
        writing manifest file 'src/cffi.egg-info/SOURCES.txt'
        copying src/cffi/_cffi_errors.h -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/_cffi_include.h -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/_embedding.h -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        copying src/cffi/parse_c_type.h -> build/lib.mingw_x86_64_ucrt_gnu-cpython-314/cffi
        running build_ext
        building '_cffi_backend' extension
        creating build/temp.mingw_x86_64_ucrt_gnu-cpython-314/src/c
        gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -march=nocona -msahf -mtune=generic -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wp,-D__USE_MINGW_ANSI_STDIO=1 -O3 -march=nocona -msahf -mtune=generic -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong -Wp,-D__USE_MINGW_ANSI_STDIO=1 -O3 -DFFI_BUILDING=1 -ID:/msys64/home/przemoc/venvs/crypto/include -ID:/msys64/ucrt64/include/python3.14 -c src/c/_cffi_backend.c -o build/temp.mingw_x86_64_ucrt_gnu-cpython-314/src/c/_cffi_backend.o
        In file included from src/c/_cffi_backend.c:9:
        src/c/misc_win32.h: In function 'cffi_atomic_load':
        src/c/misc_win32.h:279:12: warning: return discards 'volatile' qualifier from pointer target type [-Wdiscarded-qualifiers]
          279 |     return *(volatile void **)ptr;
              |            ^~~~~~~~~~~~~~~~~~~~~~
        src/c/misc_win32.h: In function 'cffi_atomic_store_ssize':
        src/c/misc_win32.h:311:33: error: passing argument 1 of '_InterlockedExchangePointer' from incompatible pointer type [-Wincompatible-pointer-types]
          311 |     _InterlockedExchangePointer(ptr, value);
              |                                 ^~~
              |                                 |
              |                                 Py_ssize_t * {aka long long int *}
        In file included from D:/msys64/ucrt64/include/winnt.h:27,
                         from D:/msys64/ucrt64/include/minwindef.h:163,
                         from D:/msys64/ucrt64/include/windef.h:9,
                         from D:/msys64/ucrt64/include/windows.h:69,
                         from src/c/_cffi_backend.c:8:
        D:/msys64/ucrt64/include/psdk_inc/intrin-impl.h:1770:51: note: expected 'void * volatile*' but argument is of type 'Py_ssize_t *' {aka 'long long int *'}
         1770 | void *_InterlockedExchangePointer(void *volatile *Target,void *Value) {
              |                                   ~~~~~~~~~~~~~~~~^~~~~~
        src/c/misc_win32.h:311:38: error: passing argument 2 of '_InterlockedExchangePointer' makes pointer from integer without a cast [-Wint-conversion]
          311 |     _InterlockedExchangePointer(ptr, value);
              |                                      ^~~~~
              |                                      |
              |                                      Py_ssize_t {aka long long int}
        D:/msys64/ucrt64/include/psdk_inc/intrin-impl.h:1770:64: note: expected 'void *' but argument is of type 'Py_ssize_t' {aka 'long long int'}
         1770 | void *_InterlockedExchangePointer(void *volatile *Target,void *Value) {
              |                                                          ~~~~~~^~~~~
        src/c/misc_win32.h: In function 'cffi_atomic_store_uint8':
        src/c/misc_win32.h:321:5: error: implicit declaration of function '_InterlockedExchange8'; did you mean '_InterlockedExchange'? [-Wimplicit-function-declaration]
          321 |     _InterlockedExchange8(ptr, value);
              |     ^~~~~~~~~~~~~~~~~~~~~
              |     _InterlockedExchange
        src/c/_cffi_backend.c: In function '_my_PyLong_AsUnsignedLongLong':
        src/c/_cffi_backend.c:955:13: warning: '_PyLong_Sign' is deprecated [-Wdeprecated-declarations]
          955 |             if (_PyLong_Sign(ob) < 0)
              |             ^~
        In file included from D:/msys64/ucrt64/include/python3.14/longobject.h:171,
                         from D:/msys64/ucrt64/include/python3.14/Python.h:92,
                         from src/c/_cffi_backend.c:2:
        D:/msys64/ucrt64/include/python3.14/cpython/longobject.h:38:37: note: declared here
           38 | Py_DEPRECATED(3.14) PyAPI_FUNC(int) _PyLong_Sign(PyObject *v);
              |                                     ^~~~~~~~~~~~
        src/c/_cffi_backend.c: In function '_my_PyObject_AsBool':
        src/c/_cffi_backend.c:4083:9: warning: '_PyLong_Sign' is deprecated [-Wdeprecated-declarations]
         4083 |         return _PyLong_Sign(ob) != 0;
              |         ^~~~~~
        D:/msys64/ucrt64/include/python3.14/cpython/longobject.h:38:37: note: declared here
           38 | Py_DEPRECATED(3.14) PyAPI_FUNC(int) _PyLong_Sign(PyObject *v);
              |                                     ^~~~~~~~~~~~
        src/c/_cffi_backend.c: In function 'b_do_dlopen':
        src/c/_cffi_backend.c:4658:22: warning: 'Py_FileSystemDefaultEncoding' is deprecated [-Wdeprecated-declarations]
         4658 |                      Py_FileSystemDefaultEncoding, &filename_or_null, &flags))
              |                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        In file included from D:/msys64/ucrt64/include/python3.14/Python.h:110:
        D:/msys64/ucrt64/include/python3.14/fileobject.h:22:46: note: declared here
           22 | Py_DEPRECATED(3.12) PyAPI_DATA(const char *) Py_FileSystemDefaultEncoding;
              |                                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        In file included from src/c/cffi1_module.c:3,
                         from src/c/_cffi_backend.c:8173:
        src/c/realize_c_type.c: In function 'init_global_types_dict':
        src/c/realize_c_type.c:32:16: warning: unused variable 'i' [-Wunused-variable]
           32 |     Py_ssize_t i;
              |                ^
        src/c/misc_win32.h: At top level:
        src/c/misc_win32.h:319:13: warning: 'cffi_atomic_store_uint8' defined but not used [-Wunused-function]
          319 | static void cffi_atomic_store_uint8(uint8_t *ptr, uint8_t value)
              |             ^~~~~~~~~~~~~~~~~~~~~~~
        src/c/misc_win32.h:314:13: warning: 'cffi_atomic_store' defined but not used [-Wunused-function]
          314 | static void cffi_atomic_store(void **ptr, void *value)
              |             ^~~~~~~~~~~~~~~~~
        src/c/misc_win32.h:309:13: warning: 'cffi_atomic_store_ssize' defined but not used [-Wunused-function]
          309 | static void cffi_atomic_store_ssize(Py_ssize_t *ptr, Py_ssize_t value)
              |             ^~~~~~~~~~~~~~~~~~~~~~~
        src/c/misc_win32.h:298:19: warning: 'cffi_atomic_load_ssize' defined but not used [-Wunused-function]
          298 | static Py_ssize_t cffi_atomic_load_ssize(Py_ssize_t *ptr)
              |                   ^~~~~~~~~~~~~~~~~~~~~~
        src/c/misc_win32.h:287:16: warning: 'cffi_atomic_load_uint8' defined but not used [-Wunused-function]
          287 | static uint8_t cffi_atomic_load_uint8(uint8_t *ptr)
              |                ^~~~~~~~~~~~~~~~~~~~~~
        src/c/misc_win32.h:276:14: warning: 'cffi_atomic_load' defined but not used [-Wunused-function]
          276 | static void *cffi_atomic_load(void **ptr)
              |              ^~~~~~~~~~~~~~~~
        src/c/misc_win32.h:265:12: warning: 'cffi_atomic_compare_exchange' defined but not used [-Wunused-function]
          265 | static int cffi_atomic_compare_exchange(void **ptr, void **expected,
              |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        error: command 'D:\\msys64\\ucrt64\\bin/gcc.EXE' failed with exit code 1
        [end of output]

        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for cffi
      Failed to build cffi
      error: failed-wheel-build-for-install

      Failed to build installable wheels for some pyproject.toml based projects

      cffi
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed to build 'cryptography' when installing build dependencies for cryptography

I had wild idea that maybe I could transplant system cffi into venv:

Symlinking system cffi into venv

(crypto)
przemoc@NUCPHKi7C UCRT64 ~
$ ln -s /ucrt64/lib/python3.14/site-packages/cffi venvs/crypto/lib/python3.14/site-packages/
(crypto)
przemoc@NUCPHKi7C UCRT64 ~
$ ln -s /ucrt64/lib/python3.14/site-packages/cffi-2.0.0.dist-info venvs/crypto/lib/python3.14/site-packages/

but it didn't change anything. pip does not rely on that.

Searching for some ideas I stumbled upon --no-build-isolation, but it's kind of nuke option putting all burden of python package management on the user, which does not scale well depending on the dependencies of stuff we want to install in venv (which directly or indirectly relies on cffi).

Any ideas?

I suspect cffi may be not the only one python package that requires patching to be successfully built in any of MSYS2 environments, so in the long-term some form of hooking to patch sources (based on patches in MINGW-packages repo) before pip builds them could be potentially considered. But for now any simpler solution or workaround that doesn't require changing pip install X into series of pip install Xdep (or transplants when needed) followed by pip install --no-build-isolation X would be appreciated.

[mlt]

Here is the crazy workaround idea: use --system-site-packages with venv. I believe I had to do it with something but I have a hard time recalling what for duckduckgo MCP due to pydantic-core. I think it is usually a handful of packages that one usually doesn't want to pollute global environment, but for msys2 supplied package it might be alright.

[przemoc]

Here is the crazy workaround idea: use --system-site-packages with venv.

Unfortunately it does not work. Like with symlinking system cffi into venv - building local cffi is attempted anyway.

[lazka]

It was fixed some time ago in python-cffi/cffi#198 but cffi never had a release since then.

[lazka]

I've pinged upstream in the PR. Not much we can do besides that.