Ship the merged CPU kernels as their own library

[shoumikhin]

The CPU operator kernels are compiled into whichever component links them, so
the Python bindings extension carries its own full copy. That makes the
extension large, and it means a C++ application cannot get the same operators
without building them from source. It also means two components that both want
CPU operators end up with two independent registrations of the same operator
set in one process.

Build the merged CPU kernels as a shared library and ship it in the wheel, so
the operators live in one place and both the Python bindings and a C++
application can link the same copy.

The library type is opt-in rather than a global change: gen_operators_lib has
many callers that want an ordinary static library, and only the one shipped in
the wheel needs to be shared. Callers that do not pass SHARED are unaffected,
and the whole change is gated on the existing EXECUTORCH_BUILD_SHARED option,
so iOS, Android, and embedded builds keep linking static libraries exactly as
before.

Test plan:

The wheel smoke test now asserts that exactly one shipped library defines the
CPU kernels, alongside the existing backend-registry and thread-pool
assertions, so a change that reintroduces a second copy fails in CI rather than
aborting at startup with a duplicate operator registration. The symbol used by
the assertion was confirmed to exist in the shipped libraries first, so the
check cannot pass by matching nothing.

Built the wheel from a clean checkout and verified against a fresh virtual
environment with a normal dependency-resolving install:

• The wheel ships the kernels as their own versioned library next to the
  runtime and the thread pool.
• nm -DC across every shipped shared object shows exactly one definition of a
  representative CPU operator, in the new library rather than in the bindings
  extension.
• The backend-registry and thread-pool assertions still hold with the new
  library loaded.
• import executorch, the registered backend list, and .pte execution
  through the Python bindings are unchanged, with outputs matching eager
  PyTorch.
• With EXECUTORCH_BUILD_SHARED off, the kernels remain a static library and
  no new shared object is produced, so every build that does not opt in is
  unaffected.

[shoumikhin]

Stack from ghstack (oldest at bottom):

• Ship the CUDA delegate as its own library #21531
• Ship the XNNPACK delegate as its own library #21526
• -> Ship the merged CPU kernels as their own library #21524
• Ship the thread pool as its own library so a process has one pool #21522
• Ship a linkable shared runtime and make the Python bindings use it #21514

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21524

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 263 Pending

As of commit e06b763 with merge base d632341 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.