[SPEC] Spec mthreads root python code#784
Closed
zhzhcookie wants to merge 45 commits into
Closed
Conversation
- git mv 12 files with genuine mthreads-specific changes from third_party/mthreads/python/triton/ to backend/spec/triton/ - Add spec_path(__path__) to triton/, compiler/, runtime/, language/, tools/ __init__.py for runtime path specialization - Add max_shared_mem to main compiler/__init__.py and triton/__init__.py - Remove monkey-patch code from mthreads.py (_wrap_setup, _patch_setup_for_mthreads_python_root), keep build-time package merging logic and expose apply_mthreads_setup_args() - Update setup.py to call apply_mthreads_setup_args for non-editable installs when FLAGTREE_BACKEND=mthreads
- Remove runtime/adjust_kernel_param.py (just an older upstream version) - Remove experimental/gluon/__init__.py (missing spec_path setup from upstream) - Restore both files to third_party/mthreads/python/triton/
…only - Remove merge_mthreads_packages, merge_mthreads_package_dir, patch_mthreads_cmdclass, and apply_mthreads_setup_args - get_package_dir() now returns empty dict (no root package override) - Remove apply_mthreads_setup_args call from setup.py - Both editable and non-editable installs now use the standard spec_path mechanism via backends/mthreads/spec/triton/
…/__init__.py - git mv language/core.py and language/standard.py to spec (have mthreads-specific functions: squeeze, unsqueeze, to_tensor, _experimental_descriptor_load, _experimental_descriptor_store) - Add these symbols to main python/triton/language/__init__.py imports and __all__ - Remove async_task import (not in mthreads version) - Revert setup.py to direct setup() call (no intermediate variable)
Spec core.py imports _tuple_create from triton._utils. The mthreads _utils.py has apply_with_path, is_iterable, and _tuple_create that are not in the main version.
Spec core.py is missing upstream async_task, causing ImportError. These files have too much upstream divergence to spec cleanly. Instead, mthreads-specific additions will be merged to main files.
- Add squeeze and unsqueeze functions to python/triton/language/standard.py (mthreads-specific functions now merged to main) - Remove async_task, _experimental_descriptor_load, _experimental_descriptor_store from language/__init__.py (not in main core.py)
…_init__.py - Revert squeeze/unsqueeze from standard.py (stay in mthreads fork) - Revert _experimental_descriptor_* from core.py (stay in mthreads fork) - Keep only spec_path(__path__) in language/__init__.py - Restore language/__init__.py imports to match main upstream
…_language
- Remove third_party/mthreads/python/triton/runtime/adjust_kernel_param.py (AI hallucination, not needed)
- Remove third_party/mthreads/python/triton/experimental/gluon/__init__.py (identical to main, not callable from third_party)
- Add third_party/mthreads/backend/spec/__init__.py with init_language() for mthreads-specific symbols
- Update python/triton/__init__.py to explicitly call spec('init_language')
Co-Authored-By: Claude <noreply@anthropic.com>
These two files contain mthreads-specific changes that init_language() alone cannot cover. By placing them at backend/spec/triton/language/, spec_path() will automatically use them when FLAGTREE_BACKEND=mthreads. Co-Authored-By: Claude <noreply@anthropic.com>
flagcx_wrapper.py and flagcx/include may not exist in all flagcx repo versions. Check existence before copying to avoid FileNotFoundError. Co-Authored-By: Claude <noreply@anthropic.com>
Move code_generator, semantic, jit, autotuner, interpreter, cache, and _async_compile from the old monkey-patch location into third_party/mthreads/backend/spec/triton/ so the spec_path import mechanism finds the correct mthreads-customized versions instead of falling back to the generic upstream originals. This fixes near-universal test failures caused by the baseline state where the old monkey-patch was disabled but critical files were missing from the spec directory. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
_libtriton.so_ lives under , not . Adjust to compute both the spec overlay path and the base triton path, walking spec first for compiler/language and falling back to base for _C and backends which only exist in the base package. Co-Authored-By: Claude <noreply@anthropic.com>
_libtriton.so_ is a compiled C extension installed under the main triton package root (triton/_C/), not under the mthreads backend directory. Use sys.modules['triton'].__file__ to locate the correct path for _C, backends, compiler, and language base files. Co-Authored-By: Claude <noreply@anthropic.com>
- Fix _constexpr annotation error in squeeze/unsqueeze by importing constexpr at module level in backend spec __init__.py, so the JIT code generator can resolve the name from fn.__globals__. - Add shutil.which fallback in both _filecheck.py modules so FileCheck is resolved from system PATH when the local binary is missing, with a clear error message when neither is found. Co-Authored-By: Claude <noreply@anthropic.com>
The module-level raise FileNotFoundError prevented pytest from collecting any tests in test_frontend.py and test_line_info.py. Move the check into a lazy _get_filecheck_path() helper so the module can be imported without FileCheck installed, and only tests that actually invoke FileCheck will fail. Co-Authored-By: Claude <noreply@anthropic.com>
CMake copies FileCheck to ${TRITON_WHEEL_DIR}/FileCheck (the triton
package root), but the mthreads _filecheck.py lives 4 levels deeper at
triton/backends/mthreads/spec/triton/. Add _filecheck_wheel as a
fallback search path between the local directory and system PATH.
Co-Authored-By: Claude <noreply@anthropic.com>
This change (5884c75) was unrelated to the mthreads spec_path migration and should not be part of this PR. Co-Authored-By: Claude <noreply@anthropic.com>
All 66 files under third_party/mthreads/python/triton/ were deleted: - 38 byte-identical to python/triton/ (pure duplicates) - 28 with version-skew diffs (older/stale copies or non-mthreads gluon code) The 7 genuinely mthreads-specific files were already git mv'd to backend/spec/triton/ and are loaded via spec_path. The remaining copies served no purpose — this directory is not on sys.path and not referenced by any build system. Retained: third_party/mthreads/python/src/ (C++ sources for CMake) and third_party/mthreads/python/test/ (mthreads-specific tests). Co-Authored-By: Claude <noreply@anthropic.com>
… python/triton/ The previous approach (e9e222f) merged mthreads-specific changes from deleted third_party/mthreads/python/triton/ files directly into shared python/triton/ files. This broke other backends because those changes depend on mthreads' own libtriton modifications. This fix reverts to the correct spec_path approach: - Only add spec_path(__path__) to 4 __init__.py files (safe no-op for other backends — spec_path returns early when spec dir doesn't exist) - Put all 17 mthreads-specific file versions in spec/triton/ where they are only activated when FLAGTREE_BACKEND=mthreads - Add missing to_tensor to triton.language __all__ No shared python/triton/ non-init files are modified. Other backends will use their own versions of _core.py, _layouts.py, _semantic.py, compile.py, ragged_tma.py, etc. exactly as before. Co-Authored-By: Claude <noreply@anthropic.com>
Upstream triton.experimental.gluon.language.__init__.py imports thread_barrier from ._core, but the mthreads spec_path copy only had the older barrier() function. This caused 10 test collection errors across test_frontend and tle tests. Co-Authored-By: Claude <noreply@anthropic.com>
zhzhcookie
requested review from
Galaxy1458,
menchunlei and
sunnycase
as code owners
zhzhcookie
force-pushed
the
feat/mthreads-spec-path-third-party.20260713
branch
from
d12cc7d to
d457bff
Compare
zhzhcookie
force-pushed
the
feat/mthreads-spec-path-third-party.20260713
branch
from
10966a1 to
8b94063
Compare
zhzhcookie
changed the title
zhzhcookie
force-pushed
the
feat/mthreads-spec-path-third-party.20260713
branch
from
defb62b to
97551a6
Compare
zhzhcookie
force-pushed
the
feat/mthreads-spec-path-third-party.20260713
branch
from
69afcbe to
6876360
Compare
zhzhcookie
force-pushed
the
feat/mthreads-spec-path-third-party.20260713
branch
from
e095c4b to
a90f20c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.