Skip to content

Refactor: inject nlocal/nbands into the LCAO dense diagonalizers - #7721

Draft
Critsium-xy wants to merge 1 commit into
deepmodeling:developfrom
Critsium-xy:refactor/remove-param-hsolver-d
Draft

Refactor: inject nlocal/nbands into the LCAO dense diagonalizers#7721
Critsium-xy wants to merge 1 commit into
deepmodeling:developfrom
Critsium-xy:refactor/remove-param-hsolver-d

Conversation

@Critsium-xy

Copy link
Copy Markdown
Collaborator

Background

Follow-up to #7706 and #7711, continuing the removal of PARAM.* reads from source_hsolver (coding rule 1 in AGENTS.md).

This is the largest single cluster: the six dense LCAO diagonalizers all sized their eigenvalue buffers from PARAM.globalv.nlocal and copied out PARAM.inp.nbands eigenvalues — 55 reads across six files.

Changes

DiagoLapack, DiagoScalapack, DiagoElpa, DiagoElpaNative, DiagoCusolver and DiagoCusolverMP each take (nlocal, nbands) through the constructor and store them as members. DiagoElpaNative additionally takes use_gpu, replacing its PARAM.inp.device == "gpu" test.

HSolverLCAO gains nbands and use_gpu (it already received nlocal in #7711) and supplies all eleven construction sites across hamiltSolvePsiK, parakSolve and parakSolve_cusolver.

The three call contexts agree on both values

Worth confirming during review, since it is what lets one (nlocal, nbands) pair serve all of them:

  • hamiltSolvePsiK (kpar_lcao == 1) uses the global ParaV directly.
  • parakSolve (kpar_lcao > 1) goes through diag_pool, which receives pool-local matrix blocks — but the distributed matrix is still nlocal x nlocal, just spread over a smaller grid. The function already feeds ParaV->get_global_row_size() into k2d.set_para_env.
  • parakSolve_cusolver gathers the matrix per k-point with nrow = ncol = ParaV->get_global_row_size().

Why nbands is injected rather than read from ParaV

HSolverLCAO already holds ParaV, and ParaV->get_nbands() does equal PARAM.inp.nbands on the LCAO path (LCAO_init_basis.cpp seeds it from there). But module_lr sets paraMat_.nbands to nocc + nvirt, so sourcing it from ParaV would plant a trap for any future caller that reaches these diagonalizers through a differently-initialised Parallel_Orbitals. It is a latent hazard rather than a live bug today — module_lr does not use these classes — but injection costs nothing and closes it.

The existing ParaV->get_nbands() reads inside parakSolve are deliberately left alone: they are not PARAM reads and are out of scope here.

Address-of sites in ScaLAPACK

Four call sites pass the address of the value to the Fortran routines rather than the value. PARAM.globalv returns a const reference, so &PARAM.globalv.nlocal was already a const int*, which matches pdsygvx_/pzhegvx_'s const int* n; &this->nlocal has the same type. This was type-checked against the real prototypes (see Testing).

Incidental cleanups in code that had to be touched anyway

Tests

The three LCAO diagonalizer tests already read nlocal/nbands from their H/S input files, so they now pass those directly. Their PARAM.sys.nlocal / PARAM.input.nbands writes are dropped, along with the #define private public / #include parameter.h / #undef private blocks that existed solely to enable those writes.

Effect

PARAM.* occurrences in source_hsolver production code: 60 → 5. What remains is DiagoIterAssist's basis_type / calculation switches, planned as a separate PR, plus one comment in simple_pexsi.cpp.

Testing

In the two earlier PRs I had to flag large unverified areas. That gap is much smaller here: by writing minimal stand-in headers for the third-party APIs that are unavailable locally, all six diagonalizers pass g++ -fsyntax-only, each in the configuration it is actually built in:

File Configuration
diago_lapack.cpp bare and __MPI
diago_scalapack.cpp __MPI with the real header; also force-included the pdsygvx_/pzhegvx_ prototypes copied verbatim from scalapack_connector.h to fully type-check the &this->nlocal sites
diago_elpa.cpp __MPI + ELPA stand-in
diago_elpa_native.cpp __MPI + ELPA stand-in with ELPA_WITH_NVIDIA_GPU_VERSION=1, so the new use_gpu branch is genuinely compiled
diago_cusolver.cpp __CUDA + CUDA/cuBLAS/cuSOLVER stand-ins
diago_cusolvermp.cpp __CUSOLVERMP + cusolverMp/NCCL stand-ins

diago_cusolvermp.cpp is worth calling out: no CI job builds __CUSOLVERMP, so this local check is the only compile coverage it gets.

hsolver_lcao.cpp was checked across six flag combinations (bare, __MPI, +__ELPA, +__CUDA, +__CUSOLVERMP, and all of them together with __PEXSI), covering every branch of the eleven construction sites. As a side effect this also closes the parakSolve gap I flagged in #7711, which sits inside #ifdef __MPI.

esolver_ks_lcao.cpp, esolver_ks_lcao_tddft.cpp, LCAO_set.cpp and cal_mw_from_lambda.cpp also pass.

Not covered locally: the three test files need gtest, which is unavailable here. Their changes are limited to dropping PARAM writes, adding two int constructor arguments, and deleting the dead include blocks; MODULE_HSOLVER_LCAO builds in the default CI matrix.

Pre-existing issue noticed, not addressed

Compiling diago_elpa.cpp without __MPI produces four errors: elpa_num_thread and lastmpinum are declared inside #ifdef __MPI in the header, but their static definitions at the bottom are unguarded. I verified against the unmodified file (git stash) that the error set is identical before and after this change. DiagoElpa requires MPI anyway, so no supported build configuration is affected, and I left it alone rather than widen this PR — happy to fix it separately if wanted.

Behaviour is intended to be unchanged throughout: every injected value is the same PARAM field that was previously read directly, sourced at the esolver layer.

🤖 Generated with Claude Code

Group D of the source_hsolver PARAM removal, and the largest single cluster:
the six dense LCAO diagonalizers all sized their eigenvalue buffers from
PARAM.globalv.nlocal and copied out PARAM.inp.nbands eigenvalues.

Each of DiagoLapack, DiagoScalapack, DiagoElpa, DiagoElpaNative, DiagoCusolver
and DiagoCusolverMP now takes (nlocal, nbands) through its constructor and
stores them as members. DiagoElpaNative additionally takes use_gpu, replacing
its PARAM.inp.device == "gpu" test.

HSolverLCAO gains nbands and use_gpu (it already received nlocal in deepmodeling#7711) and
supplies all eleven construction sites across hamiltSolvePsiK, parakSolve and
parakSolve_cusolver. The three call contexts agree on both values: parakSolve
distributes the same nlocal x nlocal matrix over a smaller pool grid, and
parakSolve_cusolver gathers it per k-point, so a single pair covers all of them.

nbands is injected rather than read from ParaV->get_nbands(). The two are equal
for the LCAO path (LCAO_init_basis.cpp seeds ParaV from PARAM.inp.nbands), but
module_lr sets paraMat_.nbands to nocc + nvirt, so sourcing it from ParaV would
plant a trap for any future caller. The existing ParaV->get_nbands() reads
inside parakSolve are left alone; they are not PARAM reads and are out of scope.

In diago_scalapack.cpp, four call sites pass the address of the value to the
Fortran routines. PARAM.globalv returns a const reference, so &PARAM.globalv.nlocal
was already a const int*, matching pdsygvx_/pzhegvx_'s const int* n; &this->nlocal
has the same type.

Also resolves the two error-message strings in diago_lapack.cpp and
diago_scalapack.cpp that embedded "PARAM.inp.nbands = ". These were deferred
from deepmodeling#7706 because removing them needed exactly the nbands member added here.

Two dead commented-out ELPA_Solver calls in diago_elpa.cpp are removed rather
than updated; each duplicated the live statement two lines below it.

The three LCAO diagonalizer tests supply their own nlocal/nbands, which they
already read from the H/S input files, so their PARAM.sys.nlocal /
PARAM.input.nbands writes and the now-dead `#define private public` include
blocks are dropped.

PARAM occurrences in source_hsolver production code: 60 -> 5. What remains is
DiagoIterAssist's basis_type/calculation switches (group E) and one comment in
simple_pexsi.cpp.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Critsium-xy
Critsium-xy marked this pull request as draft July 31, 2026 08:18
@mohanchen mohanchen added Refactor Refactor ABACUS codes The Absolute Zero Reduce the "entropy" of the code to 0 labels Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Refactor Refactor ABACUS codes The Absolute Zero Reduce the "entropy" of the code to 0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants