Fix and optimize get_S sparse output

[sgyang345]

Reminder

• I have read AGENTS.md and docs/developers_guide/agent_governance.md.
• I have linked an issue or explained why this PR does not need one.
• I have added adequate unit tests and/or case tests, or explained why not.
• I have listed the exact verification commands run and their results.
• I have described user-visible behavior changes, including INPUT parameter changes.
• I have explained core-module impact for ESolver, HSolver, ElecState, Hamilt, Operator, Psi, or other source/ changes.
• I have requested any needed governance exception below.

Linked Issue

Fix #

Unit Tests and/or Case Tests for my changes

• Commands run:
• Result summary:
• Checks not run, with reason:

What's changed?

• Example: brief summary of the user-visible or developer-facing change.

Governance Notes

• INPUT/docs changes:
• Core module impact:
• Exceptions requested:

[mohanchen]

Hello, please clarify in this PR whether it is a bug fix or an optimization of the get_overlap functionality. ABACUS supports multiple levels of code builds, so a single source code modification may pass in one built executable but fail in executables built at other levels — please keep this in mind.

[sgyang345]

Fix MPI get_S Sparse Output and Optimize CSR Generation

Summary

This pull request contains one correctness fix and two optimizations for the ABACUS calculation get_S workflow, which generates overlap matrices.

• Fix an MPI runtime failure when a rank does not own a local sparse R-block.
• Avoid allocating an unused density matrix during get_S initialization.
• Optimize text CSR output for large sparse overlap matrices.

The changes are restricted to the get_S overlap-generation path. No new INPUT parameters or public APIs are introduced.

Linked Issue

No linked issue.

The changes address issues reproduced in the calculation get_S workflow:

1. A parallel sparse-output failure caused by accessing a locally missing R-block.
2. Excessive memory consumption caused by initializing an unused density matrix.
3. Very slow text CSR output for large sparse overlap matrices.

These issues are specific to overlap-matrix generation and can be tracked independently from the general ABACUS calculation workflows.

What's Changed?

1. Fix MPI sparse-output failure for missing local R-blocks

File:

source/module_io/write_HS_sparse.cpp

The original implementation accessed each R-block with:

smat.at(R_coor)

After the nonzero count is reduced across MPI ranks, an R-block can be globally nonzero even when a particular MPI rank has no local entries for that R-block. On such a rank, std::map::at() throws std::out_of_range.

The implementation now uses find() and passes an empty sparse map when the local R-block is absent:

auto iter = smat.find(R_coor);
const auto& local_smat = (iter == smat.end()) ? empty_smat : iter->second;
output_single_R(ofs, local_smat, sparse_thr, binary, pv, reduce);

All MPI ranks still enter output_single_R(), preserving the required collective reduction order. Ranks without local entries contribute zero values instead of terminating.

This is a correctness bug fix for parallel sparse overlap output.

2. Skip unused density-matrix allocation in get_S

File:

source/module_esolver/esolver_gets.cpp

ESolver_GetS::before_all_runners() previously initialized the LCAO density matrix through init_DM().

The get_S workflow only constructs and writes overlap matrices. It does not perform an SCF density calculation and does not use the initialized density matrix.

For large LCAO/SOC systems, this unnecessary allocation can consume a very large amount of memory and cause an out-of-memory failure before overlap construction begins.

The density-matrix initialization is therefore removed from the ESolver_GetS path.

This is a memory-scalability optimization and prevents avoidable resource failures.

3. Add a block-wise fast path for text CSR output

File:

source/module_io/write_HS_R.cpp

A new save_gets_sparse_fast() routine is added for large text CSR output in get_S.

The routine:

• Counts local nonzero entries for each R-block.
• Performs the global MPI reduction of nonzero counts.
• Processes rows in blocks of 4096.
• Gathers entries block by block across MPI ranks.
• Sorts and merges duplicate (row, column) entries.
• Writes CSR values, column indices, and row pointers.

The fast path is used only when:

PARAM.inp.calculation == "get_S" && !binary

It supports both real-valued sparse overlap matrices and complex-valued SOC overlap matrices with nspin = 4.

All other cases continue to use the existing ModuleIO::save_sparse() implementation. In particular, non-get_S calculations and binary output are unchanged.

User-visible Behavior

For users running:

calculation get_S

the expected behavior is:

• MPI sparse overlap output no longer fails when a rank has no local data for a globally nonzero R-block.
• Large LCAO/SOC calculations avoid allocating an unused density matrix.
• Text CSR output uses a more memory-efficient block-wise implementation.
• The output remains in the existing CSR text format.

Calculations other than get_S retain the existing output path.

No existing INPUT parameter semantics are changed.

Core Module Impact

ESolver

The ESolver_GetS initialization path no longer initializes the density matrix.

This change is limited to get_S. Normal SCF and other electronic-structure workflows retain their existing density-matrix initialization.

ModuleIO and overlap output

The following files change sparse overlap output behavior:

source/module_io/write_HS_sparse.cpp
source/module_io/write_HS_R.cpp

MPI sparse assembly is made safe for missing local R-blocks, and text CSR serialization is optimized for get_S.

Other core modules

No direct changes were made to HSolver, Hamilt, Operator, or Psi.

No public API or input-file interface is changed.

Unit Tests and/or Case Tests

Commands Run

git diff HEAD^ HEAD --check
cmake --build build_abacus_gnu --target abacus -j2
source toolchain/abacus_env.sh
mpirun --oversubscribe -np 2 /home/sgyang/opt/abacus-develop/bin/abacus

Result Summary

Passed. No whitespace errors were reported.

The existing GNU/MPI build completed successfully:

[100%] Built target abacus

The resulting build_abacus_gnu/abacus and installed bin/abacus binaries are identical. The executable reports the expected embedded commit:

Commit: 2b0b14dda (Mon Aug 31 20:13:20 2026 +0800)

The repository integration case tests/integrate/207_NO_KP_OS was run in a temporary directory with the same relative directory layout:

# Serial case
/home/sgyang/opt/abacus-develop/bin/abacus

# MPI case
mpirun --oversubscribe -np 2 \
  /home/sgyang/opt/abacus-develop/bin/abacus

Both runs completed successfully:

serial_rc=0
mpi_rc=0
serial_SR=5708950 bytes
mpi_SR=5708950 bytes
serial_vs_mpi_SR=identical

The generated SR.csr files had the same SHA-256 checksum:

0b74428d0350b709258708ae11683cc4451cb71b7c527de4bd0abfbe0b791a43

No map::at, std::out_of_range, or other runtime error markers were observed.

Checks Not Run

The following configurations were not run in this environment and should be validated in CI or a complete ABACUS build environment:

• ENABLE_MPI=OFF
• SOC/complex-valued calculations with nspin = 4
• Existing binary output
• Release and Debug builds where available
• Full ctest suite

For the get_S case tests, the generated SR.csr files should be checked for:

• Successful completion without std::out_of_range or map::at exceptions.
• Correct nonzero counts and CSR row pointers.
• Consistent serial and MPI output for the same input.
• Successful execution for both ordinary and SOC calculations.
• No unnecessary density-matrix allocation.

Multiple Build Configurations

ABACUS supports multiple build levels and configuration combinations. The modified code should therefore be validated with MPI disabled and enabled, real and complex LCAO calculations, ordinary and SOC spin settings, text and binary output modes, and Release and Debug builds where available.

The new MPI-specific code is guarded by __MPI, and the serial path remains available when MPI is disabled. The existing generic output path is retained for non-get_S calculations and binary output.

Governance Notes

INPUT/docs Changes

None. No new parameters or documentation changes are required.

Core Module Impact

Changes are limited to ESolver_GetS initialization, sparse S(R) output, and get_S CSR serialization.

Exceptions Requested

None.