Key Concepts

Key Concepts#

This section describes the core data formats used by the DeepH project for electronic structure calculations and materials modeling. In the latest version of DeepH-pack, we have adopted a new folder layout that is more lightweight, user-friendly, and optimized for high I/O throughput.

Overview#

DeepH utilizes a standardized set of data formats to represent atomic structures, electronic properties, and force field information. These formats enable interoperability between different computational modules and ensure consistent data processing throughout the workflow.

Folder Structure#

dft
  ├── 0
  │   ├── POSCAR
  │   ├── info.json
  │   ├── overlap.h5
  │   ├── hamiltonian.h5     (optional)
  │   ├── hkb.h5             (optional, split SOC storage)
  │   ├── density_matrix.h5  (optional)
  │   ├── wavefunction_ao.h5 (optional)
  │   ├── potential_r.h5     (optional)
  │   ├── charge_density.h5  (optional)
  │   ├── force.h5           (optional)
  │   └── ...
  ├── 1
  └── ...

File Descriptions#

The root directory for all DFT raw data is named dft/.
Subfolders inside (e.g., 0, 1, or structure_001) can use free-form labels or numerical indices.

File Type	Status	Format	Description
`POSCAR`	Required	Text	Atomic structure (VASP format)
`info.json`	Required	JSON	System metadata and basis set info
`overlap.h5`	Required	HDF5	Overlap matrix (S) in sparse AO basis
`hamiltonian.h5`	Optional	HDF5	Hamiltonian matrix (H)
`hkb.h5`	Optional	HDF5	Full-spin Kleinman-Bylander component for split SOC storage
`density_matrix.h5`	Optional	HDF5	Density matrix
`wavefunction_ao.h5`	Optional	HDF5	Band eigenvalues and wavefunction coefficients in the AO basis
`potential_r.h5`	Optional	HDF5	Real-space potential matrix
`charge_density.h5`	Optional	HDF5	Charge density matrix
`force.h5`	Optional	HDF5	Atomic forces

File Types and Their Purposes#

1. POSCAR - Atomic Structure Information#

This file follows the standard POSCAR format and contains the crystal structure information:

Lattice vectors
Atomic positions
Element types

Example:

H2O POSCAR File
0
0   0.0   0.0
0  10.0   0.0
0   0.0  10.0
O  H
1
Direct
0     0.0     0.0
757   0.586   0.0
243   0.586   0.0

2. info.json - Metadata and System Information#

The info.json file stores metadata and system-specific parameters in JSON format.

Example for a Hamiltonian task (water molecule):

{
    "atoms_quantity": 3,
    "orbits_quantity": 23,
    "orthogonal_basis": false,
    "spinful": false,
    "fermi_energy_eV": -2.29107782,
    "elements_orbital_map": {
        "O": [0, 0, 1, 1, 2],
        "H": [0, 0, 1]
    }
}

Example for a force field task:

{
    "atoms_quantity": 21,
    "elements_force_rcut_map": {
        "O": 5.0,
        "H": 5.0
    },
    "max_num_neighbors": 500
}

3. HDF5 Files for Electronic Structure Properties#

DeepH uses HDF5 files to store atom-pair-resolved electronic structure properties:

Common Files#

overlap.h5 - Overlap matrices
hamiltonian.h5 - Hamiltonian matrices
hkb.h5 - Kleinman-Bylander Hamiltonian component in split SOC storage
density_matrix.h5 - Density matrices

Component Descriptions#

Each HDF5 file contains the following keys:

Key	Shape	Description
`atom_pairs`	(N, 5)	Integer matrix where N is the number of edges. Each row contains 5 integers: `[R1, R2, R3, i_atom, j_atom]`, representing a coupling between the \(i\)-th atom in the central unit cell and the \(j\)-th atom in the periodic image cell specified by the lattice vector indices \((i, j, k)\).
`chunk_boundaries`	(N+1,)	1D integer array marking boundaries for each edge’s data in the entries array
`chunk_shapes`	(N, 2)	Integer matrix where each row gives the shape of the submatrix for the corresponding edge
`entries`	(M,)	Flattened 1D array of real or complex values containing all matrix elements

atom_pairs
- Shape: N_edge × 5 array
- Stores edges/”hoppings” in format [R1, R2, R3, i_atom, j_atom]
- R1, R2, R3: Relative lattice shift along three lattice vectors
- i_atom, j_atom: Index of start/end atoms (0-indexed, matches POSCAR order)
entries
- 1-D array containing all matrix elements for edges in atom_pairs
- Blocks A_{i,j,R} are flattened and concatenated
chunk_boundaries
- Shape: (N_edge+1,) array
- Records split indexes of blocks in entries
chunk_shapes
- Shape: N_edge × 2 array
- Records shapes of each block

Spinful Systems#

For systems with spinful=true and conventional full Hamiltonian storage:

overlap.h5 remains unchanged
hamiltonian.h5 and density_matrix.h5 expand to include spin
chunk_shapes doubles in size
chunk_boundaries becomes four times larger

Each block becomes a 4-part matrix:

\[\begin{split} A_{i,j,R} = \begin{bmatrix} A_{i,j,R,\uparrow,\uparrow} & A_{i,j,R,\uparrow,\downarrow} \\ A_{i,j,R,\downarrow,\uparrow} & A_{i,j,R,\downarrow,\downarrow} \end{bmatrix} \end{split}\]

Each sub-block maintains the same size as in the non-spinful case.

Split KB Hamiltonian Storage#

For a nonmagnetic SOC calculation with fully relativistic norm-conserving pseudopotentials, info.json may contain:

"spinful": true,
"split_hkb": true

In this representation, overlap.h5 stores the spinless overlap \(S_0\), hamiltonian.h5 stores the spinless base Hamiltonian \(H_0=T+V_{\mathrm{SC}}\), and hkb.h5 stores the full spinful Kleinman-Bylander contribution, including both its spin-scalar and SOC parts. The physical matrices are reconstructed as

\[ S=I_2\otimes S_0,\qquad H=I_2\otimes H_0+H_{\mathrm{KB}}. \]

The overlap and base-Hamiltonian blocks have shape \(n_i\times n_j\); the HKB blocks have shape \(2n_i\times2n_j\). With "split_hkb": false, hamiltonian.h5 keeps the complete legacy Hamiltonian. Older data that omit the key remain in the legacy format: their existing spinful value continues to determine whether the complete matrices contain the spin degree of freedom. The former hamiltonian_storage string is accepted only as a backward-compatible input alias and is no longer written. For legacy/full spinful data, the loader accepts both overlap conventions already present in the ecosystem: scalar \(n_i\times n_j\) blocks and explicit spin-diagonal \(2n_i\times2n_j\) blocks. It infers one consistent convention from all block shapes. Split-HKB data remain strict and require scalar overlap blocks. The canonical HamiltonianObj and PETScHamiltonianObj loaders reconstruct the full physical matrices before diagonalization. AOMatrixObj can load the two raw Hamiltonian components with matrix_type="hamiltonian" and matrix_type="hkb", respectively.

Converters and analyzers that still require one complete physical hamiltonian.h5 fail before writing output when this split schema is present. This currently includes the HOPCP/PETSc and OpenMX exporters, legacy-format downgrade, Hamiltonian standardization, single-atom core correction, DeepH-to-SIESTA conversion, and Hamiltonian error analysis. Reconstruct a full Hamiltonian with a canonical loader before using one of those routes.

Important Note: The atom_pairs array must be identical across all *.h5 files within the same directory.

Code Example: Extracting Hamiltonian Matrix Elements#import h5py

def extract_hamiltonian(filepath):
    """Extract Hamiltonian matrix elements from an HDF5 file."""
    with h5py.File(filepath, 'r') as f:
        atom_pairs = f['atom_pairs'][:]
        chunk_boundaries = f['chunk_boundaries'][:]
        chunk_shapes = f['chunk_shapes'][:]
        entries = f['entries'][:]
  
    H_tb = {}
    for i, ap in enumerate(atom_pairs):
        start = chunk_boundaries[i]
        end = chunk_boundaries[i+1]
        shape = chunk_shapes[i]
        H_tb[tuple(ap)] = entries[start:end].reshape(shape)
  
    return H_tb

# Usage
H_matrices = extract_hamiltonian('hamiltonian.h5')

4. AO Wavefunction Coefficients#

The wavefunction_ao.h5 file stores band eigenvalues and the corresponding wavefunction coefficients in the atomic-orbital (AO) basis. It is written by AOWfnObj.to_h5(). If no output path is given, the default filename is wavefunction_ao.h5 under the same data directory as POSCAR and info.json. It stores dense arrays indexed by k point, band, spinor, and AO orbital.

Each wavefunction_ao.h5 file can contain the following keys:

Key	Shape	Required	Description
`efermi`	scalar	Yes	Fermi energy associated with the eigenvalues, in eV
`kpts`	`(N_k, 3)`	Yes	k-points in reduced reciprocal coordinates
`kgrid`	`(3,)`	Optional	Monkhorst-Pack-style k-grid dimensions
`eigenvalues`	usually `(N_band,)`	Yes	Band eigenvalues in eV.
`eigenvectors`	`(N_k, N_band, N_spinor, N_orb)`	Yes	AO wavefunction coefficients \(c_{n,j\beta}(\mathbf{k})\) after separating the spinor axis

The stored coefficients represent the AO coefficients in the Bloch wavefunction expansion

\[ \psi_{n \mathbf{k}}(\mathbf{r}) = \frac{1}{\sqrt{N}}\sum_{\mathbf{R}}\sum_{j\beta \in \mathrm{uc}} e^{i\mathbf{k}\cdot\mathbf{R}} c_{n,j\beta}(\mathbf{k}) \phi_{j\beta}(\mathbf{r} - \mathbf{R} - \mathcal{R}_j), \]

where \(j\) indexes atoms in POSCAR order, \(\beta\) indexes the AO functions on each atom, and N_orb is the total number of AO basis functions in the unit cell without spin duplication. For non-spinful systems, N_spinor = 1. For spinful systems, N_spinor = 2, and the spin components are stored on the separate spinor axis.

When the coefficients come from HamiltonianObj.diag(..., bands_only=False), the returned eigenvectors have shape (N_orb, N_band, N_k). Before loading them into AOWfnObj, transpose them to (N_k, N_band, N_orb) for non-spinful systems:

import numpy as np

gamma = np.array([[0.0, 0.0, 0.0]])
eigvals, eigvecs = ham.diag(gamma, bands_only=False)
wfnao = eigvecs.transpose(2, 1, 0)

wfn = AOWfnObj(data_dir, basis_dir, "openmx")
wfn.load(
    kpts=gamma,
    wfnao=wfnao,
    el=eigvals[:, 0],
    spinful=False,
    efermi=fermi_energy,
)
wfn.to_h5()

The real-space wavefunction grid can be computed from these AO coefficients and basis functions by AOWfnObj.to_real_space(). It is a derived quantity and is not stored in wavefunction_ao.h5.

5. Real-Space Grid-Resolved Properties#

These HDF5 files store properties on a real-space grid:

charge_density.h5 - Electron charge density
potential_r.h5 - Local potential

Each HDF5 file contains the following keys:

Key	Shape	Description
`shape`	(3,)	Integer array specifying grid divisions in x, y, z directions
`entries`	(M,)	Flattened 1D array that can be reshaped to `shape`

Code Example: Reading Grid Data#import numpy as np
import h5py

def read_grid_data(filepath):
    """Read and reshape real-space grid data."""
    with h5py.File(filepath, 'r') as f:
        shape = f['shape'][:]
        entries = f['entries'][:]
  
    return entries.reshape(shape)

# Usage
charge_density = read_grid_data('charge_density.h5')

6. Force Field Properties (force.h5)#

The force.h5 file contains atom-resolved force field information.

Each HDF5 file contains the following keys:

Key	Shape	Description
`cell`	(3, 3)	Lattice vectors
`energy`	scalar	Total energy of the system
`force`	(N, 3)	Forces on N atoms in x, y, z directions
`stress`	(6,)	Stress tensor components in Voigt notation

Code Example: Reading Force Data#import h5py

def read_force_data(filepath):
    """Read force field data from force.h5."""
    with h5py.File(filepath, 'r') as f:
        cell = f['cell'][:] if 'cell' in f else None
        energy = f['energy'][()] if 'energy' in f else None
        force = f['force'][:]
        stress = f['stress'][:] if 'stress' in f else None
  
    return {
        'cell': cell,
        'energy': energy,
        'force': force,
        'stress': stress
    }

# Usage
force_data = read_force_data('force.h5')

Data Flow in DeepH-dock#

Understanding these formats is crucial for working with DeepH-dock:

Input: DFT software outputs are converted to these standardized formats
Processing: DeepH modules operate on the data using these consistent representations
Output: Results are stored in the same formats for interoperability

For more detailed specifications and updates to these formats, please refer to the latest documentation and the examples/ directory in the repository.

Key Concepts

Contents

Key Concepts#

Overview#

Folder Structure#

File Descriptions#

File Types and Their Purposes#

1. POSCAR - Atomic Structure Information#

2. info.json - Metadata and System Information#

3. HDF5 Files for Electronic Structure Properties#

Common Files#

Component Descriptions#

Spinful Systems#

Split KB Hamiltonian Storage#

Code Example: Extracting Hamiltonian Matrix Elements#

4. AO Wavefunction Coefficients#

5. Real-Space Grid-Resolved Properties#

Code Example: Reading Grid Data#

6. Force Field Properties (force.h5)#

Code Example: Reading Force Data#

Data Flow in DeepH-dock#