boltzkit.evaluation.density_based.entropy

Functions

get_shannon_entropy(log_probs)	Compute the Shannon entropy of a normalized distribution.
get_tsallis_entropy(log_probs, q)	Compute the Tsallis entropy from log-probabilities.

boltzkit.evaluation.density_based.entropy.get_shannon_entropy(log_probs: ndarray) → float

Compute the Shannon entropy of a normalized distribution.

Approximates:

\[H[p] = -\mathbb{E}_{p(x)}[\log p(x)]\]

using Monte Carlo samples.

Parameters:

log_probs (np.ndarray) – Log-probabilities of samples under the normalized distribution.

Returns:

Estimated Shannon entropy.

Return type:

float

boltzkit.evaluation.density_based.entropy.get_tsallis_entropy(log_probs: ndarray, q: float) → float

Compute the Tsallis entropy from log-probabilities.

The Tsallis entropy of order \(q \neq 1\) is:

\[H_q[p] = \frac{1 - \mathbb{E}_p[p(x)^{q-1}]}{q - 1}\]

Using log-probabilities for numerical stability:

\[p(x)^{q-1} = \exp((q - 1) \log p(x))\]

Parameters:

• log_probs (np.ndarray) – Log-probabilities of samples under a normalized distribution.

• q (float) – Entropy parameter. Must satisfy \(q \neq 1\).

Returns:

Estimated Tsallis entropy.

Return type:

float

Raises:

ValueError – If q == 1.0 (use Shannon entropy instead).