boltzkit.evaluation.density_based.divergence

Functions

get_alpha_divergence_p(log_weights, alpha[, ...])	Estimate the Amari α-divergence using samples from p(x).
get_alpha_divergence_q(log_weights, alpha[, ...])	Estimate the Amari α-divergence using samples from q(x).
get_forward_logZ(log_weights)	Estimate the log-normalization constant of \(\tilde{p}(x)\) using samples from p(x).
get_kl_divergence_p(log_weights[, logZ])	Compute the forward KL using samples from p.
get_kl_divergence_q(log_weights[, logZ])	Compute the importance-weighted forward KL using samples from q.
get_reverse_logZ(log_weights)	Estimate the log-normalization constant of \(\tilde{p}(x)\) using samples from q(x).

boltzkit.evaluation.density_based.divergence.get_reverse_logZ(log_weights: ndarray) → float

Estimate the log-normalization constant of \(\tilde{p}(x)\) using samples from q(x).

For \(x \sim q(x)\),

\[\log w(x) = \log \tilde{p}(x) - \log q(x), \quad Z_r = \mathbb{E}_q[w].\]

Therefore,

\[\log Z_r = \log \mathbb{E}_q[w].\]

Parameters:

log_weights (np.ndarray) – Log importance weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim q(x)\).

Returns:

Estimated \(\log Z_r\).

Return type:

float

boltzkit.evaluation.density_based.divergence.get_forward_logZ(log_weights: ndarray) → float

Estimate the log-normalization constant of \(\tilde{p}(x)\) using samples from p(x).

For \(x \sim p(x)\),

\[\log w(x) = \log \tilde{p}(x) - \log q(x), \quad Z_f = 1/\mathbb{E}_p[1/w].\]

Therefore,

\[\log Z_f = -\log \mathbb{E}_p[1/w].\]

Parameters:

log_weights (np.ndarray) – Log weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim p(x)\).

Returns:

Estimated \(\log Z_f\).

Return type:

float

boltzkit.evaluation.density_based.divergence.get_kl_divergence_q(log_weights: ndarray, logZ: float | None = None)

Compute the importance-weighted forward KL using samples from q.

For \(x \sim q(x)\),

\[\log w(x) = \log p(x) - \log q(x), \quad \mathrm{KL}(p \| q) = \mathbb{E}_q[w \log w].\]

Parameters:

• log_weights (np.ndarray) – Unnormalized log importance weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim q(x)\).

• logZ (float | None) – Log normalization constant (or estimate thereof) of \(\tilde{p}\). If None, estimated from log_weights.

Returns:

Importance-weighted forward KL estimate.

Return type:

float

boltzkit.evaluation.density_based.divergence.get_kl_divergence_p(log_weights: ndarray, logZ: float | None = None)

Compute the forward KL using samples from p.

For \(x \sim p(x)\),

\[\log w(x) = \log p(x) - \log q(x), \quad \mathrm{KL}(p \| q) = \mathbb{E}_p[\log w].\]

Parameters:

• log_weights (np.ndarray) – Unnormalized log importance weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim p(x)\).

• logZ (float | None) – Log normalization constant (or estimate thereof) of \(\tilde{p}\). If None, estimated from log_weights.

Returns:

Forward KL estimate.

Return type:

float

boltzkit.evaluation.density_based.divergence.get_alpha_divergence_q(log_weights: ndarray, alpha: float, logZ: float | None = None)

Estimate the Amari α-divergence using samples from q(x).

For \(x \sim q(x)\),

\[\log w(x) = \log p(x) - \log q(x), \quad D_\alpha(p \| q) = \frac{\mathbb{E}_q[w^{\alpha}] - 1}{\alpha(\alpha - 1)}.\]

Using the mapping \(\alpha = (1 + \beta)/2\), this corresponds to the Amari α-divergence for \(\beta \neq -1, 1\). In the limits \(\beta \to -1\) and \(\beta \to 1\) (equivalently \(\alpha \to 0\) and \(\alpha \to 1\)), it recovers \(\mathrm{KL}(q \| p)\) and \(\mathrm{KL}(p \| q)\), respectively.

Parameters:

• log_weights (np.ndarray) – Unnormalized log importance weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim q(x)\).

• alpha (float) – The α parameter (\(\alpha \neq 0, 1\)).

• logZ (float | None) – Log normalization constant (or estimate thereof) of \(\tilde{p}\). If None, estimated from log_weights.

Returns:

Estimate of the α-divergence.

Return type:

float

boltzkit.evaluation.density_based.divergence.get_alpha_divergence_p(log_weights: ndarray, alpha: float, logZ: float | None = None)

Estimate the Amari α-divergence using samples from p(x).

For \(x \sim p(x)\),

\[\log w(x) = \log p(x) - \log q(x), \quad D_\alpha(p \| q) = \frac{\mathbb{E}_p[w^{\alpha - 1}] - 1}{\alpha(\alpha - 1)}.\]

Using the mapping \(\alpha = (1 + \beta)/2\), this corresponds to the Amari α-divergence for \(\beta \neq -1, 1\). In the limits \(\beta \to -1\) and \(\beta \to 1\) (equivalently \(\alpha \to 0\) and \(\alpha \to 1\)), it recovers \(\mathrm{KL}(q \| p)\) and \(\mathrm{KL}(p \| q)\), respectively.

Parameters:

• log_weights (np.ndarray) – Unnormalized log importance weights, i.e., \(\log \tilde{p}(x) - \log q(x)\) for samples \(x \sim p(x)\).

• alpha (float) – The α parameter (\(\alpha \neq 0, 1\)).

• logZ (float | None) – Log normalization constant (or estimate thereof) of \(\tilde{p}\). If None, estimated from log_weights.

Returns:

Estimate of the α-divergence.

Return type:

float