Behaviour Metrics

behavior_metrics.py

Utility functions for analyzing single-run black box optimization traces.

Each function expects a pandas DataFrame that contains at least:

• a column “evaluations” integer evaluation index (starting at 1 or 0)

• a column “raw_y” objective function value (the lower the better)

• coordinate columns named “x0”, “x1”, … “x{d-1}”

The helper get_coordinates(df) extracts the X matrix.

Many metrics also accept optional keyword arguments such as bounds (the search space lower/upper bounds) or radius.

All functions return a scalar float unless stated otherwise.

Author: Niki van Stein (May 2025)

iohblade.behaviour_metrics.average_convergence_rate(df: DataFrame, optimum: float | None = None) → float

Geometric mean of successive error ratios (Chen & He, 2020). ACR < 1 implies convergence; smaller = faster.

iohblade.behaviour_metrics.average_distance_to_best_so_far(df: DataFrame) → float

For each evaluation k>0, compute distance to best point found up to k-1. Average these distances lower = more exploitative.

iohblade.behaviour_metrics.average_nearest_neighbor_distance(df: DataFrame, step=10, history=1000) → float

Average Euclidean distance from each point (except the first) to its nearest previous point – proxy for sequential novelty / exploration.

iohblade.behaviour_metrics.avg_exploration_exploitation_chunked(df: DataFrame, chunk_size: int = 500, bounds: Sequence[Tuple[float, float]] | None = None, rng: Generator | None = None) → Tuple[float, float]

iohblade.behaviour_metrics.closed_form_random_search_diversity(bounds: Sequence[Tuple[float, float]]) → float

iohblade.behaviour_metrics.compute_behavior_metrics(df: DataFrame, *, bounds: Sequence[Tuple[float, float]] | None = None, radius: float | None = None, disp_samples: int = 10000)

Compute all scalar behavior metrics and return them in a dictionary.

iohblade.behaviour_metrics.coverage_dispersion(df: DataFrame, bounds: Sequence[Tuple[float, float]] = None, n_samples: int = 10000, rng: Generator | None = None) → float

Approximate dispersion: max distance from a random point in the domain to its nearest evaluated sample. Lower is better (more coverage).

iohblade.behaviour_metrics.estimate_random_search_diversity(bounds: Sequence[Tuple[float, float]], n: int = 500, samples: int = 10000, rng: Generator | None = None) → float

iohblade.behaviour_metrics.get_coordinates(df: DataFrame) → ndarray

Return (N, d) array with decision variables extracted from x-columns.

iohblade.behaviour_metrics.get_objective(df: DataFrame) → ndarray

Return 1‑D array with objective values (raw_y).

iohblade.behaviour_metrics.improvement_statistics(df: DataFrame) → Tuple[float, float]

Returns (avg_improvement, success_rate) where success_rate is the fraction of iterations that improved the best, and avg_improvement is the mean improvement magnitude on improving iterations.

iohblade.behaviour_metrics.intensification_ratio(df: DataFrame, radius: float) → float

Fraction of evaluations lying within a given radius of the final best point. Higher -> stronger intensification / exploitation near the best.

iohblade.behaviour_metrics.last_improvement_fraction(df: DataFrame) → float

Fraction of evaluations since the last improvement (0-1).

iohblade.behaviour_metrics.longest_no_improvement_streak(df: DataFrame) → int

Return the length of the longest consecutive streak with no improvement.

iohblade.behaviour_metrics.spatial_entropy(df: DataFrame, bandwidth: str | float = 'scott') → float

Differential entropy estimate of the empirical sample distribution. Uses Gaussian KDE from scipy. High entropy -> diverse sampling.