simopt.bootstrap

Bootstrapping procedures.

Module Contents

simopt.bootstrap.bootstrap_sample_all(experiments: list[list[simopt.experiment.ProblemSolver]], bootstrap_rng: mrg32k3a.mrg32k3a.MRG32k3a, normalize: bool = True, feasibility_score_method: Literal['inf_norm', 'norm'] = 'inf_norm', feasibility_norm_degree: int = 1, feasibility_two_sided: bool = False) → tuple[list[list[list[simopt.curve.Curve]]], list[list[list[simopt.curve.Curve]]]]

Generates bootstrap samples of progress and feasibility curves.

Parameters:

  • experiments (list[list[ProblemSolver]]) – Grid of problem-solver pairs, where each inner list corresponds to different problems for a given solver.

  • bootstrap_rng (MRG32k3a) – Random number generator used for bootstrapping.

  • normalize (bool, optional) – If True, normalize progress curves by optimality gaps. Defaults to True.

  • feasibility_score_method (Literal["inf_norm", "norm"], optional) – Feasibility scoring rule.

  • feasibility_norm_degree (int, optional) – Degree of the norm when feasibility_score_method == "norm".

  • feasibility_two_sided (bool, optional) – Whether to give feasible solutions a non-zero score based on the best violation.

Returns:

Bootstrapped progress/objective curves and feasibility curves for all solutions from all macroreplications, grouped by solver and problem.

Return type:

tuple[list[list[list[Curve]]], list[list[list[Curve]]]]

simopt.bootstrap.bootstrap_procedure(experiments: list[list[simopt.experiment.ProblemSolver]], n_bootstraps: int, conf_level: float, plot_type: simopt.plot_type.PlotType, beta: float | None = None, solve_tol: float | None = None, estimator: float | simopt.curve.Curve | None = None, normalize: bool = True, feasibility_score_method: Literal['inf_norm', 'norm'] = 'inf_norm', feasibility_norm_degree: int = 1, feasibility_two_sided: bool = False) → tuple[float, float] | tuple[simopt.curve.Curve, simopt.curve.Curve]

Performs bootstrapping and computes confidence intervals for progress curves.

Parameters:

  • experiments (list[list[ProblemSolver]]) – Grid of problem-solver pairs.

  • n_bootstraps (int) – Number of bootstrap samples to generate.

  • conf_level (float) – Confidence level for the interval (0 < conf_level < 1).

  • plot_type (PlotType) – Type of plot/metric for which to compute the interval.

  • beta (float, optional) – Quantile level (0 < beta < 1), used with some plot types.

  • solve_tol (float, optional) – Relative optimality gap that defines a “solved” instance.

  • estimator (float or Curve, optional) – Reference estimator for difference plot types.

  • normalize (bool, optional) – Whether to normalize progress curves. Defaults to True.

  • feasibility_score_method (Literal["inf_norm", "norm"], optional) – Feasibility scoring rule used when plot_type corresponds to feasibility metrics.

  • feasibility_norm_degree (int, optional) – Degree of the norm when feasibility_score_method == "norm".

  • feasibility_two_sided (bool, optional) – Whether to assign a non-zero score to feasible solutions based on the best violation.

Returns:

Lower and upper bounds of the CI.

Return type:

tuple[float, float] or tuple[Curve, Curve]

simopt.bootstrap.functional_of_curves(bootstrap_curves: list[list[list[simopt.curve.Curve]]], plot_type: simopt.plot_type.PlotType, beta: float | None = 0.5, solve_tol: float | None = 0.1) → float | simopt.curve.Curve

Computes a functional of bootstrapped objective or progress curves.

Parameters:

  • bootstrap_curves (list[list[list[Curve]]]) – Bootstrapped curves for all solutions across all macroreplications.

  • plot_type (PlotType) – Type of functional to compute: - PlotType.MEAN - PlotType.QUANTILE - PlotType.AREA_MEAN - PlotType.AREA_STD_DEV - PlotType.SOLVE_TIME_QUANTILE - PlotType.SOLVE_TIME_CDF - PlotType.CDF_SOLVABILITY - PlotType.QUANTILE_SOLVABILITY - PlotType.DIFFERENCE_OF_CDF_SOLVABILITY - PlotType.DIFFERENCE_OF_QUANTILE_SOLVABILITY

  • beta (float, optional) – Quantile level (0 < beta < 1). Defaults to 0.5.

  • solve_tol (float, optional) – Optimality gap for defining a solved instance (0 < solve_tol ≤ 1). Defaults to 0.1.

Returns:

The computed functional of the curves.

Return type:

Curve or float

Raises:

ValueError – If input values are invalid or unsupported for the given plot_type.

simopt.bootstrap.compute_bootstrap_conf_int(observations: list[float | int], conf_level: float, bias_correction: bool = True, overall_estimator: float | None = None) → tuple[numpy.ndarray, numpy.ndarray]

Construct a bootstrap confidence interval for an estimator.

Parameters:

  • observations (list[float | int]) – Estimators from all bootstrap instances.

  • conf_level (float) – Confidence level for confidence intervals, i.e., 1 - gamma; must be in (0, 1).

  • bias_correction (bool, optional) – Whether to use bias-corrected bootstrap CIs (via the percentile method). Defaults to True.

  • overall_estimator (float | None, optional) – The estimator to compute the CI around. Required if`bias_correction` is True.

Returns:

A tuple containing the lower and upper bounds of

the bootstrap confidence interval.

Return type:

tuple[np.ndarray, np.ndarray]

Raises:

ValueError – If conf_level is not in (0, 1), or if overall_estimator is None when bias_correction is True.