simopt.bootstrap ================ .. py:module:: simopt.bootstrap .. autoapi-nested-parse:: Bootstrapping procedures. Module Contents --------------- .. py:function:: bootstrap_sample_all(experiments: list[list[simopt.experiment.ProblemSolver]], bootstrap_rng: mrg32k3a.mrg32k3a.MRG32k3a, normalize: bool = True, feasibility_score_method: Literal['inf_norm', 'norm'] = 'inf_norm', feasibility_norm_degree: int = 1, feasibility_two_sided: bool = False) -> tuple[list[list[list[simopt.curve.Curve]]], list[list[list[simopt.curve.Curve]]]] Generates bootstrap samples of progress and feasibility curves. :param experiments: Grid of problem-solver pairs, where each inner list corresponds to different problems for a given solver. :type experiments: list[list[ProblemSolver]] :param bootstrap_rng: Random number generator used for bootstrapping. :type bootstrap_rng: MRG32k3a :param normalize: If True, normalize progress curves by optimality gaps. Defaults to True. :type normalize: bool, optional :param feasibility_score_method: Feasibility scoring rule. :type feasibility_score_method: Literal["inf_norm", "norm"], optional :param feasibility_norm_degree: Degree of the norm when ``feasibility_score_method == "norm"``. :type feasibility_norm_degree: int, optional :param feasibility_two_sided: Whether to give feasible solutions a non-zero score based on the best violation. :type feasibility_two_sided: bool, optional :returns: Bootstrapped progress/objective curves and feasibility curves for all solutions from all macroreplications, grouped by solver and problem. :rtype: tuple[list[list[list[Curve]]], list[list[list[Curve]]]] .. py:function:: bootstrap_procedure(experiments: list[list[simopt.experiment.ProblemSolver]], n_bootstraps: int, conf_level: float, plot_type: simopt.plot_type.PlotType, beta: float | None = None, solve_tol: float | None = None, estimator: float | simopt.curve.Curve | None = None, normalize: bool = True, feasibility_score_method: Literal['inf_norm', 'norm'] = 'inf_norm', feasibility_norm_degree: int = 1, feasibility_two_sided: bool = False) -> tuple[float, float] | tuple[simopt.curve.Curve, simopt.curve.Curve] Performs bootstrapping and computes confidence intervals for progress curves. :param experiments: Grid of problem-solver pairs. :type experiments: list[list[ProblemSolver]] :param n_bootstraps: Number of bootstrap samples to generate. :type n_bootstraps: int :param conf_level: Confidence level for the interval (0 < conf_level < 1). :type conf_level: float :param plot_type: Type of plot/metric for which to compute the interval. :type plot_type: PlotType :param beta: Quantile level (0 < beta < 1), used with some plot types. :type beta: float, optional :param solve_tol: Relative optimality gap that defines a "solved" instance. :type solve_tol: float, optional :param estimator: Reference estimator for difference plot types. :type estimator: float or Curve, optional :param normalize: Whether to normalize progress curves. Defaults to True. :type normalize: bool, optional :param feasibility_score_method: Feasibility scoring rule used when `plot_type` corresponds to feasibility metrics. :type feasibility_score_method: Literal["inf_norm", "norm"], optional :param feasibility_norm_degree: Degree of the norm when ``feasibility_score_method == "norm"``. :type feasibility_norm_degree: int, optional :param feasibility_two_sided: Whether to assign a non-zero score to feasible solutions based on the best violation. :type feasibility_two_sided: bool, optional :returns: Lower and upper bounds of the CI. :rtype: tuple[float, float] or tuple[Curve, Curve] .. py:function:: functional_of_curves(bootstrap_curves: list[list[list[simopt.curve.Curve]]], plot_type: simopt.plot_type.PlotType, beta: float | None = 0.5, solve_tol: float | None = 0.1) -> float | simopt.curve.Curve Computes a functional of bootstrapped objective or progress curves. :param bootstrap_curves: Bootstrapped curves for all solutions across all macroreplications. :type bootstrap_curves: list[list[list[Curve]]] :param plot_type: Type of functional to compute: - PlotType.MEAN - PlotType.QUANTILE - PlotType.AREA_MEAN - PlotType.AREA_STD_DEV - PlotType.SOLVE_TIME_QUANTILE - PlotType.SOLVE_TIME_CDF - PlotType.CDF_SOLVABILITY - PlotType.QUANTILE_SOLVABILITY - PlotType.DIFFERENCE_OF_CDF_SOLVABILITY - PlotType.DIFFERENCE_OF_QUANTILE_SOLVABILITY :type plot_type: PlotType :param beta: Quantile level (0 < beta < 1). Defaults to 0.5. :type beta: float, optional :param solve_tol: Optimality gap for defining a solved instance (0 < solve_tol ≤ 1). Defaults to 0.1. :type solve_tol: float, optional :returns: The computed functional of the curves. :rtype: Curve or float :raises ValueError: If input values are invalid or unsupported for the given plot_type. .. py:function:: compute_bootstrap_conf_int(observations: list[float | int], conf_level: float, bias_correction: bool = True, overall_estimator: float | None = None) -> tuple[numpy.ndarray, numpy.ndarray] Construct a bootstrap confidence interval for an estimator. :param observations: Estimators from all bootstrap instances. :type observations: list[float | int] :param conf_level: Confidence level for confidence intervals, i.e., 1 - gamma; must be in (0, 1). :type conf_level: float :param bias_correction: Whether to use bias-corrected bootstrap CIs (via the percentile method). Defaults to True. :type bias_correction: bool, optional :param overall_estimator: The estimator to compute the CI around. Required if`bias_correction` is True. :type overall_estimator: float | None, optional :returns: A tuple containing the lower and upper bounds of the bootstrap confidence interval. :rtype: tuple[np.ndarray, np.ndarray] :raises ValueError: If `conf_level` is not in (0, 1), or if `overall_estimator` is None when `bias_correction` is True.