simopt.experiment.multiple

Multiple problem-solver pairs.

Module Contents

simopt.experiment.multiple.model_directory
simopt.experiment.multiple.problem_directory
simopt.experiment.multiple.solver_directory
class simopt.experiment.multiple.ProblemsSolvers(solver_factors: list[dict] | None = None, problem_factors: list[dict] | None = None, solver_names: list[str] | None = None, problem_names: list[str] | None = None, solver_renames: list[str] | None = None, problem_renames: list[str] | None = None, fixed_factors_filename: str | None = None, solvers: list[simopt.base.Solver] | None = None, problems: list[simopt.base.Problem] | None = None, experiments: list[list[simopt.experiment.single.ProblemSolver]] | None = None, file_name_path: pathlib.Path | None = None, create_pair_pickles: bool = False, experiment_name: str | None = None)

Base class for running one or more solver on one or more problem.

Initialize a ProblemsSolvers object.

There are three ways to initialize a ProblemsSolvers object: 1. Provide the names of solvers and problems (for lookup in directory.py). 2. Provide lists of solver and problem objects to pair directly. 3. Provide a full list of ProblemSolver objects (as nested lists).

Parameters:

  • solver_factors (list[dict] | None) – List of solver factor dictionaries, one per design point. Requires solver_names to match the number of entries.

  • problem_factors (list[dict] | None) – List of problem/model factor dictionaries, one per design point. Requires problem_names to match the number of entries.

  • solver_names (list[str] | None) – List of solver names to look up.

  • problem_names (list[str] | None) – List of problem names to look up.

  • solver_renames (list[str] | None) – User-specified labels for solvers.

  • problem_renames (list[str] | None) – User-specified labels for problems.

  • fixed_factors_filename (str | None) – Name of a .py file containing fixed factor dictionaries.

  • solvers (list[Solver] | None) – List of Solver objects to use directly.

  • problems (list[Problem] | None) – List of Problem objects to use directly.

  • experiments (list[list[ProblemSolver]] | None) – Explicit problem-solver pairings.

  • file_name_path (Path | None) – Output path for saving the ProblemsSolvers object.

  • create_pair_pickles (bool) – Whether to create individual .pickle files for each problem-solver pair.

  • experiment_name (str | None) – Optional name to prefix output files.

property solver_names: list[str]

List of solver names.

property n_solvers: int

Number of solvers.

property problem_names: list[str]

List of problem names.

property n_problems: int

Number of problems.

property solvers: list[simopt.base.Solver]

List of solvers.

property problems: list[simopt.base.Problem]

List of problems.

property all_solver_fixed_factors: dict[str, dict]

Fixed solver factors for each solver.

property all_problem_fixed_factors: dict[str, dict]

Fixed problem factors for each problem.

property all_model_fixed_factors: dict[str, dict]

Fixed model factors for each problem.

property experiments: list[list[simopt.experiment.single.ProblemSolver]]

All problem-solver pairs.

property file_name_path: pathlib.Path

Path to the .pickle file for saving the ProblemsSolvers object.

property create_pair_pickles: bool

Whether to create pickle files for each problem-solver pair.

property experiment_name: str

Name of experiment to be appended to the beginning of output files.

check_compatibility() → str

Check whether all experiments’ solvers and problems are compatible.

Returns:

Error message in the event any problem and solver are incompatible.

Return type:

str

run(n_macroreps: int) → None

Run n_macroreps of each solver on each problem.

Parameters:

n_macroreps (int) – Number of macroreplications to run per problem-solver pair.

Raises:

ValueError – If n_macroreps is not positive.

post_replicate(n_postreps: int, crn_across_budget: bool = True, crn_across_macroreps: bool = False) → None

Runs postreplications for each problem-solver pair on all macroreplications.

Parameters:

  • n_postreps (int) – Number of postreplications per recommended solution.

  • crn_across_budget (bool, optional) – If True, use CRN across solutions from different time budgets. Defaults to True.

  • crn_across_macroreps (bool, optional) – If True, use CRN across solutions from different macroreplications. Defaults to False.

Raises:

ValueError – If n_postreps is not positive.

post_normalize(n_postreps_init_opt: int, crn_across_init_opt: bool = True) → None

Builds objective and progress curves for all experiment collections.

Parameters:

  • n_postreps_init_opt (int) – Number of postreplications at initial (x0) and optimal (x*) solutions.

  • crn_across_init_opt (bool, optional) – If True, use CRN for postreplications at x0 and x*. Defaults to True.

Raises:

ValueError – If n_postreps_init_opt is not positive.

check_postreplicate() → bool

Checks whether all experiments have been postreplicated.

Returns:

Whether all experiments have been postreplicated

Return type:

bool

check_postnormalize() → bool

Checks whether all experiments have been postnormalized.

Returns:

Whether all experiments have been postnormalized.

Return type:

bool

record_group_experiment_results() → None

Saves a ProblemsSolvers object to a .pickle file in the outputs directory.

log_group_experiment_results() → None

Creates a .txt summary of solvers and problems in the ProblemSolvers object.

The file is saved in the ‘logs/’ folder next to the experiment’s pickle file.

report_group_statistics(solve_tols: list[float] | None = None, csv_filename: str = 'df_solver_results') → None

Reports statistics for all solvers across all problems.

Parameters:

  • solve_tols (list[float], optional) – Optimality gaps defining when a problem is considered solved (values in (0, 1]). Defaults to [0.05, 0.10, 0.20, 0.50].

  • csv_filename (str, optional) – Name of the output CSV file (without ‘.csv’). Defaults to “df_solver_results”.

Raises:

ValueError – If any solve tolerance is not in the range (0, 1].

report_statistics(pair_list: list[simopt.experiment.single.ProblemSolver], solve_tols: list[float] | None = None, csv_filename: str = 'df_solver_results') → None

Calculates statistics from macroreplications and saves results to a CSV file.

Parameters:

  • pair_list (list[ProblemSolver]) – List of ProblemSolver objects.

  • solve_tols (list[float], optional) – Optimality gaps defining when a problem is considered solved (values in (0, 1]). Defaults to [0.05, 0.10, 0.20, 0.50].

  • csv_filename (str, optional) – Name of the CSV file to write results to. Defaults to “df_solver_results”.

Raises:

ValueError – If any solve tolerance is not in the range (0, 1].