baxus.benchmarks package

Submodules

baxus.benchmarks.benchmark_function module

class baxus.benchmarks.benchmark_function.Benchmark(dim: int, ub: ndarray, lb: ndarray, noise_std: float)

Bases: ABC

Abstract benchmark function.

Parameters

  • dim – dimensionality of the objective function

  • noise_std – the standard deviation of the noise (None means no noise)

  • ub – the upper bound, the object will have the attribute ub_vec which is an np array of length dim filled with ub

  • lb – the lower bound, the object will have the attribute lb_vec which is an np array of length dim filled with lb

  • benchmark_func – the benchmark function, should inherit from SyntheticTestFunction

property dim: int

The benchmark dimensionality

Returns: the benchmark dimensionality

property fun_name: str

The name of the benchmark function

Returns: The name of the benchmark function

property lb_vec: ndarray

The lower bound of the search space of this benchmark (length = benchmark dim)

Returns: The lower bound of the search space of this benchmark (length = benchmark dim)

property ub_vec: ndarray

The upper bound of the search space of this benchmark (length = benchmark dim)

Returns: The upper bound of the search space of this benchmark (length = benchmark dim)

class baxus.benchmarks.benchmark_function.BoTorchFunctionBenchmark(dim: int, noise_std: Optional[float], ub: ndarray, lb: ndarray, benchmark_func: Type[SyntheticTestFunction])

Bases: SyntheticBenchmark

A benchmark function that calls a BoTorch benchmark function

Parameters

  • dim – dimensionality of the problem

  • noise_std – noise std of the function

  • ub – the upper bound of the search space

  • lb – the lower bound of the search space

  • benchmark_func – the BoTorch benchmark function

property effective_dim: int

The effective dimensionality of the benchmark.

Returns: The effective dimensionality of the benchmark.

property optimal_value: ndarray

The optimal value of the benchmark function.

Returns: The optimal value of the benchmark function.

class baxus.benchmarks.benchmark_function.EffectiveDimBenchmark(dim: int, effective_dim: int, ub: ndarray, lb: ndarray, noise_std: float)

Bases: SyntheticBenchmark

A benchmark with a known effective dimensionality.

Note

This is an abstract class that needs an implementation.

Parameters

  • dim – the overall dimensionality of the problem

  • effective_dim – the effective dimensionality of the problem

  • ub – the upper bounds of the search space

  • lb – the lower bounds of the search space

  • noise_std – the noise std for this benchmark

class baxus.benchmarks.benchmark_function.EffectiveDimBoTorchBenchmark(dim: int, noise_std: Optional[float], effective_dim: int, ub: ndarray, lb: ndarray, benchmark_func: Type[SyntheticTestFunction], seed: int = 123)

Bases: BoTorchFunctionBenchmark

A benchmark class for synthetic benchmarks with a known effective dimensionality that are based on a BoTorch implementation.

Parameters

  • dim – int: the ambient dimensionality of the benchmark

  • noise_std – float: standard deviation of the noise of the benchmark function

  • effective_dim – int: the desired effective dimensionality of the benchmark function

  • ub – np.ndarray: the upper bound of the benchmark search space. length = dim

  • lb – np.ndarray: the lower bound of the benchmark search space. length = dim

  • benchmark_func – Type[SyntheticTestFunction]: the BoTorch benchmark function to use

  • seed – int: random seed

property dim

The dimensionality of the problem.

Returns: The dimensionality of the problem.

property effective_dim: int

the effective dimensionality of the benchmark

Returns: the effective dimensionality of the benchmark

property lb_vec: ndarray

The lower bounds of the search space.

Returns: The lower bounds of the search space.

property ub_vec: ndarray

The upper bounds of the search space.

Returns: The upper bounds of the search space.

class baxus.benchmarks.benchmark_function.SyntheticBenchmark(dim: int, ub: ndarray, lb: ndarray, noise_std: float)

Bases: Benchmark

Abstract class for synthetic benchmarks

Parameters

  • dim – the benchmark dimensionality

  • ub – np.ndarray: the upper bound of the search space of this benchmark (length = benchmark dim)

  • lb – np.ndarray: the lower bound of the search space of this benchmark (length = benchmark dim)

property optimal_value: Optional[ndarray]

Returns: Optional[Union[float, np.ndarray]]: the optimal value if known

baxus.benchmarks.benchmark_utils module

baxus.benchmarks.benchmark_utils.run_and_plot(m: OptimizationMethod, repetitions: List[int], directory: str) → None

Run an experiment for a certain number of repetitions and save the results

Parameters

  • m – the experiment to runm_x

  • repetitions – the repetitions to run

  • directory – the directory to save the results

  • split_points – whether this experiment has split points

Returns

None

baxus.benchmarks.other_methods module

class baxus.benchmarks.other_methods.OptimizationMethod(run_dir: str, conf_name: Optional[str] = None)

Bases: ABC

property conf_dict: Dict[str, Any]
optimization_results_incumbent() → Tuple[ndarray, ndarray]

Get the incumbent optimization results, i.e., optimization results such that y_2 is always less or equal to y_1.

Returns

the x-values np.ndarray: the incumbent y-values

Return type

np.ndarray

abstract optimization_results_raw() → Tuple[Optional[ndarray], ndarray]

Get the raw optimization results, i.e., the x-values, the true function values, and the additional run information.

Returns

tuple[X’s, y’s, additional_run_information]

abstract optimize() → None

Start the optimization.

Returns: None

reset() → None
class baxus.benchmarks.other_methods.RandomSearch(function: Benchmark, input_dim: int, max_evals: int, run_dir: str, lower_bounds: ndarray, upper_bounds: ndarray)

Bases: OptimizationMethod

optimization_results_raw() → Tuple[Optional[ndarray], ndarray]

Get the raw optimization results, i.e., the x-values, the true function values, and the additional run information.

Returns

tuple[X’s, y’s, additional_run_information]

optimize() → None

Run the optimization.

Returns: None

baxus.benchmarks.real_world_benchmarks module

class baxus.benchmarks.real_world_benchmarks.LassoBreastCancerBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

10-D breast cancer benchmark from https://github.com/ksehic/LassoBench

Parameters

  • noise_std – ignored

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoDNABenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

180-D DNA benchmark from https://github.com/ksehic/LassoBench

Parameters

  • noise_std – ignored

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoDiabetesBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

8-D diabetes benchmark from https://github.com/ksehic/LassoBench

Parameters

  • noise_std – ignored

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoHardBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

1000-D synthetic Lasso hard benchmark from https://github.com/ksehic/LassoBench . Effective dimensionality: 5% of input dimensionality.

Parameters

  • noise_std – if > 0: noisy version with fixed SNR, noiseless version otherwise

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoHighBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

300-D synthetic Lasso high benchmark from https://github.com/ksehic/LassoBench . Effective dimensionality: 5% of input dimensionality.

Parameters

  • noise_std – if > 0: noisy version with fixed SNR, noiseless version otherwise

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoLeukemiaBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

7129-D Leukemia benchmark from https://github.com/ksehic/LassoBench

Parameters

  • noise_std – ignored

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoMediumBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

100-D synthetic Lasso medium benchmark from https://github.com/ksehic/LassoBench . Effective dimensionality: 5% of input dimensionality.

Parameters

  • noise_std – if > 0: noisy version with fixed SNR, noiseless version otherwise

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoRCV1Benchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

19 959-D RCV1 benchmark from https://github.com/ksehic/LassoBench

Parameters

  • noise_std – ignored

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.LassoSimpleBenchmark(noise_std: Optional[float] = 0, **kwargs)

Bases: EffectiveDimBenchmark

60-D synthetic Lasso simple benchmark from https://github.com/ksehic/LassoBench . Effective dimensionality: 5% of input dimensionality.

Parameters

  • noise_std – if > 0: noisy version with fixed SNR, noiseless version otherwise

  • **kwargs –

class baxus.benchmarks.real_world_benchmarks.MoptaSoftConstraints(temp_dir: Optional[str] = None, binary_path: Optional[str] = None, noise_std: Optional[float] = 0, **kwargs)

Bases: SyntheticBenchmark

Mopta08 benchmark with soft constraints as described in https://arxiv.org/pdf/2103.00349.pdf Supports i386, x86_84, armv7l

Parameters

  • temp_dir – Optional[str]: directory to which to write the input and output files (if not specified, a temporary directory will be created automatically)

  • binary_path – Optional[str]: path to the binary, if not specified, the default path will be used

property optimal_value: Optional[ndarray]

Return the “optimal” value.

Returns

-200, some guessed optimal value we never beat

Return type

np.ndarray

class baxus.benchmarks.real_world_benchmarks.SVMBenchmark(data_folder: Optional[str] = None, noise_std: Optional[float] = 0, **kwargs)

Bases: SyntheticBenchmark

baxus.benchmarks.synthetic_benchmark_functions module

class baxus.benchmarks.synthetic_benchmark_functions.AckleyEffectiveDim(dim=200, noise_std=None, effective_dim: int = 10)

Bases: EffectiveDimBoTorchBenchmark

A benchmark function with many local minima (see https://www.sfu.ca/~ssurjano/ackley.html)

WARNING: This function has its optimum at the origin. This might give a misleading performance for AdaTheSBO as the origin will always be reachable irregardless of the embedding.

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.BraninEffectiveDim(dim: int = 200, noise_std: Optional[float] = None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The Branin function with three local minima (see https://www.sfu.ca/~ssurjano/branin.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.DixonPriceEffectiveDim(dim=200, noise_std=None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The valley shaped Dixon-Price function (see https://www.sfu.ca/~ssurjano/dixonpr.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.GriewankEffectiveDim(dim=200, noise_std=None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The Griewank function with many local minima (see https://www.sfu.ca/~ssurjano/griewank.html)

WARNING: This function has its optimum at the origin. This might give a misleading performance for AdaTheSBO as the origin will always be reachable irregardless of the embedding.

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.HartmannEffectiveDim(dim: int = 200, noise_std: Optional[float] = None, effective_dim: int = 6)

Bases: EffectiveDimBoTorchBenchmark

A valley-shape benchmark function (see https://www.sfu.ca/~ssurjano/rosen.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.LevyEffectiveDim(dim=200, noise_std=None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The Levy function with many local minima (see https://www.sfu.ca/~ssurjano/levy.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.MichalewiczEffectiveDim(dim=200, noise_std=None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The Michalewicz function with steep drops (see https://www.sfu.ca/~ssurjano/michal.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.RastriginEffectiveDim(dim=200, noise_std=None, effective_dim: int = 2)

Bases: EffectiveDimBoTorchBenchmark

The Rastrigin function with many local minima (see https://www.sfu.ca/~ssurjano/rastr.html)

WARNING: This function has its optimum at the origin. This might give a misleading performance for AdaTheSBO as the origin will always be reachable irregardless of the embedding.

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.RosenbrockEffectiveDim(dim: int = 200, noise_std: Optional[float] = None, effective_dim: int = 10)

Bases: EffectiveDimBoTorchBenchmark

A valley-shape benchmark function (see https://www.sfu.ca/~ssurjano/rosen.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

class baxus.benchmarks.synthetic_benchmark_functions.RotatedHartmann6(noise_std: Optional[float] = None, **kwargs)

Bases: EffectiveDimBoTorchBenchmark

Version of the rotated Hartmann6 function as described in https://bit.ly/3dZFVXv

Parameters

noise_std – The standard deviation of the noise

class baxus.benchmarks.synthetic_benchmark_functions.ShiftedAckley10(dim=200, noise_std=None)

Bases: EffectiveDimBoTorchBenchmark

A benchmark function with many local minima (see https://www.sfu.ca/~ssurjano/ackley.html)

Parameters

  • dim – The ambient dimensionality of the function

  • noise_std – The standard deviation of the noise

  • effective_dim – The effective dimensionality of the function

Module contents