SAMBO

Say you're a senior baker in a large pharmaceutical, tasked to ship a new medicine drug for suppressing symptoms of any of the growingly lucrative health conditions of the developed world. Among simple paper-pushing and the more menial tasks, your drug development pipeline involves figuring out the correct, for business reasons quite optimal, combination to hundreds of parameters, such as: ratios of active ingredients, bulking agents, centrifugation times, pH levels, temperature profiles at each processing step, choice of solvents and purification steps, ideal dosage forms, whether to include any of the other common processes of your establishment, and so on.

The problem is: because some processes can't be skipped or sped up, every combination you decide to try takes two of your assistant researchers nearly two weeks of laboratory work. But your new flagship drug is expected due next Thursday ...

It is easy to get a thousand prescriptions, but hard to get one single remedy.

— Chinese proverb

Thank god for SAMBO, Rambo of global optimization. It gets in and then it finds minimums of the objective criteria function quickly and efficiently, in least number of evaluations. SAMBO stands for Sequential and Model-Based Optimization. This simple optimization package consists of the following items, each with its own neat, user-friendly, Pythonic interface:

• function sambo.minimize() to drive constrained and bounded global black-box optimization, design-space exploration and model calibration, modeled after well-known Python packages SciPy and scikit-optimize,¹ supporting SOTA optimization algorithms like SHGO,² SMBO³ and SCE-UA,⁴
• class Optimizer that provides an ask-and-tell interface, additionally supporting sequential surrogate models produced by estimators like those of scikit-learn, skorch or Keras, with popular algorithms including Gaussian process and tree-based regression built in,
• SamboSearchCV, a faster drop-in replacement for GridSearchCV, RandomizedSearchCV and similar methods of hyperparameter tuning in complex ML pipelines.

See below examples for usage.

1. ¹ scikit-optimize/scikit-optimize. DOI: 10.5281/zenodo.1157319
2. ² SHGO: Simplicial homology global optimization. DOI: 10.1007/s10898-018-0645-y
3. ³ SMBO: Sequential Model-Based Optimization for General Algorithm Configuration. DOI: 10.1007/978-3-642-25566-3 40
4. ⁴ SCE-UA: Effective and efficient global optimization for conceptual rainfall-runoff models. DOI: 10.1029/91WR02985

import sambo
from sambo.plot import *
# Extras
import matplotlib.pyplot as plt
from scipy.optimize import rosen

result = sambo.minimize(
    rosen, bounds=[(-2, 2)]*2, method='shgo',
    constraints=lambda x: sum(x**2) <= 2)

plot_convergence(result)
plot_objective(result)    # Partial dependence
plot_evaluations(result)  # Sequence of evaluations
plot_regret(result)

plt.show()

<class 'sambo.OptimizeResult'>

 message: Optimization terminated successfully.
 success: True
     fun: 5.205243704996618e-08
       x: [ 9.998e-01  9.996e-01]
    nfev: 68
      xv: [[ 0.000e+00  0.000e+00]
           [ 0.000e+00  0.000e+00]
           ...
           [ 9.998e-01  9.996e-01]
           [ 9.998e-01  9.996e-01]]
    funv: [ 1.000e+00  ...  5.210e-08]

from sambo import Optimizer

def evaluate(x):
   ...  # Abstract long and laborious process
   return rosen(x)

optimizer = Optimizer(
    fun=None,  # Implies ask-and-tell interface
    bounds=[(-2, 2)]*2,
    estimator='gp',  # or bring your own
)

for i in range(100):
    suggested_x = optimizer.ask(1)
    y = [evaluate(x) for x in suggested_x]
    optimizer.tell(y)

result: OptimizeResult = optimizer.run()

# Example setup of a scikit-learn pipeline
from sklearn.datasets import load_breast_cancer
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV

X, y = load_breast_cancer(return_X_y=True)
clf = DecisionTreeClassifier()
param_grid = {
    'max_depth': list(range(1, 30)),
    'min_samples_split': [2, 5, 10, 20, 50, 100],
    'min_samples_leaf': list(range(1, 20)),
    'criterion': ['gini', 'entropy'],
    'max_features': [None, 'sqrt', 'log2'],
}
search = GridSearchCV(clf, param_grid)
# Trying all ~20k combinations takes a long time ...
search.fit(X, y)
print(search.best_params_)
print(search.best_score_)

# Alternatively ...
from sambo import SamboSearchCV
search = SamboSearchCV(clf, param_grid, max_iter=100)
search.fit(X, y)  # Fast, good enough
print(search.best_params_)
print(search.best_score_)
print(search.opt_result_)

{'criterion': 'gini',
 'max_depth': 6,
 'max_features': 'sqrt',
 'min_samples_leaf': 1,
 'min_samples_split': 5}
0.947269582406721

{'criterion': 'entropy',
 'max_depth': 20,
 'max_features': None,
 'min_samples_leaf': 5,
 'min_samples_split': 6}
0.9419940696812453

 message: Reached n_iter_no_change (=10) w/o improvement
 success: True
     fun: -0.9419940696812453
       x: [20 6 5 'entropy' None]
     nit: 26
    nfev: 77
      xv: [[15 86 ... 'gini' None]
           [1 57 ... 'entropy' 'sqrt']
           ...
           [19 5 ... 'gini' None]
           [20 8 ... 'gini' None]]
    funv: [-9.244e-01 -9.034e-01 ... -9.139e-01 -9.191e-01]
   model: [GaussianProcessRegressor()]

@software{SAMBO,
    author = {Kernc},
    title = {SAMBO: Sequential and Model-Based Optimization: Efficient global optimization in Python},
    url = {https://sambo-optimization.github.io},
    doi = {https://doi.org/10.5281/zenodo.14461363},
    year = {2024}
}