metamod.performance module

Module to access the performance of the surrogate.

metamod.performance.defined_metrics

Available metrics.

Type:dict

metamod.performance.RMSE(*args, **kwargs)

Returns the root mean squared error.

metamod.performance.benchmark_accuracy(surrogate)

Parameters:surrogate (core.Surrogate) – Trained surrogate. Returns:Benchmark accuracy statistics. Return type:diffs (dict)

metamod.performance.check_convergence(metrics)

Checks whether the metric meets the convergence criterion.

Parameters:metrics (list) – List of the convergence metrics for each iteration. Returns:Convergence status. Return type:trained (bool)

Notes

Need to add convergence if data is loaded and there is no more data to load

metamod.performance.convergence_operator()

Obtain either greater than or lower than operator based on the convergence metric type.

Returns:Direction logical operator. Return type:op (function)

metamod.performance.evaluate_metrics(inputs, outputs, predict)

Evaluates surrogate accuracy metrics based on test samples.

Parameters:

• inputs (np.array) – Test samples.
• outputs (np.array) – Test response.
• predict (SurrogateModel.predict_values) – Surrogate’s prediction method.

Returns:

Surrogate accuracy metrics.

Return type:

metrics (dict)

metamod.performance.report_divergence()

Report the problem ID if the surrogate training fails to converge with the maximal amount of training samples.

metamod.performance.retrieve_metric(surrogates)

Calculates the mean and variance of the assessed metric.

Parameters:surrogates (list) – List of cross validation surrogates. Returns:Mean and variance of the assessed metric. Return type:output_metrics (dict)