Postprocessing

Postprocessing refers to evaluations and checks performed on a model to get an understanding of its properties and estimate its quality. The BayesValidRox class bayesvalidrox.post_processing.post_processing.PostProcessing includes functions that are applicable to all types of surrogate model.

• valid_metamodel: visualizing some metamodel runs against the corresponding model runs

• check_accuracy: computing the RMSE error of the surrogate model

Postprocessing methods that are specific to a type of surrogate or training include

• plot_moments: visualizing the moments of a polynomial chaos expansion (PCE, aPCE)

• sobol_indices: calculating the Sobol’ and Total Sobol’ indices (PCE, aPCE)

• plot_seq_design_diagnostics: visualizing diagnostics that were evaluated during sequential training (surrogate trained with train_sequential)

Example

We want to compare out trained surrogate from Active learning: iteratively expanding the training set against its original model.

>>> from bayesvalidrox import PostProcessing

The postprocessing object expects the full engine. This allows it to perform both sampling, running the model and running the surrogate model.

>>> PostProc = PostProcessing(Engine_)

To gain an understanding of the approximation quality we use both valid_metamodel and check_accuracy.

>>> PostProc.valid_metamodel(n_samples=1)
>>> PostProc.check_accuracy(n_samples=10)

The visual comparison of model and metamodel results is saved in the folder Outputs_PostProcessing_calib, while the accuracy check is given as console output.

>>>>> Errors of A <<<<<

Index  |  RMSE   |  Validation Error
-----------------------------------
1  |  1.513e-07  |  2.224e-15

>>>>> Errors of B <<<<<

Index  |  RMSE   |  Validation Error
-----------------------------------
1  |  5.045e-05  |  2.406e-11