Multi-execution experiments
This tutorial covers different ways to setup a series of simulations.
Multi-execution experiments include a set of simulation cases with different parametrization that can be executed in parallel. All cases in a multi-execution experiment refer to the same model but may be structurally different if structural model parameters are changed between the cases. The multi-execution framework automatically determines if several model compilations are required for the execution.
Multi-execution experiments can be setup either:
1. declaratively via expansions. This involves using a set of predefined operators along with corresponding expansion algorithms to generate the cases.
explicitly by defining individual cases via definition extensions.
Following either of the approaches, the experiment definition can be modified to set up a series of simulations
Experiment expansions
Operators
Operators can be used to create multi-execution experiments. There are multiple operator types available:
Here is an example where an experiment definition (see Setting up an experiment section for base setup of experiments) is augmented to execute multiple cases:
from modelon.impact.client import Range
experiment_definition = experiment_definition.with_modifiers({'PI.k': Range(10, 100, 3)})
Here the Range()
operator class is used to specify the start_value
, end_value
and no_of_steps
for the parameter to sweep.
The parametrization in the examples above would configure a set of three simulation cases for the model with 3
equidistant
values set for the PI.k
parameter, with a start value of 10
and an end value of 100
.
It is also possible to create multi-execution experiments with an explicit list of parameter values to sweep:
from modelon.impact.client import Choices
experiment_definition = experiment_definition.with_modifiers({'PI.k': Choices(10, 20, 30, 40)})
Here the Choices()
operator class is used to specify an explicit list of values for the parameter to sweep. The parametrization in the
examples above would configure a set of four simulation cases for the FMU with 4
chosen values for the PI.k
parameter.
It also possible to use a combination of the range and choices operators to setup a batch run:
from modelon.impact.client import Choices
experiment_definition = experiment_definition.with_modifiers({'PI.k': Choices(10, 20),'PI.Ti': Range(10, 100, 3)})
Expansion types
The operators could be further customized with expansion algorithms. Expansion methods define how cases of an experiment are selected based on the respective operators. The following expansion algorithms are supported:
1. Full-factorial
: Creates
an experiment with all possible combinations of the respective operators. Supported operators are Range and Choices.
Note: Full-factorial is the default and does not need to be specified explicitly. Example:
from modelon.impact.client import Range, Choices
experiment_definition = experiment_definition.with_modifiers({'PI.k': Choices(10, 20),'PI.Ti': Range(10, 100, 3)})
2. Latin-hypercube
: Supported
operators are Uniform, Normal and Beta. The values for the resulting cases are produced as follows:
Each operator expression is subdivided into samples partitions of equal probability. E.g., given samples = 10, Uniform(0, 10) is subdivided into [0, 1), [1, 2), … , [9, 10). For each case, a random, but distinct, subdivision is chosen and a random (in accordance with the given probability distribution) value is chosen for the given Case.
The resulting cases are called orthogonal. Values corresponding to a given operator expression do not repeat among all cases of an Experiment.
Example:
from modelon.impact.client import Beta, Normal, LatinHypercube
experiment_definition = model.new_experiment_definition(
custom_function).with_modifiers({'inertia1.J': Beta(0.1, 0.9),
'inertia2.J': Normal(0.1, 0.5)}).with_expansion(LatinHypercube(5,0))
3. Sobol
: The expansion produces samples
cases, based on the Sobol sequence. Supported operators are Uniform, Normal and Beta. An example with Uniform
operator with Sobol expansion algorithm can be found here. Example:
from modelon.impact.client import Beta, Normal, Sobol
experiment_definition = model.new_experiment_definition(
custom_function).with_modifiers({'inertia1.J': Beta(0.1, 0.9),
'inertia2.J': Normal(0.1, 0.5)}).with_expansion(Sobol(5))
Experiment extensions
The experiment extensions approach provides a more flexible and highly parametrizable way to create a multi-execution scenario.
The extensions could be defined by calling the with_extensions()
method on the experiment_definition
class object with a list of SimpleExperimentExtension()
classes as input. The SimpleExperimentExtension()
could be parametrized
with inputs such as the custom_function parameters
, solver_options
, simulation_options
and simulation_log_level
:
from modelon.impact.client import SimpleExperimentExtension
experiment_extension_1 = SimpleExperimentExtension(
parameter_modifiers={'final_time': 2.0},
solver_options={'atol': 1e-9},
simulation_options=dynamic.get_simulation_options().with_values(ncp=1500),
)
experiment_extension_2 = SimpleExperimentExtension(
parameter_modifiers={'final_time': 5.0},
solver_options={'atol': 1e-10},
simulation_options=dynamic.get_simulation_options().with_values(ncp=1200),
)
This would create two simulation cases with different solver and simulation settings. It is also possible to have different
variable modifiers for each of these cases. This could be done by calling the with_modifiers()
method on the
SimpleExperimentExtension()
class:
experiment_extension_1 = experiment_extension_1.with_modifiers({'PI.k': 25})
experiment_extension_2 = experiment_extension_2.with_modifiers({'PI.Ti': 5})
The extensions could be passed on as a list of arguments to the with_extensions()
method on the experiment_definition
class object:
experiment_definition = experiment_definition.with_extensions(
[experiment_extension_1, experiment_extension_2]
)
A simpler approach for parametrization also exists for scenarios where only variable modifiers are varied for setting up
multi-execution cases. This could be done by calling the with_cases()
method on the experiment_definition
class object
with the variable modifiers as inputs:
experiment_definition = experiment_definition.with_cases([{'PI.k': 20}, {'PI.k': 30}])
Note:
It is not supported to have both range operator and experiment extensions defined for an experiment. The simulation cases
could only be set up with one of the two methods. However, it is allowed to call the with_modifiers
method on the
experiment_definition
class to specify variables to modify. The modified variable in such a scenario would be set in all
the cases defined using the with_extensions()
or with_cases()
method calls. If the same variable modifier is
set in both experiment_definition
and extensions, the one set in the extensions would gain precedence, overriding the former.