PhytoSim Sensitivity User Guide

Chapter 1: Overview
Chapter 2: Source and Target Components
Chapter 3: Sensitivities
Chapter 4: Ranking
Chapter 5: Identifiability
Chapter 6: Examples
Chapter 7: Preferences

Chapter 1
Overview

A sensitivity analysis studies the "sensitivity" of the outputs of a system (target components) to changes in the parameters or initial conditions (source components). It also allows you to rank the model components according to how much they influence the model output. Finally, a sensitivity analysis can be used to identify which model components can be estimated based on a given set of measurements.

Sensitivity Analysis Module Areas:

Right navigator: Source and target component selection and configuration.
Central area: Results area for sensitivities, ranking and identifiability.

Starting a sensitivity analysis

In order to start a calibration follow these steps:

Make sure the simulation in the Simulation module is properly configured.
Add source components to the sensitivity analysis. Source components are model components that will be changed in order to investigate their influence on the model output. Source components can be parameters or derived variable initial conditions.
Add target components to the sensitivity analysis. Target components are model components for which we want to find out how changes to the source components influence them. Target components can be algebraic or derived variables.
Configure the output interval and scaling for the target components.
Press Start to start the sensitivity analysis.

Starting an identifiability analysis

In order to start an identifiability analysis, press the drop down arrow of the start button. Identifiability calculations can be started manually after a sensitivity analysis has finished or directly following a sensitivity analysis.

Options for the sensitivity analysis start drop down menu

More Graphs

The 'More Graphs' toolbar button enables the visualisation of (1) the simulation results used to calculate the sensitivities and (2) the absolute sensitivities for each source and target component combination. Once the button is enabled, switching to the simulation results and absolute sensitivities can be done using the results switcher at the bottom.

Chapter 2
Source and Target Components

Source Components

Source components are model components that will be changed in order to investigate their influence on the model output. Source components can be parameters or derived variable initial conditions.

Target Components

Target components are model components for which we want to find out how changes to the source components influence them. Target components can be algebraic or derived variables.

Properties

Interval type: Specify the type of interval at which sensitivity analysis results will be calculated.

Fixed value: Calculate sensitivities at a fixed interval.

Interval value: The interval at which the results will be calculated.

Data input column: Calculate sensitivities at the exact time instances specified by a column of a data input from the Data I/O module.

Data column: The data input column that will control the time instances at which the sensitivities will be calculated.

Scaling type: Type of scaling that will be used to calculate relative sensitivity functions.

Component value: Normalize with the corresponding component value at each time instance.
Component average value: Normalize with the average component value calculated over the entire time period.
Fixed value: Normalize with a user-defined value, possibly the measurement error standard deviation.

Scaling value: Fixed values that will be used to scale the absolute sensitivity functions.

Chapter 3
Sensitivities

Relative sensitivity function results view

Theoretical Background

The PhytoSim Sensitivity module performs a local sensitivity analysis. The analysis is called 'local' because it is performed for a model with a given set of parameter values and initial conditions defined in the Simulation module. For that specific reference situation, the Sensitivity module calculates how much the target components change when a source component is changed by only a small amount. Mathematically, this can be described as the partial derivative of the target component to the source component:

Absolute sensitivity defined as a partial derivative

where S(t) is the time varying (absolute) sensitivity function of target component y. The target component y is called sensitive to the source component theta when a small change in theta produces significant changes in y. On the other hand, the target component y is called insensitive to theta if changes in theta produce insignificant changes in y.

The PhytoSim Sensitivity module uses a finite difference approach in order to numerically approximate the partial derivative. A central difference formula is used where each partial derivative is calculated based on two model simulations: one where the source component is increased by a small amount and one where the source component is decreased by a small amount. The results of these simulations can be viewed by enabling the 'More Graphs' toolbar button and selecting 'Simulations' from the bottom results switcher. The simulation results are shown for the combination of the last selected source and target component.

Simulation results of the forward and backward source component perturbations

Only when numerical issues are reported after the sensitivity analysis (see below) should these results be inspected. In such case, check for anomalies in the simulation results like sudden peaks or situations where the forward and backward simulations are both below or above the reference simulation. The latter is the simulation performed with the model as it is configured in the Simulation Module.

Based on the forward and backward simulations, numerical approximations of the absolute sensitivities are calculated using following formula:

Central difference finite difference approximation to the partial derivative

The amount with which the source components are perturbed is important in order to avoid nonlinearity effects and numerical rounding errors. The Sensitivity module uses a fixed amount of 1% of the source component value and has an automatic algorithm to detect numerical issues with the calculated sensitivities. If such problems should arise, the user will be notified and the results should be used with caution! Most of the time, these issues can be avoided by fine-tuning the solver settings used in the Simulation module:

For variable step size solvers: make the accuracy better (e.g. 1e-9 instead of 1e-6) or decrease the maximum step size.
For variable step size solvers: switch to a fixed step size solver.
For fixed step size solvers: decrease the maximum step size.
Feel free to contact support@phyto-it.com if the problems persist.

In case of numerical issues, the individual absolute sensitivities for each target and source component combination may also be inspected. This can be be done by enabling the 'More Graphs' toolbar button and selecting the 'Absolute Sensitivities' from the bottom results switcher. The absolute sensitivity is shown for the combination of the last selected source and target component.

Absolute sensitivity results of a target and source component combination

The number of simulations required to calculate all the sensitivity functions is determined by the amount of source components. One simulation is performed at the reference situation and two additional simulations are performed for each source component. This results in 2n+1 required model simulations (where n is the number of source components).

In order to compare the sensitivities of different target components to different source components, relative sensitivity functions are calculated:

Absolute sensitivity scaled by the source component value and target component scaling factor

The absolute sensitivities S(t) are scaled with respect to the source component value and a user defined target component scaling factor sc_i. Often used scaling factors are: (1) the target component value at each time instance the sensitivity is calculated, (2) the average target component value over the entire period or (3) the target component measurement error standard deviation. As such a dimensionless sensitivity is obtained which can be used for comparisons.

Relative Sensitivity Results and Interpretation

To view the relative sensitivity results, select 'Relative Sensitivities' from the bottom results switcher.
Selecting a source component in the right navigator will show a graph of the relative sensitivities of all target components to the selected source component.
Selecting a target component in the right navigator will show a graph of the relative sensitivities of this target component to all source components.
Sensitivity functions with similar shapes are often an indication of correlated source components.
Sensitivity functions which increase over time indicate that a change in the source component will have an ever increasing effect on the target component.

Additional Output in the Data I/O Module

Beside the sensitivity results accessible through the Sensitivity module user interface, some additional results are available in the Data I/O module.

Information Matrix: A matrix which is only calculated when the target components are scaled with a fixed value. If this scaling value is the target component measurement error standard deviation, the resulting matrix will be the so called Fisher Information Matrix.

Chapter 4
Ranking

Analysing individual sensitivity functions can be a difficult task, especially when a large number of source and target components is involved. Therefore, the PhytoSim Sensitivity module also calculates a source component importance ranking for each of the target components separately and a combined ranking for all target components.

Theoretical Background

The source components of the PhytoSim Sensitivity module are ranked according to their sensitivity index (SI) which is calculated as follows:

where j is the source component, i the target component, k the sensitivity time instance within a sensitivity function, N the number of target components and K the number of time instances for a particular sensitivity function. s_ijk^2 is the squared value of the relative sensitivity of the i'th target component to the j'th source component at a certain time instance k.

Results and Interpretation

The ranking table contains the sensitivity indices for each of the source components (rows).
The higher the sensitivity index value, the more influence a certain source component has on the target component(s).
The first column of the table provides a sensitivity index ranking based on all target components. As such it shows which source components have the highest/lowest cumulative sensitivity index (summed over all target components).
The other columns of the table provides a sensitivity index ranking based on a certain target component.
Selecting a certain table column will display a bar chart of the values above the table.
The sensitivity indices can be sorted ascending or descending by clicking on the target component header in the table.

Chapter 5
Identifiability

An identifiability analysis is a powerful technique to determine which source components can be calibrated based on data gathered from an experiment, even before the experiment is performed! The experiment is defined by 3 characteristics: (1) which measurements will be performed (which target components), (2) what is the measurement accuracy (target component measurement error) and (3) what is the measurement interval (target component measurement interval).

Theoretical Background

The basis of a practical identifiability analysis is the collinearity index gamma which is a measure for the linear interdependence of the source components:

where s_j is the concatenated vector of relative sensitivity functions of source component j for all target components, ||s_j|| the norm of s_j (square root of the sum of squared values of s_j), s_tilde the matrix composed of s_j_tilde columns corresponding to a certain set of source components and lambda_min the smallest eigenvalue of the matrix calculated as s_tilde_transposed multiplied by s_tilde.

Note: For an identifiability analysis, the scaling type of the target components should be set to 'Fixed value' and the measurement error standard deviation should be used as the scaling value. In this way, the measurement uncertainty of the target components will be taken into account. Indeed, target components with a large measurement error will not contain much information, and consequently also not contribute much to the identifiability of the source components.

Rather than only calculating the collinearity index of the combined set of source components, all possible subsets are typically analysed. For the case of 3 source components, this would mean that the collinearity indices for subsets {1}, {2}, {3}, {1,2}, {1,3}, {2,3} and {1,2,3} would be calculated (7 possible combinations).

When the collinearity index value is higher than 15, the source component subset is said to be unidentifiable (see Brun et al., 2002). In that case, the subset contains a combination of source components that are too linearly dependent. In such situation, a change of one of the source components can be (completely) negated by an appropriate change in the other source components. As a result, it will never be possible to obtain unique and accurate values for the source components through calibration based on the given quantity and quality of the data (target components).

Beside the collinearity index, the sensitivity index (see above) of each of the source component subsets also holds important information about the parameter identifiability. Not only is it important that the collinearity index of a subset is lower than 15, the sensitivity index of that subset should also be as high as possible. Indeed, high sensitivity indices indicate that the source components of the subset have a large influence on the target components which also means that the target components are ideally suited to be used for the calibration of the source components.

To sum things up: the ideal identifiable source component subset is the one which has a low collinearity index (certainly lower than 15), a high sensitivity index and contains as much source components as possible.

Note: The identifiability analysis will be stopped once all combinations of a certain subset size are found to be unidentifiable. E.g. when all subsets of size 4 are unidentifiable, all subsets of larger sizes (5, 6, ...) will also be unidentifiable. The collinearities for these subset sizes will therefore not be calculated. In such case, the subset size numbers in the 'Subset Size Selector' will be displayed in red.

References

Brun R., Kuhni M., Siegrist H., Gujer W. and Reichert P. (2002) Practical identifiability of ASM2d parameters--systematic selection and tuning of parameter subsets. Water Research, 36(16), 4113-4127.
De Pauw D.J.W., Steppe K. and De Baets B. (2008) Identifiability analysis and improvement of a tree water flow and storage model. Mathematical Biosciences, 211(2), 314-32.

Results and Interpretation

A picture is worth a thousand words: that is exactly what the identifiability analysis results graph hopes to accomplish. This graph gives for each subset the collinearity index values on the X-axis and the sensitivity index values on the Y-axes. The subset size is represented by the size of the dots on the graph. The color of the dots indicates whether a subset is identifiable (green) or unidentifiable (red).
Beside the identifiability graph, a table is also provided with the values of the collinearity and sensitivity indices for each of the subsets shown on the graph.
By default, only identifiable subsets are shown on the graph and in the table. This can be changed by clicking the 'Unidentifiable' button in the lower right corner of the identifiability results area.
Which subsets are shown on the graph can be changed in two other ways: (1) the subset size selector on the right allows selecting subsets of a certain size and (2) the source component selector on the right allows selecting which source component(s) should be part of the subsets shown on the graph.
Hovering with the mouse on a subset on the graph shows the list of source components for that subset and also highlights the corresponding row in the identifiability table.
If the identifiability analysis shows that too few source components are identifiable a few things can be done: (1) evaluate if an increase of the measurement frequency is experimentally feasible (target component interval), (2) investigate if the measurement error can somehow be reduced, e.g. by switching to a more accurate measurement technique/sensor (target component scaling value) and (3) possibly add additional measurements to the experimental setup (more target components which are sensitive to the source components).

Chapter 6
Examples

Following example workspaces can be found in the Examples/PhytoSimSensitivity folder of your PhytoSim folder:

Irrigation/Staci/Staci.psw: Shows the use of data input columns as target variable interval type.
Misc/Linear/Linear.psw: Simple example of a linear model for which the sensitivities can also be calculated by hand.
Plant Physiology/Penman-Monteith_Tree/Penman-Monteith_Tree.psw: Shows how an identifiability analysis can be used to remove an unidentifiable parameter and, as such, obtain better calibration estimates. Also have a look at the Calibration module.
Plant Physiology/RCGro/RCGro.psw: Example with slightly more source and target components.

Caution: Example workspaces installed in the Examples folder by the modules are overwritten each time PhytoSim is started. Any modification made to these files will be lost. When you want to use an example workspace as a starting point for your own work, first save a copy of it in the Workspaces folder.