Please visit the new BioModels platform to access the latest content. This website is no longer updated and will be retired on 31 May 2019.
BioModels Database logo

BioModels Database

spacer

Rateitschak et al (2012). Parameter Identifiability and Sensitivity Analysis Predict Targets for Enhancement of STAT1 Activity in Pancreatic Cancer and Stellate Cells.

June 2018, model of the month by Vincent Knight-Schrijver
Original model: BIOMD0000000585


Introduction

Model identifiability is usually of concern for large models where the structure may be too complex or the data too sparse. There are two types of model identifiability namely practical and structural identifiabilty. Practical identifiability becomes challenging when large biological models attempt to account for processes in biology which may be simply obscured by overall trends or not observable in available datasets. Additionally, structural identifiability issues can arise where coded reactions are functionally linked and may be redundant. In such cases, identifying the true values of parameters is difficult as parameter estimation algorithms may locate wide or no clear minima for parameters. This produces multiple parameter sets which fit data equally well. If we cannot confidently verify the occurrence of a reaction or locate an optimal value space for a parameter they may be non-identifiable. We can assess identifiability through identifibility analysis. This model of the month article highlights an additional use of identifiability analysis. Rateitschak et al [1] used identifiability as a tool for evaluating the potential differences in biological reactions occurring between two experimental systems: pancreatic stellate cells (PSC) and pancreatic cancer cells (PC) in vitro. Furthermore, the authors also used sensitivity analysis to suggest that these mechanistic differences may not totally isolate one cell type from another when targeted by specific therapeutics.

Model

The authors studied a simple model of Interferon gamma (IFN) signalling (Figure 1). This model included elements of STAT1 phosphorylation, dimerisation and nuclear translocation necessary for STAT1-mediated transcription. Here the authors included the transcription of SOCS1 as a negative feedback mechanism as often reported in literature for cytokine signalling cascades. Although more complex intracellular models of cytokine signalling are available in the BioModels Database ([BIOMD0000000544, BIOMD0000000151] ), the model here was detailed enough for the purposes of the authors' analysis.
Figure 1

Figure 1. The model structure of a signal transduction pathway from IFN gamma receptor to nucleus via the phosphorylation of STAT1. Figure taken from [1].

Prompting the authors' study was their observation that fitting virtually the same ODE model to either PSC or PC experimental data resulted in different parameter sets and simulated dynamics of STAT1 phosphorylation in response to IFN gamma stimulation (Figure 3). This must be explainable by examining the model's structure and parametrisation through identifiability analysis.

Results

The observed differences between the datasets were that (A), PSC simulations showed a rapid initial phosphorylation of STAT1 not seen in PC simulations; and (B), STAT1 accumulation observed in PSC simulations was not present in PC simulations (Figure 2, observations). Plotting Profile likelihood estimates (PLE) profiles made by Rateitschak et al [1] for the parameters shows key differences in the practical identifiability when fitting to both both PSC and PC data (Figure 3). Only one parameter, k1, was identifiable and different (no overlap of confidence interval boundaries) between PSC and PC fits; the parameter which regulates the rate of IFN gamma receptor activation (Figure 3). Examining the trajectories of the parameter sets where k1 was within the boundaries of the confidence interval showed that a difference in the rate of IFN gamma receptor activation was able to explain the rapid versus slower initial STAT1 phosphorylation seen between PSC and PC simulations (Figure 2, upper panels).

Figure 2

Figure 2. Left: The observation that the model fitted to PC and PSC cell data results in different STAT1 phosphorylation dynamics in response to IFN gamma stimulation. Right: The explanation shown by trajectories of phosphorylated STAT1 in cytosol and nucleus (noted by STAT1Dc and STAT1Dn respectively) (lower panels) in response parameter sets. parameter was within the confidence intervals (see original publication, figures 8 and 9 for the range of the non-identifiable parameters). Left panel produced in R (version 3.5.0) from simulations run in Copasi (version 4.22, build 170). Right panels taken and adapted from the original publication [1]


The authors pointed out that another key parameter seen to differ between datasets was k6, responsible for the nuclear import of phosphorylated STAT1. This appeared to be practically identifiable for PC data only (Figure 3). The authors used the parametrisation of k6 to explain observation B, the differences in nuclear accumulation of STAT1 as the minimum for parameter k6 is lower in PC than PSC PLE profiles (Figure 3). The authors further examined the correlation between k6 and k5 (the rate of nuclear STAT1 dephosphorylation) in PC parameter sets and showed that k6 had two main states for the range of k5 explored. This may be illustrated in the two groups of STAT1Dc trajectories using parameter sets covering a range of k5 for which k6 was either within its confidence interval or not (where "*" = k6 not within interval) ( Figure 2 ,lower right panels).

A higher k5 and thus rate of nuclear STAT1 dephosphorylation yielded a lower nuclear STAT1D concentration ( Figure 2 ,lower right panels). However, PC parameter sets for an identified value of k6 also contained k5 with a higher value and also simulated trajectories of higher cytoplasmic STAT1D in conjunction with the lower nuclear STAT1D. Alongside other parameters discussed in the article [1], this suggested that the nuclear import and not the nuclear dephosphorylation was important for the observed effect B. In the other cell type, PSC, parameter sets showed that k5 and k6 were very much correlated but neither were properly identifiable (see the publication, Figure 7 for correlation plots).

Figure 3

Figure 3. PLE profiles for five key parameters with practical identifiability differences between PSC and PC data sets. Figure was taken and adapted from the original publication [1].

In the end, a sensitivity analysis in both cell types showed that, despite the challenges of practical identifiability, variations of parameters within confidence intervals yielded similar nuclear phosphorylated STAT1 perturbations in both PC and PSC. Furthermore, it appeared as though nuclear STAT1 was robust against perturbations of k6 which makes the nuclear import a crucial reaction for distinguishing between cell types, but not as a reaction for therapy (See figure 10 and 11 from the original publication [1]). However, both cell types' pool of nuclear phosphorylated STAT1 was sensitive to k5; the authors conclude that targeting the dephosphorylation of STAT1 may be a strategy for therapeutics aimed at maximising the modulation of both PSC and PC simultaneously. Perhaps k5 can be identified with further and different experimental designs.


Conclusion

Identifiability analyses can and should be a major part of the modelling process but we often see their use confined to assessing a model's robustness and reliability for making predictions [2]. However, this model illustrates a novel and useful aspect of examining identifiability to reveal key reactions which distinguish biological systems from one another with application to experiments dealing with multiple cell types. Perhaps an analysis like this carried out by Rateitschak et al [1] would be difficult for larger models with more parameters which bring further challenges of identifiability. That said, another point is that non-identifiability does not make a model invaluable; the sensitivity analysis remains robust between PC and PSC parameter sets and shows that predictions can still be made and in fact, the analysis adds further context for confidence in the results. Finally, these analyses continue to drive the dialogue between modelling and experimental researchers where models are used to direct future experiments. In turn, the models can utilise the data to identify parameter values (or not) and jointly probe the very nature of biology.

References

  1. Rateitschak, K, Winter, F, Lange, F, Jaster, R, Wolkenhauer, O (2012). Parameter identifiability and sensitivity analysis predict targets for enhancement of STAT1 activity in pancreatic cancer and stellate cells. . PLoS Comput. Biol., 8, 12:e1002815.
  2. Villaverde, AF, Barreiro, A, Papachristodoulou, A (2016) Structural Identifiability of Dynamic Systems Biology Models . PLoS Comput. Biol., 12, 10:e1005153
spacer
spacer