Statistical Objective Functions

We implemented the Mann-Whitney U-test (MWUT) to calculate the error between experimental data and simulations. The U-test is a non-parametric statistical test that, within a confidence level, determine if a stochastic repeated measurements is identical or not to another repeated measurements.

1. We count how many times experimental data (\(exp_i\)) are larger than simulated values (\(sim_j\)):

for \(i \mathrm{\ in\ } \mathrm{range} ( \mathrm{len}(exp) )\):

for \(j \mathrm{\ in\ } \mathrm{range} ( \mathrm{len}(sim) )\):

if \(exp_{i} > sim_{j}\):

\(U_{exp} \gets U_{exp} + 1.0\)

else if \(exp_{i} < sim_{j}\):

\(U_{sim} \gets U_{sim} + 1.0\)

else:

\(U_{exp} \gets U_{exp} + 0.5\)

\(U_{sim} \gets U_{sim} + 0.5\)

2. We determine if \(U_{exp}\) is statistically significant:

   for \(i \mathrm{\ in\ } \mathrm{range} ( \mathrm{len}(exp) )\):

   if \(\mathrm{len}(exp) \times \mathrm{len}(sim) - \mathrm{min}(U_{exp}, U_{sim}) \leq U_{\mathrm{critic}}\):

   \(\mathrm{\textit{null}\ hypothesis,\ }H_{0}\mathrm{,\ is\ rejected}\)

   \(U_{\mathrm{model}} \gets U_{\mathrm{model}} + 1.0\)

Note

The U-test is the only fitness function that has known limits: For a perfect model, the U-test is zero. A complete wrong model will have a \(U_{model}\) equal to the number of Observables times the number of experimental time points. For instance, the example model we use to compare with BioNetFit has 2 Observables and 7 experimental time points, then a max \(U_{model}\) equal to 14.