Statistical Modelling 3 (2003), 1–13
A simulation-based method for model evaluation
David J. Allcroft
Biomathematics and Statistics Scotland,
JCMB, King's Buldings,
Edinburgh, EH9 3JZ, Scotland
UK.
eMail:
dave@bioss.ac.uk
Chris A. Glabey
Biomathematics and Statistics Scotland,
Edinburgh, Scotland
Abstract:
We wish to evaluate and compare models that are non-nested and fit
to data using different fitting criteria. We first estimate parameters
in all models by optimizing goodness-of-fit to a dataset. Then, to
assess a candidate model, we simulate a population of datasets from
it and evaluate the goodness-of-fit of all the models, without
re-estimating parameter values. Finally, we see whether the vector
of goodness-of-fit criteria for the original data is compatible with
the multivariate distribution of these criteria for the simulated
datasets. By simulating from each model in turn, we determine whether
any, or several, models are consistent with the data. We apply the
method to compare three models, fit at different temporal resolutions
to binary time series of animal behaviour data, concluding that a
semi-Markov model gives a better fit than latent Gaussian and hidden
Markov models.
Keywords:
BINARY TIME SERIES; HIDDEN MARKOV MODEL; LATENT GAUSSIAN MODEL;
MAHALANOBIS DISTANCE; MODEL; COMPARISON; SEMI-MARKOV MODEL.
Downloads:
Color
version of figure 3.
Illustrative
data and program in zipped archive
back