Back to the Top
Denison,
While I agree in general to your response I would not be so categorical about fitting data to the
model never being the right approach.
It is commonly accepted that "all models are wrong" but at the same time data can be wrong too. A
plausible model may be very helpful for identifying data that might be wrong e.g. outliers, and lead
to a re-evaluation of the data. This is a form of fitting the data to the model which should be
considered when the discrepancy between model and data leads to the conclusion that there could be a
problem with the data.
Best wishes,
Nick
--
Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A
University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
Back to the Top
Hi Nick,
I have never understood why "all models are wrong". And if one was to
believe Pete Bonate, the quote is actually misrepresented and taken out
of content. So why are all models wrong? Is it because we do not know
the full details of the system we are trying to describe by applying
models to data from that system? Or because the variability results in
no single model being able to predict every single sample? Or something
else?
Thanks,
Toufigh
--
Toufigh Gordi, PhD
www.tgordi.com
www.tgordi.wordpress.com
E-mail: tg.aaa.tgordi.com
Back to the Top
Hello,
I am a novice to modelling area and have a dilemma in the usage of terminology: ‘data was fit to a
model’ or ‘a model was fit to data’ – which is correct, both or always the latter?
To make a start……..am I correct in saying that the usage of ‘data was fit to a model’ suits to
situations where the underlying process responsible for the data observed is known so that the data
can be fit to a model that best explains that process (e.g. compound loss data to a mono-exponential
decay equation); whereas, usage of ‘a model was fit to data’ befits to situations where underlying
process is not known for the data observed and one is trying to build a model that best
fits/explains the data so as to gain mechanistic insight (e.g. a PKPD model linking target site
concentrations to an observed response).
Appreciate if experts in the modelling area can share their opinion on this.
Thanks,
Kasiram
Back to the Top
Dr. Katneni,
The right terminology is ‘the model was fit to the data’. You define a model that is able to
describe the underlying process and then you fit the model to your data and subsequently estimate
the parameters, goodness of fit etc.
‘Data was fit to model’ implies that you ‘altered’ your data to make it fit to your model. This is
never the right approach.
Best,
Denison J. Kuruvilla
University of Iowa, College of Pharmacy
115 S. Grand Ave., S213 PHAR
Iowa City, IA 52242
denisonjohn-kuruvilla.-at-.uiowa.edu
Back to the Top
Toufigh,
You know as well as I do that models do not perfectly describe
biological data because we do not understand the full complexity of the
system. Models are therefore always an approximation and therefore
always wrong. So I think this matches your first suggestion.
I only gave part of the original quote that was relevant to my response
to the original thread. The actual quote is
"All models are wrong but some are useful"
See: Box GEP Robustness in the strategy of scientific model building.
In: Launer RL & Wilkinson GN Robustness inStatistics. New York: Academic Press, 1979:pp. 202.
as researched by Steve Duffull inhttp://www.pharmpk.com/PK01/PK2001250.html
What did Peter Bonate say about this quote?
Best wishes,
Nick
--
Nick Holford, Professor Clinical Pharmacology
Dept Pharmacology & Clinical Pharmacology, Bldg 503 Room 302A
University of Auckland,85 Park Rd,Private Bag 92019,Auckland,New Zealand
Back to the Top
The phrase "all models are wrong" applies to statistical models and not mechanistic models.
Mechanistic models are predictive and "not wrong" based on the accuracy of the input data and
assumptions.
Back to the Top
Hi Toufigh,
I think the complete statement made by George Edward Pelham Box is:
"Essentially, all models are wrong, but some are useful"
meaning, in principle all models are simplifications of the processes they should describe. They are
a priori wrong, since they are models.
It's just semantics. But I like the Box statement. It is a little reminder about always existing
limitations, when we apply models.
Best,
Peter
Back to the Top
i all
If we momentarily ignore biological systems, then it rather depends on whether your model is a
theorem (built on lemmas and axioms and proofs) or a hypothesis (built on data). We use the word
model very loosely in PK.
I (personally) know of no PKPD models that are theorems and hence all PKPD models can be deductively
tested (and are therefore falsifiable). Whether a model is wrong because it is false is a
different philosophical argument. Perhaps "could be better" would be more appropriate.
"All models could be better but some are (currently as they are now) useful."
Cheers
Steve
--
Stephen Duffull
Professor of Clinical Pharmacy
Otago Pharmacometrics Group
School of Pharmacy, University of Otago
PO Box 56, Dunedin, New Zealand
Back to the Top
Hi all.
I like the quote too because it implies that some models are wrong and useless, and it reminds us we
may want to stop using them...
:-)
Dario
Back to the Top
Nick,
I was listening to Peter's presentation atPaSiPHIC (Pacific Coast
Statisticians and Pharmacometricians Innovation Conference) a couple of
years ago. His slides can be found here:
(http://pasiphic.calpoly.edu/2012Presentations/Why%20Hasnt%20the%20Modeling%20&%20Simulation%
20Revolution%20Happened.pdf).
He was arguing that although modelers may understand the concept of "all
models are wrong" as the models being simplifications of the system,
repeating the statement to non-modelers is not a wise idea. I think of
it as if a physician starts his or her discussion with a patient with
"You know, all our diagnostics are wrong and by the way, the drug and
the dose I am prescribing for you is also probably wrong"! I agree with
him that when we state something is "wrong", we cannot expect people to
agree with the results of the model. "How can the simulation results of
our next study be "right" if the underlying model to perform them with
is "wrong"?" would be the first reaction people will have.
And here is a longer version of the quote (copied from Peter's
presentation):
"The fact that the polynomial is an approximation does not necessarily
detract from its usefulness because all models are approximations.
Essentially, all models are wrong, but some are useful. However, the
approximate nature of the model must always be borne in mind."
Thanks,
Toufigh
--
Peter,
Most people with modeling background do realize the limitations of
models, especially when trying to describe such complex systems as
biological ones. When we develop models, we do all we can to proof they
are "wrong" and only when we fail to do so, we regard them as acceptable
models for the purpose. However, stating the fundamental block stones of
our efforts to describe a system to be "wrong" to non-modelers is
probably not the best way to promote the use of M&S. I always remind my
clients that (PK/PD) models are based on the data in hand and always
prone to data inaccuracies or our misspecifications of the model and
hence should not be viewed as the absolute answer. However, I never
state that the model I am presenting to them is "wrong".
Toufigh
Back to the Top
Hi Kasiram, Denison, Nick, Toufigh et al.,
An aside comment to the discussion:
With classical PK compartmental modeling, one typically fits the model to the data. The process is
"deterministic" since the data determines the model used.
With PBPK modeling, however, the approach is somewhat different - i.e. "a priori", since the model
may be created first and then used to make predictions (simulations) which are compared to the
actual data. The model can then be modified accordingly and its output again compared to the data
(and repeated as necessary).
Here's an interesting comment about PBPK models being wrong:
"Predictions from PBPK models have a very useful property: They can be wrong. The ability to predict
a particular outcome is a powerful tool for enhancing the information content of an experiment. In
effect, PBPK models, based on proposed mechanisms of disposition, make predictions that become
testable..."
- Andersen, M.E. et al. Chapter 1 in PHYSIOLOGICALLY BASED PHARMACOKINETIC MODELING - Science
and Applications (Editors: M. B. Reddy, R. S. Yang, H. J. Clewell & M. E. Andersen; Wiley; 2005),
page 8 This chapter is available online at:
http://samples.sainsburysebooks.co.uk/9780471478775_sample_388170.pdf
Best regards,
Peter
Peter W. Mullen, PhD, FCSFS
KEMIC BIORESEARCH
P.O. Box 878
Kentville
Nova Scotia, B4N 4H8
Canada
E-mail pmullen.at.kemic.com
Back to the Top
Toufigh,
The closest mathematical model of a physical process to being "right" that
I know of is for orbital mechanics, but even those equations are not
perfect (so they are "wrong") because they make assumptions (total vacuum
in space, two-body problem, etc.) and approximations to make the math
tenable. It's the assumptions and approximations that we use to build
models that make them "wrong" (imperfect).
But they can be wrong by a small enough amount that they remain highly
useful.
Walt Woltosz
Chairman and CEO
Simulations Plus, Inc. (NASDAQ: SLP)
and Cognigen Corp, a wholly owned subsidiary of Simulations Plus
42505 10th Street West
Lancaster, CA 93534
www.simulations-plus.com
Back to the Top
Walter,
I agree with you that most, if not all, models are imperfect and include
assumptions that are not perfectly applicable to the reality. My issue
is with the word "wrong" and how it is interpreted by non-modelers. In
essence, any model that tries to capture the full reality is "wrong" but
the same model can be "right" for the purpose. Nevertheless, I simply
don't see why we, as the modeling community, need to go out and state
that "All models are wrong" when communicating with others. I think
Peter Bonate's quote is a much better choice of word:
“All models are approximations, some perform better than others"
Toufigh
Back to the Top
Toufigh,
I think the "all models are wrong, some models are useful" is meant to be
a caveat to those who may be unfamiliar with what models are able to do. I
use it to ensure that users don't expect perfection, but realize instead
that imperfect models are used all the time because they are "adequate,
not perfect". Adequate means meeting all requirements for the intended
purpose.
Too often, I've seen people expect that the simulation line has to go
right through every data point, and that is absolutely not true. Some
software programs will force that, but it's not necessary to have a good
and useful model.
Too often in training, I've asked people to look at a set of data points
and to tell me what is Cmax and Tmax, and they pick the highest observed
data point. I then usually say something like "Really? When is the highest
observed data point Cmax?" and then I wait through the silence for a bit.
Then I emphasize that it is NEVER that point (it might be very close). The
odds of taking a plasma sample exactly at Cmax/Tmax are infinitesimally
small, so saying never may be only a slight stretch.
Good PK models often have a Cmax higher than the highest observation,
providing a much more reliable estimate of Cmax and Tmax than the data.
The model is note perfect, but it is adequate and very useful.
Best regards,
Walt
| PharmPK Discussion List Archive Index page |
Copyright 1995-2014 David W. A. Bourne (david@boomer.org)