Back to the Top
There is a rumor going around that third party validation of WinNONLIN, beyond the validation
package Phoenix offers, is required by FDA. I have been using this package since it was PC-NONLIN
and this is the first time I have heard something like this. Any comments?
Christopher J. Kemper, Ph.D.
Pharma Navigators, LL
Back to the Top
Dear Dr. Kemper and all:
I have not heard that the FDA required any such thing. It would be a very sad thing if it did.
Various people in the FDA have told me in the past that what they care about is good science. They
sure could use it themselves, as it seems to me that the state of modeling methodology has moved on
a lot in the last 30 years, but their awareness of this has not.
1. I wish the FDA and the pharmaceutical industry would stop using an overall error pattern
that totally discards the documented performance of the assays used in any study and replaces it by
use of an overall error pattern that does not distinguish between the noise in an assay and that in
the clinical environment in which the study is done. The clinical environmental uncertainty should
be separately known just as the assay error should be. I have never seen any evidence of such
concern. It is easy to do this. Simply use the very assay error polynomial as described in (1) and
then estimate lambda, an error term to describe the additional noise in the clinical environment. We
need to know them both, and separately, so we can optimize each of them.
2. I wish the FDA and the pharmaceutical industry would consider the fact that the
parametric modeling procedures they so often continue to use over the decades will never obtain the
most likely results given the data. This is because the parametric assumptions they make constrain
the shape of the model parameter distributions to be within those assumptions, and therefore limit
the maximum likelihood that can be obtained. Nonparametric modeling approaches do not constrain the
shape of the parameter distributions with any such assumptions at all, and more likely results are
found because of this (2).
3. I wish the FDA and the pharmaceutical industry would consider the issue of the precision
with which a dosage regimen can hit a desired target therapeutic goal. Again, they use parametric
modeling approaches which assume the shape of the population model parameter distributions. Use of
such parametric approaches means that the entire issue of dosage precision is never evaluated
because it can never even be perceived, and maximally precise dosage regimens (see 3,4), which are
vital in antibiotic, antiviral, transplant, and cancer chemotherapy, for example, can never be
achieved using parametric models.
I have wished for years that the FDA and the pharmaceutical industry would give some serious
attention to these issues. They are clinically most relevant, and are important for maximizing
therapeutic efficacy and safety for patients. I have discussed this many times with people both in
the FDA and in the industry. I have asked some people publicly whether it bothers them that they
continue to use methods such as NONMEM which use only an approximate, rather than an exact, method
to calculate the likelihood, and their public answer (surprisingly, to me) has been that it does not
concern them, or their reply has been a bland acknowledgement of the facts, with no evidence of any
real concern for the clinical relevance of what they do in this regard. In my view, the FDA and the
pharmaceutical industry very much need to use currently available best science and best modeling
procedures.
References are
1. Jelliffe R, Schumitzky A, Bayard D, and Neely M: Describing Assay Precision - Reciprocal of
Variance is much better than CV%. Therapeutic Drug Monitoring. 06/2015; 37(3):389-94.
DOI:10.1097/FTD.0000000000000168
2. Bustad A, Terziivanov D, Leary R, Port R, Schumitzky A, and Jelliffe R: Parametric and
Nonparametric Population Methods: Their Comparative Performance in Analysing a Clinical Data Set and
Two Monte Carlo Simulation Studies. Clin. Pharmacokinet., 45: 365-383, 2006.
3. Neely M, van Guilder M, Yamada W, Schumitzky A, and Jelliffe R: Accurate Detection of
Outliers and Subpopulations with Pmetrics, a Nonparametric and Parametric Pharmacometric Modeling
and Simulation Package for R. Therap. Drug Monit. 34: 467-476, 2012.
4. Jelliffe R, Bayard D, Milman M, Van Guilder M, and Schumitzky A: Achieving Target Goals most
Precisely using Nonparametric Compartmental Models and "Multiple Model" Design of Dosage Regimens.
Therap. Drug Monit. 22: 346-353, 2000.
Very best regards to you all,
Roger Jelliffe
Roger Jelliffe MD, Professor of Medicine Emeritus, Founder and Director Emeritus, Laboratory of
Applied Pharmacokinetics and Bioinformatics, USC School of Medicine,
Consultant in Infectious Diseases, Children’s Hospital of Los Angeles
Back to the Top
That's like the FDA demanding that all software and analytical validations be conducted by third
parties. It cannot be accurate! Where did this rumor start and what deficiency promoted this rumor
Back to the Top
Dear Chris et al.,
I do not understand the agitation. ;-)
Validation of software is not a new requirement but mandatory for quite
a while. See:
http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM070266.pdf
http://bebac.at/lectures/Prague2015.pdf
Personally I’m a little bit skeptic about relying solely (!) on a
validation package provided by the software vendor.
Helmut
--
Ing. Helmut Schütz
BEBAC - Consultancy Services for
Bioequivalence and Bioavailability Studies
Neubaugasse 36/11
1070 Vienna, Austria
Back to the Top
Hi,
Related to Helmut's comment "Personally I’m a little bit skeptic about
relying solely (!) on a validation package provided by the software
vendor", I'm part of an effort to put together a software-independent
validation package for NCA. Some details will be presented at a poster
at ACoP in October, and if interested, please contact me or check out
the GitHub page: https://github.com/NCAConsortium
Thanks,
Bill
Back to the Top
I agree with Helmut. Validation for off-the-shelf software is a simple process of ensuring that the
installed software performs as expected. Validation for internally built software (e.g. custom
scripts/connectors) requires additional effort to demonstrate that an appropriate software
development life-cycle is followed in addition to the testing.
In my experience, regulatory agencies accept a variety of validation procedures and methods as long
as they meet the specified requirements.
Best regards,
Nathan
Nathan S. Teuscher, PhD
Vice President, Pharmacometric Solutions
Certara
| PharmPK Discussion List Archive Index page |
Copyright 1995-2014 David W. A. Bourne (david@boomer.org)