Back to the Top
QUESTION:
Are there rigorously (or otherwise) established guidelines with regard
to how many samples per subject are recommended per PK parameter
estimated and for each covariate added to the model?
What references are suggested, if any?
BACKGROUND
I am planning a PK substudy nested within a clinical study of neonates
for a drug administered by intramuscular injection, q4h, and cleared
both renally and hepatically in adults. The median treatment length for
the drug is 12 days.
The sample size, calculated based on the primary clinical outcome
measure, is 40 babies (approximately 480 study observation days total).
Because of the understandable reluctance of our principal investigator
neonatologist to do too many heel sticks, I want to minimize the number
of samples I obtain during a dosing interval and overall. Accordingly,
I planned to do a population PK analysis of the data to determine CL/F
and Vd/F.
Optimally, I would like to add a few potentially informative clinical
covariates (bilirubin, Creatinine, weight, post-partum age) to the mixed
effects model.
PRACTICAL QUESTIONS
Practically, will 4 random samples per baby in the first dosing interval
be sufficient to evaluate 2 PK parameters and a few clinical covariates?
(160 sample-time points)
Could it be done with only 2 samples per baby? (80 sample-time points)
How many samples would be useful in each of the days following the
initiation of therapy if I wanted to capture time dependent changes in
PK parameters (expected in this population)?
Thanks for any,
Craig Hendrix
Back to the Top
Dear Craig,
although I do not know of explicit guidelines of the kind you need,
there is a growing literature on the subject of optimal design of
sampling schedules in populations.
This area of research builds on more established methods (e.g. D-optimal
design) for optimal schedules in individuals. Some classic references on
the latter are:
D'Argenio DZ. Optimal sampling times for pharmacokinetic experiments. J
Pharmacokinet Biopharm 9: 739-756, 1981.
Landaw EM. Optimal design for individual parameter estimation in
pharmacokinetics. In: Variability in Drug Therapy: Description,
Estimation and Control. M. Rowland et al., Eds. Raven Press, New York,
1985.
Individual D-optimal design is implemented for pharmacokinetic models in
a module of the ADAPT II software (available from http://bmsr.usc.edu).
The D-optimal solution always gives as many samples as there are
parameters, i.e. 2 samples are needed for a 2-parameter model.
Of course, the drawback of D-optimality is that the model and the
parameters have to be known ahead of time. In populations, this is not
feasible and parameters are always known with some degree of
uncertainty, which cannot be accommodated in standard D-optimal designs.
Some have tried to assess the sensitivity of D-optimal designs to
uncertainty in the model parameters:
Drusano GL, Forrest A, Snyder MJ, Reed MD. An evaluation of optimal
sampling strategy and adaptive study design. Clin Pharmacol Ther 44:
232-238, 1988.
Drusano GL, Forrest A, Yuen G, Plaisance K.Leslie J. Optimal sampling
theory: effect of error in a nominal parameter value on bias and
precision of parameter estimation. J Clin Pharmacol 34: 967-74, 1994.
Landaw EM. Robust sampling designs for compartmental models under large
prior eigenvalue uncertainties. In: Mathematics and Computers in
Biomedical Applications. J Eisenfeld and C DeLisi, Eds. Elsevier
Science, North-Holland, 1985.
=46or more recent, less studied population sampling strategies, some key
references are:
Mentr=E8 F, Mallet A, Baccar D. Optimal design in random-effects
regression models.Biometrika 84: 429-442, 1997.
Merl=E8 Y, Mentr=E8 F, Mallet A, Aurengo AH. Designing an optimal experiment
for Bayesian estimation: application to the kinetics of iodine thyroid
uptake. Stat Med Jan 30: 185-196, 1994
Merl=E8 Y, Mentr=E8 F. Bayesian design criteria: computation, comparison,
and application to a pharmacokinetic and a pharmacodynamic model. J
Pharmacokinet Biopharm 23:101-125, 1995.
Merl=E8 Y, Mentr=E8 F. Stochastic optimization algorithms of a Bayesian
design criterion for Bayesian parameter estimation of nonlinear
regression models: application in pharmacokinetics. Math Biosci 144:
45-70, 1997.
I have heard of a software tool, OSPOP, for optimization of sampling
times in populations. It reportedly allows the user to choose between a
few different optimality criteria and is based on a global optimization
algorithm. I have however never used it. The references for it are:
Tod M and J-M Rocchisani. Implementation of OSPOP, an algorithm for the
estimation of optimal sampling times in pharmacokinetics by the ED, EID
and API criteria. Computer Methods and Programs in Biomedicine 50:13-22,
1996.
Tod M and J-M Rocchisani. Comparison of ED, EID and API criteria for the
robust optimization of sampling times in pharmacokinetics. J Pharmacokin
Biopharm 25: 515-537, 1997.
Hope this helps,
Paolo
--
Paolo Vicini, Ph.D.
Resource Facility for Population Kinetics FOR COURIER SHIPMENTS USE:
Department of Bioengineering, Box 352255 Room 241 AERL Building
University of Washington University of Washington
Seattle, WA 98195-2255 Seattle, WA 98195-2255
Phone (206)685-2009
=46ax (206)543-3081
http://depts.washington.edu/bioe/people/corefaculty/vicini.html
Back to the Top
Craig
If you have some a priori information about the influence of
covariates then you can examin the likely information that
any given pop PK design will yield. You could do this by
evaluation of the population Fisher information matrix
(which can be assessed using software available at:
http://hermes.biomath.jussieu.fr/pfim.htm). From this you
might be able to get an idea of how many blood samples to
take and an idea of timing, eg a sampling window, and you
might consider splitting your neonates into two groups with
each group providing different sampling times. (NB: There
is no global exact pop PK design solution, each experiment
must be considered on a case by case basis.)
I hope this is helpful. Feel free to contact me should you
require further details.
Regards
Steve
=================
Stephen Duffull
School of Pharmacy
University of Queensland
Brisbane, QLD 4072
Australia
Ph +61 7 3365 8808
Fax +61 7 3365 1688
Back to the Top
Craig,
There is an extensive optimal sampling literature. Dave D'Argenio and Joe
DiStefano had most of the early entries via Monte Carlo simulation (DD) and
animals (JD). Our group did 4 or 5 clinical validation studies. If one is to
use classical optimal sampling, it is wise to choose D optimality
(determinant of the inverse isher Information Matrix) as the optimality
criterion. I you use this, the deterministic nature of the calculation will
provide the answer of 1 optimal time point per system parameter. This is
because deterministic optimal sampling assumes one true parameter vector.
There has been some work with stochastic design, especially useful for
population modeling. Again Dave D'Argenio has a publication in this area, as
do Tod and Rocchisani and Tod, Rocchisani and Mentre. Here, since there is a
stochastic process involved, the number of samples is determined by the
investigator. In this regard, the D'Argenio criterion ( expectation of the
log of the D-optimal design) is especially appealing, as it stays well
defined with fewer than one optimal sampling point per parameter (you have to
know the parameters you wish to have information on), making it very useful
for population PK modeling. There is also unpublished work by Schumitzky
using an Entropy criterion and some VERY elegant work on iteration in policy
space (which can be turned to optimal sampling) by Dave Bayard. Lazlo
Endrenyi has also published in this area.
For the deterministic calculations, ADAPT II (D'Argenio) is available free
and is robust and easy to use. For the others, the authors need to be
contacted directly.
Hope this helps. All the best,
George Drusano
Back to the Top
Dear PharmPK,
In my experience, you never quite get your samples when you request them.
If you request a specific time there is a strong tendency to get the
reported sample time as that time (rather than when it was actually taken).
In steep areas of the concentration-time curve this can be a dominant
source of error.
For this reason alone, I would recommend specifying a sampling window.
Furthermore, I think it is crucial to incorporate uncertainty in your
expected parameter estimates when designing a study. After all, if we knew
them exactly, we wouldn't be running a study.
D-optimality is about estimating parameters under precisely specified
circumstances. Reality requires practicality and acknowledgement of the
many different sources of error.
James Wright
Back to the Top
I suggest you simulate the trial with babies to answer your questions.
1. The simulation result is totally depending on what you enter as a PK
model. I guess you have a good idea of the PK profile of adults. You also
should know the disposition pathway so that the difference in metabolic
and/or elimination rate between adults and babies should be appropriately
accounted in the simulation. Body weight seems to affect the PK
allometrically...
2. Instead of defining a set of sampling times, I prefer several
combinations of sampline times to capture the whole phase of PK profile: -
e.g., 1, 3 hrs for 25% of babies, 2,5 hrs for another 25%, 1,5 for
another.... Trial executors may not like this design. Another approach may
be providing them a sampling block design - e.g., around 0.5 hour (=0.25 -
1 hour), around 2 hours (= 1 - 3 hours), etc, expecting natural
distributions around the specified times. Using the actual sampling times
is critical in your data analysis step after the actual baby trial is over.
3. Your "PRACTICAL QUESTIONS" above are good ones for simulation scenarios.
I hope this helps. :-)
Huicy
___
Hui C. Kimko, PhD
Center for Drug Development Science
Georgetown University Medical Center
voice: 202 687 4332
fax: 202 687 0193
Back to the Top
Dear James,
You are, of course, correct in the larger sense. However, system information
has a peak, but is not zero a small distance away (time wise). The point
estimate is helpful to guide acquisition. The actual collection is important
in that the "collector" needs to know that the actual time of acquisition is
key.
All the best,
George
Back to the Top
Dear Dr Drusano,
Yes, I agree that not having the sample at the precise time you requested
does not render it useless, if you know the true time. However, I think
error in the recorded time
will always exist and in a steep area of the curve could be dominant.
Hence, the sample may not truly occur at the recorded time and could even
be harmful to estimation (unless you are accounting for this in your
model)
I do not belief that D-optimal based on a fixed vestor of parameters is
based on reliable enough information to guide trial design alone. What
is a
D-optimal point depends crucially on how you define your vector of
parameters and even more crucially, how you define the nature of error.
Whilst you may be able to put some reasonable priors on
population parameters, a priori estimates of the variance function are
probably speculative. In fact, variance functions estimated once you have
the data
are often somewhat speculative.
I think D-optimality is a tool whose value is dependent on the
validity of your assumptions.
James Wright
Back to the Top
Dear James Wright:
As far as I know, D-optimality is a way of relating the
relationship between a change in a parameter value in a model and the
resulting change in a serum concentration (or other response). The
d-optimal times are those at which a change in a parameter value
makes the greatest change in a serum concentration (or other
response).
This can also be seen by simulation. A simple example is a 1
compartment model, with parameters V and Kel. In a setting of
intermittent IV infusion, suppose the V decreases by a certain
factor, for example, 10%. All the serum concentrations will be
correspondingly higher. Assuming a constant lab assay error, the
greatest change takes place at the true peak, at the end of the IV
infusion. If the assay error is not constant, that can be taken into
account - see Dave D'Argenio's ADAPT II program, for example. The
D-optimal time is the one at which the partial derivative of the
serum conc is maximal with respect to each relevant parameter.
In the same way, take a certain value of Kel. Run it through
several or many values of time, and compute e to the minus KT. Do the
same thing for a slightly different value of K. Look at the
differences between the 2 sets of numbers. They are greatest when e
to the minus KT is 0.36, or 1.44 half times. Why? Who knows, but
that's the way it is.
I am told that D optimality reflects this behavior, and that
this is the reason why the D optimal times for the 1 compartment
model, for intermittent IV input, are the peak, the true peak (unless
adjusted for assay error specifically) and at 1.44 half times after
the end of the IV, when the concentrations are down to 36% of the
original peak. Other more complex models are supersets of this
simplest model, and have other times which are optimal for the other
parameters. D-optimality seeks to find the optimal set of times which
maximize the overall information about the various parameters.
It is true that there is a catch-22, that you are supposed to
know the parameter values in order to compute the times at which it
is optimal to estimate them. However, if one's estimates are probably
not far off, it has been a good strategy.
If you do simulations of this type, it becomes clear that
while a certain time may be the very best, that there are usually
rather broad regions of general sensitivity of the responses to the
various parameters. There is not too much difference between the true
peak and other high levels. Similarly, there is usually a somewhat
broad region of good sensitivity of serum levels to the Kel. Because
of this, it is not too bad if one does not get the samples at just
the optimal times, just so you know when they were in fact obtained.
Yes, errors in recording the times are always present.
Because of this, many have preferred to get trough levels, for
example, as such errors make the least difference in the serum conc
found. However, this strategy leads to the deliberate choice of a
data point when it is usually the LEAST informative about the
structure and the parameter values of the model. Interesting!
D-optimality has been around for over 30 years now, and has
been well documented. I think Dr. Drusano's comments are well taken.
There are many varieties on D-optimality, as Dr. Drusano has
described.
What suggestions do you have for optimal times to get levels, and why?
Sincerely,
Roger Jellliffe
Roger W. Jelliffe, M.D. Professor of Medicine, USC
USC Laboratory of Applied Pharmacokinetics
2250 Alcazar St, Los Angeles CA 90033, USA
Phone (323)442-1300, fax (323)442-1302, email= jelliffe.at.hsc.usc.edu
Our web site= http://www.usc.edu/hsc/lab_apk
Back to the Top
I do not think you are fully correct. My understanding of
D-optimality is that it is an optimality criterion based on
minimizing the determinant of the covariance matrix of the estimates
(CME). D in D-optimality stands for determinant. The diagonal
elements of the CME are typically used to compute the precision of
the estimates. The determinant reflects the overall precision of the
parameter estimates. There are other ways to view the CME e.g by
scaling the diagonal elements by the parameter estimate then the
determinant can be viewed as reflecting the coefficient of variation
of the precision of the estimates. This has been called C-optimality
(C for coefficient of variation).
A common way of computing the CME uses the derivatives of the model
function with respect to the parameters. In the homoscedastic
(additive error) case the time when the derivative is maximal is the
optimum sampling time for a single parameter. Note that this
derivative method relies on asymptotic theory to estimate precision
and often has very poor properties when applied to non-linear
regression models. The standard errors reported by typical NLR
programs should be viewed with caution and in many cases are not
worth the electrons used to display them on your computer screen.
Even more caution should be applied to use when using this asymptotic
theory to compute D-optimality.
As I have pointed out in an earlier contribution to this thread,
parameter precision is only one way to view the information that one
may wish to optimize from a design. Even if you restrict your
interest to optimizing parameter values you should certainly also
consider the bias in the estimate as well as its precision. It is
possible to combine both bias and precision information into a single
metric - the root mean square error (see Sheiner & Beal JPB 1981;
9:503-12). Designs based on RMSE optimality will be different from
those which focussed only on precision. It is possible to estimate
bias and precision using simulation and then one does not have to
rely upon asymptotic theoretical results to evaluate potential
designs.
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, Private Bag 92019, Auckland, New Zealand
email:n.holford.at.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556
http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm
Back to the Top
Nick wrote:
> regression models. The standard errors reported
> by typical NLR
> programs should be viewed with caution and in
> many cases are not
> worth the electrons used to display them on your
> computer screen.
Agreed - however they can produce a useful guide. However I
wonder how many NONMEM runs have been discarded because the
RS matrix was reported as singular?
> Even more caution should be applied to use when
> using this asymptotic
> theory to compute D-optimality.
I also agree that D-optimality when used for some problems
may produce estimates of the standard errors (SEs) that can
differ from those seen in simulation (this may be more
problematic in population designs where linearisation
techniques are required). However in many cases the
estimates of the SEs of the parameters computed from the
inverse of the Fisher information matrix agree quite well
with those by simulation - and they are *much* easier to
compute. In addition, even if the SEs from the theoretic
approach may not be as accurate as we would like they still
remain relative to each other and therefore maximisation of
the Fisher info matrix will minimise the SEs.
> precision. It is
> possible to combine both bias and precision
> information into a single
> metric - the root mean square error (see Sheiner
> & Beal JPB 1981;
> 9:503-12).
This is perhaps not completely true. If we consider
Variance = MSE + ME^2 (where MSE = mean square error; ME =
mean error)
Then we can see that variance will be inflated if there is
bias, but that RMSE should be independent of bias - at least
in theory.
Cheers
Steve
=================
Stephen Duffull
School of Pharmacy
University of Queensland
Brisbane, QLD 4072
Australia
Ph +61 7 3365 8808
Fax +61 7 3365 1688
http://www.uq.edu.au/pharmacy/duffull.htm
Back to the Top
I've seen several references to the same problem ....i.e. not knowing the
exact time that a blood sample was withdrawn. We have solved this problem
by using an automated blood sampler which actually goes one level further.
It records the time that the blood began to be withdrawn and also the time
that the blood withdrawal ceased. It would be like recording the time you
started to pull on the syringe plunger and the time you stopped pulling the
plunger (something you'd never do with manual blood sampling). We
automatically time-stamp these two events and then take the midpoint
between them as the actual time of blood sampling. This provides much
better control of the Y-axis in a time vs concentration plot.
Candice Kissinger
V.P. Marketing
Bioanalytical Systems Inc.
Direct Line: 765-497-5810
FAX: 765-497-1102
Email: candice.-at-.bioanalytical.com
www.bioanalytical.com
www.culex.net
www.homocysteine-kit.com
Back to the Top
Steve,
"Stephen Duffull (by way of David_Bourne)" wrote:
> Nick wrote:
> > regression models. The standard errors reported
> > by typical NLR
> > programs should be viewed with caution and in
> > many cases are not
> > worth the electrons used to display them on your
> > computer screen.
>
> Agreed - however they can produce a useful guide. However I
> wonder how many NONMEM runs have been discarded because the
> RS matrix was reported as singular?
Dunno. If the COV step ever completes with the kinds of models I look at
I figure I am not being creative enough and add a few more BLOCKS to the
$OMEGA records to stop NONMEM issuing its Standard Error nonsense :-). I
believe others search their data for 'outliers' and delete them to force
NONMEM to behave and keep its rude RS messages to itself while they
search for the mythical 1% standard error in their final model.
> However in many cases the
> estimates of the SEs of the parameters computed from the
> inverse of the Fisher information matrix agree quite well
> with those by simulation - and they are *much* easier to
> compute.
"many cases"? Can you cite some published examples?
BTW for those who are following this thread the Fisher Information
Matrix and the Covariance Matrix of the Estimates (a NONMEM term) are
siblings and most of the time can be viewed as reflecting same thing.
Unless of course you try to stand the FIM on its head when NONMEM will
often reward you with rude warnings and error messages...
> In addition, even if the SEs from the theoretic
> approach may not be as accurate as we would like they still
> remain relative to each other and therefore maximisation of
> the Fisher info matrix will minimise the SEs.
And sometimes, for "numerical reasons" one of those SEs will be very
close to zero and the Determinant will be very small and you will think
you have a great design but the SEs of the other parameters are large
and in fact the design is poor.
> > precision. It is
> > possible to combine both bias and precision
> > information into a single
> > metric - the root mean square error (see Sheiner
> > & Beal JPB 1981;
> > 9:503-12).
>
> This is perhaps not completely true. If we consider
> Variance = MSE + ME^2 (where MSE = mean square error; ME =
> mean error)
> Then we can see that variance will be inflated if there is
> bias, but that RMSE should be independent of bias - at least
> in theory.
This is a terminology issue. For reasons that dont make any sense to me
(tradition?) S&B chose to define "precision" as the mean squared
prediction error (MSE). However, they do include a nice re-arrangement
of the expression you refer to:
MSE = ME^2 + 1/N*Sum((PEi-ME)^2)
or
MSE = BIAS^2 + Variance
(Note that ME is the same as BIAS). They describe the "Variance" term as
"an estimate of the variance of the prediction error". This quantity
describes the variability of the prediction error independently of the
bias. IMHO this "Variance" term is what we should be using to define
precision. With this definition Bias and Precision (i.e. standard
deviation of the prediction error) are independent quantities whose sum
makes up RMSE.
MSE = BIAS^2 + PRECISION^2
RMSE = SQRT(BIAS^2 + PRECISION^2)
A design metric based on RMSE will therefore incorporate both Bias and
Precision of the prediction.
I am afraid I do not understand why you say "RMSE should be independent
of bias - at least in theory.". According to the theory (see above) the
RMSE is very definitely NOT independent of Bias.
Nick
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, Private Bag 92019, Auckland, New Zealand
email:n.holford.-a-.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556
http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm
Back to the Top
Dear Steve,
For testing predictive performance, I thought it was conventional to define
RMSE=bias^2+Variance
where bias is Mean Prediction Error,
Root Mean Squared Error is what it says it is and
variance is is a measure of the spread of the predictions about their mean
ie discounting bias.
If we are considering an optimal design for a known target parameter value
and assessing it by simulation, then RMSE does combine both bias and
precision. (Whether or not it is the best measure is an entirely different
question.) I would have hoped D-optimal designs calculated their
covariance matrix about the true target value and thus worked with a
composite measure, if not this is extremely disturbing.
Regardless, if I fit a two compartment model to data, I may only be truly
interested in CL. Peripheral parameters are both hard to estimate and
difficult to make good use of, so I guess minimizing the covariance matrix
(however it is calculated) of all the parameters isn't all that sensible.
James
Back to the Top
Nick
> > However in many cases the
> > estimates of the SEs of the parameters
> computed from the
> > inverse of the Fisher information matrix agree
> quite well
> > with those by simulation - and they are *much*
> easier to
> > compute.
> "many cases"? Can you cite some published examples?
I have just completed some collaborative work with Sylvie
Retout and France Mentre where we reproduced some of the
work published by Al-Banna et al. JPB 1990;18:347-360, but
instead of using simulations we used a theoretic approach
based on an approximation of the population Fisher
information matrix and it worked very nicely. The work is
currently in press {Retout S, Duffull S, Mentr=E9 F.
Development and implementation of the population Fisher
information matrix for the evaluation of population
pharmacokinetic designs. Computer Methods and Programs in
Biomedicine 2000 [in press]}. Some other newer work also
supports the theoretic approach as providing comparable SEs
to simulation (this is submitted) and was presented by
Sylvie at PAGE 2000.
> > In addition, even if the SEs from the theoretic
> > approach may not be as accurate as we would
> like they still
> > remain relative to each other and therefore
> maximisation of
> > the Fisher info matrix will minimise the SEs.
> And sometimes, for "numerical reasons" one of
> those SEs will be very
> close to zero and the Determinant will be very
> small and you will think
> you have a great design but the SEs of the other
> parameters are large
> and in fact the design is poor.
Perhaps - although we have not seen this happen to date.
This would be very easy to test for. Interestingly, the
=46isher info matrix may become singular and be unable to be
inverted to produce the lower bound of the variance
covariance matrix when the design does not provide enough
information about one or more parameters or the model is
not-identifiable. As a nice test to show this if you try
and estimate bioavailability from only oral data and you
compute the Fisher info matrix for this design (including F
as a parameter to be estimated) the matrix will be singular
(or very close too). Although this is not a formal method
of testing identifiability it is useful.
I'm not saying don't do simulations - but rather suggesting
that computation of the Fisher information matrix when you
have the software is *easy* and can offer some nice
advantages including speed (cf simulations) and can be
implemented as an evaluation step in an optimisation
procedure.
You're quite right about the MSE/variance description - I
must have had a moment of acute brain failure.
Regards
Steve
Stephen Duffull
School of Pharmacy
University of Queensland
Brisbane, QLD 4072
Australia
Ph +61 7 3365 8808
=46ax +61 7 3365 1688
http://www.uq.edu.au/pharmacy/duffull.htm
Back to the Top
> Dunno. If the COV step ever completes with the kinds of models I look at
> I figure I am not being creative enough and add a few more BLOCKS to the
> $OMEGA records to stop NONMEM issuing its Standard Error nonsense :-).
recently we compared confidence intervals and standard errors given by the
NONMEM, and objective function profiling, Jackknife, and bootstrap for three
real data sets. NONMEM did remarkably good job, with standard errors and
confidence intervals being close to the SE and CI obtained by the other
methods (for all THETA, OMEGA, and SIGMA parameters). So I would not through
away "NONMEM ... Standard Error nonsense"
Leonid Gibiansky
Back to the Top
James
> Regardless, if I fit a two compartment model to
> data, I may only be truly
> interested in CL. Peripheral parameters are both
> hard to estimate and
> difficult to make good use of, so I guess
> minimizing the covariance matrix
> (however it is calculated) of all the parameters
> isn't all that sensible.
The alternative is to treat the other parameters as nuisence
parameters and consider the problem as Ds-optimal design (or
subset D-optimal design), eg see Atkinson & Donev. Optimum
Experimental Designs Clarendon Press Oxford 1992. However
these designs tend to be degenerate; ie the optimal design
becomes one where only CL can be estimated and the other
parameters cannot etc and the whole issue becomes much more
complex.
Regards
Steve
=================
Stephen Duffull
School of Pharmacy
University of Queensland
Brisbane, QLD 4072
Australia
Ph +61 7 3365 8808
Fax +61 7 3365 1688
http://www.uq.edu.au/pharmacy/duffull.htm
Back to the Top
Candice,
I wonder if you can give us some idea of the improvement in bias and
precision of pharmacokinetic parameter estimates if times are recorded
and calculated using your automated blood sampler. Given that central
blood volume mixing times are of the order of 2-3 minutes (several times
longer than the typical blood draw time) I wonder under what
circumstances the decreased bias and increased precision of sampling
time would have any detectable impact on parameter estimation.
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, Private Bag 92019, Auckland, New Zealand
email:n.holford.aaa.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556
http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm
Back to the Top
Leonid,
"Gibiansky, Leonid (by way of David_Bourne)" wrote:
>
> > Dunno. If the COV step ever completes with the kinds of models I look at
> > I figure I am not being creative enough and add a few more BLOCKS to the
> > $OMEGA records to stop NONMEM issuing its Standard Error nonsense :-).
>
> recently we compared confidence intervals and standard errors given by the
> NONMEM, and objective function profiling, Jackknife, and bootstrap for three
> real data sets. NONMEM did remarkably good job, with standard errors and
> confidence intervals being close to the SE and CI obtained by the other
> methods (for all THETA, OMEGA, and SIGMA parameters). So I would not throw
> away "NONMEM ... Standard Error nonsense"
Are you planning to publish the details of your findings? e.g. what kind
of models? design of the studies? FO or FOCE?
I caution against using OBJ func profiling and bootstrap methods as
confirmation of the appropriateness of NONMEM's Standard Error estimates
based on analysis of a real data set. The gold standard for evaluating
Standard Error estimates is by simulation so that the true values for
the Standard Errors are known. The only study I know of that has
evaluated the performance of NONMEM is Sheiner LB, Beal ST. A note on
confidence intervals with extended least squares parameter estimation.
Journal of Pharmacokinetics and Biopharmaceutics 1987;15(1):93-98. I
expect there are others. I encourage anyone who can throw further light
on adequately described reports of NONMEM's SE performance to contribute
references to this thread.
My own experience of OBJ func profiling has been that the profile can be
assymetric and that predictions of confidence intervals from NONMEM's
asymptotic SEs would not be in agreement (e.g. Holford NHG, Peace KE.
Results and validation of a population pharmacodynamic model for
cognitive effects in Alzheimer patients treated with tacrine.
Proceedings of the National Academy of Sciences of the United States of
America 1992;89(23):11471-11475, Holford NHG, Williams PEO, Muirhead GJ,
Mitchell A, York A. Population pharmacodynamics of romazarit. British
Journal of Pharmacology 1995;39:313-320.)
Bootstrap estimates of confidence intervals need at least 1000 bootstrap
replicates for a reasonable estimate of these marginal statistics
(Davison AC, Hinkley DV. Bootstrap methods and their application.
Cambridge: Cambridge University Press; 1997). Did you really do 1000
NONMEM runs on each of your 3 real data sets?
PS Have you ever wondered why Standard Errors have that name? I prefer
to try and limit the errors in my work and do not strive to standardize
my mistakes. Yet another reason to find ways to work around those
embarrassing occasions when NONMEM claims to have introduced standard
errors into your results :-)
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, Private Bag 92019, Auckland, New Zealand
email:n.holford.aaa.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556
http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm
Back to the Top
Dear Leonid,
The question is of course, which of these methods do you consider
definitive? They could all agree and all be wrong. NONMEM Se's are
calculated from the curvature of the likelihood surface (assessed by some
algorithm...) and invoke asymptotic properties for the creation of
confidence intervals. I trust them a a lot more than Nick, but less and
less the more parameters I have. I would be interested in seeing your
results when you publish...
General comments on SE calculation:
Parametric bootstrapping makes the assumption that the true underlying
parameter values are equal to those you have estimated and proceeds to
assess SE under this assumption. The term comes from a rather quaint fable
where some baron at the bottom of lake was drowning and then suddenly
realised he could "pull himself up by his own bootstraps". Small children
know this is dubious strategy. Bootstrappings recommended applications are
those where
1) there are no analytic methods (ie confidence interval for the median)
2) you don't trust the asymptotics
I am somewhat asymptotaphobic, but at least these SE's don't take all week
to generate. I don't have any particular faith in bootstrapping. I find
it hard to see how you can use such computationally intensive techniques
for anything but your final model - and then it may be too late...
Objective function profiling - well I don't believe in the chi-squared
approximation either. Profiling also assumes other parameters are fixed ie
that a cross-section of the likelihood surface actually tells you
something. Fans of likelihood profiling hold that it is a much underused
technique. Everyone else worries about ignoring the other parameters.
Can I suggest a superior alternative? No, not really Perhaps you could
assess the response of your objective function using simulated data, find
the "true cut-point" and then use this to likelihood profile. Or use
Markov Chain Monte Carlo (it too is computationally intensive) to explore
the likelihood surface.
If you are using your SEs to give an indication of how well-determined a
paramter is at that minima, it shouldn't be too misleading (and its
definitely better than nothing) in my experience.
James Wright
Back to the Top
Nick,
We (this is the joint work with Katya) showed one of this data set results
at the recent CPT2000 conference. Yes, we plan to publish it. We have used
FO for two data sets and FOCE with interaction for the third one. We did
1000 bootstrap runs, we have perl script that does it for you while you are
on vacation, so the last project was running for 12 days when we were
traveling. Out of 1000 (FOCE with interaction), 495 converged, the rest
finished with errors and were not used for the analysis. For the FO (the
other data sets) only 10 or so runs ended up with no convergence, so around
990 were used for the confidence intervals computations.
For these examples, profiling showed nearly symmetric shapes of the
OF(parameter) functions. All the data was described by the one and
two-compartment linear models, two of them with infusions, one with oral
tablets. Data sets were large, 300-700 patients each, with single infusion
or multiple dosing over several days-several weeks.
Leonid
Back to the Top
James
Without wishing to unduely prolong the discussion on SEs - I
think you have indicated the most appropriate solution...
> likelihood profile. Or use
> Markov Chain Monte Carlo (it too is
> computationally intensive) to explore
> the likelihood surface.
I don't agree that it takes too long. MCMC in my limited
experience has been a very useful tool for PK and popPK
analyses - and alleviates the need for consideration of "add
on" procedures such as asymptotics/profiling/boot strapping
etc!!! We should all become Bayesians thus eliminating
problems associated with frequentist approximations.
Kind regards
Steve
=================
Stephen Duffull
School of Pharmacy
University of Queensland
Brisbane, QLD 4072
Australia
Ph +61 7 3365 8808
Fax +61 7 3365 1688
http://www.uq.edu.au/pharmacy/duffull.htm
Back to the Top
Hi Nick,
We compared, by simulation, the NONMEM SE's from our final model with
the CI's of
multiple simulations/estimations from the final model with the original design.
NONMEM did a decent job on this model too. (One comp 1st order
absorption, lagtime,
IIV, IOV, FO). Karlsson MO, Jonsson EN, Wiltse CG, Wade JR.
Assumption testing in
population pharmacokinetic models: illustrated with an analysis of
moxonidine data
from congestive
heart failure patients. J Pharmacokinet Biopharm. 1998 Apr;26(2):207-46.
Best regards,
Mats
--
Mats Karlsson, PhD
Professor of Biopharmaceutics and Pharmacokinetics
Div. of Biopharmaceutics and Pharmacokinetics
Dept of Pharmacy
Faculty of Pharmacy
Uppsala University
Box 580
SE-751 23 Uppsala
Sweden
phone +46 18 471 4105
fax +46 18 471 4003
mats.karlsson.aaa.biof.uu.se
Back to the Top
"James (by way of David_Bourne)" wrote:
> If you are using your SEs to give an indication of how well-determined a
> paramter is at that minima, it shouldn't be too misleading (and its
> definitely better than nothing) in my experience.
I agree with the non-quantitative "indication" use of asymptotic SEs.
[although I only look at them at a single minimum :-) ]
--
Nick Holford, Dept Pharmacology & Clinical Pharmacology
University of Auckland, Private Bag 92019, Auckland, New Zealand
email:n.holford.-at-.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556
http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm
Back to the Top
Dear Steve,
I think you are right. But you don't have to be a Bayesian (and "warp the
likelihood surface with prior knowledge") to use MCMC.
When I build
models, I do a lot of runs which are sequential in nature and MCMC would
prolong this process, at present a little too much.I think there is probably a
case for using
linearization and MCMC as complementary techniques in the model-building
process.
Or if you want to get really flash, set up your Markov chain to run over
several different models and avoid a definitive, unrealistic
choice of
"model"...of course, this may be a little unwieldy to explain.
James
PharmPK Discussion List Archive Index page
Copyright 1995-2010 David W. A. Bourne (david@boomer.org)