- On 5 Jul 2000 at 21:18:04, chendrix.-at-.jhmi.edu sent the message

Back to the Top

QUESTION:

Are there rigorously (or otherwise) established guidelines with regard

to how many samples per subject are recommended per PK parameter

estimated and for each covariate added to the model?

What references are suggested, if any?

BACKGROUND

I am planning a PK substudy nested within a clinical study of neonates

for a drug administered by intramuscular injection, q4h, and cleared

both renally and hepatically in adults. The median treatment length for

the drug is 12 days.

The sample size, calculated based on the primary clinical outcome

measure, is 40 babies (approximately 480 study observation days total).

Because of the understandable reluctance of our principal investigator

neonatologist to do too many heel sticks, I want to minimize the number

of samples I obtain during a dosing interval and overall. Accordingly,

I planned to do a population PK analysis of the data to determine CL/F

and Vd/F.

Optimally, I would like to add a few potentially informative clinical

covariates (bilirubin, Creatinine, weight, post-partum age) to the mixed

effects model.

PRACTICAL QUESTIONS

Practically, will 4 random samples per baby in the first dosing interval

be sufficient to evaluate 2 PK parameters and a few clinical covariates?

(160 sample-time points)

Could it be done with only 2 samples per baby? (80 sample-time points)

How many samples would be useful in each of the days following the

initiation of therapy if I wanted to capture time dependent changes in

PK parameters (expected in this population)?

Thanks for any,

Craig Hendrix - On 6 Jul 2000 at 14:48:23, Paolo Vicini (vicini.aaa.u.washington.edu) sent the message

Back to the Top

Dear Craig,

although I do not know of explicit guidelines of the kind you need,

there is a growing literature on the subject of optimal design of

sampling schedules in populations.

This area of research builds on more established methods (e.g. D-optimal

design) for optimal schedules in individuals. Some classic references on

the latter are:

D'Argenio DZ. Optimal sampling times for pharmacokinetic experiments. J

Pharmacokinet Biopharm 9: 739-756, 1981.

Landaw EM. Optimal design for individual parameter estimation in

pharmacokinetics. In: Variability in Drug Therapy: Description,

Estimation and Control. M. Rowland et al., Eds. Raven Press, New York,

1985.

Individual D-optimal design is implemented for pharmacokinetic models in

a module of the ADAPT II software (available from http://bmsr.usc.edu).

The D-optimal solution always gives as many samples as there are

parameters, i.e. 2 samples are needed for a 2-parameter model.

Of course, the drawback of D-optimality is that the model and the

parameters have to be known ahead of time. In populations, this is not

feasible and parameters are always known with some degree of

uncertainty, which cannot be accommodated in standard D-optimal designs.

Some have tried to assess the sensitivity of D-optimal designs to

uncertainty in the model parameters:

Drusano GL, Forrest A, Snyder MJ, Reed MD. An evaluation of optimal

sampling strategy and adaptive study design. Clin Pharmacol Ther 44:

232-238, 1988.

Drusano GL, Forrest A, Yuen G, Plaisance K.Leslie J. Optimal sampling

theory: effect of error in a nominal parameter value on bias and

precision of parameter estimation. J Clin Pharmacol 34: 967-74, 1994.

Landaw EM. Robust sampling designs for compartmental models under large

prior eigenvalue uncertainties. In: Mathematics and Computers in

Biomedical Applications. J Eisenfeld and C DeLisi, Eds. Elsevier

Science, North-Holland, 1985.

=46or more recent, less studied population sampling strategies, some key

references are:

Mentr=E8 F, Mallet A, Baccar D. Optimal design in random-effects

regression models.Biometrika 84: 429-442, 1997.

Merl=E8 Y, Mentr=E8 F, Mallet A, Aurengo AH. Designing an optimal experiment

for Bayesian estimation: application to the kinetics of iodine thyroid

uptake. Stat Med Jan 30: 185-196, 1994

Merl=E8 Y, Mentr=E8 F. Bayesian design criteria: computation, comparison,

and application to a pharmacokinetic and a pharmacodynamic model. J

Pharmacokinet Biopharm 23:101-125, 1995.

Merl=E8 Y, Mentr=E8 F. Stochastic optimization algorithms of a Bayesian

design criterion for Bayesian parameter estimation of nonlinear

regression models: application in pharmacokinetics. Math Biosci 144:

45-70, 1997.

I have heard of a software tool, OSPOP, for optimization of sampling

times in populations. It reportedly allows the user to choose between a

few different optimality criteria and is based on a global optimization

algorithm. I have however never used it. The references for it are:

Tod M and J-M Rocchisani. Implementation of OSPOP, an algorithm for the

estimation of optimal sampling times in pharmacokinetics by the ED, EID

and API criteria. Computer Methods and Programs in Biomedicine 50:13-22,

1996.

Tod M and J-M Rocchisani. Comparison of ED, EID and API criteria for the

robust optimization of sampling times in pharmacokinetics. J Pharmacokin

Biopharm 25: 515-537, 1997.

Hope this helps,

Paolo

--

Paolo Vicini, Ph.D.

Resource Facility for Population Kinetics FOR COURIER SHIPMENTS USE:

Department of Bioengineering, Box 352255 Room 241 AERL Building

University of Washington University of Washington

Seattle, WA 98195-2255 Seattle, WA 98195-2255

Phone (206)685-2009

=46ax (206)543-3081

http://depts.washington.edu/bioe/people/corefaculty/vicini.html - On 6 Jul 2000 at 22:17:02, "Stephen Duffull" (sduffull.at.pharmacy.uq.edu.au) sent the message

Back to the Top

Craig

If you have some a priori information about the influence of

covariates then you can examin the likely information that

any given pop PK design will yield. You could do this by

evaluation of the population Fisher information matrix

(which can be assessed using software available at:

http://hermes.biomath.jussieu.fr/pfim.htm). From this you

might be able to get an idea of how many blood samples to

take and an idea of timing, eg a sampling window, and you

might consider splitting your neonates into two groups with

each group providing different sampling times. (NB: There

is no global exact pop PK design solution, each experiment

must be considered on a case by case basis.)

I hope this is helpful. Feel free to contact me should you

require further details.

Regards

Steve

=================

Stephen Duffull

School of Pharmacy

University of Queensland

Brisbane, QLD 4072

Australia

Ph +61 7 3365 8808

Fax +61 7 3365 1688 - On 16 Jul 2000 at 23:18:30, GLDrusano.-at-.aol.com sent the message

Back to the Top

Craig,

There is an extensive optimal sampling literature. Dave D'Argenio and Joe

DiStefano had most of the early entries via Monte Carlo simulation (DD) and

animals (JD). Our group did 4 or 5 clinical validation studies. If one is to

use classical optimal sampling, it is wise to choose D optimality

(determinant of the inverse isher Information Matrix) as the optimality

criterion. I you use this, the deterministic nature of the calculation will

provide the answer of 1 optimal time point per system parameter. This is

because deterministic optimal sampling assumes one true parameter vector.

There has been some work with stochastic design, especially useful for

population modeling. Again Dave D'Argenio has a publication in this area, as

do Tod and Rocchisani and Tod, Rocchisani and Mentre. Here, since there is a

stochastic process involved, the number of samples is determined by the

investigator. In this regard, the D'Argenio criterion ( expectation of the

log of the D-optimal design) is especially appealing, as it stays well

defined with fewer than one optimal sampling point per parameter (you have to

know the parameters you wish to have information on), making it very useful

for population PK modeling. There is also unpublished work by Schumitzky

using an Entropy criterion and some VERY elegant work on iteration in policy

space (which can be turned to optimal sampling) by Dave Bayard. Lazlo

Endrenyi has also published in this area.

For the deterministic calculations, ADAPT II (D'Argenio) is available free

and is robust and easy to use. For the others, the authors need to be

contacted directly.

Hope this helps. All the best,

George Drusano - On 16 Jul 2000 at 23:19:17, James (J.G.Wright.-at-.ncl.ac.uk) sent the message

Back to the Top

Dear PharmPK,

In my experience, you never quite get your samples when you request them.

If you request a specific time there is a strong tendency to get the

reported sample time as that time (rather than when it was actually taken).

In steep areas of the concentration-time curve this can be a dominant

source of error.

For this reason alone, I would recommend specifying a sampling window.

Furthermore, I think it is crucial to incorporate uncertainty in your

expected parameter estimates when designing a study. After all, if we knew

them exactly, we wouldn't be running a study.

D-optimality is about estimating parameters under precisely specified

circumstances. Reality requires practicality and acknowledgement of the

many different sources of error.

James Wright - On 16 Jul 2000 at 23:21:06, Hui Kimko (koh.at.compuserve.com) sent the message

Back to the Top

I suggest you simulate the trial with babies to answer your questions.

1. The simulation result is totally depending on what you enter as a PK

model. I guess you have a good idea of the PK profile of adults. You also

should know the disposition pathway so that the difference in metabolic

and/or elimination rate between adults and babies should be appropriately

accounted in the simulation. Body weight seems to affect the PK

allometrically...

2. Instead of defining a set of sampling times, I prefer several

combinations of sampline times to capture the whole phase of PK profile: -

e.g., 1, 3 hrs for 25% of babies, 2,5 hrs for another 25%, 1,5 for

another.... Trial executors may not like this design. Another approach may

be providing them a sampling block design - e.g., around 0.5 hour (=0.25 -

1 hour), around 2 hours (= 1 - 3 hours), etc, expecting natural

distributions around the specified times. Using the actual sampling times

is critical in your data analysis step after the actual baby trial is over.

3. Your "PRACTICAL QUESTIONS" above are good ones for simulation scenarios.

I hope this helps. :-)

Huicy

___

Hui C. Kimko, PhD

Center for Drug Development Science

Georgetown University Medical Center

voice: 202 687 4332

fax: 202 687 0193 - On 17 Jul 2000 at 23:25:16, GLDrusano.at.aol.com sent the message

Back to the Top

Dear James,

You are, of course, correct in the larger sense. However, system information

has a peak, but is not zero a small distance away (time wise). The point

estimate is helpful to guide acquisition. The actual collection is important

in that the "collector" needs to know that the actual time of acquisition is

key.

All the best,

George - On 21 Jul 2000 at 12:33:12, "J.G. Wright" (J.G.Wright.-at-.newcastle.ac.uk) sent the message

Back to the Top

Dear Dr Drusano,

Yes, I agree that not having the sample at the precise time you requested

does not render it useless, if you know the true time. However, I think

error in the recorded time

will always exist and in a steep area of the curve could be dominant.

Hence, the sample may not truly occur at the recorded time and could even

be harmful to estimation (unless you are accounting for this in your

model)

I do not belief that D-optimal based on a fixed vestor of parameters is

based on reliable enough information to guide trial design alone. What

is a

D-optimal point depends crucially on how you define your vector of

parameters and even more crucially, how you define the nature of error.

Whilst you may be able to put some reasonable priors on

population parameters, a priori estimates of the variance function are

probably speculative. In fact, variance functions estimated once you have

the data

are often somewhat speculative.

I think D-optimality is a tool whose value is dependent on the

validity of your assumptions.

James Wright - On 22 Jul 2000 at 22:37:51, Roger Jelliffe (jelliffe.aaa.usc.edu) sent the message

Back to the Top

Dear James Wright:

As far as I know, D-optimality is a way of relating the

relationship between a change in a parameter value in a model and the

resulting change in a serum concentration (or other response). The

d-optimal times are those at which a change in a parameter value

makes the greatest change in a serum concentration (or other

response).

This can also be seen by simulation. A simple example is a 1

compartment model, with parameters V and Kel. In a setting of

intermittent IV infusion, suppose the V decreases by a certain

factor, for example, 10%. All the serum concentrations will be

correspondingly higher. Assuming a constant lab assay error, the

greatest change takes place at the true peak, at the end of the IV

infusion. If the assay error is not constant, that can be taken into

account - see Dave D'Argenio's ADAPT II program, for example. The

D-optimal time is the one at which the partial derivative of the

serum conc is maximal with respect to each relevant parameter.

In the same way, take a certain value of Kel. Run it through

several or many values of time, and compute e to the minus KT. Do the

same thing for a slightly different value of K. Look at the

differences between the 2 sets of numbers. They are greatest when e

to the minus KT is 0.36, or 1.44 half times. Why? Who knows, but

that's the way it is.

I am told that D optimality reflects this behavior, and that

this is the reason why the D optimal times for the 1 compartment

model, for intermittent IV input, are the peak, the true peak (unless

adjusted for assay error specifically) and at 1.44 half times after

the end of the IV, when the concentrations are down to 36% of the

original peak. Other more complex models are supersets of this

simplest model, and have other times which are optimal for the other

parameters. D-optimality seeks to find the optimal set of times which

maximize the overall information about the various parameters.

It is true that there is a catch-22, that you are supposed to

know the parameter values in order to compute the times at which it

is optimal to estimate them. However, if one's estimates are probably

not far off, it has been a good strategy.

If you do simulations of this type, it becomes clear that

while a certain time may be the very best, that there are usually

rather broad regions of general sensitivity of the responses to the

various parameters. There is not too much difference between the true

peak and other high levels. Similarly, there is usually a somewhat

broad region of good sensitivity of serum levels to the Kel. Because

of this, it is not too bad if one does not get the samples at just

the optimal times, just so you know when they were in fact obtained.

Yes, errors in recording the times are always present.

Because of this, many have preferred to get trough levels, for

example, as such errors make the least difference in the serum conc

found. However, this strategy leads to the deliberate choice of a

data point when it is usually the LEAST informative about the

structure and the parameter values of the model. Interesting!

D-optimality has been around for over 30 years now, and has

been well documented. I think Dr. Drusano's comments are well taken.

There are many varieties on D-optimality, as Dr. Drusano has

described.

What suggestions do you have for optimal times to get levels, and why?

Sincerely,

Roger Jellliffe

Roger W. Jelliffe, M.D. Professor of Medicine, USC

USC Laboratory of Applied Pharmacokinetics

2250 Alcazar St, Los Angeles CA 90033, USA

Phone (323)442-1300, fax (323)442-1302, email= jelliffe.at.hsc.usc.edu

Our web site= http://www.usc.edu/hsc/lab_apk - On 23 Jul 2000 at 22:25:45, Nick Holford (n.holford.at.auckland.ac.nz) sent the message

Back to the Top

I do not think you are fully correct. My understanding of

D-optimality is that it is an optimality criterion based on

minimizing the determinant of the covariance matrix of the estimates

(CME). D in D-optimality stands for determinant. The diagonal

elements of the CME are typically used to compute the precision of

the estimates. The determinant reflects the overall precision of the

parameter estimates. There are other ways to view the CME e.g by

scaling the diagonal elements by the parameter estimate then the

determinant can be viewed as reflecting the coefficient of variation

of the precision of the estimates. This has been called C-optimality

(C for coefficient of variation).

A common way of computing the CME uses the derivatives of the model

function with respect to the parameters. In the homoscedastic

(additive error) case the time when the derivative is maximal is the

optimum sampling time for a single parameter. Note that this

derivative method relies on asymptotic theory to estimate precision

and often has very poor properties when applied to non-linear

regression models. The standard errors reported by typical NLR

programs should be viewed with caution and in many cases are not

worth the electrons used to display them on your computer screen.

Even more caution should be applied to use when using this asymptotic

theory to compute D-optimality.

As I have pointed out in an earlier contribution to this thread,

parameter precision is only one way to view the information that one

may wish to optimize from a design. Even if you restrict your

interest to optimizing parameter values you should certainly also

consider the bias in the estimate as well as its precision. It is

possible to combine both bias and precision information into a single

metric - the root mean square error (see Sheiner & Beal JPB 1981;

9:503-12). Designs based on RMSE optimality will be different from

those which focussed only on precision. It is possible to estimate

bias and precision using simulation and then one does not have to

rely upon asymptotic theoretical results to evaluate potential

designs.

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, Private Bag 92019, Auckland, New Zealand

email:n.holford.at.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556

http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm - On 24 Jul 2000 at 22:17:02, "Stephen Duffull" (sduffull.aaa.pharmacy.uq.edu.au) sent the message

Back to the Top

Nick wrote:

> regression models. The standard errors reported

> by typical NLR

> programs should be viewed with caution and in

> many cases are not

> worth the electrons used to display them on your

> computer screen.

Agreed - however they can produce a useful guide. However I

wonder how many NONMEM runs have been discarded because the

RS matrix was reported as singular?

> Even more caution should be applied to use when

> using this asymptotic

> theory to compute D-optimality.

I also agree that D-optimality when used for some problems

may produce estimates of the standard errors (SEs) that can

differ from those seen in simulation (this may be more

problematic in population designs where linearisation

techniques are required). However in many cases the

estimates of the SEs of the parameters computed from the

inverse of the Fisher information matrix agree quite well

with those by simulation - and they are *much* easier to

compute. In addition, even if the SEs from the theoretic

approach may not be as accurate as we would like they still

remain relative to each other and therefore maximisation of

the Fisher info matrix will minimise the SEs.

> precision. It is

> possible to combine both bias and precision

> information into a single

> metric - the root mean square error (see Sheiner

> & Beal JPB 1981;

> 9:503-12).

This is perhaps not completely true. If we consider

Variance = MSE + ME^2 (where MSE = mean square error; ME =

mean error)

Then we can see that variance will be inflated if there is

bias, but that RMSE should be independent of bias - at least

in theory.

Cheers

Steve

=================

Stephen Duffull

School of Pharmacy

University of Queensland

Brisbane, QLD 4072

Australia

Ph +61 7 3365 8808

Fax +61 7 3365 1688

http://www.uq.edu.au/pharmacy/duffull.htm - On 25 Jul 2000 at 10:25:48, Candice Kissinger (candice.-at-.bioanalytical.com) sent the message

Back to the Top

I've seen several references to the same problem ....i.e. not knowing the

exact time that a blood sample was withdrawn. We have solved this problem

by using an automated blood sampler which actually goes one level further.

It records the time that the blood began to be withdrawn and also the time

that the blood withdrawal ceased. It would be like recording the time you

started to pull on the syringe plunger and the time you stopped pulling the

plunger (something you'd never do with manual blood sampling). We

automatically time-stamp these two events and then take the midpoint

between them as the actual time of blood sampling. This provides much

better control of the Y-axis in a time vs concentration plot.

Candice Kissinger

V.P. Marketing

Bioanalytical Systems Inc.

Direct Line: 765-497-5810

FAX: 765-497-1102

Email: candice.-at-.bioanalytical.com

www.bioanalytical.com

www.culex.net

www.homocysteine-kit.com - On 25 Jul 2000 at 10:27:09, Nick Holford (n.holford.-at-.auckland.ac.nz) sent the message

Back to the Top

Steve,

"Stephen Duffull (by way of David_Bourne)" wrote:

> Nick wrote:

> > regression models. The standard errors reported

> > by typical NLR

> > programs should be viewed with caution and in

> > many cases are not

> > worth the electrons used to display them on your

> > computer screen.

>

> Agreed - however they can produce a useful guide. However I

> wonder how many NONMEM runs have been discarded because the

> RS matrix was reported as singular?

Dunno. If the COV step ever completes with the kinds of models I look at

I figure I am not being creative enough and add a few more BLOCKS to the

$OMEGA records to stop NONMEM issuing its Standard Error nonsense :-). I

believe others search their data for 'outliers' and delete them to force

NONMEM to behave and keep its rude RS messages to itself while they

search for the mythical 1% standard error in their final model.

> However in many cases the

> estimates of the SEs of the parameters computed from the

> inverse of the Fisher information matrix agree quite well

> with those by simulation - and they are *much* easier to

> compute.

"many cases"? Can you cite some published examples?

BTW for those who are following this thread the Fisher Information

Matrix and the Covariance Matrix of the Estimates (a NONMEM term) are

siblings and most of the time can be viewed as reflecting same thing.

Unless of course you try to stand the FIM on its head when NONMEM will

often reward you with rude warnings and error messages...

> In addition, even if the SEs from the theoretic

> approach may not be as accurate as we would like they still

> remain relative to each other and therefore maximisation of

> the Fisher info matrix will minimise the SEs.

And sometimes, for "numerical reasons" one of those SEs will be very

close to zero and the Determinant will be very small and you will think

you have a great design but the SEs of the other parameters are large

and in fact the design is poor.

> > precision. It is

> > possible to combine both bias and precision

> > information into a single

> > metric - the root mean square error (see Sheiner

> > & Beal JPB 1981;

> > 9:503-12).

>

> This is perhaps not completely true. If we consider

> Variance = MSE + ME^2 (where MSE = mean square error; ME =

> mean error)

> Then we can see that variance will be inflated if there is

> bias, but that RMSE should be independent of bias - at least

> in theory.

This is a terminology issue. For reasons that dont make any sense to me

(tradition?) S&B chose to define "precision" as the mean squared

prediction error (MSE). However, they do include a nice re-arrangement

of the expression you refer to:

MSE = ME^2 + 1/N*Sum((PEi-ME)^2)

or

MSE = BIAS^2 + Variance

(Note that ME is the same as BIAS). They describe the "Variance" term as

"an estimate of the variance of the prediction error". This quantity

describes the variability of the prediction error independently of the

bias. IMHO this "Variance" term is what we should be using to define

precision. With this definition Bias and Precision (i.e. standard

deviation of the prediction error) are independent quantities whose sum

makes up RMSE.

MSE = BIAS^2 + PRECISION^2

RMSE = SQRT(BIAS^2 + PRECISION^2)

A design metric based on RMSE will therefore incorporate both Bias and

Precision of the prediction.

I am afraid I do not understand why you say "RMSE should be independent

of bias - at least in theory.". According to the theory (see above) the

RMSE is very definitely NOT independent of Bias.

Nick

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, Private Bag 92019, Auckland, New Zealand

email:n.holford.-a-.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556

http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm - On 25 Jul 2000 at 10:31:00, James (J.G.Wright.-a-.ncl.ac.uk) sent the message

Back to the Top

Dear Steve,

For testing predictive performance, I thought it was conventional to define

RMSE=bias^2+Variance

where bias is Mean Prediction Error,

Root Mean Squared Error is what it says it is and

variance is is a measure of the spread of the predictions about their mean

ie discounting bias.

If we are considering an optimal design for a known target parameter value

and assessing it by simulation, then RMSE does combine both bias and

precision. (Whether or not it is the best measure is an entirely different

question.) I would have hoped D-optimal designs calculated their

covariance matrix about the true target value and thus worked with a

composite measure, if not this is extremely disturbing.

Regardless, if I fit a two compartment model to data, I may only be truly

interested in CL. Peripheral parameters are both hard to estimate and

difficult to make good use of, so I guess minimizing the covariance matrix

(however it is calculated) of all the parameters isn't all that sensible.

James - On 25 Jul 2000 at 21:51:04, "Stephen Duffull" (sduffull.-at-.pharmacy.uq.edu.au) sent the message

Back to the Top

Nick

> > However in many cases the

> > estimates of the SEs of the parameters

> computed from the

> > inverse of the Fisher information matrix agree

> quite well

> > with those by simulation - and they are *much*

> easier to

> > compute.

> "many cases"? Can you cite some published examples?

I have just completed some collaborative work with Sylvie

Retout and France Mentre where we reproduced some of the

work published by Al-Banna et al. JPB 1990;18:347-360, but

instead of using simulations we used a theoretic approach

based on an approximation of the population Fisher

information matrix and it worked very nicely. The work is

currently in press {Retout S, Duffull S, Mentr=E9 F.

Development and implementation of the population Fisher

information matrix for the evaluation of population

pharmacokinetic designs. Computer Methods and Programs in

Biomedicine 2000 [in press]}. Some other newer work also

supports the theoretic approach as providing comparable SEs

to simulation (this is submitted) and was presented by

Sylvie at PAGE 2000.

> > In addition, even if the SEs from the theoretic

> > approach may not be as accurate as we would

> like they still

> > remain relative to each other and therefore

> maximisation of

> > the Fisher info matrix will minimise the SEs.

> And sometimes, for "numerical reasons" one of

> those SEs will be very

> close to zero and the Determinant will be very

> small and you will think

> you have a great design but the SEs of the other

> parameters are large

> and in fact the design is poor.

Perhaps - although we have not seen this happen to date.

This would be very easy to test for. Interestingly, the

=46isher info matrix may become singular and be unable to be

inverted to produce the lower bound of the variance

covariance matrix when the design does not provide enough

information about one or more parameters or the model is

not-identifiable. As a nice test to show this if you try

and estimate bioavailability from only oral data and you

compute the Fisher info matrix for this design (including F

as a parameter to be estimated) the matrix will be singular

(or very close too). Although this is not a formal method

of testing identifiability it is useful.

I'm not saying don't do simulations - but rather suggesting

that computation of the Fisher information matrix when you

have the software is *easy* and can offer some nice

advantages including speed (cf simulations) and can be

implemented as an evaluation step in an optimisation

procedure.

You're quite right about the MSE/variance description - I

must have had a moment of acute brain failure.

Regards

Steve

Stephen Duffull

School of Pharmacy

University of Queensland

Brisbane, QLD 4072

Australia

Ph +61 7 3365 8808

=46ax +61 7 3365 1688

http://www.uq.edu.au/pharmacy/duffull.htm - On 26 Jul 2000 at 10:58:04, "Gibiansky, Leonid" (gibianskyl.aaa.globomax.com) sent the message

Back to the Top

> Dunno. If the COV step ever completes with the kinds of models I look at

> I figure I am not being creative enough and add a few more BLOCKS to the

> $OMEGA records to stop NONMEM issuing its Standard Error nonsense :-).

recently we compared confidence intervals and standard errors given by the

NONMEM, and objective function profiling, Jackknife, and bootstrap for three

real data sets. NONMEM did remarkably good job, with standard errors and

confidence intervals being close to the SE and CI obtained by the other

methods (for all THETA, OMEGA, and SIGMA parameters). So I would not through

away "NONMEM ... Standard Error nonsense"

Leonid Gibiansky - On 26 Jul 2000 at 10:58:53, "Stephen Duffull" (sduffull.-at-.pharmacy.uq.edu.au) sent the message

Back to the Top

James

> Regardless, if I fit a two compartment model to

> data, I may only be truly

> interested in CL. Peripheral parameters are both

> hard to estimate and

> difficult to make good use of, so I guess

> minimizing the covariance matrix

> (however it is calculated) of all the parameters

> isn't all that sensible.

The alternative is to treat the other parameters as nuisence

parameters and consider the problem as Ds-optimal design (or

subset D-optimal design), eg see Atkinson & Donev. Optimum

Experimental Designs Clarendon Press Oxford 1992. However

these designs tend to be degenerate; ie the optimal design

becomes one where only CL can be estimated and the other

parameters cannot etc and the whole issue becomes much more

complex.

Regards

Steve

=================

Stephen Duffull

School of Pharmacy

University of Queensland

Brisbane, QLD 4072

Australia

Ph +61 7 3365 8808

Fax +61 7 3365 1688

http://www.uq.edu.au/pharmacy/duffull.htm - On 26 Jul 2000 at 10:59:29, Nick Holford (n.holford.-at-.auckland.ac.nz) sent the message

Back to the Top

Candice,

I wonder if you can give us some idea of the improvement in bias and

precision of pharmacokinetic parameter estimates if times are recorded

and calculated using your automated blood sampler. Given that central

blood volume mixing times are of the order of 2-3 minutes (several times

longer than the typical blood draw time) I wonder under what

circumstances the decreased bias and increased precision of sampling

time would have any detectable impact on parameter estimation.

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, Private Bag 92019, Auckland, New Zealand

email:n.holford.aaa.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556

http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm - On 26 Jul 2000 at 15:26:07, Nick Holford (n.holford.aaa.auckland.ac.nz) sent the message

Back to the Top

Leonid,

"Gibiansky, Leonid (by way of David_Bourne)" wrote:

>

> > Dunno. If the COV step ever completes with the kinds of models I look at

> > I figure I am not being creative enough and add a few more BLOCKS to the

> > $OMEGA records to stop NONMEM issuing its Standard Error nonsense :-).

>

> recently we compared confidence intervals and standard errors given by the

> NONMEM, and objective function profiling, Jackknife, and bootstrap for three

> real data sets. NONMEM did remarkably good job, with standard errors and

> confidence intervals being close to the SE and CI obtained by the other

> methods (for all THETA, OMEGA, and SIGMA parameters). So I would not throw

> away "NONMEM ... Standard Error nonsense"

Are you planning to publish the details of your findings? e.g. what kind

of models? design of the studies? FO or FOCE?

I caution against using OBJ func profiling and bootstrap methods as

confirmation of the appropriateness of NONMEM's Standard Error estimates

based on analysis of a real data set. The gold standard for evaluating

Standard Error estimates is by simulation so that the true values for

the Standard Errors are known. The only study I know of that has

evaluated the performance of NONMEM is Sheiner LB, Beal ST. A note on

confidence intervals with extended least squares parameter estimation.

Journal of Pharmacokinetics and Biopharmaceutics 1987;15(1):93-98. I

expect there are others. I encourage anyone who can throw further light

on adequately described reports of NONMEM's SE performance to contribute

references to this thread.

My own experience of OBJ func profiling has been that the profile can be

assymetric and that predictions of confidence intervals from NONMEM's

asymptotic SEs would not be in agreement (e.g. Holford NHG, Peace KE.

Results and validation of a population pharmacodynamic model for

cognitive effects in Alzheimer patients treated with tacrine.

Proceedings of the National Academy of Sciences of the United States of

America 1992;89(23):11471-11475, Holford NHG, Williams PEO, Muirhead GJ,

Mitchell A, York A. Population pharmacodynamics of romazarit. British

Journal of Pharmacology 1995;39:313-320.)

Bootstrap estimates of confidence intervals need at least 1000 bootstrap

replicates for a reasonable estimate of these marginal statistics

(Davison AC, Hinkley DV. Bootstrap methods and their application.

Cambridge: Cambridge University Press; 1997). Did you really do 1000

NONMEM runs on each of your 3 real data sets?

PS Have you ever wondered why Standard Errors have that name? I prefer

to try and limit the errors in my work and do not strive to standardize

my mistakes. Yet another reason to find ways to work around those

embarrassing occasions when NONMEM claims to have introduced standard

errors into your results :-)

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, Private Bag 92019, Auckland, New Zealand

email:n.holford.aaa.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556

http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm - On 27 Jul 2000 at 10:47:27, James (J.G.Wright.-at-.ncl.ac.uk) sent the message

Back to the Top

Dear Leonid,

The question is of course, which of these methods do you consider

definitive? They could all agree and all be wrong. NONMEM Se's are

calculated from the curvature of the likelihood surface (assessed by some

algorithm...) and invoke asymptotic properties for the creation of

confidence intervals. I trust them a a lot more than Nick, but less and

less the more parameters I have. I would be interested in seeing your

results when you publish...

General comments on SE calculation:

Parametric bootstrapping makes the assumption that the true underlying

parameter values are equal to those you have estimated and proceeds to

assess SE under this assumption. The term comes from a rather quaint fable

where some baron at the bottom of lake was drowning and then suddenly

realised he could "pull himself up by his own bootstraps". Small children

know this is dubious strategy. Bootstrappings recommended applications are

those where

1) there are no analytic methods (ie confidence interval for the median)

2) you don't trust the asymptotics

I am somewhat asymptotaphobic, but at least these SE's don't take all week

to generate. I don't have any particular faith in bootstrapping. I find

it hard to see how you can use such computationally intensive techniques

for anything but your final model - and then it may be too late...

Objective function profiling - well I don't believe in the chi-squared

approximation either. Profiling also assumes other parameters are fixed ie

that a cross-section of the likelihood surface actually tells you

something. Fans of likelihood profiling hold that it is a much underused

technique. Everyone else worries about ignoring the other parameters.

Can I suggest a superior alternative? No, not really Perhaps you could

assess the response of your objective function using simulated data, find

the "true cut-point" and then use this to likelihood profile. Or use

Markov Chain Monte Carlo (it too is computationally intensive) to explore

the likelihood surface.

If you are using your SEs to give an indication of how well-determined a

paramter is at that minima, it shouldn't be too misleading (and its

definitely better than nothing) in my experience.

James Wright - On 27 Jul 2000 at 10:47:52, "Gibiansky, Leonid" (gibianskyl.-at-.globomax.com) sent the message

Back to the Top

Nick,

We (this is the joint work with Katya) showed one of this data set results

at the recent CPT2000 conference. Yes, we plan to publish it. We have used

FO for two data sets and FOCE with interaction for the third one. We did

1000 bootstrap runs, we have perl script that does it for you while you are

on vacation, so the last project was running for 12 days when we were

traveling. Out of 1000 (FOCE with interaction), 495 converged, the rest

finished with errors and were not used for the analysis. For the FO (the

other data sets) only 10 or so runs ended up with no convergence, so around

990 were used for the confidence intervals computations.

For these examples, profiling showed nearly symmetric shapes of the

OF(parameter) functions. All the data was described by the one and

two-compartment linear models, two of them with infusions, one with oral

tablets. Data sets were large, 300-700 patients each, with single infusion

or multiple dosing over several days-several weeks.

Leonid - On 27 Jul 2000 at 20:47:34, "Stephen Duffull" (sduffull.aaa.pharmacy.uq.edu.au) sent the message

Back to the Top

James

Without wishing to unduely prolong the discussion on SEs - I

think you have indicated the most appropriate solution...

> likelihood profile. Or use

> Markov Chain Monte Carlo (it too is

> computationally intensive) to explore

> the likelihood surface.

I don't agree that it takes too long. MCMC in my limited

experience has been a very useful tool for PK and popPK

analyses - and alleviates the need for consideration of "add

on" procedures such as asymptotics/profiling/boot strapping

etc!!! We should all become Bayesians thus eliminating

problems associated with frequentist approximations.

Kind regards

Steve

=================

Stephen Duffull

School of Pharmacy

University of Queensland

Brisbane, QLD 4072

Australia

Ph +61 7 3365 8808

Fax +61 7 3365 1688

http://www.uq.edu.au/pharmacy/duffull.htm - On 28 Jul 2000 at 11:23:19, Mats Karlsson (Mats.Karlsson.-at-.biof.uu.se) sent the message

Back to the Top

Hi Nick,

We compared, by simulation, the NONMEM SE's from our final model with

the CI's of

multiple simulations/estimations from the final model with the original design.

NONMEM did a decent job on this model too. (One comp 1st order

absorption, lagtime,

IIV, IOV, FO). Karlsson MO, Jonsson EN, Wiltse CG, Wade JR.

Assumption testing in

population pharmacokinetic models: illustrated with an analysis of

moxonidine data

from congestive

heart failure patients. J Pharmacokinet Biopharm. 1998 Apr;26(2):207-46.

Best regards,

Mats

--

Mats Karlsson, PhD

Professor of Biopharmaceutics and Pharmacokinetics

Div. of Biopharmaceutics and Pharmacokinetics

Dept of Pharmacy

Faculty of Pharmacy

Uppsala University

Box 580

SE-751 23 Uppsala

Sweden

phone +46 18 471 4105

fax +46 18 471 4003

mats.karlsson.aaa.biof.uu.se - On 28 Jul 2000 at 17:28:46, Nick Holford (n.holford.-at-.auckland.ac.nz) sent the message

Back to the Top

"James (by way of David_Bourne)" wrote:

> If you are using your SEs to give an indication of how well-determined a

> paramter is at that minima, it shouldn't be too misleading (and its

> definitely better than nothing) in my experience.

I agree with the non-quantitative "indication" use of asymptotic SEs.

[although I only look at them at a single minimum :-) ]

--

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, Private Bag 92019, Auckland, New Zealand

email:n.holford.-at-.auckland.ac.nz tel:+64(9)373-7599x6730 fax:373-7556

http://www.phm.auckland.ac.nz/Staff/NHolford/nholford.htm - On 30 Jul 2000 at 13:41:47, "J.G. Wright" (J.G.Wright.-a-.newcastle.ac.uk) sent the message

Back to the Top

Dear Steve,

I think you are right. But you don't have to be a Bayesian (and "warp the

likelihood surface with prior knowledge") to use MCMC.

When I build

models, I do a lot of runs which are sequential in nature and MCMC would

prolong this process, at present a little too much.I think there is probably a

case for using

linearization and MCMC as complementary techniques in the model-building

process.

Or if you want to get really flash, set up your Markov chain to run over

several different models and avoid a definitive, unrealistic

choice of

"model"...of course, this may be a little unwieldy to explain.

James

Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "How many samples per subject" as the subject

PharmPK Discussion List Archive Index page

Copyright 1995-2010 David W. A. Bourne (david@boomer.org)