- On 16 May 2005 at 17:54:06, Roger Jelliffe (jelliffe.at.usc.edu) sent the message

Back to the Top

Hi Lauren:

I don't know of workshops in WinNonLin either. But if you are

interested in real population modeling software leading to optimal

dosage regimens, which also has the guarantee that the more patients

you study the closed your results get to the truth (statistical

consistency), and which are among the most precise known (statistical

efficiency), think of the nonparametric approach with the USCPACK

software. Population modeling may have begun with NONMEM, but it

certainly has not ended there. I don't know of any currently widely

used software that beats this approach. Look at our web site and see

New advances in population modeling, and "teaching topics", and a

comparison of methods that use exact calculations of the likelihood

versus those that use approximations such as FO or FOCE. For example,

we recently presented at the3 IATDM-CT meetings in Louisville the

following poster, which is attached. Further, in a comparison of

population modeling software based on a careful Monte Carlo simulation

study,

Dr. Leary simulated a two-parameter truly Gaussian population

ranging from 25 to 800 subjects, and compared the results found with 1)

IT2B (using the FOCE approximate parametric likelihood), 2) PEM (with

an accurate parametric likelihood [42]), and 3) with NPAG and NPOD

(with exact nonparametric likelihoods). The model had parameters Vol

and Kel. Over 1000 replications were done. Populations ranged from 25

to 800 subjects. A single bolus intravenous dose, and 2 simulated serum

concentrations, each with a 10% standard deviation, were used. Results

with both NPAG and PEM were consistent, with estimates more closely

approaching the true values as the number of subjects increased. The

FOCE IT2B did not have such consistent behavior. A small bias in mean

values of 1 - 2 % was seen with FOCE. As to variances, NPAG and PEM

were again consistent, but the bias of FOCE was quite significant,

about 20 - 30%. As to correlation coefficients, consistent behavior was

again seen with NPAG and PEM. Severe bias was seen with FOCE.

Even more disturbing was the loss of statistical efficiency with the

FOCE approximation. Recently [45], this work was extended, with Dr.

Ruedi Port of the German Cancer Research Institute, to include the FO

and FOCE approximations as implemented in the parametric population

modeling program NONMEM. NONMEM FO had biases as high as 50% in

estimates of variances, and statistical efficiencies less than 2% of

those of the accurate likelihood PEM and NPAG methods for 800 subjects.

NONMEM FOCE was a modest improvement over its IT2B FOCE counterpart.

However, NONMEM FOCE still had significantly compromised statistical

efficiency, less than half that of the accurate likelihood methods, as

shown below:

Estimator Relative efficiency Relative

error

DIRECT OBSERVATION 100.0 % 1.00

PEM 75.4% 1.33

NPOD 61.4% 1.63

NONMEM FOCE 29.0% 3.45

IT2B FOCE 25.3% 3.95

NONMEM FO 0.9% 111.11

A Recent Competition. Recently an international blind trial of seven

parametric population PK/PD estimation methods was conducted under by

INSERM in Lyon, France. One hundred simulated data sets from a

sigmoidal PD dose/response model were sent in May, 2004 to a variety of

PK/PD software vendors and academic developers. Both standard (e.g.,

NONMEM and NLME) nonlinear mixed effects methods based on FOCE

likelihood approximations and new approaches (simulated likelihood,

stochastic approximation, and parametric EM methods, including our PEM)

were included. In September, 2004, participants met in Lyon and the

results were revealed. In general, methods based on more precise

likelihood evaluation techniques significantly outperformed those using

FOCE approximations. Our PEM tied for the overall best performance

among all seven methods as measured by criteria such as RMSE of the

estimated parameter values relative to the true values, and the bias of

the model predictions. In particular, PEM had the best overall

performance in correctly identifying which data sets had a significant

gender covariate dependence and which did not. In addition, when it is

not known whether the distributions are Gaussian or not, NPOD or NPAG

are very efficient and useful methods for population PK/PD modeling, as

also shown above.

Very best regards,

Roger Jelliffe

Roger W. Jelliffe, M.D. Professor of Medicine,

Division of Geriatric Medicine,

Laboratory of Applied Pharmacokinetics,

USC Keck School of Medicine

2250 Alcazar St, Los Angeles CA 90033, USA

Phone (323)442-1300, fax (323)442-1302, email= jelliffe.-a-.usc.edu

Our web site= http://www.lapk.org - On 19 May 2005 at 10:07:24, j.h.proost.-at-.rug.nl sent the message

Back to the Top

The following message was posted to: PharmPK

Dear Roger,

I would like to make a few comments to your reply to Lauren. Putting aside

the issue what you mean with 'real population modeling software', I agree

with your statement that population modeling has not ended with NONMEM.

You described an example of a simulation study by Dr. Leary. About one

year

ago I had some correspondence with you and Dr. Leary about this study and

the results. I had repeated this study with the program MW\Pharm (module

KinPop, written by myself; available from Mediware BV, The Netherlands

(www.mediware.nl)) using an Iterative Two-Stage Bayesian (ITSB) approach

for

population analysis, based on the method described by Mentre and Gomeni (J

Pharm Stat 1995; 5: 141-158) and by Bennett and Wakefield (J Pharmacokinet

Biopharm 1996; 24: 403-432).

The performance of the ITSB algorithm was quite good, and comparable to

that

of PEM and NPAG, the programs cited in your message. For example, the

efficiency was about the same as that of the value of PEM. The only

deviation was a bias of about 1% in k, even for high numbers of patients.

However, given the fact that this value is still smaller than the RMSE

using

800 patients, and clinically insignificant, this limitation is of academic

interest only.

Interestingly, the ITSB algorithm as implemented in MW\Pharm performed

much

better than the USC*PACK IT2B program, although both programs are based on

the same Bayesian approach. The reason for the apparent discrepancy

between

MW\Pharm and IT2B is unclear. Another interesting finding was the relative

good performance of the Standard Two-Stage (STS) procedure; in this case

of

two measurements and two parameters, STS does not require any fitting

procedure, and can be performed in a simple spreadsheet. STS performed

less

well than PEM, NPAG and MW\Pharm (although parameter estimates were still

rather good), and markedly better than IT2B. Also, the efficiency of STS

was

close to that of these programs. This would imply that STS performs also

better than NONMEM. This finding throws a new light on the question with

respect to the begin (STS, not NONMEM) and end of population modeling.

Best regards,

Hans Proost

Johannes H. Proost

Dept. of Pharmacokinetics and Drug Delivery

University Centre for Pharmacy

Antonius Deusinglaan 1

9713 AV Groningen, The Netherlands

tel. 31-50 363 3292

fax 31-50 363 3247

Email: j.h.proost.aaa.rug.nl

Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "Population PK software" as the subject

PharmPK Discussion List Archive Index page

Copyright 1995-2010 David W. A. Bourne (david@boomer.org)