- On 9 Apr 2003 at 17:47:08, "Susanne Quellmann" (sqpharma.aaa.web.de) sent the message

Back to the Top

Dear Colleagues,

I'm working with the software WinNonMix (using the FOCE-method at the

moment) and would like to know, whether the minimum objective function

value or the value of -2LL is the parameter to assess the goodness of

fit. I've found in several articles information about the likelihood

ratio test and that for example a decrease of more than 6.6 is

associated with a p-value of 0.01. But this drop of 6.6 as an absolut

number corresponds to -2LL or the minimum objective function value?

This is not consistent in the literature. The users guide of WinNonMix

even says that the parameter estimates are values that minimize an

objective function of the form -2log(likelihood)-nlog(2pi), where

likelihood is an estimate of the likelihood function.

Honestly, I'm confused know, which of these values (minimum objective

function value or -2LL) are associated with this signifcance level and

I hope, somebody can help me answering this question.

I thank you in advance for your assistance.

Susanne Quellmann - On 10 Apr 2003 at 15:21:40, Nick Holford (n.holford.-at-.auckland.ac.nz) sent the message

Back to the Top

The following message was posted to: PharmPK

WinNonMix like NONMEM reports a number proportional to -2 * log

likelihood as its objective function value. The -n*log(2*pi) constant

can be ignored when two obj function values are compared. The

difference between the objective function values obtained with two

models applied to the same data is commonly used as a statistic to

compare the goodness of fit.

If you want to know the probability of rejecting the null hypothesis

about 2 models using the difference in objective function (DOBJ) as a

test statistic then you need to know the distribution of this test

statistic when the null hypothesis is true. The distribution of DOBJ is

not known in general. Under some rather restrictive assumptions

(particularly assuming normal errors in the model) then it may be

reasonable to *assume* that DOBJ is approximately chi-square

distributed. However, recent empirical tests of this assumed

distribution with some simple cases have shown it cannot be relied upon

to predict the P value associated with a particular DOBJ.

If you really want to know the P value then you will need to determine

it empirically using the randomization test e.g. see

http://wfn.sourceforge.net/wfnrt.htm. This is tedious and often

impractical (especially with WinNonMix which has no support for

batching thousands of model runs which is essential to perform the

randomization test).

Wählby U, Jonsson EN, Karlsson MO. Assessment of the actual

significance levels for covariate effects in NONMEM. Journal of

Pharmacokinetics & Pharmacodynamics 2001;28:23-252

Wählby U, Bouw R, Jonsson EN, Karlsson MO. Assessment of Type I error

rates for the statistical sub-model in NONMEM. Journal of

Pharmacokinetics and Pharmacodynamics 2002;29(3):251-269.

Gobburu JVS, Lawrence J. Application of resampling techniques to

estimate exact significance levels for covariate selection during

nonlinear mixed effects model building: some inferences. Pharmaceutical

Research 2002;19(1):92-98.

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New

Zealand

email:n.holford.-a-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556

http://www.health.auckland.ac.nz/pharmacology/staff/nholford/ - On 10 Apr 2003 at 20:35:49, Noel Cranswick (noel.cranswick.aaa.rch.org.au) sent the message

Back to the Top

Dear Nick,

Greetings from another Downunder

Just for my own information, is the Winnonmix algorithm equivalent to

FOCE (I'd always assumed that Winnonmix was the same as FO)

Thanks,

Noel

Noel E Cranswick

Director, Clinical Pharmacology, Royal Children's Hospital

Director, APPRU, Royal Children's Hospital

Associate Professor, University of Melbourne

5th Floor, Main Building

Royal Children's Hospital,

Parkville

Victoria 3052

Australia

Phone: 61-3-9345 6987

Fax: 61-3-9345 5093

Mobile: 0407 512 583

www.appru.com - On 11 Apr 2003 at 07:00:03, Nick Holford (n.holford.aaa.auckland.ac.nz) sent the message

Back to the Top

The following message was posted to: PharmPK

Noel,

The default estimation method used by WinNonMix is more like FOCE than

FO. The algorithm is not identical but similar. My own evaluation of

WNM vs NONMEM indicated that they have broadly similar properties in

terms of ability to find an objective function minimum and the

associated parameter estimates.

Nick

Nick Holford, Dept Pharmacology & Clinical Pharmacology

University of Auckland, 85 Park Rd, Private Bag 92019, Auckland, New

Zealand

email:n.holford.-at-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556

http://www.health.auckland.ac.nz/pharmacology/staff/nholford/ - On 11 Apr 2003 at 06:55:53, "Mark Lovern" (mlovern.-at-.Pharsight.com) sent the message

Back to the Top

The following message was posted to: PharmPK

Dear Noel:

WinNonMix offers both the FO and conditional first-order (FOCE)

estimation methods. Our implementation for FOCE is a variant of the

Lindstrom-Bates method (Biometrics 46: 673-687). Accordingly,

WinNonMix's FOCE method is theoretically comparable to NONMEM's.

Furthermore, benchmark testing of our software has indicated that,

generally speaking, differences in the numerical results of the two

packages are negligible.

I hope this information is helpful. If there is any way that I may be

of further assistance, please let me know.

With Best Regards,

Mark

Mark R. Lovern, Ph. D.

Director

Technical Pre-Sales and Training

Phone: (919) 852-4607

Mobile (919) 622-2296

FAX: (919) 859-6871

5520 Dillard Drive, Suite 210

Cary, NC 27511 - On 14 Apr 2003 at 19:13:52, Roger Jelliffe (jelliffe.-a-.usc.edu) sent the message

Back to the Top

Dear Susanne Quellmann:

Since you bring up the subject of goodness of fit, and the

calculation of the likelihood function using the FOCE method, it might

be useful to look at the examination of the consistency and efficiency

of population modeling methods which use the FOCE approximation to

compute the likelihood, and those which do not. Exact calculation of

the likelihood function is possible with the nonparametric methods such

as the NPML of Mallet, and the NPEM of Schumitzky, which do not have to

do the integration that is needed with the parametric methods using

such approximations. Go to our web site www.lapk.org, click on New

advances in population modeling, and see the work that Robert Leary

presented at the PAGE meeting in Paris last June. Consistent methods

obtain results that get closer and closer to the true values as more

and more subjects are studied. This is not true of methods using the

FOCE approximation, as he shows there. Because of this, even when the

population parameter distributions are truly Gaussian, the means,

variances, and covariances obtained with the NP methods are more

reliable than those obtained using the FOCE approximation.

Very best regards,

Roger Jelliffe

Roger W. Jelliffe, M.D. Professor of Medicine,

Division of Geriatric Medicine,

Laboratory of Applied Pharmacokinetics,

USC Keck School of Medicine

2250 Alcazar St, Los Angeles CA 90033, USA

Phone (323)442-1300, fax (323)442-1302, email= jelliffe.at.usc.edu

Our web site= http://www.lapk.org

Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "Goodness of fit - minimum objective function value versus -2LL" as the subject

PharmPK Discussion List Archive Index page

Copyright 1995-2010 David W. A. Bourne (david@boomer.org)