# PharmPK Discussion - Goodness of fit - minimum objective function value versus -2LL

PharmPK Discussion List Archive Index page
• On 9 Apr 2003 at 17:47:08, "Susanne Quellmann" (sqpharma.aaa.web.de) sent the message
`Dear Colleagues,I'm working with the software WinNonMix (using the FOCE-method at themoment) and would like to know, whether the minimum objective functionvalue or the value of -2LL is the parameter to assess the goodness offit. I've found in several articles information about the likelihoodratio test and that for example a decrease of more than 6.6 isassociated with a p-value of 0.01. But this drop of 6.6 as an absolutnumber corresponds to -2LL or the minimum objective function value?This is not consistent in the literature. The users guide of WinNonMixeven says that the parameter estimates are values that minimize anobjective function of the form -2log(likelihood)-nlog(2pi), wherelikelihood is an estimate of the likelihood function.Honestly, I'm confused know, which of these values (minimum objectivefunction value or -2LL) are associated with this signifcance level andI hope, somebody can help me answering this question.I thank you in advance for your assistance.Susanne Quellmann`
Back to the Top

• On 10 Apr 2003 at 15:21:40, Nick Holford (n.holford.-at-.auckland.ac.nz) sent the message
`The following message was posted to: PharmPKWinNonMix like NONMEM reports a number proportional to -2 * loglikelihood as its objective function value.  The -n*log(2*pi) constantcan be ignored when two obj function values are compared. Thedifference between the objective function values obtained with twomodels applied to the same data is commonly used as a statistic tocompare the goodness of fit.If you want to know the probability of rejecting the null hypothesisabout 2 models using the difference in objective function (DOBJ) as atest statistic then you need to know the distribution of this teststatistic when the null hypothesis is true. The distribution of DOBJ isnot known in general. Under some rather restrictive assumptions(particularly assuming normal errors in the model) then it may bereasonable to *assume* that DOBJ is approximately chi-squaredistributed. However, recent empirical tests of this assumeddistribution with some simple cases have shown it cannot be relied uponto predict the P value associated with a particular DOBJ.If you really want to know the P value then you will need to determineit empirically using the randomization test e.g. seehttp://wfn.sourceforge.net/wfnrt.htm. This is tedious and oftenimpractical (especially with WinNonMix which has no support forbatching thousands of model runs which is essential to perform therandomization test).Wählby U, Jonsson EN, Karlsson MO. Assessment of the actualsignificance levels for covariate effects in NONMEM. Journal ofPharmacokinetics & Pharmacodynamics 2001;28:23-252Wählby U, Bouw R, Jonsson EN, Karlsson MO. Assessment of Type I errorrates for the statistical sub-model in NONMEM. Journal ofPharmacokinetics and Pharmacodynamics 2002;29(3):251-269.Gobburu JVS, Lawrence J. Application of resampling techniques toestimate exact significance levels for covariate selection duringnonlinear mixed effects model building: some inferences. PharmaceuticalResearch 2002;19(1):92-98.Nick Holford, Dept Pharmacology & Clinical PharmacologyUniversity of Auckland, 85 Park Rd, Private Bag 92019, Auckland, NewZealandemail:n.holford.-a-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556http://www.health.auckland.ac.nz/pharmacology/staff/nholford/`
Back to the Top

• On 10 Apr 2003 at 20:35:49, Noel Cranswick (noel.cranswick.aaa.rch.org.au) sent the message
`Dear Nick,Greetings from another DownunderJust for my own information, is the Winnonmix algorithm equivalent toFOCE (I'd always assumed that Winnonmix was the same as FO)Thanks,NoelNoel E Cranswick Director, Clinical Pharmacology, Royal Children's HospitalDirector, APPRU, Royal Children's HospitalAssociate Professor, University of Melbourne5th Floor, Main BuildingRoyal Children's Hospital,ParkvilleVictoria 3052AustraliaPhone: 61-3-9345 6987Fax: 61-3-9345 5093Mobile: 0407 512 583www.appru.com`
Back to the Top

• On 11 Apr 2003 at 07:00:03, Nick Holford (n.holford.aaa.auckland.ac.nz) sent the message
`The following message was posted to: PharmPKNoel,The default estimation method used by WinNonMix is more like FOCE thanFO. The algorithm is not identical but similar. My own evaluation ofWNM vs NONMEM indicated that they have broadly similar properties interms of ability to find an objective function minimum and theassociated parameter estimates.NickNick Holford, Dept Pharmacology & Clinical PharmacologyUniversity of Auckland, 85 Park Rd, Private Bag 92019, Auckland, NewZealandemail:n.holford.-at-.auckland.ac.nz tel:+64(9)373-7599x86730 fax:373-7556http://www.health.auckland.ac.nz/pharmacology/staff/nholford/`
Back to the Top

• On 11 Apr 2003 at 06:55:53, "Mark Lovern" (mlovern.-at-.Pharsight.com) sent the message
`The following message was posted to: PharmPKDear Noel:WinNonMix offers both the FO and conditional first-order (FOCE)estimation methods.  Our  implementation for FOCE is a variant of theLindstrom-Bates method (Biometrics 46: 673-687).  Accordingly,WinNonMix's FOCE method is theoretically comparable to NONMEM's.Furthermore, benchmark testing of our software has indicated that,generally speaking, differences in the numerical results of the twopackages are negligible.I hope this information is helpful.  If there is any way that I may beof further assistance, please let me know.With Best Regards,MarkMark R. Lovern, Ph. D.DirectorTechnical Pre-Sales and TrainingPhone: (919) 852-4607Mobile (919) 622-2296FAX: (919) 859-68715520 Dillard Drive, Suite 210Cary, NC 27511`
Back to the Top

• On 14 Apr 2003 at 19:13:52, Roger Jelliffe (jelliffe.-a-.usc.edu) sent the message
`Dear Susanne Quellmann:Since you bring up the subject of goodness of fit, and thecalculation of the likelihood function using the FOCE method, it mightbe useful to look at the examination of the consistency and efficiencyof population modeling methods which use the FOCE approximation tocompute the likelihood, and those which do not. Exact calculation ofthe likelihood function is possible with the nonparametric methods suchas the NPML of Mallet, and the NPEM of Schumitzky, which do not have todo the integration that is needed with the parametric methods usingsuch approximations. Go to our web site www.lapk.org, click on Newadvances in population modeling, and see the work that Robert Learypresented at the PAGE meeting in Paris last June. Consistent methodsobtain results that get closer and closer to the true values as moreand more subjects are studied. This is not true of methods using theFOCE approximation, as he shows there. Because of this, even when thepopulation parameter distributions are truly Gaussian, the means,variances, and covariances obtained with the NP methods are morereliable than those obtained using the FOCE approximation.Very best regards,Roger JelliffeRoger W. Jelliffe, M.D. Professor of Medicine,Division of Geriatric Medicine,Laboratory of Applied Pharmacokinetics,USC Keck School of Medicine2250 Alcazar St, Los Angeles CA 90033, USAPhone (323)442-1300, fax (323)442-1302, email= jelliffe.at.usc.eduOur web site= http://www.lapk.org`
Back to the Top

Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "Goodness of fit - minimum objective function value versus -2LL" as the subject

Copyright 1995-2010 David W. A. Bourne (david@boomer.org)