# PharmPK Discussion - Standard errors of Bayesian parameter estimates

PharmPK Discussion List Archive Index page
• On 10 May 2009 at 19:06:40, "andreas lindauer" (lindauer.-a-.uni-bonn.de) sent the message
`Dear All,I have some questions regarding the calculation of standard errors (orconfidence intervals) of Bayesian (PK-)parameter estimates.Suppose the following expression is to be minimized:SSE=sum((observations-predictions)^2/sigma^2))+sum((individual_parameter-population_parameter)^2/omega^2))Where observations are from a single patient only and predictions areobtained from some nonlinear equation (e.g. a PK-model).1) Is it (mathematically) correct to calculate SEs for the parameterestimates like this:     M=number of parameters     n=number of observations     MSE=SSE/(n+M-M)  # Mean squared error     SE = sqrt(MSE.*diagonal(inverse(hessian)))2) However, 1) gives only asymptotic SEs which are known to have somedisadvantages. Therefore in population modeling non-parametricresampling techniques (bootstrap, jack-knife) are increasinglyapplied. Such methods, however, wouldn't make much sense for single-subject data with only few observations.Are there other methods that could be used for such situations inorder to calculate confidence intervals? Maybe log-likelihood-profiling?Or, can inference statistics be applied on the posterior distributionof a parameter?3) How can a confidence band around the predicted curve be constructed?Thanks and regards, Andreas.--Andreas LindauerDepartment of Clinical PharmacyInstitute of PharmacyUniversity of BonnAn der Immenburg 4D-53121 Bonn`
Back to the Top

• On 11 May 2009 at 06:43:20, "Ed O'Connor" (Bioconsul09.-a-.cox.net) sent the message
`The following message was posted to: PharmPKAndreas:  I cannot see how you would calculate a SSE for an individualpatient when the data collected are released as a single value for each timepoint, which is the norm.   Please explain.`
Back to the Top

• On 11 May 2009 at 17:51:23, "andreas lindauer" (lindauer.aaa.uni-bonn.de) sent the message
`The following message was posted to: PharmPKEd,Maybe my nomenclature was confusing. With SSE I essentially meant aBayesianleast squares criterion i.e. an Objective function value that isminimizedby some nonlinear regression algorithm (e.g. Levenberg-Marquardt) whichprovides a Hessian matrix.There should be no problem with data obtained at different time points.Sorry for the confusion, Andreas.`
Back to the Top

• On 11 May 2009 at 17:43:37, "Roger Jelliffe" (jelliffe.aaa.usc.edu) sent the message
`Dear Ed and Andreas:           You are asking a very good question. It seems to me thatyou wish to have some idea of the uncertainties present in anindividual patient's  Bayesian posterior parameter distribution. Idon't quite understand why you wish to do this for a parameterdistribution which is assumed to have some particular shape. It doesnot get you the model likely posterior distribution. It just gets youthe estimated means and covariances. Also why do you wish to do thisat all? Just to know, or perhaps to use this information somehow incalculating the uncertainties which will result from a dosage regimenbased on those posterior parameter estimates and those variances. Itseems to me that you can do better, and can also see the resultsgraphically, by using nonparametric (NP) approaches.           First, use an NP approach to make a population PK/PD model.There are theorems by Mallet, Lindsay, and Carathodory that prove thatyou do not have to examine a continuous distribution over all itspossible parameter spaces, but that the maximum likelihooddistribution "can be found" in a discrete set of points thatconstitute the most likely joint parameter distribution. In this way,no assumptions need to be made about the shape of any continuousfunction such as normal, lognormal, etc. The NP approach finds themost likely set of support points given the raw data and the errormodel used. You wind up with up to 1 support point per subject studiedin the population. You get the entire discrete distribution. You willsee unsuspected subpopulations such as faster and slower metabolizers,for example, at that early stage of analysis.           What does this do for you? You don't need to assume anydistribution. You don't need to obtain the "estimators" (means andcovariances) of a distribution. They will come anyway. What you get isthe entire distribution by estimating its NP support points. Eachsupport points consists of a point estimate for each model parameter(vol, clearance, rate constants) and an estimate of the probabilityassociated with each set of estimates. So now you have many models(support points) rather than just one, as with parametric approaches.You also now have a tool with which to estimate the errors associatedwith future predictions of serum concentrations, for example, thatresult from a dosage regimen.           How are you going to develop an initial dosage regimen whenyou can see that some patients are fast metabolizers and others areslow, and others you don't even know about yet? How do you controlMOST PRECISELY a patient when all you now about him/her is that s/heis a member of such a population? How do you CONTROL a patient mostprecisely who at this stage is represented only by a vaguely known"kinda - sorta" multimodal parameter distribution?           Using the NP joint parameter distribution, it is easy toget multiple predictions of future serum concentrations and otherresponses, one from each model support point, weighted by theprobability of that support point. These weighted predictions can becompared with the desired target goal to be achieved at the desiredtime. You can compute the weighted squared error of the failure ofthat regimen to hit the target. You can then find the regimen whichspecifically minimizes that error. This is the most precise dosageregimen you can develop to hit any target goal. It is the optimal wayto use all the information based on the data you have obtained up tonow. This called the Multiple Model (MM) approach. This approach hasbeen and is widely used in the aerospace community for flight controland spacecraft guidance systems, and for tracking (and hitting)hostile targets taking evasive action. Our lab uses it for developingdrug dosage regimens. See www.lapk.org, and click on new developments,teaching topics, and software.           Now we are getting there. We have a joint parameterdistribution in the form of these NP distributions. We give thepatient a dosage regimen and do TDM, monitoring the serumconcentrations at various times, to obtain a Bayesian posteriorparameter distribution for that individual patient. We are now gettingat the key point of the discussion. Now, instead of using the maximuma posteriori probability (MAP) Bayesian approach, we simply start withthe prior probability of each support in the population model, examinethe data, and compute the Bayesian posterior probability of eachsupport point given the data. Most support points do not fit the datawell at all. Their posterior probability becomes very small ornegligible. The few that do predict the patient's data well becomemuch more likely. You now have a few, (or maybe only one), posteriorsupport points. These points constitute the Bayesian posterior jointNP parameter distribution. You can see graphically the much narrower,more precise, bands of prediction of the patient's data. You can seegraphically what you have learned about the patient by the TDM youhave done. You will also get the patient's Bayesian posteriorparameter means, medians, modes, and covariances as well.           You can now use this patient's entire Bayesian posteriorparameter distribution to develop the same kind of MM dosage regimenwe described before,  but now using this much more precise posteriormodel to get the most precise hit on the desired target goal.           The NP approaches do not yet give you rigorous confidencebands, but they do give you easily seen 95 percentile distancesconcerning the estimates of the past and also of the expected futureconcentrations. There is no need to use parametric approaches when youdon't even know what the shape of the parameter distribution is. And,as you say, the parametric approached, give only asymptotic confidenceestimates.           Our clinical software does the job I think you areinterested in. Go to www.lapk.org. Click on teaching topics, on newdevelopments, and on software. You can download demo versions of thisMM-USCPACK software for population modeling and for clinicallypractical approached to dosage optimizations for patients. I hope youlike it.Very best regards,Roger Jelliffe`
Back to the Top

• On 12 May 2009 at 09:03:03, "andreas lindauer" (lindauer.-at-.uni-bonn.de) sent the message
`The following message was posted to: PharmPKRoger,Thank you very much for your extensive comment on this. You are right,someidea of parameter uncertainty is what I would like to have. I think in aparametric Bayesian estimation scenario one would like to have not onlypoint estimates of the most probable PK-parameters, but also some ideaifthe estimate is precise enough to give an adequate dose recommendation.The NP approach really has some advantages over parametric methods. Youexplained them very convincingly in your last mail. Generally, I ammuch infavour of the NP approach for therapeutic drug monitoring, however,there isone major drawback. You mentioned it in your second paragraph:   "First, use an NP approach to make a population PK/PD model."Suppose the pharmacy department of a hospital wants to implement TDMfor anew drug (not one already included in the USC*PACK ;-) ). In order tousethe NP method for TDM they first had to perform a Population PK-studyto geta sufficient number of support points. For the parametric approach,however,the necessary information may be already published in a PopPK-Analysisofthis drug using e.g. NONMEM.I think as long as 99% of PopPK-Studies are done with parametricmethods theuse of the NP method for TDM will be limited, despite its obviousadvantages.Very best regards, Andreas.`
Back to the Top

• On 12 May 2009 at 16:11:54, "J.H.Proost" (J.H.Proost.-at-.rug.nl) sent the message
`The following message was posted to: PharmPKDear Andreas,You asked: > 1) Is it (mathematically) correct to calculate SEs for the parameter > estimates like this: > >     M=number of parameters >     n=number of observations >     MSE=SSE/(n+M-M)  # Mean squared error > >     SE = sqrt(MSE.*diagonal(inverse(hessian)))  From numerous Monte Carlo simulations I concluded that SEs can becalculated accurately from the latter equation, if MSE is calculatedfrom:MSE = SSE/(n+M) > 2) However, 1) gives only asymptotic SEs which are known to have >some  disadvantages. Therefore in population modeling non-parametric > resampling techniques (bootstrap, jack-knife) are increasingly > applied. Such methods, however, wouldn't make much sense for single- >subject data with only few observations.Asymptotic SEs have indeed some disadvantages, but they still may beuseful and sufficiently precise to estimate confidence intervals.You may find some useful information in my paper:Proost JH, Eleveld DJ. Performance of an Iterative Two-Stage Bayesiantechnique for population pharmacokinetic analysis of rich data sets.Pharm Res 2006; 23: 2748-2759 (Erratum in Pharm Res 2007; 24: 1599).Please see the Erratum for the correct version of Eq. 6. I can sendyou the pdfs. > Are there other methods that could be used for such situations in > order to calculate confidence intervals? Maybe >log-likelihood-profiling?This is indeed a good alternative approach. Fix all parameters exceptfor one, and find the parameter values where the objective function-2*LogLikelihood is increased by the critical value of the Fdistribution with df1 = 1 and df2 = infinity (or better: the dfassociated with the prior distribution). For a 95% confidenceinterval, use alpha = 0.025 (critical value 3.84 (equal to 1.96^2)). > Or, can inference statistics be applied on the posterior >distribution  of a parameter?Yes, but this requires that the posterior distribution is known, andthat was the problem. > 3) How can a confidence band around the predicted curve be >constructed?See the abovementioned paper (Eq. 15).best regards,Johannes H. ProostDept. of Pharmacokinetics and Drug DeliveryUniversity Centre for PharmacyAntonius Deusinglaan 19713 AV Groningen, The NetherlandsEmail: j.h.proost.-at-.rug.nl`
Back to the Top

• On 12 May 2009 at 11:14:58, "G. Scott Lett, Ph.D." (slett.aaa.bioanalyticsgroup.com) sent the message
`The following message was posted to: PharmPKI've been following this discussion with interest.It seems clear that posterior parameter estimates (mean, variance,covariance) are possible, even straightforward.   Having come from theaerospace industry, I've been surprised that methods such as Kalmanfilters(and their extensions) don't seem to be very commonplace in PK.Regarding nonparametric methods discussed here; I love theirgenerality butit seems that there are two barriers to using them for everyday work:1.   Don't the nonparametric estimate require much more data? That is,   If the parametric assumptions (approximately) hold, parametricmethods   require much less data.2.  If the methods are not well known or accepted, then it may be achallenge to get your work technically reviewed, published, oraccepted bydecision-makers (management, regulators).Can you folks please comment these? I'd be interested in the currentthinking.Thanks,ScottG. Scott Lett, Ph.D.The BioAnalytics Group241 Forsgate Dr.Suite 209Jamesburg, NJ 08831`
Back to the Top

• On 12 May 2009 at 18:25:48, "Roger Jelliffe" (jelliffe.aaa.usc.edu) sent the message
`Dear Andreas:           Thanks for your note and your comments. Yes. It is not onlya problem of knowing the errors in the Bayesian posterior parameterestimates, it is also, as you say, the ability to predict, based onthe Bayesian posterior joint parameter density, the precision withwhich a dosage regimen is likely to hit the target, and then,specifically, to maximize that precision. That is what our approachesdo.           For getting population parameter estimates from theliterature, you can do that, and use a Monte Carlo simulator togenerate fictive patient data files. You can also state the errorswith which doses are prepared and given, the errors in recording theirtiming, the assay errors, and the timing errors of getting thesamples. From the resulting data files, you can then make an NPAGpopulation model, and then put this model in the clinical software touse it. Michael Neely has done this for several HIV drugs.           I am at a loss to understand why so many people continue touse pop methods that are only approximate, not statisticallyconsistent, and which have no way to estimate the precision with whicha regimen will hit a target.           Anyone who wishes to can go to our web site www.lapk.org,click on software, and download our demonstration software to evaluateit both for its ability to make a NP pop model, but also to plan,monitor, and adjust maximally precise dosage regimens for patient care.Very best regards,Roger Jelliffe`
Back to the Top

• On 12 May 2009 at 20:27:37, "Roger Jelliffe" (jelliffe.-a-.usc.edu) sent the message
`Dear Scott:           Thanks for your very good comments. I am also surprisedthat Kalman filtering is not more widely understood in the PKcommunity. We use an extended Kalman filter in our clinical softwarefor multiple model (MM) Bayesian adaptive control of drug dosageregimens. It works very well where appropriate. We really do optimalopen-loop feedback stochastic adaptive control.           As to the nonparametric (NP) methods, there are no realbarriers.1.                Most modelers, though, really do not care aboutmath. They are accustomed to using structural models in a certainsoftware package. They almost never ask what such a package reallydoes for them. Some sort of results are usually enough. How theresults are obtained, they don't really care. If it gets means withinabout 2%, that is fine. Variances and correlations are usually notconsidered as to their reliability. Go to almost any modelingworkshop. This is usually the case.2.                The NP methods do not require more data. Indeed theywork better than parametric methods, especially in data poor settings.However, you are quite correct, that no methods do well with sparse oruninformative data. You can see the results degrade as you withholddata points from an analysis. However, you never see the actualdistributions with parametric methods - only the means, SD's, andcovariances. With the NP methods it is much easier, since you areestimating the entire distribution, you can see the plots of theestimated densities degrade as the data, and the experimental design,become less informative. You never get away from the issues of properexperimental design. You cannot make a silk purse from a sow's ear. Iguess with many policies for therapeutic drug monitoring (TDM) as theyare today, you only put lipstick on a pig. Also, with the NP approach,there are NO outliers, as there is no assumed shape of thedistribution. This is very useful.3.                Yes. It has been quite difficult to get our workaccepted by conventional statisticians who review many papers. It hasbeen extremely hard to put across the difference between Bayesianforecasting (which usually does not specify a target, but shows youwhat the regimen predicts), and really optimized Bayesian stochasticadaptive CONTROL of a system, where a specific target is selected, andthe regimen to hit it most precisely is developed.4.                We have talked with the FDA in many settings, butthe usual response there and almost everywhere else, has beensomething like"Yes, OK,  ............"   and then inertia, except in one instance.Very disappointing. Clinical Pharmacology is dead. It used to be areal clinical specialty. Then came PK and quantitative approaches, Thephysicians of the time couldn't understand them. Now they are dead orretired, and the specialty has become one oriented 99.44% about thedrug industry. Good clinical pharmacists do what they can. A fewphysicians do what they can. But the practical, good and precise useof drugs is hardly thought of, and is not taught effectively in anymedical school I have seen, except 1 or 2. NOT here, for example. Lookat the discussion we are having now. And you are the only one withoutparametric statistical blinders! Good for you!5.                The PK/PD community needs to see that their work isnot done in cultural isolation just for the drug companies. Models ofpotentially toxic drugs need to be used in settings of optimal controlof PK/PD systems for maximally precise patient care. It is NOT enoughto go to meetings and talk to each other about what a great thing popmodeling is. It is NOT enough to look up the literature about howdrugs age dosed today. We need real SOFTWARE TOOLS to do the job ofBayesian adaptive control optimally. We need to reach the physicianswho strongly resist any applications to therapy which are more thansimple memorized rituals that some committee who never saw thatphysician's patient has issued some guidelines for. Nobody knows his/her patient better than the clinician at the bedside. Clinicians needto be trained to set their own target goals for each patient accordingto THAT patient's perceived need, to develop regimens which hit thosetargets with minimum error, and to take the responsibility for whatthey do. They need to take the responsibility as the patient's bestadvocate, and not evade it by following some guideline set up bypeople who never saw that patient, or knew the infecting bug, forexample.6.                Look at the way the aerospace industry works. Lookwhat they do! Air travel is so safe now that it would be mostdifficult to make ANY "statistically significant" reduction in thenumber of air crashes.  But Rolls-Royce, I hear, monitor all theirengines in flight by satellite. If there is any trouble over theAtlantic, for example, the pilots and the controllers hear it rightNOW, and can discuss it and what to do about it. This is clearly NOT"cost-effective" in the medical culture.7.                But did you know that an episode of grade 3-4 graftversus host disease in a child with a bone marrow transplant costs anextra 100,000 euros to treat, and costs only 30,000 euros extra tomonitor the child and avoid it? Nathalie Bleyzac writes poorly inEnglish as she is French, in Lyon, and so her work doesn't getpublished well. But this is her work.  Sander Vinks and his groupshowed in The Hague that model-based, goal-oriented TDM also improvedcare and reduced hospital stay by 6 days. I think it is AthanniosIliadis in Marseille who now has patients with testicular CA treatedwith cisplatin, using Bayesian adaptive control, some of whom have 15year follow-ups! When complications are reduced by good Bayesianadaptive control, all the rest that goes with it also gets reduced.The medical community today would NEVER spend the extra money tomonitor by satellite, but I sure feel better flying knowing that theengines are looked after like that!8.                The way things are now is that the pharmacy and thelaboratory see only the extra costs of monitoring and adjusting thetherapy with toxic drugs. They do NOT see the results of what they do.I think the hospital administrators should see this, but what do theyknow yet? They sure don't read this chatbox.9.                Further, the medical schools never teach this way touse drugs. They give lip service to it by teaching baby PK with linearregression on logs of levels, but never anything clinically useful, inany medical school I have visited worldwide. Especially anythingBayesian is too hard" and "too much work for what it is worth". Thesewere the student comments I got when I was teaching such an electivecourse for 3 and 4 year students. That is what my students said to thecurriculum committee. I have seen many student applicants to medschool who had good math backgrounds and who had the ability tounderstand these approaches. When I would see them again in 3rd year,all that ability was gone. It had been brainwashed out of them by themedical culture of intuitive judgment and memorization. It wasdropped. No one taches decision theory in med school either. What atragedy! At least you can go to Tufts and get a fellowship in it. Goodfor Tufts! Most of the good work is done by thoughtful pharmacists,but most do not have the clinical training to really use theirclinical judgment, in addition to the analyses, to responsibly selecta target goal. Some do, most don't. Hardly any physician does, that'sfor sure. This is a terrible indictment of the medical, and also someof the pharmaceutical communities, as well as the drug industry, whenit comes to suggesting dosage regimens of toxic drugs. Marketing adrug needing TDM is currently the kiss of death.10.           Again, go to our web site www.lapk.org. Click around onteaching topics, new advances, and software. Let's talk more.All the best, and many thanks  again!Roger Jelliffe`
Back to the Top

 Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "Standard errors of Bayesian parameter estimates" as the subject Support PharmPK by using thelink to buy books etc.from Amazon.com

Copyright 1995-2011 David W. A. Bourne (david@boomer.org)