Back to the Top
I was cleaning out some old messages and looked at this one again. Can
you point to some literature that discusses the robustness of the
Simplex algorithm?
Thanks
Craig Hendrix
Johns Hopkins
Back to the Top
[Two replies - db]
X-Sender: st005899.-a-.brandywine.otago.ac.nz
Mime-Version: 1.0
Date: Tue, 25 Aug 1998 11:32:20 +1200
To: PharmPK.aaa.pharm.cpb.uokhsc.edu
From: Robert Purves
Subject: Re: PharmPK Robustness of the Simplex method
I do not know of any published comparisons. The following extracts from the
instruction manual for Minim are relevant. (Minim is a non-linear least
squares program for the Macintosh family of computers). The name "Simplex"
is apt to be misleading, since there is an unrelated numerical programming
technique with that name, and I prefer "Nelder-Mead" or "polytope method".
---------------------------
The Nelder-Mead (polytope) method is an ingeniously formulated series of
trials-and-errors. Derivatives are not used and no assumption is made
about the form of the objective function. The method is therefore immune
to most of the pathological conditions affecting Gauss-Newton-Marquardt and
Gauss-Newton-SVD. This property makes it useful for improving bad starting
values prior to using a Gauss-Newton method, and for confirming the outcome
of a Gauss-Newton fit. It can be used for the entire fit, but is
distressingly slow to converge if there are more than 3 or 4 parameters.
The Gauss-Newton methods are usually much faster for least-squares fitting,
because they "know" much more about the function to be fitted, whereas
Nelder-Mead knows only the values of the objective function.
Like many others, I have tinkered with the algorithm but have not found a
way to overcome its main vice: a tendency to premature convergence while
still far from the solution. This unfortunate behaviour occurs after a
series of early successes have expanded the polytope along some parameter
axes but not others. A drastic change in shape of the polytope is then
required when the expansion reaches a valley floor. In the course of
rearrangement the hitherto unexpanded parameters do not always expand
quickly enough for the algorithm to notice that progress can be made down
the valley. The only ways around this problem are to give good starting
guesses and to specify a sufficiently small value of e [the convergence
criterion]
----------------------------
Robert D Purves
Department of Pharmacology
University of Otago
PO Box 913, Dunedin
New Zealand
---
X-Sender: jelliffe.at.hsc.usc.edu
Date: Mon, 24 Aug 1998 23:12:17 -0700
To: PharmPK.-at-.pharm.cpb.uokhsc.edu
From: Roger Jelliffe
Subject: Re: Robustness of the Simplex method
Mime-Version: 1.0
Dear Craig:
I am not sure what you mean by robust. However, the Nelder-Mead simplex
algorithm seems never to blow up. It always gets at least a local minimum.
There are no derivatives to calculate. It is slow, dumb, and plodding, but
it gets there, and is safe, I believe, to use by someone not well versed in
fitting. This is why we have chosen it in our USC*PACK programs, for
example, for about 20 years. It is well discussed in Numerical Recipes,
1986, Cambridge University Press, page 276, and pp. 289-293.
Best regards,
Roger Jelliffe
************************************************
Roger W. Jelliffe, M.D.
USC Lab of Applied Pharmacokinetics
CSC 134-B, 2250 Alcazar St, Los Angeles CA 90033
**Note our new area codes below, since 6/15/98!**
Phone **(NOTE NEW AREA CODE AND PREFIX)** (323)442-1300, Fax (323)442-1302
email=jelliffe.-at-.hsc.usc.edu
************************************************
You might also look at our Web page for announcements of
new software and upcoming workshops and events. It is
http://www.usc.edu/hsc/lab_apk/
************************************************
PharmPK Discussion List Archive Index page
Copyright 1995-2010 David W. A. Bourne (david@boomer.org)