- On 24 Aug 1998 at 11:26:14, "Craig Hendrix. M.D." (chendrix.-at-.erols.com) sent the message

Back to the Top

I was cleaning out some old messages and looked at this one again. Can

you point to some literature that discusses the robustness of the

Simplex algorithm?

Thanks

Craig Hendrix

Johns Hopkins - On 25 Aug 1998 at 10:45:12, David_Bourne (david.-a-.pharm.cpb.uokhsc.edu) sent the message

Back to the Top

[Two replies - db]

X-Sender: st005899.-a-.brandywine.otago.ac.nz

Mime-Version: 1.0

Date: Tue, 25 Aug 1998 11:32:20 +1200

To: PharmPK.aaa.pharm.cpb.uokhsc.edu

From: Robert Purves

Subject: Re: PharmPK Robustness of the Simplex method

I do not know of any published comparisons. The following extracts from the

instruction manual for Minim are relevant. (Minim is a non-linear least

squares program for the Macintosh family of computers). The name "Simplex"

is apt to be misleading, since there is an unrelated numerical programming

technique with that name, and I prefer "Nelder-Mead" or "polytope method".

---------------------------

The Nelder-Mead (polytope) method is an ingeniously formulated series of

trials-and-errors. Derivatives are not used and no assumption is made

about the form of the objective function. The method is therefore immune

to most of the pathological conditions affecting Gauss-Newton-Marquardt and

Gauss-Newton-SVD. This property makes it useful for improving bad starting

values prior to using a Gauss-Newton method, and for confirming the outcome

of a Gauss-Newton fit. It can be used for the entire fit, but is

distressingly slow to converge if there are more than 3 or 4 parameters.

The Gauss-Newton methods are usually much faster for least-squares fitting,

because they "know" much more about the function to be fitted, whereas

Nelder-Mead knows only the values of the objective function.

Like many others, I have tinkered with the algorithm but have not found a

way to overcome its main vice: a tendency to premature convergence while

still far from the solution. This unfortunate behaviour occurs after a

series of early successes have expanded the polytope along some parameter

axes but not others. A drastic change in shape of the polytope is then

required when the expansion reaches a valley floor. In the course of

rearrangement the hitherto unexpanded parameters do not always expand

quickly enough for the algorithm to notice that progress can be made down

the valley. The only ways around this problem are to give good starting

guesses and to specify a sufficiently small value of e [the convergence

criterion]

----------------------------

Robert D Purves

Department of Pharmacology

University of Otago

PO Box 913, Dunedin

New Zealand

---

X-Sender: jelliffe.at.hsc.usc.edu

Date: Mon, 24 Aug 1998 23:12:17 -0700

To: PharmPK.-at-.pharm.cpb.uokhsc.edu

From: Roger Jelliffe

Subject: Re: Robustness of the Simplex method

Mime-Version: 1.0

Dear Craig:

I am not sure what you mean by robust. However, the Nelder-Mead simplex

algorithm seems never to blow up. It always gets at least a local minimum.

There are no derivatives to calculate. It is slow, dumb, and plodding, but

it gets there, and is safe, I believe, to use by someone not well versed in

fitting. This is why we have chosen it in our USC*PACK programs, for

example, for about 20 years. It is well discussed in Numerical Recipes,

1986, Cambridge University Press, page 276, and pp. 289-293.

Best regards,

Roger Jelliffe

************************************************

Roger W. Jelliffe, M.D.

USC Lab of Applied Pharmacokinetics

CSC 134-B, 2250 Alcazar St, Los Angeles CA 90033

**Note our new area codes below, since 6/15/98!**

Phone **(NOTE NEW AREA CODE AND PREFIX)** (323)442-1300, Fax (323)442-1302

email=jelliffe.-at-.hsc.usc.edu

************************************************

You might also look at our Web page for announcements of

new software and upcoming workshops and events. It is

http://www.usc.edu/hsc/lab_apk/

************************************************

Want to post a follow-up message on this topic? If this link does not work with your browser send a follow-up message to PharmPK@boomer.org with "Robustness of the Simplex method" as the subject

PharmPK Discussion List Archive Index page

Copyright 1995-2010 David W. A. Bourne (david@boomer.org)