Back to the Top
Dear all
A Bioequivalence question ...................
Our guidance says that if Confidence interval is with in 80-125 %
then products are bioequivalent. Thus our conclusions are entirely
based on confidence interval.
But there can be cases where CI falls within prespecified limits
though ANOVA shows highly significant difference between formulations
( p = .00n). I have come across such cases many a
times ............... I
How would you conclude your results in such situations ? How
would you justify your decision based on CI ?
Regards
Sulagna
Associate Scientist ( Biostatistics)
Back to the Top
The following message was posted to: PharmPK
Dear Sulagna,
I'm extremely interested of a practical case where, as you say,
" ...CI falls within prespecified limits
though ANOVA shows highly significant difference between formulations
( p = .00n). I have come across such cases many a
times ............... "
Best regards
Stefano Porzio
Pharmacokinetic and Tox. Dept.
Inpharzam Ricerche SA - ZAMBON-GROUP
Taverne - Switzerland
Back to the Top
The following message was posted to: PharmPK
I think you just answered your question. BE are based on CI limits
80-125% and not on the F test.
Regards,
Dominique Paccaly
Syngenor Pharma Consulting
Back to the Top
Hello Sulagna, interesting question.
The bioequivalence limits were selected from a regulatory standpoint
rather than a statistical one. They are a criteria rather than a
statistical test. Thus two products need not demonstrate to be
similar in an ANOVA, only pass the bioequivalence limits. The
conclusions are based on the bioequivalence limits, not the p-values
of the ANOVA. Thus your conclusions would state that the 90%
confidence intervals of the parameters of interest fell within the
bioequivalence limits as defined by your regulatory body.
One corollary to what your saying is that you may have enough power
to determine a difference between a Cmax of 100 ug/mL (Test) and 102
ug/mL (Reference), but the question remains, is the difference
physiologically significant? Small differences between formulations
(p<0.05) may not be important. In addition, there may be considerable
variability between formulations precluding the ability to determine
a significant statistical difference, yet the ratios may well fall
outside the confidence limits. Large differences between formulations
(p>0.05) may be important.
If you have seen many studies then you know that there are
considerable intra- and inter-individual differences with the same
drug, and you have to draw lines somewhere. A problem with using the
confidence interval approach is that if you were a drug manufacturer
with infinite resources, as long as your Test/Reference ratios are
within 80-125, you can find enough subjects to get your drug to pass,
as the more subjects you have, the smaller the confidence intervals
will become. However, we reconcile this by the fact that it is not
terribly economically feasible to have football stadium sized Phase I
BE studies.
I found one article (Rani S and Pargal A., Indian J Pharmacol (2004);
36(4):209-216) which answers your question better than I could:
"In ANOVA, the ratio of the formulations' mean and sum of squares to
the error mean sum of squares gives an F-statistic to test the null
hypothesis Ho: uT=uR. This provides a test of whether the mean amount
of drug absorbed from the test formulation is identical to the mean
abount of drug absorbed from the reference. The test of this simple
null hypothesis of identity is of little interest in bioequivalence
studies, since the answer is always negative. This is because we
cannot expect the mean amounts of drug absorbed from two different
formulations or two different batches of the same formulation to be
identical. They may be very nearly equal, but not identical. Also, if
the trial is run under tightly controlled conditions (resulting in a
small error mean sum of squares in the analysis) and if the number of
subjects is large enough, no matter how small the difference between
the formulations, it will be detected as significant.
Thus the detection of the difference (which as indicated above,
will always exist) becomes simply a function of sample size, and
since the probable magnitude of the difference is the critical
factor, this gives rise to two anamalies:
1. A large difference between two formulations which is
nevertheless not statistically significant if error variability is
high and/or sample size not large enough.
2. A small difference, probably of no therapeutic importance
whatsoever, that is shown to be statistically significant if error
variability is minimal and/or sample size adequately large.
The first case suggests a lack of sensitivity in the analysis,
and the second an excess of it. Consequently, any practice that
increases the variability of the study (sloppy designs, assay
variability and within formulation variability) would reduce the
chances of finding a significant difference and hence improve the
cances of concluding bioequivalence."
David Dubins, B.A.Sc., Ph.D.
Pharmacokinetic Scientist
Back to the Top
Sulagna: we are asked to use Ln transformed data: it is my
understanding that the confidence intervals approach is an effort to
quantitative the magnitude of the difference between the LS means
ratio (test vs reference) of the exposure parameter of interest.
Therefore this is the criterion fro BE with the ranges quoted
(0.8-1.25, after back transformation) that is to be followed not the
p value of the F test from the ANOVA.
Hope above helps,
Angus McLean, Ph.D.
8125 Langport Terrace,
Suite 100,
Gaithersburg,
MD 20877
Tel 301-869-1009
fax 301-869-5737
BioPharm Global Inc.
Back to the Top
Dear Stefano
I might have overstated by saying that " ANOVA shows highly significant
difference between formulations " . Actually p values for significance
were slightly less than p=.05 at times even though CI was well within
0.80-1.25 .
I think, if p value (for formulations only) is highly
significant
then CI is bound to fall outside limits. This is because we take the
ratio
of formulations during calculation of CI. However this will not be
true for
other effects like period and sequence. Inspite of being highly
significant CI can lie within 80-125%.
Period and Sequence effects only add to the total residual
variability and give us an explaination as to what contributed to the
total
error.
Regards
Sulagna
Associate Scientist ( Biostatistics)
Back to the Top
The following message was posted to: PharmPK
Dear Sulagna,
You wrote:
> I think, if p value (for formulations only) is highly significant
> then CI is bound to fall outside limits. This is because we take
> the ratio of formulations during calculation of CI.
This is not correct. The p-value and CI have a completely different
background and a completely different meaning.
The p-value refers to the test of the null hypothesis that both
formulations
are identical, i.e that their ratio is 1. If p<0.05, this null
hypothesis is
rejected, and we conclude that both formulations are significantly
different. If the null hypothesis cannot be rejected, one cannot make a
conclusion. Note that one should not conclude that the formulations are
bioequivalent: this would be a pertinently wrong conclusion.
The confidence interval is used as a statistic to test the null
hypothesis
that the formulations are bioinequivalent, i.e. that their true ratio is
outside the interval 0.8 to 1.25 (either below 0.8, or above 1.25;
hence the
'two one-sided tests', hence the 90% confidence interval). If the entire
confidence interval is within the interval 0.8 to 1.25, the null
hypothesis
is rejected, and the formulations are considered bioequivalent. If
the null
hypothesis cannot be rejected, one cannot make a conclusion. Actually
one
should not conclude that the formulations are bioequivalent (e.g. a
larger
samples might have resulted in a confidence interval within the
interval 0.8
to 1.25). It is better to say that the formulation failed the test on
bioequivalence.
Please note that the first test is not relevant in bioequivalence
testing.
It is really not important whether or not two formulations are
significantly
different or not. The second test gives what we want to know: that a
product
passing the test will have a bioavailability within 0.8 and 1.25 of
that of
the reference product.
Period and sequence effects does not necessarily play a role here.
Best regards,
Hans Proost
Johannes H. Proost
Dept. of Pharmacokinetics and Drug Delivery
University Centre for Pharmacy
Antonius Deusinglaan 1
9713 AV Groningen, The Netherlands
tel. 31-50 363 3292
fax 31-50 363 3247
Email: j.h.proost.at.rug.nl
Back to the Top
The following message was posted to: PharmPK
The usual test of the null hypothesis is Ho: B=0, a p-value less than
0.05 would imply that you have rejected the null hypothesis at the
alpha-level of significance of 0.05.
The corresponding 95% CI would not include the value of zero in the
interval.
So, if you have a UMP test then you can construct a corresponding UMA
CI.
Therefore, its quite possible to have a significant treatment difference
p < 0.05 and still claim bioequivalence because the confidence interval
could be within .80 to 1.25, but not include one.
Note that, the two-one sided t-test for BE is computationally equivalent
to a 90% confidence interval.
mike
Back to the Top
Johannes: I agree with your previous message, but I would to add one
detail to a sentence in your message: please see addition in
parenthesis below.
The confidence interval is used as a statistic to test the null
hypothesis (p value set to 0.05) that the formulations are
bioinequivalent, i.e., that their true ratio is outside the interval
0.8 to 1.25 (either below 0.8, or above 1.25; hence the 'two one-
sided tests', hence the 90% confidence interval). If the entire
confidence interval is within the interval 0.8 to 1.25, the null
hypothesis.
In other words for the second null hypothesis the p value is set to
0.05.
Do you agree with this addition?
Angus McLean, Ph.D.
8125 Langport Terrace,
Suite 100,
Gaithersburg,
MD 20877
Tel 301-869-1009
fax 301-869-5737
BioPharm Global Inc.
Back to the Top
It is possible to obtain a 90% confidence interval (for the ratio of
geometric means) within the ranges like 85-95% or 105-120%. Under
these situations, we can claim the bioequivalence (since the 90% CI
falls entirely within the range of 80-125%), but the p-value will be
<0.05 (significant, ratio does not include 1).
This is why we need to rely on the confidence interval (not p-value)
to make the judgment in BE study.
Thanks,
CQ Deng, PhD
http://www.talecris.com
Back to the Top
What if your 90% CI ranges from for eg. say 75% - 130%? It includes
80%-125% and also includes mean ratio 1
Back to the Top
On 9/29/05, Hans Proostwrote:
> Dear Sulagna,
> You wrote:
> > I think, if p value (for formulations only) is highly significant
> > then CI is bound to fall outside limits. This is because we take
> > the ratio of formulations during calculation of CI.
> This is not correct. The p-value and CI have a completely different
> background and a completely different meaning.
Be careful with this one -- p-values and CI's are mirror images of
each other, for appropriately matched computations. Modulo regularity
conditions on the hypothesis test (which some common ones fail), any
hypothesis test (through its p-values) can be used to generate
confidence intervals, and any confidence interval can be used to
create a hypothesis test and corresponding p-value.
It does get tricky, though -- for instance, take the wilcoxon rank sum
(2-sample) and signed-rank tests. They don't qualify, because they
don't satisfy a transitivity property. It is possible to find real
data sets such that A > B (hypothesis of one being larger is
evidenced) and B > C, and C > A.
For tests like the t-test, it's easy to show that you can match
confidence intervals with p-values. When they don't match, then you
generally are using a better formula for one than the other (i.e. CI
for difference of 2 means under the assumption of non-equal variances,
vs. a T-test assuming equal variances, or similar mis-matches).
Now the interpretations are different (though related) and the
backgrounds/history are different, but if you look at them from a big
enough picture (and compute enough of them, in a sense -- i.e. for
different level sets, or different hypotheses "close" to the original,
you get the same results).
best,
-tony
blindglobe.-at-.gmail.com
Muttenz, Switzerland.
Back to the Top
The following message was posted to: PharmPK
Dear Tony,
On my comment to Sulagna:
> > This is not correct. The p-value and CI have a completely
different
> > background and a completely different meaning.
You wrote:
> Be careful with this one -- p-values and CI's are mirror images of
> each other, for appropriately matched computations. Modulo
regularity
> conditions on the hypothesis test (which some common ones fail), any
> hypothesis test (through its p-values) can be used to generate
> confidence intervals, and any confidence interval can be used to
> create a hypothesis test and corresponding p-value.
Thank you for pointing to my badly phrased statement. I fully agree with
your comments.
What I meant was: 'The statistical inference from a p-value obtained
from
ANOVA (i.e. aiming at the difference between the two formulations) is
completely different from the statistical inference from the 90%
confidence
interval in bioequivalence testing'. This difference is due to the
different
aim of the test, resulting in a different null hypothesis, as
explained in
my earlier message.
Two additional notes:
1) The 90% confidence interval is not appropriate to test the null
hypothesis that both formulations are equal with a p-value of 0.05;
to this
purpose a 95% confidence interval should be used.
2) In bioequivalence testing (i.e. null hypothesis: the formulations are
bioinequivalent, i.e. their true ratio is
outside the interval 0.8 to 1.25) the 90% confidence interval does not
restrict the type I error (the risk of falsely rejecting the null
hypothsis,
i.e. concluding bioequivalence whereas the true ratio is outside the
interval 0.8 and 1.25) to 0.05 in all cases; in some cases the true
value is
slightly higher. As far as I remember the 90% confidence interval was
adopted by FDA for practical reasons as a simple alternative to 'exact'
procedures.
Best regards,
Hans Proost
Johannes H. Proost
Dept. of Pharmacokinetics and Drug Delivery
University Centre for Pharmacy
Antonius Deusinglaan 1
9713 AV Groningen, The Netherlands
tel. 31-50 363 3292
fax 31-50 363 3247
Email: j.h.proost.-at-.rug.nl
Back to the Top
The following message was posted to: PharmPK
Dear Rajkumar,
You wrote:
> What if your 90% CI ranges from for eg. say 75% - 130%? It includes
> 80%-125% and also includes mean ratio 1
Null hypothesis of bioinequivalence cannot be rejected. The test
formulation
failed the test on bioequivalence.
Best regards,
Hans Proost
Johannes H. Proost
Dept. of Pharmacokinetics and Drug Delivery
University Centre for Pharmacy
Antonius Deusinglaan 1
9713 AV Groningen, The Netherlands
tel. 31-50 363 3292
fax 31-50 363 3247
Email: j.h.proost.aaa.rug.nl
Back to the Top
The following message was posted to: PharmPK
Dear Angus,
You added to my comment:
> (p value set to 0.05)
> In other words for the second null hypothesis the p value is set to
> 0.05.
> Do you agree with this addition?
You are right. I tacitly assumed a p-value of 0.05. To be even more
correctly, it is better to say either 'p < 0.05' or 'alpha = 0.05', to
discriminate between the cut-off value for the type I error (alpha =
0.05)
and the actual value (p-value, p < 0.05) of the probability of falsely
rejecting the null hypothesis.
Best regards,
Hans Proost
Johannes H. Proost
Dept. of Pharmacokinetics and Drug Delivery
University Centre for Pharmacy
Antonius Deusinglaan 1
9713 AV Groningen, The Netherlands
tel. 31-50 363 3292
fax 31-50 363 3247
Email: j.h.proost.at.rug.nl
PharmPK Discussion List Archive Index page
Copyright 1995-2010 David W. A. Bourne (david@boomer.org)