1. Home
  2. FDRBONFERRONI procedure


Estimates false discovery rates by a Bonferroni-type procedure (A.I. Glaser).


PRINT = string token Controls printed output (pi0); default pi0
METHOD = string token Controls the method used for calculating π0 (smoother, bootstrap); default smoo
LOGP = string token Whether to take logs of π0 when METHOD=smoother (yes, no); default no
DF = scalar Degrees of freedom for smoothing spline; default 3
PLOT = string token Controls plots (phistogram, qhistogram, pi0vslambda, qvsp, tests, expfalsepositive, inference, loginference); default phis, qhis, pi0v, qvsp, test, expf, infe, logi
WINDOW = scalar Window for the graphs; default 1
KEYWINDOW = scalar Window for the key (zero for none); default 2


PROBABILITIES = variates Significance values, must lie between 0 and 1
LAMBDA = scalars or variates Values of tuning parameter λ, equivalent to significance levels at which to test the PROBABILITIES; default !(0, 0.05....0.9)
FDR = variates Saves the False Discovery Rates (i.e. q-values) at the sorted p-values in PROBABILITIES
FRR = variates Saves the False Rejection Rates at the sorted p-values in PROBABILITIES
POWER = variates Saves the power estimates as a function of the sorted p-values in PROBABILITIES
PI0 = scalars Saves the value of π0, i.e. the maximum value of the FDR
LOWER = scalars Lower bound of q-values to use with PLOT settings qvsp, tests and expfalsepositive; default 0
UPPER = scalar Upper bound of q-values to to use with PLOT settings qvsp, tests and expfalsepositive; default 1, which indicates maximum q-value


When testing m multiple hypotheses there are various outcomes that can occur, summarized in the table below

Decision on null hypothesis: Accept Reject Total
Situation: null true U V m0
alternative true T S m1
Total W R m

where R is the total number of rejected hypotheses and W = mR. The proportion of tests that are truly null, π0, is m0 divided by m. The false discovery rate (FDR), also known as the q-value of a test, is a commonly used error measure in multiple-hypotheses, defined as

FDR = E(V/R | R > 0) × Pr(R > 0)

i.e. the expected proportion of false positives findings among all the rejected hypotheses multiplied by the probability of making at least one rejection; the FDR is zero when R = 0. Similarly the false rejection rate (FRR) is defined as

FDR = E(T/W | W > 0) × Pr(W > 0),

i.e. the expected proportion of false negatives findings among all the accepted hypotheses times the probability of accepting at least one test. We also define the power to be equal to E(S/m1 | m1 > 0) × Pr(m1 > 0).

The p-values from the multiple hypotheses are supplied, in a variate, using the PROBABILITIES parameter. The analysis uses a Bonferroni-type multiple-testing procedure to calculate the corresponding q-values. The p-values are assumed independent, or may be weakly dependent if there are many of them. The parameter π0 is calculated using the method of Storey (2002). This involves a tuning parameter λ, which can be set using the LAMBDA parameter; the default is a variate containing the numbers 0, 0.05, … 0.9. Λ can be thought of as the value beyond which the individual p-values are considered null. As λ gets larger the bias of π0 gets smaller, but its variance increases. If you set LAMBDA to a scalar, π0 is estimated by dividing the number of null tests (i.e. the number of p-values greater than λ) by the expected number of null tests m × (1 – Λ). If you set LAMBDA to a variate with several values, two methods are available, selected by the following settings of the METHOD option:

    smoother fits a smoothing spline of λ onto initial estimates of π0 calculated as for a single λ value, and takes the estimate of π0 as the value corresponding to the largest value of λ;
    bootstrap estimates π0 by bootstrap sampling from the variate of p-values.

The default is smoother, as the bootstrap method may be time-consuming when there are many p-values. The number of degrees of freedom to use in the smoothing is specified by the DF option (default 3). Also, you can set option LOGP=yes to do the smoothing on log-transformed π0 values.

The PRINT option controls printed output, using the settings:

    pi0 prints π0, the estimate of the expected proportion of null p-values corresponding to the largest q-value.

By default PRINT=pi0.

Various graphs can be selected by the following settings of the PLOT option:

    phistogram histogram of p-values;
    qhistogram histogram of q-values (i.e. the FDR values);
    pi0vslambda λ against π0 (when only one value of λ is specified the default values of LAMBDA are used);
    qvsp q-values against p-values;
    tests plot of the sorted tests against q-values;
    expfalsepositive plot of the number of expected false positives against the sorted tests;
    inference plot of the FDR, FRR and power statistics against the sorted p-values; and
    loginference plots the FDR, FRR and power statistics on log scales, against the sorted p-values restricted to p<0.5, with a background grid to enable estimates to be read for specific probability values. Due to the small numbers used in this plot the p-values and FDR, FRR & power statistics are displayed as percentages.

By default all the plots are produced. The WINDOW option specifies the window for the graphs, and the KEYWINDOW option species the window for keys.




FDRBONFERRONI uses the method of Storey (2002), with the definitions of FRR and power given in Genovese & Wasserman (2002).

Action with RESTRICT

The PROBABILITIES parameter can be restricted. All output estimates will then be based only on those unrestricted units.


Storey, J.D. (2002). A direct approach to false discovery rates. Journal of the Royal Statistical Society Series B, 64, 479-498.

Genovese, C. & Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. Journal of the Royal Statistical Society Series B, 64, 499-518.

See also


Commands for: Microarray data.


SCALAR        ntests,seed,delta,phi,sampsize; VALUE=\
" Create ntest data sets (rows), each of size sampsize (columns)."
GRANDOM       [NVALUES=1; SEED=seed] dum
POINTER       [NVALUES=sampsize] x
CALCULATE     x[] = GRNORMAL(ntests; 0; 1)
" Add an offset, delta, to a fraction, (1-phi), of the samples.
  Derive test statistic, z, for Test Ho: mean=0 against Ha:mean0
  and significance, p."
CALCULATE     nHa = ROUND(ntests * (1-phi))
&             nHo = ntests - nHa
VARIATE       [VALUES=#nHo(0),#nHa(delta)] vdelta
CALCULATE     xbar = vmean(x) + vdelta
&             t = ABS(xbar * SQRT(sampsize))
&             p = 2 * CUNORMAL(t; 0; 1)
" Derive FDR "
FDRBONFERRONI p; FDR=qvals; LAMBDA=!(0,0.01...0.99); UPPER=0.4
Updated on March 8, 2019

Was this article helpful?