We make minimal assumptions about the data streams which may differ in distribution and dimension. In Section 3 we introduce such a procedure which we call the Sequential BH Procedure that controls FDR as well as its type II analog, the false nondiscovery rate (FNR, both defined below), under independence of data streams, and under arbitrary dependence with a small logarithmic inflation of the prescribed values these results mirror the conditions under which Benjamini and Hochberg (1995) proved FDR control in their original paper. Our goal is to introduce an FDR-controlling procedure with as much flexibility as the Benjamini-Hochberg (BH) procedure but tailored for sequential data, by allowing for accept/reject decisions in between sequential sampling of data streams. However, the particular needs of sequential data have largely been neglected in the FDR literature and most papers adopt Benjamini and Hochberg’s (1995) starting point, a set of p-values arising from fixed sample size hypothesis tests. Other areas of application involving multiple data streams include data from pharmacovigilance drug side effect databases (e.g., Avery et al., 2011), testing for disease clusters over a spatial area (e.g., Sonesson, 2007) and closely related problems of industrial quality control (see Woodall, 2006), and testing for a signal in a noisy image (e.g., Siegmund and Yakir, 2008).
Sequential testing ab testing trial#
One such area is in certain types of clinical trials, in particular the setting discussed by Berry and Berry (2004) in which treatments are compared on the basis of a long list of adverse events affecting the patients during the trial multiple endpoint clinical trials such as this are discussed in more detail in Sections 2 and 5.3. One characteristic of the data in some of the application areas mentioned above is that it arrives sequentially in time, or as a data stream. (2001), Efron and Tibshirani (2002), Genovese and Wasserman (2002), Storey (2002), Newton et al. Since Benjamini and Hochberg’s (1995) seminal paper introducing FDR and proving that Simes’ (1986) earlier step-up procedure controls FDR, the topic and related problems such as Empirical Bayes have been active areas of research see Efron et al. (2007, Section 1) give a variety of examples. These areas include high throughput gene and protein expression data, brain imaging, and astrophysics Müller et al. Multiple testing error metrics based on the false discovery proportion - such as its expectation, the false discovery rate (FDR) - are widely used in applications involving large or high dimensional data sets or when many comparisons are needed.
We prove the procedure’s error control and give some tips for implementation in commonly encountered testing situations. The procedure is a natural extension of Benjamini and Hochberg’s (1995) widely-used fixed sample size procedure to the domain of sequential data, with the added benefit of simultaneous FDR and FNR control that sequential sampling affords. The procedure can be used with sequential, group sequential, truncated, or other sampling schemes.
All that is needed is a test statistic for each data stream that controls its conventional type I and II error probabilities, and no information or assumptions are required about the joint distribution of the statistics or data streams. We propose a general and flexible procedure for testing multiple hypotheses about sequential (or streaming) data that simultaneously controls both the false discovery rate (FDR) and false nondiscovery rate (FNR) under minimal assumptions about the data streams which may differ in distribution, dimension, and be dependent.
0 kommentar(er)
