In 2004, Garcia-Berthou and Alcaraz published "Incongruence between test statistics and P values in medical papers," a critique of statistical errors that received a tremendous amount of attention. One of their observations was that the final reported digit of p-values in articles published in the journal Nature departed substantially from the uniform distribution that they suggested should be expected. In 2006, Jeng critiqued that critique, observing that the statistical analysis of those terminal digits had been based on comparing the actual distribution to a uniform continuous distribution, when digits obviously are discretely distributed. Jeng corrected the calculation and reported statistics that did not so clearly support the claim of a digit preference. However delightful it may be to read a critique of statistical errors in a critique of statistical errors, we nevertheless found several aspects of the whole exchange to be quite troubling, prompting our own meta-critique of the analysis. The previous discussion emphasized statistical significance testing. But there are various reasons to expect departure from the uniform distribution in terminal digits of p-values, so that simply rejecting the null hypothesis is not terribly informative. Much more importantly, Jeng found that the original p-value of 0.043 should have been 0.086, and suggested this represented an important difference because it was on the other side of 0.05. Among the most widely reiterated (though often ignored) tenets of modern quantitative research methods is that we should not treat statistical significance as a bright line test of whether we have observed a phenomenon. Moreover, it sends the wrong message about the role of statistics to suggest that a result should be dismissed because of limited statistical precision when it is so easy to gather more data. In response to these limitations, we gathered more data to improve the statistical precision, and analyzed the actual pattern of the departure from uniformity, not just its test statistics. We found variation in digit frequencies in the additional data and describe the distinctive pattern of these results. Furthermore, we found that the combined data diverge unambiguously from a uniform distribution. The explanation for this divergence seems unlikely to be that suggested by the previous authors: errors in calculations and transcription.

Background: Only four out of 31 completed randomized controlled trials (RCTs) of HIV prevention strategies against sexual transmission have shown significant efficacy. Poor adherence may have contributed to the lack of effect in some of these trials. In this paper we explore the impact of various levels of adherence on measured efficacy within an RCT. Analysis We used simple quantitative methods to illustrate the impact of various levels of adherence on measured efficacy by assuming a uniform population in terms of sexual behavior and the binomial model for the transmission probability per partnership. At 100% adherence the measured efficacy within an RCT is a reasonable approximation of the true biological efficacy. However, as adherence levels fall, the efficacy measured within a trial substantially under-estimates the true biological efficacy. For example, at 60% adherence, the measured efficacy can be less than half of the true biological efficacy. Conclusions: Poor adherence during a trial can substantially reduce the power to detect an effect, and improved methods of achieving and maintaining high adherence within trials are needed. There are currently 12 ongoing HIV prevention trials, all but one of which require ongoing user-adherence. Attention must be given to methods of maximizing adherence when piloting and designing RCTs and HIV prevention programmes.

IntroductionCluster surveys are frequently used to measure key nutrition and health indicators in humanitarian emergencies. The survey design of 30 clusters of 7 children (30x7) was initially proposed by the World Health Organization for measuring vaccination coverage, and later a design of 30 clusters of 30 children (30x30) was introduced to measure acute malnutrition in emergency settings. Recently, designs of 33 clusters of 6 children (33x6) and 67 clusters of 3 children (67x3) have been proposed as alternatives that enable measurement of several key indicators with sufficient precision, while offering substantial savings in time. This paper explores expected effects of using 67x3, 33x6, or 30x7 designs instead of a "standard" 30x30 design on precision and accuracy of estimates, and on time required to complete the survey. Analysis The 67x3, 33x6, and 30x7 designs are expected to be more statistically efficient for measuring outcomes having high design effects (e.g., vaccination coverage, vitamin A distribution coverage, or access to safe water sources), and less efficient for measuring outcomes with more within-cluster variability, such as global acute malnutrition or anemia. Because of small sample sizes, these designs may not provide sufficient levels of precision to measure crude mortality rates. Given the small number (3 to 7) of survey subjects per cluster, it may be hard to select representative samples of subjects within clusters. The smaller sample size in these designs will likely result in substantial time savings. The magnitude of the savings will depend on several factors, including the average travel time between clusters. The 67x3 design will provide the least time savings. The 33x6 and 30x7 designs perform similarly to each other, both in terms of statistical efficiency and in terms of time required to complete the survey. Conclusion: Cluster designs discussed in this paper may offer substantial time and cost savings compared to the traditional 30x30 design, and may provide acceptable levels of precision when measuring outcomes that have high intracluster homogeneity. Further investigation is required to determine whether these designs can consistently provide accurate point estimates for key outcomes of interest. Organizations conducting cluster surveys in emergency settings need to build their technical capacity in survey design to be able to calculate context-specific sample sizes individually for each planned survey.

The conventional method to collect data on the health, nutrition, and food security status of a population affected by an emergency is a 30x30 cluster survey. This sampling method can be time and resource intensive and, accordingly, may not be the most appropriate one when data are needed rapidly for decision making. In this study, we compare the precision, time and cost of the 30x30 cluster survey with two alternative sampling designs: a 33x6 cluster design (33 clusters, 6 observations per cluster) and a 67x3 cluster design (67 clusters, 3 observations per cluster). Data for each sampling design were collected concurrently in West Darfur, Sudan in September-October 2005 in an emergency setting. Results of the study show the 30x30 design to provide more precise results (i.e. narrower 95% confidence intervals) than the 33x6 and 67x3 design for most child-level indicators. Exceptions are indicators of immunization and vitamin A capsule supplementation coverage which show a high intra-cluster correlation. Although the 33x6 and 67x3 designs provide wider confidence intervals than the 30x30 design for child anthropometric indicators, the 33x6 and 67x3 designs provide the opportunity to conduct a LQAS hypothesis test to detect whether or not a critical threshold of global acute malnutrition prevalence has been exceeded, whereas the 30x30 design does not. For the household-level indicators tested in this study, the 67x3 design provides the most precise results. However, our results show that neither the 33x6 nor the 67x3 design are appropriate for assessing indicators of mortality. In this field application, data collection for the 33x6 and 67x3 designs required substantially less time and cost than that required for the 30x30 design. The findings of this study suggest the 33x6 and 67x3 designs can provide useful time- and resource-saving alternatives to the 30x30 method of data collection in emergency settings.