An efficiency comparison of the test statistics for testing the mean difference between two independent populations

Abstract

The objectives of this study were to calculate the probability of type I error, power of a test, and to compare the performance of three test statistics: Z, Student’s t, and Welch’s t. Including the recommending the appropriate test statistic. Classification of the population according to five distribution. The ratios of mean group 1 per group 2 were three levels. The ratios of variance group 1 per group 2 were three levels. The sample sizes of two populations were nine sizes. The significance levels that to study were 0.01 and 0.05. The data were simulated by using SAS programming with 1,000 replicates. Results of the research were Z test statistic could control the probability of type I error and had higher power of a test than other test statistics, but this test is necessary to know the variances of the two groups. Therefore, it was appropriate to apply the other test statistics, i.e., Student’s t or Welch’s t. If the variances of the two groups were equal, Student’s t was a more appropriate test statistic than Welch’s t. But, if the variances of the two groups were not equal, Welch’s t was a more appropriate. When both sample sizes were large and the population variances were equal, we could use any tests since there is no difference in the performance.