A new multivariate two-sample test using regular minimum-weight spanning subgraphs

Abstract

A new nonparametric test is proposed for the multivariate two-sample problem. Similar to Rosenbaum’s cross-match test, each observation is considered to be a vertex of a complete undirected weighted graph; interpoint distances are edge weights. A minimum-weight, r-regular subgraph is constructed, and the mean cross-count test statistic is equal to the number of edges in the subgraph containing one observation from the first group and one from the second, divided by r. Unequal distributions will tend to result in fewer edges that connect vertices between different groups. The mean cross-count test is sensitive to a wide range of distribution differences and has impressive power characteristics. We derive the first and second moments of the mean cross-count test, and note that simulation studies suggest this test statistic is asymptotically normal regardless of underlying data distributions. A small simulation study compares the power of the mean cross-count test to Hotelling’s T² test and to the cross-match test. This new test is a more powerful generalization of Rosenbaum’s test (the cross-match test is the case r = 1) and constitutes a noteworthy addition to the class of multivariate, nonparametric two-sample tests.

Background

Objective

Consider N = m + n independent multivariate observations Y₁,…, Y _m and Y_{m + 1},…, Y _N , where each Y _i is drawn from distribution F for 1≤i≤m and from distribution G for m + 1≤i≤N. The dimension of the observations does not depend on N. The covariates may be quantitative or categorical; there need only exist some function, d, that measures distance between observations. The null hypothesis is that F = G. The objective is a two-sample test that has little or no dependence on the underlying distribution of the data. Furthermore, this test should have sufficient power to be useful for applications.

Motivation

We follow in the vein of graph-theoretic tests for homogeneity: Consider each observation to be a vertex of a complete, undirected, weighted graph, $\mathcal{G}$, and assign interpoint distances as edge weights. The distribution of these distances is sensitive to departures from homogeneity; Maa et al. ([1996]) prove that two distributions are identical if and only if the distributions of inter-point distances within and between observations sampled from the two populations are the same. Friedman and Rafsky ([1979], [1981]) fit a minimum spanning tree to $\mathcal{G}$ and count the number of edges in the tree that connect vertices from different groups to test whether the sampling distributions are the same. Schilling ([1986]), Henze ([1988]), and Hall and Tajvidi ([2002]) examine properties of nearest-neighbor subgraphs of $\mathcal{G}$ to test for homogeneity.

Rosenbaum ([2005]) provides a novel approach to this problem: Suppose N is even. Find a minimum-weight non-bipartite matching on $\mathcal{G}$, which is the lowest-weight spanning subgraph for which the degree of each vertex with respect to the subgraph is one and which consists of N/2 non-adjacent edges. Rosenbaum’s cross-match statistic, A₁, counts the number of edges in the matching that include one vertex from each of the two groups. Under the null hypothesis of no group difference each vertex is equally likely to be paired with any other vertex. Rosenbaum ([2005]) shows that the exact null distribution of A₁ is found by combinatorial argument to be

$$P\left(A_1=a_1\right)=\frac{2^{a_1}\left(N/2\right)!}{\left(\begin{array}{c}\hfill N\hfill \\ \hfill m\hfill \end{array}\right)\left(\frac{m-a_1}{2}\right)!a_1!\left(\frac{n-a_1}{2}\right)!}$$

(1)

for a₁∈{0, 2, …, min(m, n)} and m and n even, or a₁∈{1, 3, …, min(m, n)} and m and n odd; P(A₁ = a₁) = 0 otherwise. In the denominator of (1), $\frac{1}{2}\left(m-a_1\right)$ is the number of edges in the matching where both vertices are in the group of size m and $\frac{1}{2}\left(n-a_1\right)$ is the number of edges in the matching where both vertices are in the group of size n.

When the two groups are drawn from different distributions the number of within-group pairs tends to be higher than for the null case, so the null hypothesis of homogeneity is rejected if A₁ is sufficiently small. For odd N, this procedure may be simply modified by introducing a pseudo-observation, Y₀ such that d(Y₀, Y _i ) = 0 for all i∈{1, …, N}, and randomly assigning it to one of the two groups. Then find a minimum-weight non-bipartite matching on this resulting graph with N + 1 vertices and compute A₁ with respect to observations Y₀, …, Y _N .

That the exact null distribution of A₁ is known, regardless of the underlying data distribution, is a particularly attractive property for a multivariate two-sample test. Furthermore, the asymptotic normality of A₁ facilitates testing for large-sample problems. However, the cross-match test has relatively low power. Since only a single non-bipartite matching is considered in this test, information contained in the proximity of many pairs of points is ignored. Friedman and Rafsky ([1979], [1981]) observe that the power of their single-tree test is enhanced by evaluating successive disjoint low-weight spanning trees. Similarly, Ruth and Koyak ([2011]) show that ensembles of disjoint low-weight non-bipartite matchings carry significant information regarding whether a distributional change occurs over a sequence of independent observations. A drawback associated with examining collections of such subgraphs is that null distributions are extremely difficult to determine. Mindful of this caveat, we offer an extension of the cross-match test which exploits the information contained in the distances between many pairs of points.

Methods

Illustrating example

Consider the bivariate sample of size N = 20 listed in Table 1 and displayed in Figure 1. The sample consists of independent observations in groups 1 (○) and 2 (△); observations within groups are identically distributed. For the purposes of this example, these data were simulated from distributions whose locations differ by one unit in each dimension. Figure 1 also shows the minimum-weight non-bipartite matching associated with this sample with respect to Euclidean distance. The present goal is to identify the distribution difference between these groups, making no assumptions about the underlying distributions.

Table 1 Bivariate data for illustrating example

Figure 1

Bivariate data for illustrating example with optimal non-bipartite matching on groups 1 (○) and 2 (△) with m = n = 10. Cross-group pairs are connected by solid lines; within-group pairs by dotted lines.

The cross-match test is applicable here; for this example the value of the cross-match statistic is A₁ = 4 with a corresponding p-value = 0.433. So, the cross-match test is insufficiently powerful to identify a distribution difference in this case. In the next section, we introduce an extension of the cross-match test that enhances test power significantly.

The mean cross-count (MCC) test

As before, we assume an even number N of observations forming a complete, undirected, weighted graph, $\mathcal{G}$. Rather than find a minimum-weight non-bipartite matching, we find a minimum-weight r-regular spanning subgraph of$\mathcal{G}$, where 1≤r≤N-2, denoted ${\mathcal{G}}_r^{*}$. That is, ${\mathcal{G}}_r^{*}$ is a subgraph of $\mathcal{G}$ with the following properties:

• a) Every vertex in $\mathcal{G}$ is also in ${\mathcal{G}}_r^{*}$.

• b) Every vertex in ${\mathcal{G}}_r^{*}$ has degree r.

• c) The total weight of all edges in ${\mathcal{G}}_r^{*}$ is the lowest among all subgraphs of $\mathcal{G}$ which satisfy properties (a) and (b).

In graph theory, an r- regular spanning subgraph of $\mathcal{G}$ is sometimes called an r-factor of$\mathcal{G}$. Note that ${\mathcal{G}}_1^{*}$ is the special case of a minimum-weight non-bipartite matching used by Rosenbaum ([2005]), and ${\mathcal{G}}_{N-1}^{*}$ is identical to $\mathcal{G}$. In practice, we are mainly interested in 2≤r≤N/2, although the theoretical details are not so constrained. Minimum-weight r- factors may be computed as follows: For any subgraph of $\mathcal{G}$, let x _ij be an indicator variable equal to 1 if the edge connecting vertices i and j is included in the subgraph and let d _ij be the distance between vertex i and vertex j. Then the edges of ${\mathcal{G}}_r^{*}$ solve following the combinatorial optimization problem:

$$\begin{array}{c}\underset{\mathbf{x}}{min}{\displaystyle \sum _{j=2}^N}{\displaystyle \sum _{i=1}^{j-1}}{d}_{ij}{x}_{ij}\\ \mathrm{subject}\mathrm{t}\mathrm{o}{\displaystyle \sum _{i=1}^{k-1}}{x}_{ik}+{\displaystyle \sum _{j=k+1}^N}{x}_{kj}=r\forall k\in \left\{1,\dots ,N\right\}\\ x_{ij}\in \left\{0,1\right\}\forall j\in \left\{i+1,\dots ,N\right\},\forall i\in \left\{1,\dots ,N-1\right\}\text{.}\end{array}$$

(2)

Anderson ([1972]) assures the existence of a solution for r≤N/2. Solutions for r > N/2 are guaranteed by the fact that the complement of an r-regular subgraph of $\mathcal{G}$ is an (N-1-r) -regular graph. For this paper, solutions are found in R using the package “lpSolve” for N≤400. For N > 400, solutions are found in R using the package “gurobi” due to the computational complexity of larger problems.

Similar to the cross-match test, we count the number of edges A _r in ${\mathcal{G}}_r^{*}$ that include a vertex from each group. We call T _r = A _r /r the mean cross-count (MCC) statistic. The idea here is that the number of within-group edges in ${\mathcal{G}}_r^{*}$ will be higher for cases of a distribution difference than for the null case. So, small values of T _r are evidence against the null hypothesis. Note that T₁ = A₁ is the cross-match statistic as before. One could use the total cross-count, A _r , as an equivalent test-statistic; however, we choose to scale this value to give some notion of “average cross-count per vertex degree” (hence the name “mean cross-count”). For odd N, randomly introduce a pseudo-observation in the same manner as the r = 1 case discussed in Section 1.2.

Illustrating example, continued

Figure 2 shows a minimum-weight 3-factor, ${\mathcal{G}}_3^{*}$, for the data in Table 1 with respect to Euclidean distance. Cross-group edges are shown with solid lines. For this case, A₃ = 12⇒T₃ = 4, so the test statistic value here is the same as Rosenbaum’s cross-match test statistic. A discussion of the distribution of T _r is in Section 3.1; for this example, we estimate the p-value for T _r by permutation test on the observation vertex labels. Using 10,000 permutations yields an estimated p-value = 0.146. While not enough evidence to conclude a group difference, this reduction in p-value relative to the r = 1 case (p-value = 0.433) suggests that considering minimum-weight r- factors for r > 1 may improve test power. In Section 3.2 we demonstrate significant power advantages that are realized for the MCC statistic.

Figure 2

Bivariate data for illustrating example with optimal 3-factor on groups 1 (○) and 2 (△) with m = n = 10. Cross-group pairs are connected by solid lines; within-group pairs by dotted lines.

Results and discussion

MCC moments and normal approximation

For the following discussion we assume N is even, adopting the convention that if the number of observations is odd then we will consider N to be the number of observations including a pseudo-observation as previously discussed. To find the mean and variance of T _r under the null hypothesis, we proceed as follows: Let $\mathcal{G}$ be the complete undirected graph (Z _N , E _N ) where the vertex set Z _N consists of the indices 1, 2, …, N and the edge set consists of all N(N - 1)/2 pairs of vertices; by convention, write the pairs with smaller vertex first, so E _N = {(i, j) : 1≤i < j≤N}. Partition Z _N into two sets S and T, with |S| = m and |T| = n, so m + n = N. Denote $E_N^{\left(S,T\right)}$ as the set of all edges with one vertex in S and the other in T. Let X _ij be the random variable that indicates whether edge (i, j) is included in a minimum-weight r-regular subgraph, ${\mathcal{G}}_r^{*}$, with 1≤r≤N-2. By the r-regularity of ${\mathcal{G}}_r^{*}$, for each i∈Z _N we have $r={\displaystyle \sum _{j=1}^{i-1}}{X}_{ji}+{\displaystyle \sum _{j=i+1}^N}{X}_{ij}\text{,}$ and so

$$\begin{array}{l}r=E\left[r\right]=E\left[{\displaystyle \sum _{j=1}^{i-1}}{X}_{ji}+{\displaystyle \sum _{j=i+1}^N}{X}_{ij}\right]={\displaystyle \sum _{j=1}^{i-1}}E\left[X_{ji}\right]+{\displaystyle \sum _{j=i+1}^N}E\left[X_{ij}\right]\\ ={\displaystyle \sum _{j=1}^{i-1}}P\left(X_{ji}=1\right)+{\displaystyle \sum _{j=i+1}^N}P\left(X_{ij}=1\right).\end{array}$$

(3)

But under the null hypothesis, each edge is equally likely to be included in ${\mathcal{G}}_r^{*}$, so

r = (N-1)P(X₁₂ = 1). Therefore, for all (i, j)∈E _N

$$E\left[X_{ij}\right]=P\left(X_{ij}=1\right)=\frac{r}{N-1}$$

(4)

and

$$\mathrm{V}\mathrm{a}\mathrm{r}\left[X_{ij}\right]=P\left(X_{ij}=1\right)P\left(X_{ij}=0\right)=\frac{r\left(N-1-r\right)}{{\left(N-1\right)}^2}\text{.}$$

(5)

The total cross-count, A _r , may be written

$${\mathit{A}}_r={\displaystyle \sum _{\left(i,j\right)\in E_N^{\left(S,T\right)}}}{X}_{ij}$$

(6)

resulting in

$$\begin{array}{c}E\left[T_r\right]=\frac{1}{r}E\left[A_r\right]=\frac{1}{r}E\left[{\displaystyle \sum _{\left(i,j\right)\in E_N^{\left(S,T\right)}}}{X}_{ij}\right]=\frac{mn}{r}E\left[X_{ij}\right]\\ =\frac{mn}{N-1}\text{.}\end{array}$$

(7)

Finding the variance of T _r is slightly more involved. First take

$$\mathrm{V}\mathrm{a}\mathrm{r}\left[A_r\right]=\mathrm{V}\mathrm{a}\mathrm{r}\left[{\displaystyle \sum _{\left(i,j\right)\in E_N^{\left(S,T\right)}}}{X}_{ij}\right]={\displaystyle \sum _{\left(i,j\right)\in E_N^{\left(S,T\right)}}}\mathrm{V}\mathrm{a}\mathrm{r}\left[X_{ij}\right]+{\displaystyle \sum _{\left(i,j\right),\left(k,l\right)\in E_N^{\left(S,T\right)}\left(i,j\right)\ne \left(k,l\right)}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{\mathit{kl}}\right]\text{.}$$

(8)

The sum of variances is computed directly as

$${\displaystyle \sum _{\left(i,j\right)\in E_N^{\left(S,T\right)}}}\mathrm{V}\mathrm{a}\mathrm{r}\left[X_{ij}\right]=mn\mathrm{V}\mathrm{a}\mathrm{r}\left[X_{ij}\right]=\frac{mnr\left(N-1-r\right)}{{\left(N-1\right)}^2}\text{.}$$

(9)

The sum of covariances may be partitioned into terms that include pairs of adjacent edges and terms that include disjoint (i.e., non-adjacent) edges:

$${\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill j,l\in T\hfill \\ \hfill \left(i,j\right)\ne \left(k,l\right)\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{\mathit{kl}}\right]={\displaystyle \sum _{\begin{array}{c}\hfill i\in S\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{il}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j\in T\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{kj}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{\mathit{kl}}\right]$$

(10)

For any two adjacent edges (k, l) and (i, j),

$$\begin{array}{l}P\left(X_{ij}{X}_{\mathit{kl}}=1\right)=P\left(X_{\mathit{kl}}=1\left|X_{ij}=1\right.\right)P\left(X_{ij}=1\right)=\frac{\left(r-1\right)r}{\left(N-2\right)\left(N-1\right)}\\ =E\left[X_{ij}{X}_{\mathit{kl}}\right]\text{,}\end{array}$$

(11)

so

$$\begin{array}{l}{\displaystyle \sum _{\begin{array}{c}\hfill i\in S\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{il}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j\in T\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{kj}\right]\\ =\left(mn\left(n-1\right)+m\left(m-1\right)n\right)\left(\frac{\left(r-1\right)r}{\left(N-2\right)\left(N-1\right)}-{\left(\frac{r}{N-1}\right)}^2\right)\\ =-\frac{mnr\left(N-1-r\right)}{{\left(N-1\right)}^2}\text{.}\end{array}$$

(12)

For any two disjoint edges (k, l) and (i, j),

$$\begin{array}{l}P\left(X_{ij}{X}_{\mathit{kl}}=1\right)=P\left(X_{\mathit{kl}}=1\left|X_{ij}=1\right.\right)P\left(X_{ij}=1\right)=\frac{\left(r\left(N-4\right)+2\right)}{\left(N-3\right)\left(N-2\right)}\frac{r}{\left(N-1\right)}\\ =E\left[X_{ij}{X}_{\mathit{kl}}\right]\text{,}\end{array}$$

(13)

So

$$\begin{array}{l}{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{\mathit{kl}}\right]=m\left(m-1\right)n\left(n-1\right)\left(\frac{\left(r\left(N-4\right)+2\right)}{\left(N-3\right)\left(N-2\right)}\frac{r}{\left(N-1\right)}-{\left(\frac{r}{N-1}\right)}^2\right)\\ =\frac{2m\left(m-1\right)n\left(n-1\right)\left(N-1-r\right)r}{\left(N-3\right)\left(N-2\right){\left(N-1\right)}^2}\text{.}\end{array}$$

(14)

Combining terms yields

$$\begin{array}{l}\mathrm{V}\mathrm{a}\mathrm{r}\left[A_r\right]={\displaystyle \sum _{\begin{array}{c}\hfill i\in S\hfill \\ \hfill j\in T\hfill \end{array}}}\mathrm{V}\mathrm{a}\mathrm{r}\left[X_{ij}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i\in S\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{il}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j\in T\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{kj}\right]+{\displaystyle \sum _{\begin{array}{c}\hfill i,k\in S\hfill \\ \hfill i\ne k\hfill \\ \hfill j,l\in T\hfill \\ \hfill j\ne l\hfill \end{array}}}\mathrm{C}\mathrm{o}\mathrm{v}\left[X_{ij},X_{\mathit{kl}}\right]\\ =\frac{mnr\left(N-1-r\right)}{{\left(N-1\right)}^2}-\frac{mnr\left(N-1-r\right)}{{\left(N-1\right)}^2}\\ +\frac{2m\left(m-1\right)n\left(n-1\right)\left(N-1-r\right)r}{\left(N-3\right)\left(N-2\right){\left(N-1\right)}^2}\\ =\frac{2m\left(m-1\right)n\left(n-1\right)\left(N-1-r\right)r}{\left(N-3\right)\left(N-2\right){\left(N-1\right)}^2}\text{.}\end{array}$$

(15)

Therefore,

$$\mathrm{V}\mathrm{a}\mathrm{r}\left[T_r\right]=\frac{1}{r^2}\mathrm{V}\mathrm{a}\mathrm{r}\left[A_r\right]=\frac{2m\left(m-1\right)n\left(n-1\right)\left(N-1-r\right)}{r\left(N-3\right)\left(N-2\right){\left(N-1\right)}^2}\text{.}$$

(16)

We note in particular that the first and second moment results in (7) and (16) match the results in Rosenbaum ([2005]) for the special case r = 1.

Simulation suggests that the null distribution of T _r is negatively skewed, but that for sufficiently large N and possibly certain conditions on r this distribution is asymptotically normal, independent of distribution function F. Rosenbaum ([2005]) proves that T _r is asymptotically normal for r = 1 for any distribution function; proof of this conjecture for r > 1 remains an open problem. This conjecture is supported by the normal QQ-plots shown in Figures 3 and 4 for 1,000 simulated values of $\left(T_r-E\left[T_r\right]\right)/\sqrt{\mathrm{V}\mathrm{a}\mathrm{r}\left(T_r\right)}$ at r = 5, 50 and N = 150, 600 with m/n = 1/2, under sampling from uniform distributions on [-1, 1]⁵ and [-1, 1]²⁰. For the smaller sample size (N = 150), negative skewness is stronger for lower dimension and for higher r, with r = 50 and Dim = 5 being the most strongly skewed case shown. For the larger sample size (N = 600), skewness effects appear to vanish for all but the r = 50 and Dim = 5 case, and even in this case skewness is vastly diminished compared to the smaller sample size. Other distribution families and other values of m/n produce similar results.

Figure 3

Normal QQ-plots of 1,000 simulated values of T _r for m = 50, n = 100 and r = 5, 50 . Panels (a) and (b) are from independent samples of Unif [-1, 1]⁵ variates. Panels (c) and (d) are from independent samples of Unif [-1, 1]²⁰ variates.

Figure 4

Normal QQ-plots of 1,000 simulated values of T _r for m = 200, n = 400 and r = 5, 50 . Panels (a) and (b) are from independent samples of Unif [-1, 1]⁵ variates. Panels (c) and (d) are from independent samples of Unif [-1, 1]²⁰ variates.

Future work remains to bound rejection region probabilities in terms of sample size, dimension, and choice of r. In the absence of such theoretical bounds, for practical purposes a permutation test on observation indices serves as a suitable method to estimate p-values for the MCC test in cases where a normal approximation cannot be justified.

Small simulation study

We compare power characteristics of tests for two different location-shift scenarios. For each case, 1000 simulations are conducted for each shift in location parameter, group sizes are m = 20 and n = 40, and tests are conducted at significance level α = .05. Distances are Euclidean. Estimated power is shown for MCC tests with r = 1, 4, 10, and 30 (where r = 1 is the cross-match test), and the performance of these tests is compared directly to that of Hotelling’s T² test. Critical values for the MCC test were estimated through simulation for this study. All simulations were performed in R.

For the first example, the smaller group is drawn from a multivariate normal distribution with mean vector 0, identity covariance matrix, and dimension 5. The larger group is drawn from the same family, but the location vector of the second group is Δ, where Δ ranges in magnitude from 0 to 1.5 by increments of 0.3. Hotelling’s T² test is known to be the uniformly most powerful invariant test for location shift under these conditions (Bilodeau and Brenner [1999]) and the exact power of the test is known for all location alternatives.

Figure 5 displays the estimated power results. We notice immediately that a modest increase of r = 1 to r = 4 substantially improves on the power of the cross-count test. As r continues to increase, MCC performance is even more impressive; the r = 30 = N/2 case performs nearly as well as Hotelling’s T² test. Power estimates for cases r > N/2 are not shown. Not surprisingly, test power generally decreases as r increases beyond N/2 toward N-1; in the extreme case T_N-1 takes the fixed value $\frac{mn}{N-1}$ and hence the MCC test with r = N-1 has power equal to zero against all alternatives.

Figure 5

Power estimates for the MCC test at r = 1, 4, 10, and 30 and exact power for Hotelling’s T²statistic for 5-variate normal mean alternatives with m = 20 and n = 40. The horizontal dashed line is test level α = .05.

For the second example, the first group is drawn from the multivariate log-normal distribution, where each of the 5 dimensions consists of independent, univariate log-normal draws with location parameter 0 and scale parameter 1. As before, the second group is drawn from the same family, but the location parameter vector for the second group is Δ, where the magnitude of Δ ranges from 0 to 1.5 by increments of 0.3 and each dimension of Δ takes equal value. The lognormal distribution is considered here to examine the effects of a skewed distribution on the tests in question. Since the underlying distributions are no longer multivariate normal, the power of Hotelling’s T² test is estimated by simulation for this example.

Figure 6 displays the estimated power results. As before, we see that the power of the MCC test with r = 4 is much better than for r = 1. It is particularly noteworthy that for sufficiently large r the MCC test outperforms Hotelling’s T² test.

Figure 6

Power estimates for the MCC test at r = 1, 4, 10, and 30 and estimated power for Hotelling’s T²statistic for 5-variate lognormal location parameter alternatives with m = 20 and n = 40. The horizontal dashed line is test level α = .05.

Conclusions

The mean cross-count test is a powerful, non-parametric multivariate two-sample test that is applicable to any case where a notion of distance between observations exists. While this paper considers only location shifts, other simulations show that the MCC test has power in a variety of alternative cases as well. A shortcoming of the MCC test is that the null distribution for T _r is not simple (and perhaps not possible) to compute for all r > 1 and is not exactly distribution-free in these cases; in contrast, the test upon which it is based, the cross-match test with r = 1, has a known distribution that is independent of the distribution being tested.

It is known that T₁ is asymptotically normal, and while the mean and variance of T _r are derived herein and simulation suggests that the normal approximation for T _r is appropriate for sufficiently large N with r > 1, this property remains to be proven. This proof is part of ongoing work, as is sharpening the normal approximation via Edgeworth expansion based on higher moments of T _r . Likewise, finding useful criteria for choosing r is another area for future work. This choice is subject to competing factors: On the one hand, the power of T _r appears to improve as r increases to N/2 when group sizes are equal (i.e., m = n = N/2); therefore, r = N/2 seems a good choice for equal group sizes. On the other hand, the normal approximation appears to worsen as r increases; thus it may be desirable to restrict the size of r for this sake. Furthermore, an additional effect exists when group sizes are different. For example, assume m < n. If r≥m, then at least one edge in ${\mathcal{G}}_r^{*}$ contains a vertex from each group and contributes to the cross-count, increasing the value of T _r . This is true even if the two groups are very different. Since a higher cross-count weakens the evidence against a group difference, this consideration suggests choosing r < min(m, n). A similar effect exists for multimodal distributions, suggesting that the size of r might be restricted as the number of modes grows. In any case, the best choice of r in practice clearly depends upon application specifics.