Recent developments on the construction of bivariate distributions with fixed marginals

Gwo Dong Lin; Xiaoling Dou; Satoshi Kuriki; Jin-Sheng Huang

doi:10.1186/2195-5832-1-14

Review
Open Access
Published: 17 June 2014

Recent developments on the construction of bivariate distributions with fixed marginals

Gwo Dong Lin¹,
Xiaoling Dou²,
Satoshi Kuriki³ &
Jin-Sheng Huang⁴

Journal of Statistical Distributions and Applications volume 1, Article number: 14 (2014) Cite this article

1591 Accesses
9 Citations
Metrics details

Abstract

Constructing a bivariate distribution with specific marginals and correlation has been a challenging problem since 1930s. In this survey we shall focus on the recent developments on the FGM-related distributions, including Sarmanov and Lee’s distributions, Baker’s distributions and Bayramoglu’s distributions. This complements the most recent works of (i) the review by Sarabia and Gómez-Déniz (2008, SORT) and (ii) the monograph by Balakrishnan and Lai (2009, Springer). Some new results are provided.

Mathematics Subject Classification (2000)

62H20; 62H86; 62G30; 60E05; 62E10

1 Introduction and a brief history

In both statistical theory and practice, we often need to construct a joint distribution with specific marginals and correlation. Ever since Wigner (1932) and Eyraud (1936), it has been an active and challenging topic. Its field of applications is wide ranging. It encompasses physics, economics, engineering, risk analysis, medicine etc. There is no dearth of interesting real life examples. Only recently Takeuchi (2010) constructed a bivariate distribution having specific marginals and certain degrees of correlation to model the joint behavior of far-infrared and far-ultraviolet galaxy luminosity. Danaher and Smith (2011) studied the interaction between a firm’s total amount of purchase (log-normal distribution) and the time since last purchase (exponential distribution). For applications in reliability theory, see for example Li et al. (2013) and the references therein.

Since 1990 there have been seven conferences devoted to this topic: 1990 (Rome, Italy), 1993 (Seattle, USA), 1996 (Prague, Czech Republic), 2000 (Barcelona, Spain), 2004 (Québec, Canada), 2007 (Tartu, Estonia), and 2010 (São Paulo, Brazil), averaging one in every three to four years. The papers presented in the conferences during the period 1990-2000 have all appeared in monographs, edited by Dall’Aglio et al. (1991), Rüschendorf et al. (1996), Beněs and Štěṕan (1997), and Cuadras et al. (2002), respectively. As for the 2004 and 2007 ones, they were published as special issues of Insurance: Mathematics and Economics (August 2005), The Canadian Journal of Statistics (September 2005) and Journal of Statistical Planning and Inference (November 2009). We have yet to find any documentation for the 2010 conference.

There were three survey papers: Lai (2004; 2006) and Sarabia and Gómez-Déniz (2008). The latest monograph by Balakrishnan and Lai (2009) provided an excellent overview. As for the continuous multivariate version, there were Kotz et al. (2000) and Kotz and Nadarajah (2004). For the discrete version, see Johnson et al. (1997), Gómez-Déniz et al. (2012) and Sarabia and Gómez-Déniz (2011), among others. Then there were Cuadras (1992) and Dolati and Úbeda-Flores (2005) for constructing distributions with given multivariate marginals and given dependence structure.

Section 2 is a review of some basic properties of the bivariate distributions with fixed marginals. Then, in Sections 3 to 6, we will focus on the recent developments on the FGM-related distributions, including Sarmanov and Lee’s distributions, Baker’s distributions and Bayramoglu’s distributions. Some new results are provided. This complements the most recent works mentioned above. Finally, in Section 7 we briefly discuss some other related distributions.

2 The natural bounds of the correlations

We first recall an important fundamental result due to Hoeffding (1940) and Fréchet (1951) independently. Let H (x,y) = Pr(X ≤x, Y ≤ y) be the joint distribution of any pair (X,Y) of random variables whose marginals are F (x) = Pr(X ≤ x) and G (y) = Pr(Y ≤ y). We write (X,Y) ∼ H, X ∼ F, Y ∼ G. Then the bivariate distribution H satisfies the following inequality:

max {0, F (x) + G (y) - 1} ≡ H - (x, y) ≤ H (x, y) ≤ H + (x, y) ≡ min {F (x), G (y)}, x, y ∈ R ≡ (- ∞, ∞),

(1)

where the extremal distributions H₊ and H_- are known as the Fréchet–Hoeffding upper and lower bounds, respectively.

To generate a pair of random variables obeying the extremal distributions, we start with Z∼ U(0,1), the uniform distribution on (0,1), and note that (F^-1(Z), G^-1(Z)) ∼ H₊ and (F^{- 1}(Z), G^{- 1}(1 - Z)) ∼ H_- , where F^{- 1}(t) = inf {x : F (x) ≥ t}, t ∈ (0,1), is the quantile function of F. Take for example F = G = U (0,1). Then (Z,Z) ∼ H₊ with H₊(x,y) = min {x,y} and (Z, 1 - Z)∼H_- with H_-(x,y)= max{0,x+y-1}. We see that for this special case, both H₊ and H_- are singular bivariate distributions, with the entire mass concentrated on the lines y = x or y = - x, respectively (see Lin and Huang 2010, and the references therein).

The inequality (1) for H implies that Cov(H_-) ≤ Cov (H) ≤ Cov(H₊) by the Hoeffding representation for covariance:

Cov (H) ≡ Cov (X, Y) = ∫ - ∞ ∞ ∫ - ∞ ∞ [H (x, y) - F (x) G (y)] dxdy,

(2)

whenever the double integral exists and is finite. It then follows that

ρ (H -) ≤ ρ (H) ≤ ρ (H +),

(3)

where ρ (H) denotes the correlation of any pair of random variables (X,Y) ∼ H (or, simply, the correlation of H). Writing μ and σ for the mean and the standard deviation of the random variable appearing as a subscript, we see that the maximum correlation can be expressed explicitly in terms of the quantile functions of the marginals:

ρ (H +) = ∫ 01 F - 1 (t) G - 1 (t) dt - μ X μ Y / (σ X σ Y) ≤ 1 .

Moreover, ρ (H₊) = 1 iff the marginals F and G are of the same type, namely, F(x) = G (a x + b) on R for some a > 0 and b ∈ R. Similarly, we have

ρ (H -) = ∫ 01 F - 1 (t) G - 1 (1 - t) dt - E (X) E (Y) / (σ X σ Y) ≥ - 1,

and ρ (H_-) = - 1 iff the distributions of X and - Y are of the same type.

3 FGM distributions

To construct a bivariate distribution with fixed marginals, a well known method is the FGM distribution, dating back to Eyraud (1936), Farlie (1960), Gumbel (1960) and Morgenstern (1956). Let (X,Y) ∼ H, X ∼ F and Y ∼ G. The bivariate distribution

H (x, y) = F (x) G (y) {1 + α F ¯ (x) G ¯ (y)}, x, y ∈ R,

(4)

is called the FGM distribution, where $F ¯ = 1 - F, G ¯ = 1 - G$ and α is a parameter such that H is a bona fide bivariate distribution.

The advantage of the FGM distribution is its simplicity. Its drawback is the lack of flexibility. As was pointed out by Schucany et al. (1978), for absolutely continuous marginals the correlation in FGM is restricted to the interval [-1/3,1/3], which is a far cry from the full range [-1,1] enjoyed by the bivariate normal. Schucany et al.’s condition of absolute continuity was later relaxed by Lin (1987) to just continuity.

To alleviate the deficiency, Johnson and Kotz (1975) proposed the ‘iterated’ FGM distribution:

H k (x, y) = F (x) G (y) + ∑ j = 1 k α j (F (x) G (y)) ⌊ j / 2 ⌋ + 1 (F ¯ (x) G ¯ (y)) ⌊ (j + 1) / 2 ⌋, x, y ∈ R,

where ⌊z⌋ denotes the greatest integer less than or equal to z, and k = 1,2,3,….

For k = 1, it reverts back to the plain FGM, where |α₁| ≤ 1 if both F and G are continuous. For k = 2 the iterated FGM takes the form (Huang and Kotz 1984, 1999)

H (x, y) = F (x) G (y) + α 1 F (x) G (y) F ¯ (x) G ¯ (y) + α 2 [F (x) G (y)] 2 F ¯ (x) G ¯ (y), x, y ∈ R.

To ensure that H be a bona fide distribution function, the range of the parameters, however, is more complicated, namely, for continuous marginals,

| α 1 | ≤ 1, α 1 + α 2 ≥ - 1, α 2 ≤ 2 - 1 3 - α 1 + (9 - 6 α 1 - 3 α 12) 1 / 2 .

(See also Lin 1987.) Take F = G = U (0,1) for the sake of comparison, the iteration succeeded in extending the range to $- 13 ≤ ρ ≤ 0.434$ from the previous $- 13 ≤ ρ ≤ 13$ of the plain FGM.

The iteration, however, was little studied beyond the k = 2 case due to the difficulty in determining the admissible range of the parameters α_j.

There were several other extensions of the FGM distribution, but the improvement in correlation was still limited (see, e.g., Balakrishnan and Lai 2009). To accommodate higher flexibility we turn to two other classes: (i) Sarmanov and Lee’s distribution (Sarmanov 1966, Lee 1996) and its generalization and (ii) Baker’s distribution (Baker 2008) and its generalization.

4 Sarmanov and Lee’s distributions

The joint density of a Sarmanov–Lee distribution is given by

h (x, y) = f (x) g (y) {1 + α θ 1 (x) θ 2 (y)}, x, y ∈ R,

(5)

where f and g are the densities of the marginals F and G, while θ₁ and θ₂ are measurable functions satisfying the condition

1 + α θ 1 (x) θ 2 (y) ≥ 0 and E (θ 1 (X)) = E (θ 2 (Y)) = 0,

which serves to ensure that h is a bona fide joint density.

When the marginals F and G are absolutely continuous, setting θ₁ = 1 - 2F and θ₂ = 1 - 2G shows that FGM is a special case of the Sarmanov–Lee.

Shubina and Lee (2004) showed that the maximum positive correlation of (5) is effected by concentrating all its mass on the (NE-SW) quadrants: {(x,y):(x - x₀)(y - y₀) ≥ 0}, and for the negative, on the (NW-SE) quadrants: {(x,y):(x - x₀)(y - y₀) ≤ 0}, for some real numbers x₀ and y₀.

The improvement in correlation is substantial. The maximum correlation, for the uniform marginals case, is 3/4 as opposed to 1/3 of the plain FGM and 0.434 of the (k=2) iterated FGM.

To further extend the Sarmanov–Lee family (5), a ‘generalized’ Sarmanov–Lee was proposed by Bairamov et al. (2001),

h (x, y) = f (x) g (y) 1 + αT (F (x), G (y)), x, y ∈ R,

(6)

where the product function θ₁(x) θ₂(y) in (5) is replaced by T (F (x),G (y)) in which T is an integrable bivariate function on [0,1]², satisfying

∫ 01 T (u, v) du = ∫ 01 T (u, v) dv = 0 and 1 + α T ≥ 0 on [0, 1] 2,

(7)

and the parameter α satisfies

- max (u, v) ∈ D + T (u, v) - 1 ≤ α ≤ max (u, v) ∈ D - (- T (u, v)) - 1,

(8)

where D⁺ = {(u,v) : T (u,v) > 0} and D^- = {(u,v) : T (u,v) < 0}. Again, the constraints (7) and (8) together guarantee that the function h in (6) is a bona fide bivariate density with marginal densities f and g.

The new class (6) is so rich it contains members arbitrarily close to H₊ and H_-, the two extremal distributions (1) of Fréchet–Hoeffding. It is thus flexible enough to accommodate nearly the maximum positive (negative) correlation. Some simple examples help demonstrate the idea (more can be found in Lin and Huang 2011).

Example 1.

The generalized Sarmanov–Lee density (3 × 3) with uniform marginals. Consider the partitioning of the unit square into 3 × 3= 9 subsquares, induced by the lines x,y = 1/3,2/3. Let the function T be defined by

T (x, y) = 2, 0 < x, y < 13 or 13 < x, y < 23 or 23 < x, y < 1, - 1, elsewhere .

Then the joint density becomes

h (x, y) = 3, 0 < x, y < 13 or 13 < x, y < 23 or 23 < x, y < 1, 0, elsewhere .

Its correlation is ρ (h) = 8/9, which exceeds the maximum, ρ_max = 3/4, of the plain Sarmanov–Lee (with uniform marginals).

Example 2.

The generalized Sarmanov–Lee density (n × n) with uniform marginals. For n = 2,3,4,…, let

T n (x, y) = n - 1, i - 1 n < x, y < i n, i = 1, …, n, - 1, elsewhere,

and

h n (x, y) = n, i - 1 n < x, y < i n, i = 1, …, n, 0, elsewhere .

We have the correlation ρ (h_n) = 1 - n^-2 → 1 as n → ∞.

Example 3.

The generalized Sarmanov–Lee density (n×n) with exponential marginals. Let

h n (x, y) = n e - x - y, ln n n - i + 1 < x, y < ln n n - i, i = 1, …, n, 0, elsewhere,

where ln(n/0) ≡ ∞. Our calculations show that ρ(h_n) ≈ 1 - 1.08/n → 1 as n → ∞.

The convergence to the maximal correlation is not limited to those with uniform or the exponential marginals. It holds for more general cases. The following Theorems 1 through 3 are successively stronger. Only Theorem 3 will be proved, since the others appeared in Lin and Huang (2011).

Let F and G be arbitrary distributions (not necessarily identical nor ‘of the same type’) with densities F^′= f and G^′= g. Define the joint density of (X_n,Y_n) by

h n (x, y) = nf (x) g (y), (x, y) ∈ F - 1 (i - 1 n), F - 1 (i n) × G - 1 (i - 1 n), G - 1 (i n), i = 1, 2, …, n, 0, elsewhere,

(9)

where $F - 1 (0) ≡ lim t → 0 + F - 1 (t)$ and $F - 1 (1) ≡ lim t → 1 - F - 1 (t)$ are the left and right extremities of the distribution F, respectively. It qualifies as a generalized Sarmanov–Lee bivariate density (with marginal densities f and g).

Theorem 1.

For F = G, (9) becomes

h n (x, y) = nf (x) f (y), F - 1 (i - 1 n) < x, y < F - 1 (i n), i = 1, 2, …, n, 0, elsewhere .

If F has a finite variance σ², then the correlation is

ρ (h n) = 1 σ 2 n ∑ i = 1 n ∫ F - 1 (i - 1 n) F - 1 (i n) xf (x) dx 2 - (E (X)) 2,

which converges to ρ (H₊), which is 1 in this case, as n → ∞.

Theorem 2.

If in (9), X and Y satisfy any of the conditions:

F^{- 1} or G^{- 1} is uniformly continuous on (0,1),
a≤X, a^′ ≤ Y a.s., (iii) X ≤b, Y ≤ b^′a.s.,
X ≥ a, Y ≤ b a.s. and F^{- 1},G^{- 1} have continuous derivatives, where a, b, a^′, b^′ ∈ R,

then ρ (h_n) converges to ρ (H₊) as n → ∞.

Remarks 1.

Theorem 1 follows immediately from a general result, which is interesting in its own right: For any X ∼ F with finite E (X²),

S n ≡ n ∑ i = 1 n ∫ i - 1 n i n F - 1 (t) dt 2 → ∫ 01 F - 1 (t) 2 dt = E (X 2) as n → ∞ .

The proofs of Remarks 1 and Theorem 2 are based on the following two lemmas.

Lemma 1.

(Chebyshev’s inequality for integrals.) Let f₁, f₂ : (a,b) → R be both increasing or both decreasing, and let p : (a,b) → (0,∞) be an integrable function. Then

∫ a b p (x) f 1 (x) dx ∫ a b p (x) f 2 (x) dx ≤ ∫ a b p (x) dx ∫ a b p (x) f 1 (x) f 2 (x) dx,

provided that all integrals exist and are finite.

Remarks 2.

An extension of this inequality can be rephrased in probabilistic terms: Let X be any random variable and let f₁, f₂ be both increasing or both decreasing, then the covariance Cov(f₁(X), f₂(X)) = E [f₁(X)f₂(X)] - E [f₁(X)]E [f₂(X)] is non-negative, provided that the expectations E[f₁(X)], E [f₂(X)] and E [f₁(X)f₂(X)] exist.

Lemma 2.

(Euler–Maclaurin summation formula.) Let m < n be positive integers and let f be a real-valued function on [m,n]. Then we have(i) if f has a continuous derivative on [m,n],

∑ k = m n f (k) = ∫ m n f (x) dx + 12 (f (m) + f (n)) + ∫ m n f ′ (x) (x - ⌊ x ⌋ - 12) dx;

(ii) if f has a continuous derivative of order 4,

∑ k = m n f (k) = ∫ m n f (x) dx + 12 (f (m) + f (n)) + 112 (f ′ (n) - f ′ (m)) + R (f),

where the remainder $R (f) = 124 ∫ m n f (4) (x) (B 4 - B 4 (x - ⌊ x ⌋)) dx$ in which $B 4 (x) = x 4 - 2 x 3 + x 2 - 130$ is the Bernoulli polynomial and B₄ = B₄(0) = - 1/30 the Bernoulli number.

Theorem 3.

For arbitrary marginals F and G with densities F^′= f and G^′= g, we have (i) the distribution H_n of (9) converges weakly to H₊ as n → ∞, and (ii) the correlation of H_n converges to that of H₊ as n → ∞, provided that F and G have finite variances.

Proof.

Note that the support of (9) is contained in the region bounded by the two curves $G (y) = F (x) + 1 n$ and $G (y) = F (x) - 1 n .$ Therefore for any (x,y) in the region G (y) > F (x), we have Pr(X_n< x, Y_n> y) = 0 for all large n (n ≥ (G (y) - F (x))^-1). This implies that H_n(x,y) = Pr(X_n≤ x, Y_n≤ y)= Pr(X_n≤ x) - Pr (X_n≤ x, Y_n> y) = Pr (X_n≤ x) = F (x) = min {F (x),G (y)} = H₊(x,y) for all n ≥ (G (y) - F (x))^-1. Likewise, for each (x,y) in the region G (y) < F(x) we have H_n(x,y) = G (y) = min {F (x),G (y)} = H₊(x,y) for all large n. Finally, for each (x,y) with G (y) = F(x), $F (x) - 1 n ≤ H n (x, y) ≤ F (x)$ for all n, and $lim n → ∞ H n (x, y) = F (x) = G (y) = H + (x, y) .$ All together, we see that in all three cases, we have $lim n → ∞ H n (x, y) = H + (x, y)$ . This proves part (i). Part (ii) follows from the next lemma which can be proved by using Hölder’s inequality (see, e.g., the proof of Theorem 4 in Dou et al. 2013). □

Lemma 3.

Let (X_n,Y_n) ∼ H_n be a sequence of bivariate random variables and X_n∼ F, Y_n∼ G for all n ≥ 1. Assume that (X_n,Y_n) converges in distribution to (X₀,Y₀) ∼ H₀ as n tends to infinity. If, in addition, E[|X_n|^{p + q}], E[|Y_n|^{p + q}] < ∞ for some positive integers p and q, then

lim n → ∞ E [X n p Y n q] = E [X 0 p Y 0 q] .

Remarks 3.

We wondered if the convergence of Theorem 3(ii) is monotone. Indeed, ρ (H_m) ≥ ρ (H_n) if m is a multiple of n (say, m = k n). To see this, write from (9)

E (X n Y n) = n ∑ i = 1 n ∫ (i - 1) / n i / n F - 1 (t) dt ∫ (i - 1) / n i / n G - 1 (t) dt ≡ n ∑ i = 1 n A i B i,

where

A i B i = ∑ j = 1 k ∫ (i - 1) / n + (j - 1) / (kn) (i - 1) / n + j / (kn) F - 1 (t) dt ∑ j = 1 k ∫ (i - 1) / n + (j - 1) / (kn) (i - 1) / n + j / (kn) G - 1 (t) dt ≡ ∑ j = 1 k a ij ∑ j = 1 k b ij ≤ k ∑ j = 1 k a ij b ij,

by Chebyshev’s sum inequality (see, e.g., Dou et al. 2013, Lemma 1). This in turn implies that E (X_mY_m) ≥ E(X_nY_n) and hence ρ (H_m) ≥ ρ(H_n). That the sequence ${ρ (H n)} n = 1 ∞$ fails to be monotonically increasing can be seen from the following counterexample. Let f = g,f (x) = 1/6 if 0 ≤ a ≤ |x| ≤ a + 3, and f (x) = 0 otherwise. We have ρ (H₃) < ρ (H₂) for $a > (- 1 + 6) / 2$ .

For certain marginals, the rate of convergence of Theorem 3(ii) can be determined (Lin and Huang 2011 and Theorem 4 below). The calculation is based on the following. Note that the correlation of two random variables is location-scale invariant.

Lemma 4.

For positive integer n > 1, we have (i) $∑ k = 1 n k ln k = 12 n 2 ln n - 14 n 2 + 12 n ln n + 112 ln n + R 1 (n), where | R 1 (n) | < 512;$ (ii) $∑ k = 1 n k 2 (ln k) 2 = 13 n 3 (ln n) 2 - 29 n 3 ln n + 227 n 3 + 12 n 2 (ln n) 2 + 16 n (ln n) 2 + 16 n ln n + R 2 (n)$ , where R₂(n) = (1) as n → ∞; (iii) $∑ k = 1 n k (k + 1) (ln k) ln (k + 1) = 13 n 3 (ln n) ln (n + 1) - 19 n 3 (ln n + ln (n + 1)) + 227 n 3 + n 2 (ln n) ln (n + 1) - 14 n 2 ln (n + 1) - 112 n 2 ln n + 23 n (ln n) ln (n + 1) + 19 n 2 + 112 (ln n) ln (n + 1) + 14 n ln n + 112 (n + 1) ln (n + 1) - 1136 n - 112 (ln n) 2 + 536 ln (n + 1) + R 3 (n), where R 3 (n) = O (1)$ as n → ∞.

Remarks 4.

Recall that the Euler constant $γ = lim n → ∞ ∑ k = 1 n 1 / k - ln n ≈ 0.57721 .$ Interestingly, each of the three remainder terms in Lemma 4 also converges to a real constant as n → ∞, say $lim n → ∞ R i (n) = R i, i = 1, 2, 3 .$ In fact, R₁ = lnA ≈ 0.24875, where A ≈ 1.28243 is the so-called Glaisher–Kinkelin constant, and lnA can be represented as

ln A = 14 + 124 ∫ 1 ∞ f 1 (4) (x) h (x) dx = 14 + 124 ∑ k = 1 ∞ ∫ 01 f 1 (4) (x + k) [- x 2 (x - 1) 2] dx = 14 + 112 ∑ k = 1 ∞ {6 k + 3 - (6 k 2 + 6 k + 1) ln (1 + 1 / k)},

where f₁(x) = x lnx and h (x) = B₄- B₄(x - ⌊ x ⌋). Similarly,

R 2 = - 227 + 124 ∫ 1 ∞ f 2 (4) (x) h (x) dx ≈ - 0.06576, R 3 = 118 ln 2 + 13108 + π 2 72 + 124 ∫ 1 ∞ f 3 (4) (x) h (x) dx = 0.295956 + 124 ∫ 1 ∞ f 3 (4) (x) h (x) dx ≈ 0.303,

where f₂(x) = x²(lnx)² and f₃(x) = x (x + 1)(lnx) ln(x + 1).

Theorem 4.

(i) If F is uniform and if G is a power function distribution, then the convergence rate of ρ (h_n) in (9) is 1/n² as n → ∞. (ii) If F = U(0,1) and if G is exponential, then the convergence rate of ρ (h_n) is (lnn)/n² as n → ∞. More precisely, $ρ (h n) = 3 / 2 - (23) - 1 (ln n) / n 2 + O (n - 2)$ as n → ∞. (iii) If F = U(0,1) and if G is logistic, then the convergence rate of ρ (h_n) is (lnn)/n² as n → ∞. More precisely, ρ (h_n) = 3/π - π^{- 1}(lnn)/n²+ (n^{- 2}) as n → ∞. (iv) If F = G is exponential, then ρ(h_n) = 1+(n^{- 1}) as n → ∞.

It is interesting to characterize the FGM and Sarmanov–Lee distributions by minimizing the χ² divergence. Let h be the joint density of X and Y with marginal densities f = F^′ and g = G^′. Define the χ² divergence (distance) between the joint density h and the product density fg (of independent random variables) by

χ 2 (h; f, g) = ∫ ∫ S F × S G h (x, y) f (x) g (y) - 1 2 f (x) g (y) dxdy,

(10)

where S_F and S_G are the supports of F and G, respectively.

Nelsen (1994) obtained a characterization of the FGM distributions by minimizing the χ² divergence (10). Huang and Lin (2011) extended Nelsen’s (1994) result to the case of Sarmanov–Lee distributions. For i=1,2, consider the functions $θ i ∗ : [0, 1] → [- 1, 1]$ satisfying

∫ 01 θ i ∗ (u) du = 0, ∫ 01 (θ i ∗ (u)) 2 du = c i > 0, sup u ∈ [0, 1] θ i ∗ (u) = 1, inf u ∈ [0, 1] θ i ∗ (u) = - h i,

(11)

where h_i∈ (0,1]. Then we have the following.

Theorem 5.

Among all absolutely continuous bivariate distributions with marginal densities f = F^′ and g = G^′, the one whose joint density is closest to the product density of independent random variables (in the sense of minimizing the χ² divergence) subject to the constraint $E θ 1 ∗ (F (X)) θ 2 ∗ (G (Y)) = c 0,$ where $- 1 ≤ c 0 c 1 c 2 ≤ max {h 1, h 2} - 1$ and $θ i ∗, c i, h i, i = 1, 2,$ are given in (11), is the Sarmanov–Lee distribution having joint density

h (x, y) = f (x) g (y) 1 + c 0 c 1 c 2 θ 1 ∗ (F (x)) θ 2 ∗ (G (y)), x, y ∈ R .

The Sarmanov–Lee distribution and its generalization have been used in actuarial science, financial markets, electrical engineering and quantum statistical mechanics (see the references in Lin and Huang 2011). For example, Hernández-Bastida and Fernández-Sánchez (2012) applied a Sarmanov–Lee family to the Bayes premium in a collective risk model. In the analysis of longitudinal data, Cole et al. (1995) used a Sarmanov–Lee bivariate distribution for transition probabilities in a two-state Markov model and developed an empirical Bayes estimation methodology. Recently, Pelican and Vernic (2013a; 2013b) studied the parameter estimation problems for the Sarmanov–Lee distribution.

5 Baker’s distributions

Write X_k,n∼ F_k,n for the k th smallest order statistic of the random sample ${X i} i = 1 n$ from F. Likewise, let Y_k,n and G_k,n stand for another sequence Y_i∼ G, ${Y i} i = 1 n$ independent of ${X i} i = 1 n$ . For maximal correlation, Baker (2008) proposed the bivariate distribution

H + (n) (x, y) = 1 n ∑ k = 1 n F k, n (x) G k, n (y), x, y ∈ R,

(12)

and for the minimum,

H - (n) (x, y) = 1 n ∑ k = 1 n F k, n (x) G n - k + 1, n (y), x, y ∈ R .

(13)

Clearly, both $H + (n)$ and $H - (n)$ satisfy the constraint of having F and G as the marginals. The following convex combination of $H + (n)$ (or $H - (n)$ ), n = 1,2,

H q ± (x, y) = (1 - q) H ± (1) (x, y) + qH ± (2) (x, y) = F (x) G (y) {1 ± q F ¯ (x) G ¯ (y)}, x, y ∈ R, q ∈ [0, 1],

is an FGM distribution.

As a generalization of (12) and (13), Baker (2008) also introduced

H r (n) (x, y) = ∑ k = 1 n ∑ ℓ = 1 n r k, ℓ F k, n (x) G ℓ, n (y), x, y ∈ R,

(14)

where (r_k,ℓ) = r ∈ which consists of all r satisfying: r_k,ℓ≥ 0 and $∑ k = 1 n r k, ℓ = ∑ ℓ = 1 n r k, ℓ = 1 / n$ for all k,ℓ = 1,2,…,n; namely, n r is a doubly stochastic matrix.

If P {(X_n,Y_n) = (X_k,n,Y_ℓ,n)}=r_k,ℓ for all k,ℓ, then $(X n, Y n) ∼ H r (n) .$ A simple application of majorization theory leads to the inequality:

H - (x, y) ≤ H - (n) (x, y) ≤ H r (n) (x, y) ≤ H + (n) (x, y) ≤ H + (x, y), x, y ∈ R,

and hence

Cov (H -) ≤ Cov (H - (n)) ≤ Cov (H r (n)) ≤ Cov (H + (n)) ≤ Cov (H +),

ρ (H -) ≤ ρ (H - (n)) ≤ ρ (H r (n)) ≤ ρ (H + (n)) ≤ ρ (H +),

provided the variances of F and G are positive and finite.

Unlike the Sarmanov–Lee, all Baker’s distributions retain the same support as that of F×G (and so does FGM). The former also admits discrete F and G.

For (X,Y) ∼ H with continuous marginals F and G, let ρ_s(H)(= 12 E [F (X)G (Y)]-3) and τ(H)(= 4 E [H (X,Y)]-1) be its Spearman’s rho and Kendall’s tau coefficients, respectively (see, e.g., Joe 2001, pp. 31-32). Baker (2008) proved that $lim n → ∞ ρ s (H + (n)) = 1$ and $lim n → ∞ τ (H + (n)) = 1$ . On the other hand, Lin and Huang (2010) found conditions under which Pearson’s correlation $ρ (H + (n))$ converges to ρ(H₊) as n → ∞. It is also possible to calculate the convergence rate of $ρ (H + (n))$ for some specific marginals. For instance, we have

Theorem 6.

(i) If F is uniform and if G a power function distribution, then the convergence rate of $ρ (H + (n))$ in (12) is 1/n as n → ∞. (ii) If F = U (0,1) and if G is exponential, then the convergence rate of $ρ (H + (n))$ is 1/n as n → ∞. More precisely, $ρ (H + (n)) = 3 / 2 - 3 / n + O (n - 2)$ as n → ∞. (iii) If F = U(0,1) and if G is logistic, then the convergence rate of $ρ (H + (n))$ is 1/n as n → ∞. More precisely, $ρ (H + (n)) = 3 π - 1 - 6 π - 1 / n + O (n - 2)$ as n → ∞. (iv) If F = G is exponential, then the convergence rate of $ρ (H + (n))$ is (lnn)/n as n→ ∞. More precisely, $ρ (H + (n)) = 1 - 1 n ∑ k = 1 n 1 / k = 1 - (ln n) / n + O (n - 1)$ as n → ∞.

For $H + (n)$ in (12), Dou et al. (2013) established the weak convergence and the product-moment convergence as well as the TP ₂ property described below. The results for $H - (n)$ can be derived similarly and are omitted. The next theorem is of interest in its own right.

Theorem 7.

Let $(X n, Y n) ∼ H + (n)$ in (12) with marginal densities f and g, and let $U n = n (G (Y n) - F (X n))$ . Then, as n → ∞, (X_n,U_n) converges in distribution to $(X ~, U ~)$ having joint density

k (x, u) = 1 2 π F (x) F ¯ (x) exp - u 2 4 F (x) F ¯ (x) f (x) .

The conditional distribution of $U ~$ given $X ~ = x$ is the normal distribution with mean 0 and variance $2 F (x) F ¯ (x)$ , namely,

U ~ | X ~ = x ∼ N (0, 2 F (x) F ¯ (x)) .

By Theorem 7, we see that for uniform marginals F = G = U (0,1), Baker’s distribution $H + (n)$ converges weakly to H₊ (with support on the diagonal line y = x) as n → ∞. This can be further extended to the following general case by using the monotone property of distribution functions.

Theorem 8.

For general marginals F and G, Baker’s distribution $H + (n)$ converges weakly to the Fréchet–Hoeffding upper bound H₊ as n tends to infinity.

Theorem 9.

Let $(X n, Y n) ∼ H + (n)$ with general marginals F and G, and let $(X ~, Y ~) = (F - 1 (Z), G - 1 (Z))$ , where Z ∼ U (0,1). If, in addition, E [|X_n|^{p + q}], E [|Y_n|^{p + q}] < ∞ for some positive integers p and q, then

lim n → ∞ E [X n p Y n q] = E [(X ~) p (Y ~) q] = ∫ 01 F - 1 (t) p G - 1 (t) q dt .

Corollary 1.

If F and G have finite non-zero variances, $lim n → ∞ ρ (H + (n)) = ρ (H +)$ .

The main tool in the proof of Theorem 7 is the following generalization of the so-called local limit theorem for binomial distribution, which is also of interest in itself.

Lemma 5.

Let 0 < p,q < 1 be constants such that p + q = 1, and let u ∈ R be a constant. Let ψ be a function such that ψ (n) = o (n^1/6) as n → ∞. Write $k = np + y k npq ∈ {0, 1, 2, …, n}$ . Then, as n → ∞,

n k p + u n k q - u n n - k ≈ 1 2 π npq exp - 12 y k - u pq 2

(15)

(asymptotically) uniformly in y_k such that |y_k| ≤ ψ (n). Namely, the left-hand side a_n(k) and the right-hand side b_n(y_k) in (15) satisfy

sup {k : | y k | ≤ ψ (n)} | a n (k) / b n (y k) - 1 | → 0 as n → ∞ .

Recall that a real-valued function k on R² is totally positive of order two (TP ₂), a notion of strong positive dependence, if k (x₁,y₁)k (x₂,y₂)≥ k(x₁,y₂)k (x₂,y₁) for all x₁≤ x₂, y₁≤ y₂.

Theorem 10.

For general distributions F and G, Baker’s distribution $H + (n)$ is TP ₂.

Finally, if we allow the two sample sizes in (14) to be different, say m and n, then the resulting bivariate distribution is

H r (m, n) (x, y) = ∑ k = 1 m ∑ ℓ = 1 n r k, ℓ F k, m (x) G ℓ, n (y) = C (F (x), G (y); r),

(16)

where C is a Bernstein copula and the parameters r_k,ℓ satisfy

∑ ℓ = 1 n r k, ℓ = 1 m, ∑ k = 1 m r k, ℓ = 1 n, r k, ℓ ≥ 0, 1 ≤ k ≤ m, 1 ≤ ℓ ≤ n .

Because of the Weierstrass approximation theorem, we can use $H r (m, n)$ in (16) to approximate smooth bivariate distributions. Moreover, since (16) is a finite mixture distribution, the EM algorithm applies to the estimation of the parameters r_k,ℓ of $H r (m, n) .$ Dou et al. (2014) took these advantages and succeeded in fitting Baker’s distributions to some practical data sets including the Illinois state education data.

6 Bayramoglu’s distributions

In this section, we extend Baker’s distributions by an alternative approach. Starting with any joint distribution H with marginals F and G (e.g., the FGM distribution (4)), let (X₁,Y₁),…,(X_n,Y_n) be a random sample of size n from H. As before, sorting ${X ℓ} ℓ = 1 n$ and ${Y ℓ} ℓ = 1 n$ , we obtain the order statistics X_1,n≤ X_2,n≤ ⋯ ≤ X_n,n and Y_1,n≤ Y_2,n≤ ⋯ ≤ Y_n,n, respectively, and again, write X_r,n∼ F_r,n, Y_s,n∼ G_s,n.

Instead of Baker’s $H + (n)$ and $H - (n)$ , Bairamov and Bayramoglu (2013) proposed

K + (n) (x, y) = 1 n ∑ r = 1 n Pr (X r, n ≤ x, Y r, n ≤ y), x, y ∈ R,

(17)

K - (n) (x, y) = 1 n ∑ r = 1 n Pr (X r, n ≤ x, Y n - r + 1, n ≤ y), x, y ∈ R,

(18)

both having marginals F and G. We see that if H (x,y) = F (x) G (y) for all x,y, namely, if X and Y are independent, then Pr(X_r,n≤ x, Y_s,n≤ y) = F_r,n(x)G_s,n(y) for all x,y, and hence (17) and (18) reduce to (12) and (13), respectively. Therefore, the generalization admits a wider range of correlation.

When X and Y are not independent, the computation of the joint distribution of bivariate order statistics is complicated (see David 1981, p. 26, or David and Nagaraja 2003, p. 25):

K r, s (n) (x, y) ≡ Pr (X r, n ≤ x, Y s, n ≤ y) = ∑ i = r n ∑ j = s n ∑ k f k, i, j (n) (x, y), x, y ∈ R,

where k takes all integers such that k, i - k, j - k, n - i - j + k ≥ 0, the summand

f k, i, j (n) (x, y) = n! k! (i - k)! (j - k)! (n - i - j + k)! H k (x, y) (F (x) - H (x, y)) i - k · (G (y) - H (x, y)) j - k (H ¯ (x, y)) n - i - j + k, if the exponents k, i - k, j - k, n - i - j + k ≥ 0,

and $f k, i, j (n) (x, y) = 0$ , otherwise. Here $H ¯ (x, y) ≡ Pr (X > x, Y > y) = 1 - F (x) - G (y) + H (x, y)$ .

Huang et al. (2013) established the following properties of $K r, s (n), K + (n)$ and $K - (n)$ . Hereafter, we shall consider x,y as fixed, and for simplicity, write F,G, H and $H ¯$ for F (x),G (y), H (x,y) and $H ¯ (x, y)$ , respectively.

Theorem 11.

For fixed marginals F, G and 1 ≤ r, s ≤ n, the joint distribution $K r, s (n)$ of bivariate order statistics (X_r,n,Y_s,n) is increasing in H. More precisely, for given x,y,F and G, if H₁(x,y) ≤ H₂(x,y), then $K r, s, 1 (n) (x, y) ≤ K r, s, 2 (n) (x, y)$ , where $K r, s, i (n), i = 1, 2,$ is the distribution of (X_r,n,Y_s,n) generated from H_i with marginals F and G.

Theorem 12.

For given H with marginals F and G, Bayramoglu’s distribution $K + (n)$ in (17) is increasing in n. More precisely,

K + (n) = K + (n - 1) + 1 n - 1 ∑ i = 1 ⌊ n / 2 ⌋ n - 1 i, i - 1, n - 2 i (F - H) i (G - H) i H + H ¯ n - 2 i, n ≥ 2 .

(19)

Theorem 13.

For given H with marginals F and G, Bayramoglu’s distribution $K - (n)$ in (18) is decreasing in n. More precisely,

K - (n) = K - (n - 1) - 1 n - 1 ∑ i = 1 ⌊ n / 2 ⌋ n - 1 i, i - 1, n - 2 i (H H ¯) i (1 - H - H ¯) n - 2 i, n ≥ 2 .

(20)

Recall that the bivariate distribution H with marginals F and G is positive quadrant dependent (PQD), if H ≥ F G, that is, H (x,y) ≥ F (x) G(y) for all x,y, and that H is negative quadrant dependent (NQD) if H ≤ F G. As immediate consequences of Theorems 11–13, we have the following corollary (assuming $σ X 2, σ Y 2 ∈ (0, ∞)$ if necessary).

Corollary 2.

(i) For fixed n and marginals F, G, both $K + (n)$ and $K - (n)$ are increasing in H;(ii) $ρ (K + (n)) ≥ ρ (H + (n))$ if H is PQD, and $ρ (K - (n)) ≤ ρ (H - (n))$ if H is NQD;(iii) $K + (2) = FG$ if H = H_-, and in general, $0 ≤ Cov (K + (2)) ≤ Cov (K + (3)) ≤ ⋯$ ;(iv) $K - (2) = FG$ if H = H₊, and in general, $0 ≥ Cov (K - (2)) ≥ Cov (K - (3)) ≥ ⋯$ .

Remarks 5.

The first parts of Corollaries 2(iii) and 2(iv) can be proved directly without invoking the identities (19) and (20), but the proof is complicated. For instance, that of Corollary 2(iv) can be proceeded as follows. Let Z₁ and Z₂ be two independent copies of Z ∼ U (0,1), and let Z_1,2≤ Z_2,2 be the corresponding order statistics. Then (X_i,Y_i) = (F^-1(Z_i), G^{- 1}(Z_i)), i = 1,2, are independent and have common distribution H₊(x,y) = min {F (x), G (y)}, x,y ∈ R. In this case, we have

K - (2) (x, y) = 12 Pr (X 1, 2 ≤ x, Y 2, 2 ≤ y) + Pr (X 2, 2 ≤ x, Y 1, 2 ≤ y) = 12 Pr (F - 1 (Z 1, 2) ≤ x, G - 1 (Z 2, 2) ≤ y) + Pr (F - 1 (Z 2, 2) ≤ x, G - 1 (Z 1, 2) ≤ y) = 12 Pr (Z 1, 2 ≤ F (x), Z 2, 2 ≤ G (y)) + Pr (Z 2, 2 ≤ F (x), Z 1, 2 ≤ G (y)) = 12 Pr (Z 1 ≤ F (x), Z 2 ≤ G (y) | Z 1 ≤ Z 2) + Pr (Z 1 ≤ F (x), Z 2 ≤ G (y) | Z 1 > Z 2) = Pr (Z 1 ≤ F (x), Z 2 ≤ G (y)) = F (x) G (y), x, y ∈ R,

the penultimate equality following from the fact that Pr(Z₁≤ Z₂) = Pr(Z₁> Z₂) = 1/2 and the law of total probability.

In the rest of this section, we will mention some new results (Theorems 14–17) regarding (a) weak convergence, (b) product-moment convergence and (c) convergence of correlations (including Pearson’s correlation, Spearman’s rho and Kendall’s tau) of Bayramoglu’s distributions. The proofs are given in the Appendix: Proof of the new results in Section 6. For proving the weak convergence, we offer an alternative approach different from that (Theorem 7) for Baker’s distributions.

Theorem 14.

Let H be any bivariate distribution with marginals F and G. (i) Bayramoglu’s distribution (17) converges weakly to H₊ as n → ∞. Moreover, $lim n → ∞ ρ (K + (n)) = ρ (H +)$ , provided both F and G have finite nonzero variances. (ii) Bayramoglu’s distribution (18) converges weakly to H_- as n → ∞. Moreover, $lim n → ∞ ρ (K - (n)) = ρ (H -)$ , provided both F and G have finite nonzero variances.

Theorem 15.

Let (X,Y) ∼ H with marginals F and G and E [|X|^{p + q}],E [|Y|^{p + q}] < ∞ for some positive integers p and q. (i) If $(X n, Y n) ∼ K + (n)$ and $(X ~, Y ~) = (F - 1 (Z), G - 1 (Z)) ∼ H +$ , then

lim n → ∞ E [X n p Y n q] = E [(X ~) p (Y ~) q] = ∫ 01 F - 1 (t) p G - 1 (t) q dt .

(ii) If $(X n ∗, Y n ∗) ∼ K - (n)$ and $(X ∗ ~, Y ∗ ~) = (F - 1 (Z), G - 1 (1 - Z)) ∼ H -$ , then

lim n → ∞ E [(X n ∗) p (Y n ∗) q] = E [(X ∗ ~) p (Y ∗ ~) q] = ∫ 01 F - 1 (t) p G - 1 (1 - t) q dt .

For non-negative random variables X and Y, we can consider the general functions α, β of X, Y, respectively.

Theorem 16.

Let (X,Y) ∼ H, defined on $R + 2 ≡ [0, ∞) × [0, ∞),$ and let X ∼ F, Y ∼ G. Let α and β be two increasing and left-continuous functions on R₊. Assume also that E [|α (X) β (Y)|], E[|α (X)|] and E [|β (Y)|] are finite. (i) If $(X n, Y n) ∼ K + (n)$ and $(X ~, Y ~) = (F - 1 (Z), G - 1 (Z)) ∼ H +$ , then

lim n → ∞ E [α (X n) β (Y n)] = E [α (X ~) β (Y ~)] = ∫ 01 α (F - 1 (t)) β (G - 1 (t)) dt .

(ii) If $(X n ∗, Y n ∗) ∼ K - (n)$ and $(X ∗ ~, Y ∗ ~) = (F - 1 (Z), G - 1 (1 - Z)) ∼ H -$ , then

lim n → ∞ E [α (X n ∗) β (Y n ∗)] = E [α (X ∗ ~) β (Y ∗ ~)] = ∫ 01 α (F - 1 (t)) β (G - 1 (1 - t)) dt .

For Spearman’s ρ_s and Kendall’s τ, we have

Theorem 17.

If H has continuous marginals F and G then (i) $lim n → ∞ ρ s (K + (n)) = 1$ and $lim n → ∞ ρ s (K - (n)) = - 1$ ; (ii) $lim n → ∞ τ (K + (n)) = 1$ and $lim n → ∞ τ (K - (n)) = - 1$ .

Remarks 6.

By the monotone property of $K + (n)$ and $K - (n)$ (Theorems 12 and 13), we can further conclude, in addition to Theorems 14 and 17, that all the sequences ${ρ (K + (n))}$ , ${ρ (K - (n))}$ , ${ρ s (K + (n))}$ , ${ρ s (K - (n))}$ , ${τ (K + (n))}$ and ${τ (K - (n))}$ are monotone (see, e.g., Joe 2001, p. 54).

To prove the new results, we need two more lemmas. Lemma 6 below is an extension of the classic Hoeffding’s (1940) identity (2) for Cov (X,Y) to Cov (α (X),β (Y)), and Lemma 7 gives an explicit form of the expectation E [α (X) β (Y)] for (X,Y) ∼ H on $R + 2$ . Our assumptions on the marginals (F,G) are weaker than those in Cuadras (2002), pp. 19-20, and the proof differs from that of Beare (2009).

Lemma 6.

Let (X,Y) ∼ H, X ∼ F and Y ∼ G. Let α,β : R→R be two left-continuous functions and be of bounded variation on each compact subset of R. Further, assume that the expectations E [|α (X) β (Y)|], E [|α (X)|], E[|β (Y)|] < ∞. Then

Cov (α (X), β (Y)) = E [α (X) β (Y)] - E [α (X)] E [β (Y)] = ∫ - ∞ ∞ ∫ - ∞ ∞ [H ¯ (x, y) - F ¯ (x) G ¯ (y)] d α (x) d β (y) = ∫ - ∞ ∞ ∫ - ∞ ∞ [H (x, y) - F (x) G (y)] d α (x) d β (y) .

Lemma 7.

Let (X,Y) ∼ H defined on $R + 2$ and X ∼ F, Y ∼ G. Let α and β be two increasing and left-continuous functions on R₊. Then

E [α (X) β (Y)] = ∫ 0 ∞ ∫ 0 ∞ H ¯ (x, y) dα (x) dβ (y) - α (0) β (0) + α (0) E [β (Y)] + β (0) E [α (X)],

(21)

provided the expectations exist.

Corollary 3.

Let (X,Y) ∼ H defined on $R + 2$ , and assume that the expectations E[X^rY^s], E [X^r], E [Y^s] < ∞ for some real numbers r,s > 0. Then

E [X r Y s] = rs ∫ 0 ∞ ∫ 0 ∞ H ¯ (x, y) x r - 1 y s - 1 dxdy.

Lemma 7 is a strengthening of Theorem 3.1 of Gupta et al. (2008). Then as a result, our corollary here extends their Corollary 2 in that no assumption of any smoothness of the bivariate distribution H is required.

7 Other related distributions

In addition to the above Sarmanov–Lee and Baker approaches, we mention some others.

For constructing bivariate distributions H with marginals F and G, Johnson and Tenenbein (1981) proposed the following steps:(i) Let $X 1 ′$ and $X 2 ′$ be two i.i.d. random variables with a density k.(ii) Define the correlated random variables

(X 1, X 2) = (X 1 ′, α X 1 ′ + (1 - α) X 2 ′), α ∈ [0, 1] .

(iii) Find the copula C (u,v) of (X₁,X₂), which is a function of α and k. (iv) Define H (x,y) = C (F (x),G (y)), x,y ∈ R.

In general, the constructed copula C in (iii) is too complicated to have a closed form.

Takeuchi (2010) suggested using FGM distributions for low correlation, and the Gaussian copula for high correlation:

C N (u, v; ρ) = Φ 2 (Φ - 1 (u), Φ - 1 (v); ρ), u, v ∈ [0, 1],

where Φ₂ is a two-dimensional normal distribution with correlation ρ, and Φ^-1 is the quantile function of the standard normal distribution. For example, for constructing a bivariate (FIR-FUV) galaxy luminosity distribution, he proposed H (x,y) = C_N(F (x),G (y); ρ), x,y ∈ R, where X ∼ F is far-infrared luminosity and Y ∼ G is far-ultraviolet luminosity. Note that there is no explicit form for the quantile function Φ^{- 1}.

Larralde (2012) gave the maximum-entropy bivariate distribution with normal marginals. Dukic and Marić (2013) considered the convex combinations of the independent case FG and the extreme H₊ or H_-. But these are not what we really want because both H₊ and H_- are composed of singular distributions.

In conclusion, the most convenient unified approach to the above-mentioned problem is probably by way of a linear combination of the joint distributions of bivariate order statistics.

Appendix: Proof of the new results in Section 6

Proof of Lemma 6.

Let (X₁,Y₁) and (X₂,Y₂) be two independent copies of (X,Y). Then

E [α (X i) β (Y i)] = ∫ - ∞ ∞ ∫ - ∞ ∞ α (x) β (y) dH (x, y) for i = 1, 2,

and

E [α (X i) β (Y j)] = ∫ - ∞ ∞ ∫ - ∞ ∞ α (x) β (y) dF (x) G (y) for i ≠ j .

Define the function I on R² by

I (u, x) = 1, if u < x, 0, otherwise

(note that the function I here is a slight modification of those defined in Lehmann 1966, Lemma 2, and Quesada-Molina 1992, Theorem 2.1, and the references therein). Since α is left-continuous on R and is of bounded variation on each compact subset of R, we have

∫ - ∞ ∞ [I (u, x 1) - I (u, x 2)] d α (u) = - ∫ [x 1, x 2) d α (u) = α (x 1) - α (x 2) if x 1 ≤ x 2,

and

∫ - ∞ ∞ [I (u, x 1) - I (u, x 2)] dα (u) = ∫ [x 2, x 1) dα (u) = α (x 1) - α (x 2) if x 2 ≤ x 1 .

Hence, in either case, $∫ - ∞ ∞ [I (u, x 1) - I (u, x 2)] dα (u) = α (x 1) - α (x 2),$ and

∫ - ∞ ∞ ∫ - ∞ ∞ [I (u, x 1) - I (u, x 2)] [I (v, y 1) - I (v, y 2)] dα (u) dβ (v) = [α (x 1) - α (x 2)] [β (y 1) - β (y 2)] .

On the other hand, note that E [I (u,X_i) I (v,Y_j)] = Pr(X_i> u,Y_j> v). This implies that

E [I (u, X i) I (v, Y j)] = Pr (X > u, Y > v) if i = j, and E [I (u, X i) I (v, Y j)] = Pr (X > u) Pr (Y > v) if i ≠ j.

Therefore, we have

2 {E [α (X) β (Y)] - E [α (X)] E [β (Y)]} = E [α (X 1) β (Y 1) - α (X 1) β (Y 2) - α (X 2) β (Y 1) + α (X 2) β (Y 2)] = E ∫ - ∞ ∞ ∫ - ∞ ∞ {I (u, X 1) - I (u, X 2)} {I (v, Y 1) - I (v, Y 2)} d α (u) d β (v) = ∫ - ∞ ∞ ∫ - ∞ ∞ E {I (u, X 1) - I (u, X 2)} {I (v, Y 1) - I (v, Y 2)} d α (u) d β (v) = ∫ - ∞ ∞ ∫ - ∞ ∞ 2 Pr (X > u, Y > v) - Pr (X > u) Pr (Y > v) d α (u) d β (v) = 2 ∫ - ∞ ∞ ∫ - ∞ ∞ [H ¯ (x, y) - F ¯ (x) G ¯ (y)] d α (x) d β (y) = 2 ∫ - ∞ ∞ ∫ - ∞ ∞ [H (x, y) - F (x) G (y)] d α (x) d β (y),

last equality following from the fact that $H (x, y) - F (x) G (y) = H ¯ (x, y) - F ¯ (x) G ¯ (y)$ .

Proof of Lemma 7.

By Lemma 6, we have

E [α (X) β (Y)] - E [α (X)] E [β (Y)] = ∫ 0 ∞ ∫ 0 ∞ H ¯ (x, y) - F ¯ (x) G ¯ (y) d α (x) d α (y) = ∫ 0 ∞ ∫ 0 ∞ H ¯ (x, y) d α (x) d β (y) - ∫ 0 ∞ F ¯ (x) d α (x) ∫ 0 ∞ G ¯ (y) d β (y) .

(22)

On the other hand, we have

E [α (X)] = ∫ 0 ∞ α (x) dF (x) = - ∫ 0 ∞ α (x) d F ¯ (x) = - lim b → ∞ ∫ [0, b] α (x) d F ¯ (x) = - lim b → ∞ α (b) F ¯ (b) - α (0 -) F ¯ (0 -) - ∫ 0 b F ¯ (x) d α (x) = α (0) + ∫ 0 ∞ F ¯ (x) d α (x) .

(23)

The last equality is due to the assumption E [|α (X)|] < ∞ and the monotone property of the function α in case $lim x → ∞ α (x) = ∞$ . Similarly,

E [β (Y)] = β (0) + ∫ 0 ∞ G ¯ (y) d β (y) .

(24)

Therefore, it follows from (23) and (24) that

∫ 0 ∞ F ¯ (x) d α (x) ∫ 0 ∞ G ¯ (y) d β (y) = {E [α (X)] - α (0)} {E [β (Y)] - β (0)} = E [α (X)] E [β (Y)] - α (0) E [β (Y)] - β (0) E [α (X)] + α (0) β (0) .

(25)

Combining (22) and (25), we prove the identity (21).

Proof of Theorem 14.

We first prove the following two facts.

Fact A. If H = H_- with uniform marginals,

lim n → ∞ Cov (K + (n)) = 112 = Cov (H +) .

Fact B. If H = H₊ with uniform marginals,

lim n → ∞ Cov (K - (n)) = - 112 = Cov (H -) .

Proof of Fact A. For F = G = U (0,1), and H (x,y) = H_-(x,y) = max {0,x + y - 1}, we have, by (19),

K + (n) = H + ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i (x - H) i (y - H) i 1 - x - y + 2 H m - 2 i = ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i x i y i 1 - x - y m - 2 i, if x + y ≤ 1, x + y - 1 + ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i (1 - x) i (1 - y) i (x + y - 1) m - 2 i, if x + y > 1 .

Then, by (2),

Cov K + (n) = ∫ 01 ∫ 0 1 - y - xy + ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i x i y i 1 - x - y m - 2 i dxdy + ∫ 01 ∫ 1 - y 1 - (1 - x) (1 - y) + ∑ m = 2 n 1 m - 1 · ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i (1 - x) i (1 - y) i x + y - 1 m - 2 i dxdy = - ∫ 01 ∫ 0 1 - y xydxdy - ∫ 01 ∫ 1 - y 1 (1 - x) (1 - y) dxdy + ∫ 01 ∫ 0 1 - y ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i x i y i 1 - x - y m - 2 i dxdy + ∫ 01 ∫ 1 - y 1 ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i (1 - x) i (1 - y) i x + y - 1 m - 2 i dxdy = - 112 + ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i ∫ 01 ∫ 0 1 - y x i y i 1 - x - y m - 2 i dxdy + ∫ 01 ∫ 1 - y 1 (1 - x) i (1 - y) i x + y - 1 m - 2 i dxdy

(26)

= - 112 + ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i i! i! (m - 2 i)! (m + 2)! + i! i! (m - 2 i)! (m + 2)! = - 112 + 2 ∑ m = 2 n 1 (m - 1) m (m + 1) (m + 2) ∑ i = 1 ⌊ m / 2 ⌋ i .

(27)

The two double integrals in (26) are equal by changing variables and in (27) we apply the representations for generalized (or multinomial) Beta function:

∫ 01 ∫ 0 1 - y x i y i 1 - x - y m - 2 i dxdy = B (i + 1, i + 1, m - 2 i + 1) = Γ (i + 1) Γ (i + 1) Γ (m - 2 i + 1) Γ (m + 3) = i! i! (m - 2 i)! (m + 2)! .

Note that

∑ m = 2 n ∑ i = 1 ⌊ m / 2 ⌋ = ∑ m = 2, even n ∑ i = 1 ⌊ m / 2 ⌋ + ∑ m = 2, odd n ∑ i = 1 ⌊ m / 2 ⌋ = ∑ m = 2, even n ∑ i = 1 m / 2 + ∑ m = 3, odd n ∑ i = 1 (m - 1) / 2

and that

∑ i = 1 ⌊ m / 2 ⌋ i = 12 m 2 m 2 + 1 = 18 m (m + 2), m = even, 12 m - 1 2 m - 1 2 + 1 = 18 (m - 1) (m + 1), m = odd .

We have

∑ m = 2 n 1 (m - 1) m (m + 1) (m + 2) ∑ i = 1 ⌊ m / 2 ⌋ i = ∑ m = 2, even n 1 (m - 1) m (m + 1) (m + 2) m (m + 2) 8 + ∑ m = 3, odd n 1 (m - 1) m (m + 1) (m + 2) (m - 1) (m + 1) 8 = 18 ∑ m = 2, even n 1 (m - 1) (m + 1) + ∑ m = 3, odd n 1 m (m + 2) = 18 ∑ m = 2, even n 1 (m - 1) (m + 1) + ∑ m = 3, odd n - 1 1 m (m + 2), n = even, 18 ∑ m = 2, even n - 1 1 (m - 1) (m + 1) + ∑ m = 3, odd n 1 m (m + 2), n = odd = 18 ∑ k = 1 n / 2 1 (2 k - 1) (2 k + 1) + ∑ k = 1 n / 2 - 1 1 (2 k + 1) (2 k + 3), n = even, 18 ∑ k = 1 (n - 1) / 2 1 (2 k - 1) (2 k + 1) + ∑ k = 1 (n - 1) / 2 1 (2 k + 1) (2 k + 3), n = odd = 18 12 - 1 2 n + 1 + 16 - 1 2 n + 1 = 124 2 n - 1 n + 1, n = even, 18 12 - 1 2 n + 16 - 1 2 n + 2 = 124 (n - 1) (2 n + 3) n n + 2, n = odd.

The last equality follows from the two identities:

∑ k = 1 N 1 (2 k - 1) (2 k + 1) = 12 ∑ k = 1 N 1 2 k - 1 - 1 2 k + 1 = 12 - 1 2 (2 N + 1), ∑ k = 1 N 1 (2 k + 1) (2 k + 3) = 12 ∑ k = 1 N 1 2 k + 1 - 1 2 k + 3 = 16 - 1 2 (2 N + 3) .

Thus

Cov K + (n) = - 112 + 2 (124 2 n - 1 n + 1) = 112 n - 2 n + 1, n = even, - 112 + 2 (124 (n - 1) (2 n + 3) n n + 2) = 112 n 2 - n - 3 n n + 2, n = odd,

and in either case

lim n → ∞ 112 n - 2 n + 1 = 112, lim n → ∞ 112 n 2 - n - 3 n n + 2 = 112 .

Therefore, $Cov K + (n) → 112 = Cov (H +)$ as n → ∞.

Proof of Fact B. The case of $K - (n)$ from H (x,y) = H₊(x,y) = min {x,y} requires no new proof by noting that

Cov K - (n) = 112 - ∑ m = 2 n 1 m - 1 ∑ i = 1 ⌊ m / 2 ⌋ m - 1 i, i - 1, m - 2 i ∫ 01 ∫ 0 y x i (1 - y) i y - x m - 2 i dxdy + ∫ 01 ∫ y 1 y i (1 - x) i x - y m - 2 i dxdy

(28)

and that the two integrals in (28) are identical to those of (26) (by a simple change of variables).

Proof of Theorem 14 (continued).

(i) We divide the proof of part (i) into two steps:

(1a) Consider arbitrary H with uniform marginals, and prove that $K + (n)$ converges weakly to H₊ as n → ∞ and $lim n → ∞ σ (K + (n)) = σ (H +)$ .

(1b) Extend the result (1a) to any H with general fixed marginals.

Proof of (1a). For arbitrary H with uniform marginals, since ${K + (n)} n = 1 ∞$ is a bounded and increasing sequence, its limit exists, say K_{+ ∞}. Then $K + (n) ≤ K + ∞ ≤ H +$ for all n ≥ 1. Also, note that $K + (n) ≥ K + (n) (H -)$ due to Theorem 11. We have, since $K + (n)$ increases in n,

0 ≤ ∫ 01 ∫ 01 H + (x, y) - K + ∞ (x, y) dxdy = ∫ 01 ∫ 01 H + (x, y) - lim n → ∞ K + (n) (x, y) dxdy = lim n → ∞ ∫ 01 ∫ 01 H + (x, y) - K + (n) (x, y) dxdy = lim n → ∞ ∫ 01 ∫ 01 (H + (x, y) - xy) - (K + (n) (x, y) - xy) dxdy = lim n → ∞ Cov (H +) - Cov (K + (n)) ≤ lim n → ∞ Cov (H +) - Cov (K + (n) (H -)) = 0 .

The last equality follows from Fact A. Therefore $K + (n)$ converges weakly to H₊ as n → ∞ and $lim n → ∞ σ (K + (n)) = σ (H +)$ .

Proof of (1b). We now extend the result of part (1a) to any H with general fixed marginals F and G. First, let C be a copula of H. We rewrite the result of part (1a) in terms of random variables. Let $(Z i, Z i ∗), i = 1, 2, …, n$ , be n independent copies of (Z,Z^∗) ∼ C. And let Z_(k) and $Z (k) ∗$ denote the k th smallest order statistics of ${Z i} i = 1 n$ and ${Z i ∗} i = 1 n$ , respectively. On the other hand, let K_n be a discrete random variable obeying the uniform distribution on {1,2,…,n}. Then $(Z (K n), Z (K n) ∗) ∼ K + (n) (C)$ (the bivariate distribution (17) generated from C) and by part (1a), we have that as n → ∞,

(Z (K n), Z (K n) ∗) → d (Z, Z) ∼ C + (the upper bound with uniform marginals) .

Proceeding on similar lines in the proof of Theorem 3 of Dou et al. (2013), we conclude that the bivariate $(F - 1 (Z (K n)), G - 1 (Z (K n) ∗))$ converges in distribution to (F^{- 1}(Z),G^{- 1}(Z)) as n → ∞ (Serfling 1980, p. 24), namely, $K + (n)$ converges weakly to H₊ as n → ∞. This together with Corollary 2(iii) completes the proof.

(ii) We also divide the proof of part (ii) into two steps:

(2a) Consider arbitrary H with uniform marginals, and prove that $K - (n)$ converges weakly to H_- as n → ∞, and $lim n → ∞ σ (K - (n)) = σ (H -)$ .

(2b) Extend the result (2a) to any H with general fixed marginals.

Proof of (2a). For arbitrary H with uniform marginals, since ${K - (n)} n = 1 ∞$ is a bounded and decreasing sequence, its limit exists, say K_{- ∞}. Then $H - ≤ K - ∞ ≤ K - (n)$ for all n≥1. Also, note that $K - (n) ≤ K - (n) (H +)$ by Theorem 11. We have, since $K - (n)$ decreases in n,

0 ≤ ∫ 01 ∫ 01 K - ∞ (x, y) - H - (x, y) dxdy = ∫ 01 ∫ 01 lim n → ∞ K - (n) (x, y) - H - (x, y) dxdy = lim n → ∞ ∫ 01 ∫ 01 K - (n) (x, y) - H - (x, y) dxdy = lim n → ∞ ∫ 01 ∫ 01 (K - (n) (x, y) - xy) - (H - (x, y) - xy) dxdy = lim n → ∞ Cov (K - (n)) - Cov (H -) ≤ lim n → ∞ Cov (K - (n) (H +)) - Cov (H -) = 0 .

The last equality is due to Fact B. Therefore $K - (n)$ converges weakly to H_- as n → ∞. This together with Corollary 2(iv) completes the proof.

Proof of (2b). This is similar to the proof of part (1b) and is omitted.

Proof of Theorem 15.

Apply Lemma 3 to Theorem 14.

Proof of Theorem 16.

(i) Note that as n → ∞, $K + (n)$ converges weakly to H₊, the Fréchet–Hoeffding upper bound with the same marginals F and G, and that (F^{- 1}(Z), G^{- 1}(Z)) ∼ H₊. Then the required result follows from Lemma 7 above and the Generalized Monotone Convergence Theorem (see, e.g., Royden 1988, p. 265), because

E [α (X n) β (Y n)] = ∫ 0 ∞ ∫ 0 ∞ K ¯ + (n) (x, y) d α (x) d β (y) - α (0) β (0) + α (0) E [β (Y)] + β (0) E [α (X)],

and $K ¯ + (n)$ converges monotonically to $H ¯ +$ as n → ∞ (see Theorems 12 and 14(i)).

(ii) The proof is similar to that of part (i), by noting that (F^{- 1}(Z), G^{- 1}(1 - Z)) ∼ H_- and that $K ¯ - (n)$ converges monotonically to $H ¯ -$ as n → ∞ (see Theorems 13 and 14(ii)).

Proof of Theorem 17.

It suffices to consider the case of uniform marginals. Let $(X n, Y n) ∼ K + (n)$ with marginals F = G = U (0,1), and $(X n ∗, Y n ∗) ∼ K - (n)$ with marginals F = G = U (0,1).

(i) From Theorem 15 it follows that

lim n → ∞ E [F (X n) G (Y n)] = lim n → ∞ E [X n Y n] = 13, lim n → ∞ E [F (X n ∗) G (Y n ∗)] = lim n → ∞ E [X n ∗ Y n ∗] = 16 .

This in turn implies that $lim n → ∞ ρ s (K + (n)) = 12 lim n → ∞ E [X n Y n] - 3 = 1$ and $lim n → ∞ ρ s (K - (n)) = 12 lim n → ∞ E [X n ∗ Y n ∗] - 3 = - 1$ . (Note that these results also follow from Theorem 14; see, e.g., Balakrishnan and Lai 2009, p. 156.)

(ii) Let $(X ~, Y ~) = (Z, Z) ∼ H +$ . Recall that $K + (n)$ are bounded and continuous functions, and that $K + (n)$ converges increasingly to H₊ as n → ∞ (see Theorems 12 and 14). Moreover, (X_n,Y_n) converges in distribution to $(X ~, Y ~)$ as n → ∞. By the Generalized Lebesgue Dominated Convergence Theorem (see, e.g., Royden 1988, p. 270, or Hernández-Lerma and Lasserre 2000), we have

lim n → ∞ E [