Conditioning and Subsetting

Conditioning

This page introduces conditional distributions under a copula model and shows how to construct them programmatically using condition. The same interface works either on the uniform scale (copula only) or on the original scale (via SklarDist).

Overview

Take a D-variate copula $C$ and consider a split of the indices $1,..,d$ into $J=1,...,p$ and $I=p+1,..,d$ wihtout loss of generality (reorder the marignals if necessary). The conditional joint distribution of the I's given the J's is given, on the uniform scale, by the function

$$H_{I\mid J}({\mathbf{u}}_I\mid {\mathbf{u}}_J):=\frac{{\partial }^{p}C({\mathbf{u}}_I,{\mathbf{u}}_J)}{\partial {\mathbf{u}}_J}/\frac{{\partial }^{p}C({\mathbf{1}}_I,{\mathbf{u}}_J)}{\partial {\mathbf{u}}_J},$$

which is a proper distribution on $[0,1{]}^{|I|}$. Each conditional marginal (uniform scale) can moreover be expressed as a “distortion” $H_{i|J}(·|u_J)$, which is the distribution function of a random variable with support $[0,1]$ (but non-uniform):

$$H_{i\mid J}(u\mid {\mathbf{u}}_J):=\frac{{\partial }^{p}C({\mathbf{u}}^{(i)}(u_I),{\mathbf{u}}_J)}{\partial {\mathbf{u}}_J}/\frac{{\partial }^{p}C(\mathbf{1},{\mathbf{u}}_J)}{\partial {\mathbf{u}}_J},$$

where ${\mathbf{u}}^{(i)}(u_I)$ has coordinate u at index i and 1 elsewhere in I.

On the original scale for a compound distribution X = SklarDist(C, (X_1,…,X_D)), conditioning on values $x_J$ is obtained by mapping to uniforms $u_J=(F_j(x_j){)}_{j\in J}$ then pushing forward each distortion through the corresponding marginal:

$$F_{X_i\mid X_J}(x\mid {\mathbf{x}}_J)=H_{i\mid J}(F_i(x)\mid {\mathbf{u}}_J).$$

The copula of the conditional vector $U_I|U_J=u_J$ is a genuine copula denoted $C_{I|J}(·|u_J)$, which is the copula of $H_{I|J}$. The public entry point is condition:

• condition(C::Copula, js, u_js) returns the conditional distribution on the uniform scale for I = setdiff(1:D, js). If length(I) == 1, the result is a univariate distribution supported on [0, 1]; otherwise it is a multivariate distribution implementing the usualDistributions.jl` interface.

• condition(X::SklarDist, js, x_js) returns the conditional distribution on the original scale by pushing forward each distortion through the corresponding marginal.

• Known parametric families may use specialized representations, but their concrete types are implementation details and do not change this contract.

Missing fast-paths?

If you find a conditional that should admit a faster closed-form or semi-analytic path but currently falls back to the generic construction, please open an issue, we’ll happily implement it 😃

Examples

Let us visualize a given univariate distortion:

using Copulas, Distributions, Plots, StatsBase
C = ClaytonCopula(2, 1.5)
D = condition(C, 2, 0.3)  # distortion for U₁ | U₂ = 0.3
ts = range(0.0, 1.0; length=401)
plt = plot(ts, cdf.(Ref(D), ts);
          xlabel="u", ylabel="H_{1|2}(u | 0.3)",
          title="Conditional CDF on the uniform scale",
          legend=false)
plt

Confirm the result by overlaying the empirical cdf of a sample:

N = 2000
αs = rand(N)
us = Distributions.quantile.(Ref(D), αs)
ECDF = ecdf(us)
plot!(ts, ECDF.(ts); seriestype=:steppost, label="empirical", alpha=0.6, color=:black)
plot!(ts, cdf.(Ref(D), ts); label="analytic", color=:blue)
plt

The same thing can be done on marginal scales using SklarDist:

C = ClaytonCopula(2, 1.5)
X = SklarDist(C, (Normal(), Normal()))
X1_given_X2 = condition(X, 2, 0.0) # distribution of X₁ | X₂ = 0.0
cdf(X1_given_X2, 1.0), quantile(X1_given_X2, 0.95)

(0.8472337930319087, 1.5990201973152853)

xs = rand(X1_given_X2, 2000)
Fx = ecdf(xs)
xs_grid = range(quantile(X1_given_X2, 0.001), quantile(X1_given_X2, 0.999); length=401)
plot(xs_grid, Distributions.cdf.(Ref(X1_given_X2), xs_grid);
  xlabel="x", ylabel="F_{X₁|X₂}(x|0)", title="Original-scale conditional CDF", label="analytic")
plot!(xs_grid, Fx.(xs_grid); seriestype=:steppost, label="empirical", alpha=0.6, color=:black)

When conditioning on less than $D-1$ dimensions, we obtain a multivariate object, usually a SklarDist:

H = condition(ClaytonCopula(4, 4.2), (2, 3), (0.25, 0.8))

SklarDist{ArchimedeanCopula{2, Copulas.TiltedGenerator{Copulas.ClaytonGenerator{Float64}, Float64}}, Tuple{Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}, Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}}}(
C: ArchimedeanCopula{2, Copulas.TiltedGenerator{Copulas.ClaytonGenerator{Float64}, Float64}}(θ = 4.2, sJ = 80.55877526114394)
m: (Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}(
G: Copulas.ClaytonGenerator{Float64}(4.2)
p: 2
sJ: 80.55877526114394
den: 1.1276684721576129e-5
)
, Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}(
G: Copulas.ClaytonGenerator{Float64}(4.2)
p: 2
sJ: 80.55877526114394
den: 1.1276684721576129e-5
)
)
)

plot(H)

Relation to the conditional copula

The conditional copula $C_{I|J}(·|u_J)$ is the copula of the conditional distribution $H_{I|J}(·|u_J)$. A multivariate result currently follows the SklarDist interface, so its copula and margins can be inspected as follows. Code should nevertheless rely on the public distribution interface rather than on a particular internal wrapper type:

H.C # the copula

ArchimedeanCopula{2, Copulas.TiltedGenerator{Copulas.ClaytonGenerator{Float64}, Float64}}(θ = 4.2, sJ = 80.55877526114394)

H.m # the marginals

(Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}(
G: Copulas.ClaytonGenerator{Float64}(4.2)
p: 2
sJ: 80.55877526114394
den: 1.1276684721576129e-5
)
, Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64}(
G: Copulas.ClaytonGenerator{Float64}(4.2)
p: 2
sJ: 80.55877526114394
den: 1.1276684721576129e-5
)
)

Implementation

Copulas.condition Function

    condition(C::Copula{D}, js, u_js)
    condition(X::SklarDist, js, x_js)

Construct conditional distributions with respect to a copula, either on the uniform scale (when passing a Copula) or on the original data scale (when passing a SklarDist).

Arguments

• C::Copula{D}: D-variate copula

• X::SklarDist: joint distribution with copula X.C and marginals X.m

• js: indices of conditioned coordinates (tuple, NTuple, or vector)

• u_js: values in [0,1] for U_js (when conditioning a copula)

• x_js: values on original scale for X_js (when conditioning a SklarDist)

• j, u_j, x_j: 1D convenience overloads for the common p = 1 case

Returns

• If the number of remaining coordinates d = D - length(js) is 1:

  • condition(C, js, u_js) returns a Distortion on [0,1] describing U_i | U_js = u_js.

  • condition(X, js, x_js) returns an unconditional univariate distribution for X_i | X_js = x_js, computed as the push-forward D(X.m[i]) where D = condition(C, js, u_js) and u_js = cdf.(X.m[js], x_js).

• If d > 1:

  • condition(C, js, u_js) returns the conditional joint distribution on the uniform scale as a SklarDist(ConditionalCopula, distortions).

  • condition(X, js, x_js) returns the conditional joint distribution on the original scale as a SklarDist with copula ConditionalCopula(C, js, u_js) and appropriately distorted marginals D_k(X.m[i_k]).

Notes

• For best performance, pass js and u_js as NTuple to keep p = length(js) known at compile time. The specialized method condition(::Copula{2}, j, u_j) exploits this for the common D = 2, d = 1 case.

• Specializations are provided for many copula families (Independent, Gaussian, t, Archimedean, several bivariate families). Others fall back to an automatic differentiation based construction.

• This function returns the conditional joint distribution H_{I|J}(· | u_J). The “conditional copula” is ConditionalCopula(C, js, u_js), i.e., the copula of that conditional distribution.

source

Copulas.Distortion Type

Distortion <: Distributions.ContinuousUnivariateDistribution

Abstract super-type for objects describing the (uniform-scale) conditional marginal transformation U_i | U_J = u_J of a copula.

Subtypes implement cdf/quantile on [0,1]. They are not full arbitrary distributions; they model how a uniform variable is distorted by conditioning. They can be applied as a function to a base marginal distribution to obtain the conditional marginal on the original scale: if D::Distortion and X::UnivariateDistribution, then D(X) is the distribution of X_i | U_J = u_J.

source

See also

• condition — reference documentation with all calling syntaxes

• SklarDist — compound distributions via Sklar’s theorem

• rosenblatt — sequential transforms (related but different)

Subsetting

Subsetting extracts the dependence structure among a subset of coordinates. Given a copula C of dimension d and an index tuple dims::NTuple{p,Int}, the function subsetdims returns a copula on those p dimensions that preserves the original dependence restricted to dims.

There are two entry points:

• subsetdims(C::Copula, dims) returns a Copula{p} (or Uniform() when p == 1).

• subsetdims(X::SklarDist, dims) returns a SklarDist with copula subsetdims(C, dims) and marginals (m[i] for i in dims).

The concrete representation is family-dependent. Some families return a natural reduced-parameter form, while the generic path uses an internal delegating representation. Both implement the same public copula interface:

using Copulas, Distributions
C = GaussianCopula([1.0 0.6 0.2; 0.6 1.0 0.3; 0.2 0.3 1.0])
S = subsetdims(C, (1,3))    # 2D copula on coordinates 1 and 3
length(S), typeof(S)

(2, GaussianCopula{2, Matrix{Float64}})

X = SklarDist(C, (Normal(), Normal(1,2), LogNormal()))
X13 = subsetdims(X, (1,3))  # keeps marginals (Normal(), LogNormal()) and reduces the copula
length(X13.C), length(X13.m)

(2, 2)

The exact result type is not part of the contract. Specialized forms may provide better performance or clearer display, while every result remains usable through the same copula API.

Subsetting and conditioning commute in the obvious way: conditioning on coordinates J and then extracting a subset of the remaining coordinates is equivalent to subsetting the base copula first and then conditioning on the corresponding indices.

Examples

# Archimedean example
C = ClaytonCopula(3, 2.0)
S = subsetdims(C, (1,2))        # still a ClaytonCopula with the same parameter
rand(S, 3)                      # sample 3 points
cdf(S, [0.7, 0.9])

0.6629375642933931

# Survival example with flips remapped
base = GaussianCopula([1.0 0.7 0.2; 0.7 1.0 0.1; 0.2 0.1 1.0])
S = SurvivalCopula(base, (2,))
S13 = subsetdims(S, (1,3))      # flip on 2 drops; no flips remain
typeof(S13), S13 isa SurvivalCopula

(SurvivalCopula{2, GaussianCopula{2, Matrix{Float64}}}, true)

Implementation

Copulas.subsetdims Function

subsetdims(C::Copula, dims::NTuple{p, Int})
subsetdims(D::SklarDist, dims)

Return a new copula or Sklar distribution corresponding to the subset of dimensions specified by dims.

Arguments

• C::Copula: The original copula object.

• D::SklarDist: The original Sklar distribution.

• dims::NTuple{p, Int}: Tuple of indices representing the dimensions to keep.

Returns

• A SubsetCopula or a new SklarDist object corresponding to the selected dimensions. If p == 1, returns a Uniform distribution or the corresponding marginal.

Details

This function extracts the dependence structure among the specified dimensions from the original copula or Sklar distribution. Specialized methods exist for some copula types to ensure efficiency and correctness.

source

Rosenblatt transformations

Definition and usefulness

Definition: Rosenblatt transformation

The Rosenblatt transformation considers a random vector $X$ distributed according to a certain multivariate cumulative distribution function $F_X(x)$, and maps it back to a uniform distribution on the unit hypercube.

More formally, consider the map $R_X(x)$ defined as follows:

$$R_X(x_1,...,x_d)=(r_1=F_{X_1}(x_1),r_2=F_{X_2|X_1}(x_2|x_1),...,r_d=F_{X_d|X_1,...,X_{d-1}}(x_d|x_1,...,x_{d-1}))$$

References:

• [13] Rosenblatt, M. (1952). Remarks on a multivariate transformation. Annals of Mathematical Statistics, 23(3), 470-472.

• [4] Joe, H. (2014). Dependence Modeling with Copulas. CRC Press. (Section 2.10)

• [14] McNeil, A. J., & Nešlehová, J. (2009). Multivariate Archimedean copulas, d-monotone functions and ℓ 1-norm symmetric distributions.

In certain circumstances, in particular for Archimedean copulas, this map simplifies to tractable expressions. It has a few nice properties:

• $R_X(X)\sim \mathtt{\text{Uniform(Unit Hypercube)}}$

• $R_X$ is a bijection.

These two properties are leveraged in some cases to construct the inverse Rosenblatt transformations, which map random noise to proper samples from the copula. In some cases, this is the best sampling algorithm available.

Implementation

As soon as the random vector $X$ is represented by an object X that subtypes SklarDist or Copula, you have access to the rosenblatt(X, x) and inverse_rosenblatt(X, x) operators, which both have a straightforward interpretation from their names.

Copulas.rosenblatt Function

rosenblatt(C::Copula, u)

Computes the rosenblatt transform associated to the copula C on the vector u. Formally, assuming that U ∼ C, the result should be uniformely distributed on the unit hypercube. The importance of this transofrmation comes from its bijectivity: inverse_rosenblatt(C, rand(d)) is equivalent to rand(C). The interface proposes faster versions for matrix inputs u.

Generic Rosenblatt transform using conditional distortions: S₁ = U₁, S_k = H_{k|1:(k-1)}(U_k | U₁:U_{k-1}). Specialized families may provide faster overrides.

• [13] Rosenblatt, M. (1952). Remarks on a multivariate transformation. Annals of Mathematical Statistics, 23(3), 470-472.

• [4] Joe, H. (2014). Dependence Modeling with Copulas. CRC Press. (Section 2.10)

• [14] McNeil, A. J., & Nešlehová, J. (2009). Multivariate Archimedean copulas, d-monotone functions and ℓ 1-norm symmetric distributions.

source

Copulas.inverse_rosenblatt Function

inverse_rosenblatt(C::Copula, u)

Computes the inverse rosenblatt transform associated to the copula C on the vector u. Formally, assuming that U ∼ Π, the independence copula, the result should be distributed as C. Also look at rosenblatt(C, u) for the inverse transformation. The interface proposes faster versions for matrix inputs u.

Generic inverse Rosenblatt using conditional distortions: U₁ = S₁, U_k = H_{k|1:(k-1)}^{-1}(S_k | U₁:U_{k-1}). Specialized families may provide faster overrides.

References:

• [13] Rosenblatt, M. (1952). Remarks on a multivariate transformation. Annals of Mathematical Statistics, 23(3), 470-472.

• [4] Joe, H. (2014). Dependence Modeling with Copulas. CRC Press. (Section 2.10)

• [14] McNeil, A. J., & Nešlehová, J. (2009). Multivariate Archimedean copulas, d-monotone functions and ℓ 1-norm symmetric distributions.

source

Once again, since the rosenblatt transform leverages the conditioning mechanisme, some fast-paths might be missing in the implementation.

Sanity check plot

You can validate that the Rosenblatt transform maps samples to independent uniforms by checking the marginal ECDFs against the 45° line.

using Copulas, Plots, StatsBase
# pick a nontrivial copula
C = ClaytonCopula(3, 1.5)

# draw samples and apply Rosenblatt transform coordinate-wise
U = rand(C, 3000)                 # size (3, N)
S = reduce(hcat, (rosenblatt(C, U[:, i]) for i in 1:size(U, 2)))  # size (3, N)

ts = range(0.0, 1.0; length=401)
layout = @layout [a b c]
plt = plot(layout=layout, size=(900, 280), legend=false)
for k in 1:3
  Ek = ecdf(S[k, :])
  plot!(plt[k], ts, Ek.(ts); seriestype=:steppost, color=:black,
      title="ECDF of $(k)", xlabel="u", ylabel="ECDF")
  plot!(plt[k], ts, ts; color=:blue, alpha=0.7)
end
plt

References

2. H. Joe. Dependence Modeling with Copulas (CRC press, 2014).

3. M. Rosenblatt. Remarks on a multivariate transformation. Annals of Mathematical Statistics 23, 470–472 (1952).

4. A. J. McNeil and J. Nešlehová. Multivariate Archimedean Copulas, $d$-Monotone Functions and ${\ell }_1$-Norm Symmetric Distributions. The Annals of Statistics 37, 3059–3097 (2009).