Data Processing Layer

The module data_processing provides implementations for the marked independence atoms: mCITs and indicator relations, and their sub-components. Additionally, it specifies interfaces that custom implementations of an underlying CIT, homogeneity test or weak-regime test should expose.

See also

The conditional independence test (CIT) implementations themselves are logically also part of the data processing layer. Considering that CIT-implementations, similar to underlying CD-algorithms, will typically be employed from third-party sources, the partial correlation implementation currently included with this code is factored into the separate file cit. Details on the specification and implementation of custom CITs (or integration of third-party implementations) can be found at Interfaces for CIT implementations.

Test implementations

• The class mCIT composes an underlying CIT (exposing ITestCI), a homogeneity test (exposing ITestHomogeneity) and a weak-regime test (exposing ITestWeakRegime) into a regime-marked conditional independence test (mCIT), providing ITestMarkedCI, as described by §5 of [RR25].

• The class Homogeneity_Binomial provides a homogeneity test (as specified by ITestHomogeneity) based on a binomal test with block-quantiles, as described by §5.2.2 of [RR25].

• The class WeakRegime_AcceptanceInterval provides a weak-regime test (as specified by ITestWeakRegime) based on the acceptance-interval method, as described by §5.2.3 of [RR25].

• The class IndicatorImplication_AcceptanceInterval provides a indicator-implication test based on the acceptance-interval method, as described by §D.4.7 of [RR25].

• The helper TruncatedNormal provides some basic properties of truncated normal distributions, used by the acceptance-interval tests.

Interfaces for Custom Stages

• The underlying CIT implementations can provide information through different interfaces:

  • The interface ITestCI is mandatory, it specifies how to expose CI-testing functionality itself.

  • The interface IProvideAnalyticQuantilesForCIT is optional (if not implemented, quantiles will be bootstrapped), it specifies how to provide quantile-estimates for a specific CIT. It is required only for Homogeneity_Binomial.

  • The interface IProvideVarianceForCIT is manatory, it specifies how to provide variance-estimates for a specific CIT. It is required only for WeakRegime_AcceptanceInterval and IndicatorImplication_AcceptanceInterval.

• The interface ITestMarkedCI specifies how to provide a custom marked conditional independence test.

• The interface ITestHomogeneity specifies how to provide a custom homogeneity test.

• The interface ITestWeakRegime specifies how to provide a custom weak-regime test.

• The interface ITestIndicatorImplications specifies how to provide a custom indicator-relation test.

• The interface IProvideHyperparamsForRobustCIT specifies how to provide N and dim(Z) dependent hyper-parameter sets for robust CI testing (cf. mCIT.__init__()).

• The interface IProvideHyperparamsForBinomial specifies how to provide N and dim(Z) dependent hyper-parameter sets for Homogeneity_Binomial.

• The interface IProvideHyperparamsForAcceptanceInterval specifies how to provide N and dim(Z) dependent hyper-parameter sets for WeakRegime_AcceptanceInterval and IndicatorImplication_AcceptanceInterval.

CIT Implementations

We currently include an implementation for z-score represented partial correlation as cit.ParCorr.