Dominance Analysis

The focus of the {domir} package is, currently, on dominance analysis (DA). DA can be thought of as an extension of Shapley value decomposition from Cooperative Game Theory (see Grömping, 2007 for a discussion) which seeks to find solutions of how to subdivide payoffs to players in a game based on their relative contribution to the payoff.

This methodology can be applied to predictive modeling in a conceptually straightforward way. Predictive models are, in a sense, a game in which independent variables (IVs)/predictors/features cooperate to produce a payoff in the form of predicting the dependent variable (DV)/outcome/response. The component of the decomposition/the proportion of the payoff ascribed to each IV can then be interpreted as the IVs importance in the context of the model as that is the contribution it makes to predicting the DV.

In application, DA determines the relative importance of IVs in a predictive model based on each IV’s contribution to an overall model fit statistic—a value that describes the entire model’s predictions on a dataset at once. DA’s goal extends beyond just the decomposition of the focal model fit statistic. In fact, DA produces three different results that it uses to compare the contribution each IV makes in the predictive model against the contributions attributed to each other IV. These three results, and their combined use to compare IVs, is the reason DA is an extension of Shapley value decomposition. The three different results are discussed in greater detail in the context of an example discussed below after a brief introduction to the domin function.

DA Implemtation with domir::domin

The domin function of the {domir} package is an API for applying DA to predictive modeling functions in R. The sections below will use this function to illustrate how DA is implemented and discuss conceptual details of each computation.

Details on how to work with domin in terms of programming and how the API functions are reported on in the domir details vignette.

Fit Statistic Results

#> Overall Fit Statistic:      0.8113023 

The first result domin prints is related to the overall fit statistic value for the model. In game theory terms, this value is the total payoff all players/IVs produced in the cooperative game/model. Recall that the value reported was

The value produced serves as the fit statistic “to be decomposed” by the DA and is limiting value for how much each IV will be able to explain. The DA will ascribe the three IVs in this model separate components of this ~\(.8113\) value related to their contributions to predicting mpg.

Other fit statistic value adjustments are reported in this section as well in particular those associated with the all subsets and constant model adjustments (discussed further below) when used in the DA.

General Dominance Statistics

#> General Dominance Statistics:
#>      General Dominance Standardized Ranks
#> am           0.2156848    0.2658501     2
#> cyl          0.4173094    0.5143698     1
#> carb         0.1783081    0.2197801     3

The second result printed reports the general dominance statistics related to how the overall fit statistic’s value is divided up among the IVs. These also represent the Shapley value decompositions of the fit statistic showing how each player/IV is ascribed a component of the payoff/fit statistic from the game/model.

The General Dominance column of statistics can be interpreted in terms of the fit metric it decomposed. For example, am has a value of ~\(0.2157\) which means am is associated with an \(R^2\) of about twenty-two percentage points of mpg’s variance given the predictive model and other IVs.

The Standardized column of statistics expresses the general dominance statistic value as a percentage of the overall fit statistic value and thus sum to 100%. am’s contributions to the \(R^2\)’s total value is ~27% (i.e., \(\frac{.2157}{.8113} = .2659\)).

The final Ranks column is most relevant to the focal purpose of determining the relative importance of the IVs in this model and provides a rank ordering of the IVs based on the general dominance statistics. It is here that DA moves beyond Shapley value decomposition in that DA, and this rank ordering based on general dominance statistics, allows for applying labels to the relationships between IVs in terms of their importance.

am is ranked second because it has a smaller general dominance statistic than the first ranked cyl. As such, am “is generally dominated by” cyl. By contrast, am is ranked higher than third ranked carb and thus am “generally dominates” carb. These labels are known as general dominance designations.

Conditional Dominance Statistics

#> Conditional Dominance Statistics:
#>         IVs: 1    IVs: 2     IVs: 3
#> am   0.3597989 0.2164938 0.07076149
#> cyl  0.7261800 0.4181185 0.10762967
#> carb 0.3035184 0.1791172 0.05228872

The third section reported on by domin prints the conditional dominance statistics associated with each IV. Each IV has a separate conditional statistic related to the number of IVs that are in a sub-model; why this matrix is useful delves into the computation of these statistics which is discussed later. For the time being it suffices to note that, conceptually, these values can be thought of as Shapley values for a specific number of players in the game. Thus, the IVs: 1 column reports on the value of the fit statistic/payoff when the IV is “playing alone” (i.e., by itself in the model with no other IVs). Similarly, the IVs: 2 column reports on a general value of the fit statistic/payoff when the IV is “playing with one other” (i.e., with another IV irrespective of which) and so on until the column with all IVs in the model (here IVs: 3).

The primary utility of the conditional dominance matrix is that it can be used to designate importance in a way that is more stringent/harder to achieve than the general dominance statistics. Unfortunately, this matrix does not have a ranking like the general dominance statistics and, indeed, it is not necessarily obvious as to how one might use this matrix for determining importance. The way this matrix is used is ‘row-wise’, comparing the results of an entire row/IV against those of another row/IV. If the value of each entry for a row/IV is greater than the value of another row/IV at the same position (i.e., comparing models with the same number of IVs) than an IV is said to “conditionally dominate” another IV. The matrix above shows that am “is conditionally dominated by” cyl as its conditional dominance statistics are smaller than cyl’s at positions 1, 2, and 3. Conversely, am “conditionally dominates” carb as its conditional dominance statistics are greater than carb’s at positions 1, 2, and 3.

Complete Dominance Designations

#> Complete Dominance Designations:
#>              Dmnated?am Dmnated?cyl Dmnated?carb
#> Dmnates?am           NA       FALSE         TRUE
#> Dmnates?cyl        TRUE          NA         TRUE
#> Dmnates?carb      FALSE       FALSE           NA

The fourth section reported on by domin prints the complete dominance designations associated with each IV pair. Each IV is compared to each other IV and has two entries in this matrix. The row labels denote that the IV noted in the label “completely dominates” the IV noted in the column label. By contrast, the column labels denote that the IV in the label “is completely dominated by” the IV in the row label. Each designation is then assigned a logical value or NA (when no such designation can be made given the model results).

The complete dominance designations are useful beyond the general and conditional dominance results as they are the most stringent sets of comparisons. Complete dominance reflects the relative performance of each IV to another IV in all the sub-models where their relative predictive performance can be compared. The results of this section are then not statistics but are an extensive series of inequality comparisons (i.e., in the mathematical sense \(>\), \(<\)) of individual sub-models expressed as logical designations.

Computational Implementation: DA as an Experiment

Shapley value decomposition and DA have traditionally been implemented by treating the cooperative game/predictive model as though it was an experimental design which seeks to evaluate the impact of the players/IVs on the payoff/fit statistic. When designing the experiment applied to the model, it is assumed that the only factors are the IVs and that they all have two levels: 1) the IV is included in the sub-model or 2) the IV is excluded from the sub-model—all other potential inputs to the model are constant.

The specific type of design applied to the model is a full-factorial design where all possible combinations of the IVs included or excluded are estimated as sub-models. When there are \(p\) IVs in the model there will be \(2^p\) sub-models estimated. The full-factorial design required for computing dominance statistics and designations is why the traditional approach is considered computationally expensive—each additional IV added to the model results in a geometric increase in the number of requires sub-models to estimate.

DA Results: The Full-Factorial Design

The DA results related to the lm model with three IVs discussed above is composed of 8 sub-models and their \(R^2\) values. The domin function, if supplied a predictive modeling function that can record each sub-model’s results, can be adapted to capture each sub-model’s \(R^2\) value along with the IVs that comprise it.

The code below constructs a wrapper function to export results from each sub-model to an external data frame. The code to produce these results is complex and each line is commented to note its purpose. This wrapper function then replaces lm in the call to domin so that, as the DA is executed, all the sub-models’ data are captured for the illustration to come.

lm_capture <- 
  function(formula, ...) { # wrapper program that accepts formula and ellipsis arguments
    count <<- count + 1 # increment counter in enclosing environment
    lm_obj <- lm(formula, ...) # estimate 'lm' model and save object
    DA_results[count, "formula"] <<- 
      deparse(formula) # record string version of formula passed in 'DA_results' in enclosing environment
    DA_results[count, "R^2"] <<- 
      summary(lm_obj)[["r.squared"]] # record R^2 in 'DA_results' in enclosing environment
    return(lm_obj) # return 'lm' class-ed object
  }

count <- 0 # initialize the count indicating the row in which the results will fill-in

DA_results <- # container data frame in which to record results
  data.frame(formula = rep("", times = 2^3-1), 
             `R^2` = rep(NA, times = 2^3-1), 
             check.names = FALSE)

lm_da <- domin(mpg ~ am + cyl + carb, # implement the DA with the wrapper
               lm_capture, 
               list(summary, "r.squared"), 
               data = mtcars)

DA_results
#>                 formula       R^2
#> 1              mpg ~ am 0.3597989
#> 2             mpg ~ cyl 0.7261800
#> 3        mpg ~ am + cyl 0.7590135
#> 4            mpg ~ carb 0.3035184
#> 5       mpg ~ am + carb 0.7036726
#> 6      mpg ~ cyl + carb 0.7405408
#> 7 mpg ~ am + cyl + carb 0.8113023

The printed result from the DA_results container data frame shows that domin runs 7 sub-models; each a different combination of the IVs. Note that, by default, the sub-model where all IVs are excluded is assumed to result in a fit statistic value of 0 and is not estimated directly (which can be changed with the consmodel argument).

The \(R^2\) values recorded in DA_results are used to compute the dominance statistics and designations reported on above.

lm_da$Complete_Dominance
#>              Dmnated_am Dmnated_cyl Dmnated_carb
#> Dmnates_am           NA       FALSE         TRUE
#> Dmnates_cyl        TRUE          NA         TRUE
#> Dmnates_carb      FALSE       FALSE           NA

lm_da$Conditional_Dominance
#>          IVs_1     IVs_2      IVs_3
#> am   0.3597989 0.2164938 0.07076149
#> cyl  0.7261800 0.4181185 0.10762967
#> carb 0.3035184 0.1791172 0.05228872

lm_da$General_Dominance
#>        am       cyl      carb 
#> 0.2156848 0.4173094 0.1783081

knitr::kable(data.frame(IV = names(lm_da$General_Dominance), `general dominance` = lm_da$General_Dominance, ranks = lm_da$Ranks, check.names = FALSE), row.names = FALSE, caption = "General Dominance Designations", digits = 3)

Strongest Dominance Designations

Whereas all dominance designations have been made in the example above, the strongest designation between two IVs is likely of primary interest as the strongest designation, as is noted above, implies all weaker designations.

To access the strongest dominance designations, the DA object can be submitted to the summary function.

summary(lm_da)$Strongest_Dominance
#>                                                                            
#>  "am"                         "am"                   "cyl"                 
#>  "is completely dominated by" "completely dominates" "completely dominates"
#>  "cyl"                        "carb"                 "carb"

The result the summary function produces in the Strongest_Dominance element is consistent with expectation in that all three IV interrelationships have complete dominance designations between them.