Gradient boosted PIK and propensity score weighting within the RDC

Introduction

The twangRDC R package is a streamline version of the twang R package created specifically for use in the RDCs. It utilizes gradient boosted models to estimate weights to handle imbalanced data. Results using twangRDC will not necessarily be reproduced using twang due to important differences in implementation. The twangRDC package allows for much larger datasets and many more covariates, but users should note that with smaller datasets, the twang package is computationally more efficient.

Methodology

The twangRDC package utilizes gradient boosted models to derive weights for nonequivalent groups. The algorithm alternates between adding iterations to the gradient boosted model (increasing its complexity) and evaluating balance. The algorithm automatically stops when additional iterations no longer improve balance. The package allows the user to generate weights for two different scenarios: missing data and construction of a comparison group.

Motivation

We will highlight two uses of the twangRDC R package. First, we will generate weights that ensure individuals with Protected Identification Keys (PIKs) are representative of their geographic region. PIKs are linking keys used by the Census Bureau to match person records across datasets¹. For example, using a PIK, a respondent in the 2000 Decennial Census can be linked to their response to the 2010 Decennial Census. PIKs are assigned through a probabilistic matching process that links survey or census records containing Personally Identifiable Information (PII) like name, address, date of birth and place of birth, to a Social Security reference file. Not all survey and census records can be linked to the reference file and as such not all records will be assigned a PIK. Linkage failures may occur if the PII used to match to the reference file is of insufficient quality or if an individual has no matching record in the Social Security reference file. PIKs can be assigned for the large majority of 2000 Decennial Census records. A Decennial Census record must have a PIK in order to link it to administrative records that contain other information, such as morality or annual residence.

Second, we will highlight using the twangRDC R package to generate propensity score weights such that a group of southern metropolitan areas can be used as a comparison group for residents of Orleans Parish.

The Data

A simulated data file was created for use in this vignette. It contains simulated records for residents of Orleans Parish, Louisiana, and other metropolitan areas in the South census region. We built the file exclusively from public use data, but it mirrors the structure of restricted versions of the 2000 Decennial Census available through the FSRDC network². Each simulated record includes basic individual demographic characteristics, basic household characteristics, and a set of neighborhood-level characteristics. Each record also includes two important indicators, one to simulate whether the individual record received a PIK and another to denote whether the individual lived in Orleans Parish.

The data was created by extracting all “short form” variables for households and individuals from the 5% Integrated Public Use Microdata Sample (IPUMS³) of the 2000 Decennial Census for the city of New Orleans (Orleans Parish) and for a selection of other southern metropolitan areas. We extracted contextual information from public use tract level tabulations of the 2000 Decennial Census (distributed by NHGIS⁴) and created select factor-based measures. We simulated assignment of households to census tracts and attach tract identifiers and characteristics.

We also simulated an indicator of PIK assignment (PIK=yes/no) to person records. Public use data do not include PIK assignment, so we estimated a predicted probability of receiving a PIK by using estimated regression parameters from Bond et al. 2013⁵. We added a random error to the predicted probability so that the PIK assignment status is not deterministic and convert the predicted probability into a dichotomous PIK=yes/no variable.

Lastly, we pooled Orleans Parish records with those from other southern metropolitan areas, create an indicator for Orleans Parish residence and, for the purposes of this vignette, sampled the data to shrink the size of the dataset. The simulated file contains individual records for 4,606 residents of Orleans Parish, LA and 9,519 individual records for residents from select southern metropolitan areas.

Table 1 provides a description of the data element included in the simulated data file.

PIK weighting

As previously mentioned, we will generate weights that ensure individuals with PIKs are representative of their geographic region. To keep the computational time of this vignette down, we focus only on a subset of Orleans parish. First, we load the twangRDC package and our simulated dataset.

  library(twangRDC) 
#> Loading required package: xgboost
#> Loading required package: data.table
#> Loading required package: ggplot2
  data(nola_south)

Next, it is important that the variables of the dataset are coded as intended. In this case, we convert several of variables to factors.

  # factors need to be coded as such
  nola_south$metarea = as.factor(nola_south$metarea)
  nola_south$c00_age12 = as.factor(nola_south$c00_age12)
  nola_south$c00_race = as.factor(nola_south$c00_race)

In a final data preparation step, we limit the dataset to Orleans parish and select only 25 of the Census tracts.

  # only consider Orleans parish 
  nola_only = subset(nola_south , metarea==556)

  # keep only 10 tracts for computational speed
  to.keep = unique(nola_only$tract_id_str)[1:10]
  nola_only = nola_only[ nola_only$tract_id_str %in% to.keep, ]

In this case, we wish to generate weights to ensure that individuals with PIKs are representative of their entire Census tract. That is, for each Census tract, we want to generate weights such that the observations with PIKs within the Census tract are representative of the tract’s population. This is achieved by specifying population=TRUE, which tells the function that the full data is representative of the underlying population, and strata="tract_id_str", which tells the function that we wish to generate weights that are representative within strata defined by Census tracts. The formula sim_pik ~ c00_age12 + c00_race + c00_sex identifies the observations with PIK on the left hand side and the characteristics that we want to consider when estimating the weights the right hand side.

  # set the model parameters
  params = list(eta = .01 , max_depth = 10 , subsample = 1 , 
                max_delta_step=0 , gamma=0 , lambda=0 , alpha=0, 
                min_child_weight=5 , objective = "binary:logistic")

  # fit the xgboost model
  res.pik = ps.xgb(sim_pik ~ c00_age12 + c00_race + c00_sex , 
               strata="tract_id_str",
               data=nola_only,
               params=params,
               max.steps=10,
               iters.per.step=500,
               min.iter=500,
               id.var="id",
               population = TRUE)

The parameters of the underlying xgboost model are specified in params. These were described in detail in a previous section. The id.var="id" provides a unique identifier for observations such that the generated weights can easily be merged back in with the original data. The other options specified in the ps.xgb function control how frequently the algorithm checks for convergence and how many iterations should be considered before stopping. First, iters.per.step=50 tells the algorithm to only consider every 50-th iteration when evaluating convergence. Larger values improve computational time by reducing the number of balance evaluations, while smaller values may achieve slightly better balance. Next, min.iter=50 tells the algorithm that at least 50 iterations must be used before stopping for convergence. Larger values ensure that more complex models are evaluated before determining the optimal set of weights. Finally, max.steps=2 indicates that the algorithm should only evaluate the balance of the weights twice before stopping. The maximum number of iterations of the xgboost model is given by max.steps*iters.per.setp, which in this case is 100. In general, this value should be large to ensure that the optimum set of weights is achieved. The default value is max.steps=Inf, which will continue adding iterations to the model until the convergence criteria is met. Due to computational concerns, we recommend testing your code with values of iters.per.step and max.steps such that the total number of iterations is small (1000 to 10000). Once you have determined the model is working as intended, set max.steps to Inf.

Now that the weights have been estimated, we can evaluate their quality. First, we need to ensure that a sufficient number of iterations have been used such that the balance criteria is minimized.

plot(res.pik)

Figure 1: Your figure caption.

The plot function provides a plot of the balance critera, which in this case is the average of the strata-specific maximum standardized difference of the covariates, versus the number iterations. As shown in this figure, we are verifying that the algorithm has run for a sufficient number of iterations such that a clear minimum has been achieved.

  bal.table(res.pik)

  bal.table(res.pik , type='strata' , include.var=TRUE , n=3 , decreasing = T)

Extracting weights

The get.weights function extracts the weights at the optimal iteration. The resulting data contains the weights and the ID variable specified in id.var. The weights can then be merged back in with the original data using the id.var. Note that base R merge function is slow compared to modern alternatives. If your data is large, consider data.table or dplyr.

  # extract weights
  w = get.weights(res.pik)
  
  # merge weights into data -- use data.table because its merge is faster than base R
  dta=merge(dta , w , by='id' , all=TRUE)

Comparison group construction

Our second example use of twangRDC will generate a comparison group for Orleans Parish consisting of residents of other southern metropolitan areas. The steps of the process remain the same as the PIK weighting example, but with minor adjustments to the interpretation and specification of the model. First, population = FALSE specifies that we are no longer interested in weights that represent a population, but instead, we wish to generate a comparison group using propensity score weights. Note that ps.xgb only estimates the average treatment effect on the treated, with the treatment group identified by records with value 1 in the left hand side of the formula. Our formula now includes three tract-level summaries that we wish to balance in addition to the three person-level characteristics. Finally, we no longer specify a stratification variable, as we are not attempting to create a comparison group for each tract of Orleans parish, but instead, a single comparison group that is representative of Orleans parish.

  # set the model parameters
  params = list(eta = .1 , max_depth = 10 , subsample = 1 , 
                max_delta_step=0 , gamma=0 , lambda=0 , alpha=0, 
                min_child_weight=5 , objective = "binary:logistic")

  # fit the xgboost model
  res.ps = ps.xgb(nola_rec ~ c00_age12 + c00_race + c00_sex + concdis + res_stab + imm_conc , 
               data=nola_south,
               params=params,
               max.steps=10,
               iters.per.step=500,
               min.iter=500,
               id.var="id",
               population = FALSE)

plot(res.ps)

Figure 2: Your figure caption.

  bal.table(res.pik)

  bal.table(res.ps , type='strata' , include.var=TRUE , n=3 , decreasing = T)