Introduction to Rating Tables

A rating table is an exportable CSV representation of a Generalized Additive Model. It contains information about the features and coefficients used to make predictions. Users can influence predictions by downloading and editing values in a rating table, then uploading the table and using it to create a new model. See the page about interpreting Generalized Additive Model output in the Datarobot user guide for more details on how to interpret and edit rating tables.

Connect to DataRobot

To explore rating tables, let’s first connect to DataRobot. First, you must load the DataRobot R package library.

library(datarobot)

If you have set up a credentials file, library(datarobot) will initialize a connection to DataRobot automatically. Otherwise, you can specify your endpoint and apiToken as in this example to connect to DataRobot directly. For more information on connecting to DataRobot, see the “Introduction to DataRobot” vignette.

endpoint <- "https://<YOUR DATAROBOT URL GOES HERE>/api/v2"
apiToken <- "<YOUR API TOKEN GOES HERE>"
ConnectToDataRobot(endpoint = endpoint, token = apiToken)

Retrieving Rating Tables

You can retrieve a rating table from the list of rating tables in a project:

projectId <- "59dab74bbd2a54035786bfc0"
ratingTables <- ListRatingTables(projectId)
ratingTable <- ratingTables[[1]]
print(ratingTable)

$validationJobId NULL

$validationError [1] “”

$projectId [1] “59dab74bbd2a54035786bfc0”

$ratingTableName [1] “Rating Table for 59dab774bd2a54035d157fa7”

$parentModelId [1] “59dab774bd2a54035d157fa7”

$modelJobId NULL

$id [1] “59dab7a06f42a6df428bc14c”

$originalFilename [1] “rating_table.csv”

$modelId [1] “59dab774bd2a54035d157fa7”

attr(,“class”) [1] “dataRobotRatingTable”

Or you can retrieve a rating table from a specific model. The model must already have a rating table.

projectId <- "59dab74bbd2a54035786bfc0"
ratingTableModels <- ListRatingTableModels(projectId)
ratingTableModel <- ratingTableModels[[1]]
ratingTableId <- ratingTableModel$ratingTableId
ratingTable <- GetRatingTable(projectId, ratingTableId)
print(ratingTable)

$validationJobId NULL

$validationError [1] “”

$projectId [1] “59dab74bbd2a54035786bfc0”

$ratingTableName [1] “Rating Table for 59dab774bd2a54035d157fa7”

$parentModelId [1] “59dab774bd2a54035d157fa7”

$modelJobId NULL

$id [1] “59dab7a06f42a6df428bc14c”

$originalFilename [1] “rating_table.csv”

$modelId [1] “59dab774bd2a54035d157fa7”

attr(,“class”) [1] “dataRobotRatingTable”

Or retrieve model by id. The model must have a rating table.

projectId <- "59dab74bbd2a54035786bfc0"
modelId <- "59dd0b01d9575702bec96e4"
ratingTableModel <- GetRatingTableModel(projectId, modelId)
ratingTableId <- ratingTableModel$ratingTableId
ratingTable <- GetRatingTable(projectId, ratingTableId)
print(ratingTable)

$validationJobId NULL

$validationError [1] “”

$projectId [1] “59dab74bbd2a54035786bfc0”

$ratingTableName [1] “Rating Table for 59dab774bd2a54035d157fa7”

$parentModelId [1] “59dab774bd2a54035d157fa7”

$modelJobId NULL

$id [1] “59dab7a06f42a6df428bc14c”

$originalFilename [1] “rating_table.csv”

$modelId [1] “59dab774bd2a54035d157fa7”

attr(,“class”) [1] “dataRobotRatingTable”

Downloading Rating Tables

Once you have a rating table, you can download the contents to a CSV.

DownloadRatingTable(projectId, ratingTableId, "myRatingTable.csv")

Modifying Rating Tables

You can then modify the values in the CSV and re-upload a new rating table back to DataRobot.

DownloadRatingTable(projectId, ratingTableId, "myRatingTable.csv")
newRatingTableJobId <- CreateRatingTable(project,
                                         modelId,
                                         "myRatingTable.csv",
                                         ratingTableName = "Modified File")
newRatingTable <- GetRatingTableFromJobId(project, newRatingTableJobId)
print(newRatingTable)

$validationJobId NULL

$validationError [1] “”

$projectId [1] “59dab74bbd2a54035786bfc0”

$ratingTableName [1] “Rating Table for 59dab774bd2a54035d157fa7”

$parentModelId [1] “59dab774bd2a54035d157fa7”

$modelJobId NULL

$id [1] “59dab7a06f42a6df428bc14c”

$originalFilename [1] “rating_table.csv”

$modelId [1] “59dab774bd2a54035d157fa7”

attr(,“class”) [1] “dataRobotRatingTable”

Making New GAMs from New Rating Tables

You can then take the new rating tables you make and create new models from them.

newModelJobId <- RequestNewRatingTableModel(project, newRatingTable)
newRatingTableModel <- GetRatingTableModelFromJobId(project, newModelJobId)
print(newRatingTableModel)

$featurelistId [1] “59dd4731c0d33327b8f55610”

$processes [1] “One-Hot Encoding”
[2] “Ordinal encoding of categorical variables”
[3] “Missing Values Imputed”
[4] “Matrix of word-grams occurrences”
[5] “Generalized Additive Model”
[6] “Text fit on Residuals (L2 / Binomial Deviance)”

$featurelistName [1] “Informative Features”

$projectId [1] “59dd4723d957570407bc37b4”

$modelType [1] “Generalized Additive Model”

$samplePct [1] 64.041

$isFrozen [1] FALSE

$metrics $metrics$AUC $metrics$AUC$backtesting NULL

$metrics$AUC$holdout NULL

$metrics$AUC$backtestingScores NULL

$metrics$AUC$crossValidation NULL

$metrics$AUC$validation [1] 0.77283

$metrics$Rate@Top5% $metrics$Rate@Top5%$backtesting NULL

$metrics$Rate@Top5%$holdout NULL

$metrics$Rate@Top5%$backtestingScores NULL

$metrics$Rate@Top5%$crossValidation NULL

$metrics$Rate@Top5%$validation [1] 1

$metrics$Rate@TopTenth% $metrics$Rate@TopTenth%$backtesting NULL

$metrics$Rate@TopTenth%$holdout NULL

$metrics$Rate@TopTenth%$backtestingScores NULL

$metrics$Rate@TopTenth%$crossValidation NULL

$metrics$Rate@TopTenth%$validation [1] 1

$metrics$RMSE $metrics$RMSE$backtesting NULL

$metrics$RMSE$holdout NULL

$metrics$RMSE$backtestingScores NULL

$metrics$RMSE$crossValidation NULL

$metrics$RMSE$validation [1] 0.41992

$metrics$LogLoss $metrics$LogLoss$backtesting NULL

$metrics$LogLoss$holdout NULL

$metrics$LogLoss$backtestingScores NULL

$metrics$LogLoss$crossValidation NULL

$metrics$LogLoss$validation [1] 0.50643

$metrics$FVE Binomial $metrics$FVE Binomial$backtesting NULL

$metrics$FVE Binomial$holdout NULL

$metrics$FVE Binomial$backtestingScores NULL

$metrics$FVE Binomial$crossValidation NULL

$metrics$FVE Binomial$validation [1] 0.23955

$metrics$Gini Norm $metrics$Gini Norm$backtesting NULL

$metrics$Gini Norm$holdout NULL

$metrics$Gini Norm$backtestingScores NULL

$metrics$Gini Norm$crossValidation NULL

$metrics$Gini Norm$validation [1] 0.54566

$metrics$Rate@Top10% $metrics$Rate@Top10%$backtesting NULL

$metrics$Rate@Top10%$holdout NULL

$metrics$Rate@Top10%$backtestingScores NULL

$metrics$Rate@Top10%$crossValidation NULL

$metrics$Rate@Top10%$validation [1] 1

$modelCategory [1] “model”

$blueprintId [1] “24ea7590216323555b8fe51fe006dfba”

$ratingTableId [1] “59dd4778e643feefd5d87c9b”

$id [1] “59dd474bd95757040120a8e8”

attr(,“class”) [1] “dataRobotRatingTableModel”

Introduction to Rating Tables

Peter Hurford

2018-01-17

Connect to DataRobot

Retrieving Rating Tables

Downloading Rating Tables

Modifying Rating Tables

Making New GAMs from New Rating Tables