Get batch predictions and explanations

BigQuery table

If you choose a BigQuery table as the input, you must ensure the following:

• BigQuery data source tables must be no larger than 100 GB.
• If the table is in a different project, you must grant the BigQuery Data Editor role to the Vertex AI service account in that project.

CSV file

If you choose a CSV object in Cloud Storage as the input, you must ensure the following:

• The data source must begin with a header row with the column names.
• Each data source object must not be larger than 10 GB. You can include multiple files, up to a maximum amount of 100 GB.
• If the Cloud Storage bucket is in a different project, you must grant the Storage Object Creator role to the Vertex AI service account in that project.
• You must enclose all strings in double quotation marks (").

Google Cloud console

Use the Google Cloud console to request a batch prediction.

1. In the Google Cloud console, in the Vertex AI section, go to the Batch predictions page.

   Go to the Batch predictions page

2. Click Create to open the New batch prediction window.
3. For Define your batch prediction, complete the following steps:
   1. Enter a name for the batch prediction.
   2. For Model name, select the name of the model to use for this batch prediction.
   3. For Version, select the model version to use for this batch prediction.
   4. For Select source, select whether your source input data is a CSV file on Cloud Storage or a table in BigQuery.
      • For CSV files, specify the Cloud Storage location where your CSV input file is located.
      • For BigQuery tables, specify the project ID where the table is located, the BigQuery dataset ID, and the BigQuery table or view ID.
   5. For the Output, select CSV or BigQuery.
      • For CSV, specify the Cloud Storage bucket where Vertex AI stores your output.
      • For BigQuery, you can specify a project ID or an existing dataset:
        • To specify the project ID, enter the project ID in the Google Cloud project ID field. Vertex AI creates a new output dataset for you.
        • To specify an existing dataset, enter its BigQuery path in the Google Cloud project ID field, such as bq://projectid.datasetid.
      • Optional. You can request a prediction with explanations (also called feature attributions) to see how your model arrived at a prediction. The local feature importance values tell you how much each feature contributed to the prediction result. Feature attributions are included in Vertex AI predictions through Vertex Explainable AI.
        

        To enable feature attributions, select Enable feature attributions for this model. This option is available if your output destination is BigQuery or JSONL on Cloud Storage. Feature attributions are not supported for CSV on Cloud Storage.

4. Optional: Model Monitoring analysis for batch predictions is available in Preview. See the Prerequisites for adding skew detection configuration to your batch prediction job.
   1. Click to toggle on Enable model monitoring for this batch prediction.
   2. Select a Training data source. Enter the data path or location for the training data source that you selected.
   3. Optional: Under Alert thresholds, specify thresholds at which to trigger alerts.
   4. For Notification emails, enter one or more comma-separated email addresses to receive alerts when a model exceeds an alerting threshold.
   5. Optional: For Notification channels, add Cloud Monitoring channels to receive alerts when a model exceeds an alerting threshold. You can select existing Cloud Monitoring channels or create a new one by clicking Manage notification channels. The Console supports PagerDuty, Slack, and Pub/Sub notification channels.
5. Click Create.

API: BigQuery

POST https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs

{
  "displayName": "BATCH_JOB_NAME",
  "model": "MODEL_ID",
  "inputConfig": {
    "instancesFormat": "bigquery",
    "bigquerySource": {
      "inputUri": "INPUT_URI"
    }
  },
  "outputConfig": {
    "predictionsFormat": "bigquery",
    "bigqueryDestination": {
      "outputUri": "OUTPUT_URI"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "MACHINE_TYPE",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT,
    "maxReplicaCount": MAX_REPLICA_COUNT
  },

  "generateExplanation": GENERATE_EXPLANATION

}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs" | Select-Object -Expand Content

{
  "name": "projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs/67890",
  "displayName": "batch_job_1 202005291958",
  "model": "projects/12345/locations/us-central1/models/5678",
  "state": "JOB_STATE_PENDING",
  "inputConfig": {
    "instancesFormat": "bigquery",
    "bigquerySource": {
      "inputUri": "INPUT_URI"
    }
  },
  "outputConfig": {
    "predictionsFormat": "bigquery",
    "bigqueryDestination": {
        "outputUri": bq://12345
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "n1-standard-32",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": 2,
    "maxReplicaCount": 6
  },
  "manualBatchTuningParameters": {
    "batchSize": 4
  },
  "generateExplanation": false,
  "outputInfo": {
    "bigqueryOutputDataset": "bq://12345.reg_model_2020_10_02_06_04
  }
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-09-30T02:58:44.341643Z",
  "updateTime": "2020-09-30T02:58:44.341643Z",
}

import com.google.cloud.aiplatform.v1.BatchPredictionJob;
import com.google.cloud.aiplatform.v1.BigQueryDestination;
import com.google.cloud.aiplatform.v1.BigQuerySource;
import com.google.cloud.aiplatform.v1.JobServiceClient;
import com.google.cloud.aiplatform.v1.JobServiceSettings;
import com.google.cloud.aiplatform.v1.LocationName;
import com.google.cloud.aiplatform.v1.ModelName;
import com.google.gson.JsonObject;
import com.google.protobuf.Value;
import com.google.protobuf.util.JsonFormat;
import java.io.IOException;

public class CreateBatchPredictionJobBigquerySample {

  public static void main(String[] args) throws IOException {
    // TODO(developer): Replace these variables before running the sample.
    String project = "PROJECT";
    String displayName = "DISPLAY_NAME";
    String modelName = "MODEL_NAME";
    String instancesFormat = "INSTANCES_FORMAT";
    String bigquerySourceInputUri = "BIGQUERY_SOURCE_INPUT_URI";
    String predictionsFormat = "PREDICTIONS_FORMAT";
    String bigqueryDestinationOutputUri = "BIGQUERY_DESTINATION_OUTPUT_URI";
    createBatchPredictionJobBigquerySample(
        project,
        displayName,
        modelName,
        instancesFormat,
        bigquerySourceInputUri,
        predictionsFormat,
        bigqueryDestinationOutputUri);
  }

  static void createBatchPredictionJobBigquerySample(
      String project,
      String displayName,
      String model,
      String instancesFormat,
      String bigquerySourceInputUri,
      String predictionsFormat,
      String bigqueryDestinationOutputUri)
      throws IOException {
    JobServiceSettings settings =
        JobServiceSettings.newBuilder()
            .setEndpoint("us-central1-aiplatform.googleapis.com:443")
            .build();
    String location = "us-central1";

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (JobServiceClient client = JobServiceClient.create(settings)) {
      JsonObject jsonModelParameters = new JsonObject();
      Value.Builder modelParametersBuilder = Value.newBuilder();
      JsonFormat.parser().merge(jsonModelParameters.toString(), modelParametersBuilder);
      Value modelParameters = modelParametersBuilder.build();
      BigQuerySource bigquerySource =
          BigQuerySource.newBuilder().setInputUri(bigquerySourceInputUri).build();
      BatchPredictionJob.InputConfig inputConfig =
          BatchPredictionJob.InputConfig.newBuilder()
              .setInstancesFormat(instancesFormat)
              .setBigquerySource(bigquerySource)
              .build();
      BigQueryDestination bigqueryDestination =
          BigQueryDestination.newBuilder().setOutputUri(bigqueryDestinationOutputUri).build();
      BatchPredictionJob.OutputConfig outputConfig =
          BatchPredictionJob.OutputConfig.newBuilder()
              .setPredictionsFormat(predictionsFormat)
              .setBigqueryDestination(bigqueryDestination)
              .build();
      String modelName = ModelName.of(project, location, model).toString();
      BatchPredictionJob batchPredictionJob =
          BatchPredictionJob.newBuilder()
              .setDisplayName(displayName)
              .setModel(modelName)
              .setModelParameters(modelParameters)
              .setInputConfig(inputConfig)
              .setOutputConfig(outputConfig)
              .build();
      LocationName parent = LocationName.of(project, location);
      BatchPredictionJob response = client.createBatchPredictionJob(parent, batchPredictionJob);
      System.out.format("response: %s\n", response);
      System.out.format("\tName: %s\n", response.getName());
    }
  }
}

from google.cloud import aiplatform_v1beta1
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value


def create_batch_prediction_job_bigquery_sample(
    project: str,
    display_name: str,
    model_name: str,
    instances_format: str,
    bigquery_source_input_uri: str,
    predictions_format: str,
    bigquery_destination_output_uri: str,
    location: str = "us-central1",
    api_endpoint: str = "us-central1-aiplatform.googleapis.com",
):
    # The AI Platform services require regional API endpoints.
    client_options = {"api_endpoint": api_endpoint}
    # Initialize client that will be used to create and send requests.
    # This client only needs to be created once, and can be reused for multiple requests.
    client = aiplatform_v1beta1.JobServiceClient(client_options=client_options)
    model_parameters_dict = {}
    model_parameters = json_format.ParseDict(model_parameters_dict, Value())

    batch_prediction_job = {
        "display_name": display_name,
        # Format: 'projects/{project}/locations/{location}/models/{model_id}'
        "model": model_name,
        "model_parameters": model_parameters,
        "input_config": {
            "instances_format": instances_format,
            "bigquery_source": {"input_uri": bigquery_source_input_uri},
        },
        "output_config": {
            "predictions_format": predictions_format,
            "bigquery_destination": {"output_uri": bigquery_destination_output_uri},
        },
        # optional
        "generate_explanation": True,
    }
    parent = f"projects/{project}/locations/{location}"
    response = client.create_batch_prediction_job(
        parent=parent, batch_prediction_job=batch_prediction_job
    )
    print("response:", response)

API: Cloud Storage

POST https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs

{
  "displayName": "BATCH_JOB_NAME",
  "model": "MODEL_ID",
  "inputConfig": {
    "instancesFormat": "csv",
    "gcsSource": {
      "uris": [
        URI1,...
      ]
    },
  },
  "outputConfig": {
    "predictionsFormat": "csv",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_URI_PREFIX"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "MACHINE_TYPE",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": STARTING_REPLICA_COUNT,
    "maxReplicaCount": MAX_REPLICA_COUNT
  },

  "generateExplanation": GENERATE_EXPLANATION

}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION_ID-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION_ID/batchPredictionJobs" | Select-Object -Expand Content

{
  "name": "projects/PROJECT__ID/locations/LOCATION_ID/batchPredictionJobs/67890",
  "displayName": "batch_job_1 202005291958",
  "model": "projects/12345/locations/us-central1/models/5678",
  "state": "JOB_STATE_PENDING",
  "inputConfig": {
    "instancesFormat": "csv",
    "gcsSource": {
      "uris": [
        "gs://bp_bucket/reg_mode_test"
      ]
    }
  },
  "outputConfig": {
    "predictionsFormat": "csv",
    "gcsDestination": {
      "outputUriPrefix": "OUTPUT_URI_PREFIX"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "n1-standard-32",
      "acceleratorCount": "0"
    },
    "startingReplicaCount": 2,
    "maxReplicaCount": 6
  },
  "manualBatchTuningParameters": {
    "batchSize": 4
  }
  "outputInfo": {
    "gcsOutputDataset": "OUTPUT_URI_PREFIX/prediction-batch_job_1 202005291958-2020-09-30T02:58:44.341643Z"
  }
  "state": "JOB_STATE_PENDING",
  "createTime": "2020-09-30T02:58:44.341643Z",
  "updateTime": "2020-09-30T02:58:44.341643Z",
}

BigQuery

Output dataset

If you are using BigQuery, the output of batch prediction is stored in an output dataset. If you had provided a dataset to Vertex AI, the name of the dataset (BQ_DATASET_NAME) is the name you had provided earlier. If you did not provide an output dataset, Vertex AI created one for you. You can find its name (BQ_DATASET_NAME) with the following steps:

1. In the Google Cloud console, go to the Vertex AI Batch predictions page.
   

   Go to the Batch predictions page

2. Select the prediction you created.
3. The output dataset is given in Export location. The dataset name is formatted as follows: prediction_MODEL_NAME_TIMESTAMP

The output dataset contains one or more of the following three output tables:

• Predictions table
  

  This table contains a row for every row in your input data where a prediction was requested (that is, where TARGET_COLUMN_NAME = null).

• Errors table
  

  This table contains a row for each non-critical error encountered during batch prediction. Each non-critical error corresponds with a row in the input data that Vertex AI could not return a forecast for.

Predictions table

The name of the table (BQ_PREDICTIONS_TABLE_NAME) is formed by appending `predictions_` with the timestamp of when the batch prediction job started: predictions_TIMESTAMP

To retrieve predictions, go to the BigQuery page.

Go to BigQuery

Classification:

SELECT predicted_TARGET_COLUMN_NAME.classes AS classes,
predicted_TARGET_COLUMN_NAME.scores AS scores
FROM BQ_DATASET_NAME.BQ_PREDICTIONS_TABLE_NAME

classes is the list of potential classes, and scores are the corresponding confidence scores.

Regression:

SELECT predicted_TARGET_COLUMN_NAME.value,
predicted_TARGET_COLUMN_NAME.lower_bound,
predicted_TARGET_COLUMN_NAME.upper_bound
FROM BQ_DATASET_NAME.BQ_PREDICTIONS_TABLE_NAME

If you enabled feature attributions, you can find them in the predictions table as well. To access attributions for a feature BQ_FEATURE_NAME, run the following query:

SELECT explanation.attributions[OFFSET(0)].featureAttributions.BQ_FEATURE_NAME FROM BQ_DATASET_NAME.BQ_PREDICTIONS_TABLE_NAME
  

Errors table

1. In the console, go to the BigQuery page.

   Go to BigQuery

2. Run the following query:

   SELECT * FROM BQ_DATASET_NAME.BQ_ERRORS_TABLE_NAME
         

• errors_TARGET_COLUMN_NAME.code
• errors_TARGET_COLUMN_NAME.message

Cloud Storage

If you specified Cloud Storage as your output destination, the results of your batch prediction request are returned as CSV objects in a new folder in the bucket you specified. The name of the folder is the name of your model, prepended with "prediction-" and appended with the timestamp of when the batch prediction job started. You can find the Cloud Storage folder name in the Batch predictions tab for your model.

• Prediction objects

  The prediction objects are named `predictions_1.csv`, `predictions_2.csv`, and so on. They contain a header row with the column names, and a row for every prediction returned. In the prediction objects, Vertex AI returns your prediction data and creates one or more new columns for the prediction results based on your model type:

  • Classification: For each potential value of your target column, a column named TARGET_COLUMN_NAME_VALUE_score is added to the results. This column contains the score, or confidence estimate, for that value.
  • Regression: The predicted value for that row is returned in a column named predicted_TARGET_COLUMN_NAME. The prediction interval is not returned for CSV output.
• Error objects

  The error objects are named `errors_1.csv`, `errors_2.csv`, and so on. They contain a header row, and a row for every row in your input data that Vertex AI could not return a prediction (for example, if a non-nullable feature was null) for.

Note: If the results are large, it is split into multiple objects.

Feature attributions are not available for batch prediction results returned in Cloud Storage.

Classification

Classification models return a confidence score.

The confidence score communicates how strongly your model associates each class or label with a test item. The higher the number, the higher the model's confidence that the label should be applied to that item. You decide how high the confidence score must be for you to accept the model's results.

Regression

Regression models return a prediction value. For BigQuery destinations, they also return a prediction interval. The prediction interval provides a range of values that the model has 95% confidence contain the actual result.