Configure example-based explanations

To use example-based explanations, you must configure explanations by specifying an explanationSpec when you import or upload the Model resource into the Model Registry.

Then, when you request online explanations, you can override some of those configuration values by specifying an ExplanationSpecOverride in the request. You cannot request batch explanations; they are not supported.

This page describes how to configure and update these options.

Configure explanations when importing or uploading the model

Before you begin, make sure that you have following:

1. A Cloud Storage location that contains your model artifacts. Your model either needs to be a deep neural network (DNN) model where you provide the name of a layer, or signature, whose output can be used as the latent space, or you can provide a model that directly outputs embeddings (latent space representation). This latent space captures the example representations that are used for generating explanations.

2. A Cloud Storage location that contains the instances to be indexed for approximate nearest neighbor search. For more information, see input data requirements.

{
  "inputs": {
    "my_input": {
      "inputTensorName": "INPUT_TENSOR_NAME",
      "encoding": "IDENTITY",
    },
    "id": {
      "inputTensorName": "id",
      "encoding": "IDENTITY"
    }
  },
  "outputs": {
    "embedding": {
      "outputTensorName": "OUTPUT_TENSOR_NAME"
    }
  }
}

{
  "contentsDeltaUri": "",
  "config": {
      "dimensions": 50,
      "approximateNeighborsCount": 10,
      "distanceMeasureType": "SQUARED_L2_DISTANCE",
      "featureNormType": "NONE",
      "algorithmConfig": {
          "treeAhConfig": {
              "leafNodeEmbeddingCount": 1000,
              "fractionLeafNodesToSearch": 1.0
          }
      }
    }
  }

gcloud ai models upload \
    --region=LOCATION \
    --display-name=MODEL_NAME \
    --container-image-uri=IMAGE_URI \
    --artifact-uri=MODEL_ARTIFACT_URI \
    --explanation-method=examples \
    --uris=[URI, ...] \
    --explanation-neighbor-count=NEIGHBOR_COUNT \
    --explanation-metadata-file=explanation-metadata.json \
    --explanation-modality=IMAGE|TEXT|TABULAR \
    --explanation-query=PRECISE|FAST \
    --explanation-nearest-neighbor-search-config-file=search_config.json

POST https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/models:upload

{
  "model": {
    "displayName": "my-model",
    "artifactUri": "gs://your-model-artifact-folder",
    "containerSpec": {
      "imageUri": "us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-11:latest",
    },
    "explanationSpec": {
      "parameters": {
        "examples": {
          "gcsSource": {
            "uris": ["gs://your-examples-folder"]
          },
          "neighborCount": 10,
          "presets": {
            "modality": "image"
          }
        }
      },
      "metadata": {
        "outputs": {
            "embedding": {
                "output_tensor_name": "embedding"
            }
        },
        "inputs": {
          "my_fancy_input": {
            "input_tensor_name": "input_tensor_name",
            "encoding": "identity",
            "modality": "image"
          },
          "id": {
            "input_tensor_name": "id",
            "encoding": "identity"
          }
        }
      }
    }
  }
}

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/models:upload"

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://LOCATION-aiplatform.googleapis.com/v1/projects/PROJECT/locations/LOCATION/models:upload" | Select-Object -Expand Content

{
  "name": "projects/PROJECT_NUMBER/locations/LOCATION/models/MODEL_ID/operations/OPERATION_ID",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.aiplatform.v1.UploadModelOperationMetadata",
    "genericMetadata": {
      "createTime": "2022-01-08T01:21:10.147035Z",
      "updateTime": "2022-01-08T01:21:10.147035Z"
    }
  }
}

gcloud ai operations describe <operation-id>

NearestNeighborSearchConfig

The following JSON request body demonstrates how to specify the full NearestNeighborSearchConfig (instead of presets) in an upload request.

{
  "model": {
    "displayName": displayname,
    "artifactUri": model_path_to_deploy,
    "containerSpec": {
      "imageUri": DEPLOY_IMAGE,
    },
    "explanationSpec": {
      "parameters": {
        "examples": {
          "gcsSource": {
            "uris": [DATASET_PATH]
          },
          "neighborCount": 5,
          "nearest_neighbor_search_config": {
            "contentsDeltaUri": "",
            "config": {
              "dimensions": dimensions,
              "approximateNeighborsCount": 10,
              "distanceMeasureType": "SQUARED_L2_DISTANCE",
              "featureNormType": "NONE",
              "algorithmConfig": {
                  "treeAhConfig": {
                      "leafNodeEmbeddingCount": 1000,
                      "fractionLeafNodesToSearch": 1.0
                  }
                }
              }
          }
        }
      },
      "metadata": { ... }
    }
  }
}

The tables below lists the fields for NearestNeighborSearchConfig.

DistanceMeasureType

FeatureNormType

TreeAhConfig

These are the fields to select for the tree-AH algorithm (Shallow tree + Asymmetric Hashing).

BruteForceConfig

This option implements the standard linear search in the database for each query. There are no fields to configure for a brute force search. To select this algorithm, pass an empty object for BruteForceConfig to algorithmConfig.

Input data requirements

Upload your dataset to a Cloud Storage location. Make sure the files in the JSON Lines format.

The files must be in JSON Lines format. The following sample is from the image classification example-based explanations notebook:

{"id": "0", "bytes_inputs": {"b64": "..."}}
{"id": "1", "bytes_inputs": {"b64": "..."}}
{"id": "2", "bytes_inputs": {"b64": "..."}}

Update the index or configuration

Vertex AI lets you update a model's nearest neighbor index or Example configuration. This is useful if you'd like to update your model without re-indexing its dataset. For example, if your model's index contains 1,000 instances, and you'd like to add 500 more instances, you can call UpdateExplanationDataset to add to the index without re-processing the original 1,000 instances.

To update the explanation dataset:

def update_explanation_dataset(model_id, new_examples):
    response = clients["model"].update_explanation_dataset(model=model_id,  examples=new_examples)
    update_dataset_response = response.result()
    return update_dataset_response

PRESET_CONFIG = {
    "modality": "TEXT",
    "query": "FAST"
}
NEW_DATASET_FILE_PATH = "new_dataset_path"
NUM_NEIGHBORS_TO_RETURN = 10

EXAMPLES = aiplatform.Examples(presets=PRESET_CONFIG,
                                    gcs_source=aiplatform.types.io.GcsSource(uris=[NEW_DATASET_FILE_PATH]),
                                      neighbor_count=NUM_NEIGHBORS_TO_RETURN)

MODEL_ID = 'model_id'
update_dataset_response = update_explanation_dataset(MODEL_ID, EXAMPLES)

Usage notes:

• The model_id remains unchanged after the UpdateExplanationDataset operation.

• The UpdateExplanationDataset operation affects only the Model resource; it does not update any associated DeployedModels. This means that a deployedModel's index contains the dataset at the time it was deployed. To update a deployedModel's index, you must re-deploy the updated model to an endpoint.

Override the configuration when getting online explanations

When you request an explanation, you can override some of the parameters on the fly by specifying the ExplanationSpecOverride field.

Depending on the application, some constraints might be desirable on the kind of explanations that are returned. For example, to ensure diversity of explanations, a user can specify a crowding parameter which dictates that no single type of examples are over-represented in the explanations. Concretely, if a user is trying to understand why a bird was labeled as a plane by their model, they might not be interested in seeing too many bird examples as explanations to better investigate the root cause.

The following table summarizes the parameters that can be overridden for an example-based explanation request:

The Vector Search Filtering describes these parameters in more details.

Here's an example of a JSON request body with overrides:

{
  "instances":[
    {
      "id": data[0]["id"],
      "bytes_inputs": {"b64": bytes},
      "restricts": "",
      "crowding_tag": ""
    }
  ],
  "explanation_spec_override": {
    "examples_override": {
      "neighbor_count": 5,
      "crowding_count": 2,
      "restrictions": [
        {
          "namespace_name": "label",
          "allow": ["Papilloma", "Rift_Valley", "TBE", "Influenza", "Ebol"]
        }
      ]
    }
  }
}

What's next

• Deploy your Model to an Endpoint and get example-based explanations.

Here's an example of the response from an example-based explain request:

[
   {
      "neighbors":[
         {
            "neighborId":"311",
            "neighborDistance":383.8
         },
         {
            "neighborId":"286",
            "neighborDistance":431.4
         }
      ],
      "input":"0"
   },
   {
      "neighbors":[
         {
            "neighborId":"223",
            "neighborDistance":471.6
         },
         {
            "neighborId":"55",
            "neighborDistance":392.7
         }
      ],
      "input":"1"
   }
]

Pricing

See the section on example-based explanations in the pricing page.