Stream configurations

Android includes features allowing camera clients to choose optimal camera streams for specific use cases and to ensure that certain stream combinations are supported by the camera device. A stream configuration refers to a single camera stream configured in the camera device and a stream combination refers to one or more sets of streams configured in the camera device. For more on these features, see recommended stream configurations and API to query feature combinations.

Reference implementation

There is a vendor-side reference implementation of the recommended configuration streams and the API to query stream combination features. You can find this implementation at QCamera3HWI.cpp

Camera vendors can advertise recommended stream configurations for specific use cases to camera clients. These recommended stream configurations, which are subsets of StreamConfigurationMap, can help camera clients choose optimal configurations.

Although StreamConfigurationMap provides exhaustive stream configuration information to camera clients, it doesn't provide any information about the efficiency, power, or performance impacts of choosing one stream over another. Camera clients can freely choose from all the possible stream configurations but in many cases, this leads to clients using sub-optimal camera configurations and apps making time-consuming exhaustive searches.

For example, although some processed YUV formats are required and must be supported, the camera device might not have native support for the formats. This results in an additional processing pass for the format conversion and reduces efficiency. The size and corresponding aspect ratio can also have a similar impact making particular dimensions preferable in terms of power and performance.

Your recommended stream configuration maps aren't required to be exhaustive compared to the StreamConfigurationMap. The suggested configuration maps must follow the requirements in the implementation section and can include any of the available formats, sizes, or other values found in StreamConfigurationMap. Hidden formats, sizes, or other values not found in StreamConfigurationMap can't be included in recommended stream configuration maps.

All tests remain unchanged and aren't relaxed depending on the recommended stream configurations.

The recommended stream configurations provided by the camera implementation are optional and the camera client can ignore them.

Implementation

Follow these steps to implement this feature.

Metadata entries

To enable this feature the Camera HAL must populate the following static metadata entries:

  • android.scaler.availableRecommendedStreamConfigurations: The recommended subset of stream configurations for specific use cases. The declaration uses bitmaps indicating the suggested use cases in the form of [1 << PREVIEW | 1 << RECORD..]. The use cases extend the regular (format, width, height, input) tuple with one additional entry. Nonexisting public use cases or any other bits set within the range [PUBLIC_END, VENDOR_START] are prohibited.

    This information is stored in the availableRecommendedStreamConfigurations metadata tag.

    The following example shows an array for a recommended stream configuration for a camera device that only supports 4K and 1080p, where both resolutions are preferred for video recording but only 1080p is suggested for preview.

    [3840, 2160, HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED,
ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT,
(1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_RECORD |
1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_SNAPSHOT |
1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_VIDEO_SNAPSHOT),
1920, 1080, HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED,
ANDROID_SCALER_AVAILABLE_STREAM_CONFIGURATIONS_OUTPUT,
(1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_PREVIEW |
1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_RECORD |
1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_SNAPSHOT |
1 << ANDROID_SCALER_AVAILABLE_RECOMMENDED_STREAM_CONFIGURATIONS_VIDEO_SNAPSHOT)]

  • android.depth.availableRecommendedDepthStreamConfigurations (available only if supported by device): The recommended depth dataspace stream configurations suggested for this camera device. Similar to the above metadata entry, an additional use case bitmap indicates the suggested use cases.

    This information is stored in the availableRecommendedInputOutputFormatsMap metadata tag.

  • android.scaler.availableRecommendedInputOutputFormatsMap (available only if supported by device): The mapping of recommended image formats that are suggested for this camera device for input streams, to their corresponding output formats.

    This information is stored in the availableRecommendedDepthStreamConfigurations metadata tag.

This information is available to camera clients through the RecommendedStreamConfigurationMap API.

Required use cases

Recommended stream configurations must be provided for the following use cases and meet the corresponding requirements:

Use case Requirement
PREVIEW A preview must only include nonstalling processed stream configurations with output formats such as YUV_420_888 and IMPLEMENTATION_DEFINED.
RECORD A video record must include stream configurations that match the advertised supported media profiles with the IMPLEMENTATION_DEFINED format.
VIDEO_SNAPSHOT A video snapshot must include stream configurations that are at least as large as the maximum RECORD resolutions and only with the BLOB + DATASPACE_JFIF format/dataspace combination (JPEG). The configurations shouldn't cause preview glitches and should be able to run at 30 fps.
SNAPSHOT Snapshot stream configurations must include at least one with a size close to android.sensor.info.activeArraySize with the BLOB + DATASPACE_JFIF format/dataspace combination (JPEG). Taking into account restrictions on aspect ratio, alignment, and other vendor-specific restrictions, the area of the maximum suggested size shouldn't be less than 97% of the sensor array size area.
ZSL (if supported) If supported by the camera device, recommended input stream configurations must only be advertised together with other processed or stalling output formats.
RAW (if supported) If supported by the camera device, recommended raw stream configurations must only include RAW based output formats.

Other use cases

You can provide additional recommended configuration streams for use cases specific to your implementation.

Validation

To test your implementation of the recommended configuration streams, run the following CTS and VTS tests:

API to query feature combinations

Starting in Android 15, the Android platform provides an API to query feature combinations. This API lets camera clients query whether a specified combination of features can be supported by the device. This API is necessary because the camera2 API models different features such as 4k, 60fps, HDR video, UltraHDR, Ultrawide zoom, and stabilization as orthogonal controls.

Requirements

To support the API to query feature combinations, the camera HAL must implement version 3 of the ICameraDevice interface. For details, see the Implementation section.

When the API is supported, preview stabilization must be orthogonal to other features. This means that for a camera device that supports preview stabilization, the return value of isStreamCombinationWithSettingsSupported for a particular combination must be the same value when preview stabilization is on or off. This reduces the search space for feature combination queries.

In addition, for media performance class 15, the primary rear camera must support preview stabilization with 10-bit HLG10 preview for 1080p and 720p preview and maximum size JPEGs. For more details on these requirements, see section 2.2.7.2. Camera of the CDD.

Implementation

To support the API to query feature combinations, implement the following feature combination query APIs in version 3 of ICameraDevice:

For versions lower than version 3 of the ICameraDevice interface, the HAL should implement the isStreamCombinationSupported method.

For more information on the feature combinations queried by the API, see the documentation on sessionConfigurationQueryVersion in system/media/camera/docs/metadata_definitions.xml.

For a reference implementation of this feature, see hardware/google/camera/devices/EmulatedCamera/hwl/.

Public APIs

Apps can use the following public APIs to query supported feature combinations for the device:

  • CameraDevice.CameraDeviceSetup: A limited representation of CameraDevice that can be used to query feature combinations without requiring a CameraDevice instance.

  • getCameraDeviceSetup: Acquires a CameraDeviceSetup object for a given camera ID if isCameraDeviceSetupSupported returns true.

  • INFO_SESSION_CONFIGURATION_QUERY_VERSION: Supports feature combination queries if this value is VANILLA_ICE_CREAM or higher.

  • OutputConfiguration: A class describing camera output, which can contain a deferred surface for the purpose of low latency feature combination queries.

  • SessionConfiguration: A utility class describing the session configuration including stream combinations and session parameters, which can be used for feature combination queries.

Validation

To validate your implementation of this feature, use the following VTS, CTS, and Camera ITS (CTS Verifier) tests:

VTS

CTS

Camera ITS