HolisticLandmarker class

Performs holistic landmarks detection on images.

Signature:

export declare class HolisticLandmarker extends VisionTaskRunner

Extends: VisionTaskRunner

Properties

Property Modifiers Type Description
FACE_LANDMARKS_CONTOURS static Connection[] Landmark connections to draw the face's contour.
FACE_LANDMARKS_FACE_OVAL static Connection[] Landmark connections to draw the face's oval.
FACE_LANDMARKS_LEFT_EYE static Connection[] Landmark connections to draw the connection between a face's left eye.
FACE_LANDMARKS_LEFT_EYEBROW static Connection[] Landmark connections to draw the connection between a face's left eyebrow.
FACE_LANDMARKS_LEFT_IRIS static Connection[] Landmark connections to draw the connection between a face's left iris.
FACE_LANDMARKS_LIPS static Connection[] Landmark connections to draw the connection between a face's lips.
FACE_LANDMARKS_RIGHT_EYE static Connection[] Landmark connections to draw the connection between a face's right eye.
FACE_LANDMARKS_RIGHT_EYEBROW static Connection[] Landmark connections to draw the connection between a face's right eyebrow.
FACE_LANDMARKS_RIGHT_IRIS static Connection[] Landmark connections to draw the connection between a face's right iris.
FACE_LANDMARKS_TESSELATION static Connection[] Landmark connections to draw the face's tesselation.
HAND_CONNECTIONS static Connection[] An array containing the pairs of hand landmark indices to be rendered with connections.
POSE_CONNECTIONS static Connection[] An array containing the pairs of pose landmark indices to be rendered with connections.

Methods

Method Modifiers Description
createFromModelBuffer(wasmFileset, modelAssetBuffer) static Initializes the Wasm runtime and creates a new HolisticLandmarker based on the provided model asset buffer.
createFromModelPath(wasmFileset, modelAssetPath) static Initializes the Wasm runtime and creates a new HolisticLandmarker based on the path to the model asset.
createFromOptions(wasmFileset, holisticLandmarkerOptions) static Initializes the Wasm runtime and creates a new HolisticLandmarker from the provided options.
detect(image, callback) Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode image.
detect(image, imageProcessingOptions, callback) Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode image.
detect(image) Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode image.
detect(image, imageProcessingOptions) Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode image.
detectForVideo(videoFrame, timestamp, callback) Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode video.
detectForVideo(videoFrame, timestamp, imageProcessingOptions, callback) Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the holisticLandmarker is created with running mode video.
detectForVideo(videoFrame, timestamp) Performs holistic landmarks detection on the provided video frame and returns the result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode video.
detectForVideo(videoFrame, timestamp, imageProcessingOptions) Performs holistic landmarks detection on the provided video frame and waits synchronously for the response. Only use this method when the HolisticLandmarker is created with running mode video.
setOptions(options) Sets new options for this HolisticLandmarker.Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined.

HolisticLandmarker.FACE_LANDMARKS_CONTOURS

Landmark connections to draw the face's contour.

Signature:

static FACE_LANDMARKS_CONTOURS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_FACE_OVAL

Landmark connections to draw the face's oval.

Signature:

static FACE_LANDMARKS_FACE_OVAL: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_EYE

Landmark connections to draw the connection between a face's left eye.

Signature:

static FACE_LANDMARKS_LEFT_EYE: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_EYEBROW

Landmark connections to draw the connection between a face's left eyebrow.

Signature:

static FACE_LANDMARKS_LEFT_EYEBROW: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LEFT_IRIS

Landmark connections to draw the connection between a face's left iris.

Signature:

static FACE_LANDMARKS_LEFT_IRIS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_LIPS

Landmark connections to draw the connection between a face's lips.

Signature:

static FACE_LANDMARKS_LIPS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_EYE

Landmark connections to draw the connection between a face's right eye.

Signature:

static FACE_LANDMARKS_RIGHT_EYE: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_EYEBROW

Landmark connections to draw the connection between a face's right eyebrow.

Signature:

static FACE_LANDMARKS_RIGHT_EYEBROW: Connection[];

HolisticLandmarker.FACE_LANDMARKS_RIGHT_IRIS

Landmark connections to draw the connection between a face's right iris.

Signature:

static FACE_LANDMARKS_RIGHT_IRIS: Connection[];

HolisticLandmarker.FACE_LANDMARKS_TESSELATION

Landmark connections to draw the face's tesselation.

Signature:

static FACE_LANDMARKS_TESSELATION: Connection[];

HolisticLandmarker.HAND_CONNECTIONS

An array containing the pairs of hand landmark indices to be rendered with connections.

Signature:

static HAND_CONNECTIONS: Connection[];

HolisticLandmarker.POSE_CONNECTIONS

An array containing the pairs of pose landmark indices to be rendered with connections.

Signature:

static POSE_CONNECTIONS: Connection[];

HolisticLandmarker.createFromModelBuffer()

Initializes the Wasm runtime and creates a new HolisticLandmarker based on the provided model asset buffer.

Signature:

static createFromModelBuffer(wasmFileset: WasmFileset, modelAssetBuffer: Uint8Array): Promise<HolisticLandmarker>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
modelAssetBuffer Uint8Array A binary representation of the model.

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.createFromModelPath()

Initializes the Wasm runtime and creates a new HolisticLandmarker based on the path to the model asset.

Signature:

static createFromModelPath(wasmFileset: WasmFileset, modelAssetPath: string): Promise<HolisticLandmarker>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
modelAssetPath string The path to the model asset.

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.createFromOptions()

Initializes the Wasm runtime and creates a new HolisticLandmarker from the provided options.

Signature:

static createFromOptions(wasmFileset: WasmFileset, holisticLandmarkerOptions: HolisticLandmarkerOptions): Promise<HolisticLandmarker>;

Parameters

Parameter Type Description
wasmFileset WasmFileset A configuration object that provides the location of the Wasm binary and its loader.
holisticLandmarkerOptions HolisticLandmarkerOptions The options for the HolisticLandmarker. Note that either a path to the model asset or a model buffer needs to be provided (via baseOptions).

Returns:

Promise<HolisticLandmarker>

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter Type Description
image ImageSource An image to process.
callback HolisticLandmarkerCallback The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource, imageProcessingOptions: ImageProcessingOptions, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter Type Description
image ImageSource An image to process.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference.
callback HolisticLandmarkerCallback The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource): HolisticLandmarkerResult;

Parameters

Parameter Type Description
image ImageSource An image to process. The landmarker result. Any masks are copied to avoid lifetime limits. The detected pose landmarks.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detect()

Performs holistic landmarks detection on the provided single image and waits synchronously for the response. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode image.

Signature:

detect(image: ImageSource, imageProcessingOptions: ImageProcessingOptions): HolisticLandmarkerResult;

Parameters

Parameter Type Description
image ImageSource An image to process. The landmarker result. Any masks are copied to avoid lifetime limits. The detected pose landmarks.
imageProcessingOptions ImageProcessingOptions

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms.
callback HolisticLandmarkerCallback The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and invokes the callback with the response. The method returns synchronously once the callback returns. Only use this method when the holisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, imageProcessingOptions: ImageProcessingOptions, callback: HolisticLandmarkerCallback): void;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference.
callback HolisticLandmarkerCallback The callback that is invoked with the result. The lifetime of the returned masks is only guaranteed for the duration of the callback.

Returns:

void

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and returns the result. This method creates a copy of the resulting masks and should not be used in high-throughput applications. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number): HolisticLandmarkerResult;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms. The landmarker result. Any masks are copied to extend the lifetime of the returned data.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.detectForVideo()

Performs holistic landmarks detection on the provided video frame and waits synchronously for the response. Only use this method when the HolisticLandmarker is created with running mode video.

Signature:

detectForVideo(videoFrame: ImageSource, timestamp: number, imageProcessingOptions: ImageProcessingOptions): HolisticLandmarkerResult;

Parameters

Parameter Type Description
videoFrame ImageSource A video frame to process.
timestamp number The timestamp of the current frame, in ms.
imageProcessingOptions ImageProcessingOptions the ImageProcessingOptions specifying how to process the input image before running inference. The detected holistic landmarks.

Returns:

HolisticLandmarkerResult

HolisticLandmarker.setOptions()

Sets new options for this HolisticLandmarker.

Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined.

Signature:

setOptions(options: HolisticLandmarkerOptions): Promise<void>;

Parameters

Parameter Type Description
options HolisticLandmarkerOptions The options for the holistic landmarker.

Returns:

Promise<void>