How can I force the model to show labels completely?

[amir-sbg]

Hi,

I use yolov8 model for object detection ( the code is here ).
I have a problem that when the number of objects increases, my model does not display the label format that I specified, and only displays the object code, how can I force the mode to display the specified label in any case completely?

Thanks

[SkalskiP]

Hi @amir-sbg 👋🏻 Could you upload a cingle frame that results in that unexpected behavior?

[amir-sbg]

Hello, Tank you for your response. Yes, for example in this frame all labels are just the type of object. ​While in the code the label is:

On Wed, 31 Khordad 1402 08:19 AM, Piotr Skalski ***@***.***> wrote:

Hi @amir-sbg 👋🏻 Could you upload a cingle frame that results in that unexpected behavior? —

Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- This email was Anti Virus checked by Security Gateway.

[amir-sbg]

​Hello, Tank you for your response. Yes, for example in this frame all labels are just the type of object. While the code of labels is: labels = [ f"#{tracker_id} {model.model.names[class_id]} {confidence:0.2f}" for _, confidence, class_id, tracker_id in detections] when the number of objects are a few it shows the complete code but other wise like this frame it only shows the object numbers. the frame is attached. thanks, Amir

On Wed, 31 Khordad 1402 08:19 AM, Piotr Skalski ***@***.***> wrote:

Hi @amir-sbg 👋🏻 Could you upload a cingle frame that results in that unexpected behavior? —

Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- This email was Anti Virus checked by Security Gateway.

[amir-sbg]

[SkalskiP]

I look at the code you included, and it makes no sense. I'd love to understand what is happening.

Could you upload a video you use for inference? I'll try to debug it live (have your code and your data).

[amir-sbg]

​My full code for this part (without object tracking part) is: ```python import ultralytics import cv2 import argparse from ultralytics import YOLO import supervision as sv import numpy as np def parse_arguments() -> argparse.Namespace: parser = argparse.ArgumentParser(description="YOLOv8 live") parser.add_argument( "--webcam-resolution", default=[640, 360], nargs=2, type=int ) args = parser.parse_args() return args def object_detection_setting(): args = parse_arguments() frame_width, frame_height = args.webcam_resolution # cap_outside = cv2.VideoCapture(0) cap_outside = cv2.VideoCapture("videoplayback - Trim.mp4") cap_outside.set(cv2.CAP_PROP_FRAME_WIDTH, frame_width) cap_outside.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_height) model = YOLO("yolov8l.pt") ZONE_POLYGON = [ np.array([ [0, 0], [0.25, 0], [0.25, 1], [0, 1] ]), np.array([ [0.25, 0], [0.75, 0], [0.75, 1], [0.25, 1] ]), np.array([ [0.75, 0], [1, 0], [1, 1], [0.75, 1] ]), ] zone_polygons = (ZONE_POLYGON * np.array(args.webcam_resolution)).astype(int) zones = [ sv.PolygonZone( polygon=polygon, frame_resolution_wh=tuple(args.webcam_resolution) ) for polygon in zone_polygons] colors = sv.ColorPalette.default() zone_annotators = [ sv.PolygonZoneAnnotator( zone=zone, color=colors.by_idx(index), thickness=2, text_thickness=1, text_scale=0.5 ) for index, zone in enumerate(zones)] box_annotators = [ sv.BoxAnnotator( color=colors.by_idx(index), thickness=1, text_thickness=1, text_scale=0.3 ) for index in range(len(zone_polygons))] return cap_outside,model,zones,zone_annotators,box_annotators def needed_detecteions_picker(detections): person = detections.class_id == 0 bicycle = detections.class_id == 1 car = detections.class_id == 2 motorcycle = detections.class_id == 3 bus = detections.class_id == 5 truck = detections.class_id == 7 traffic_light = detections.class_id == 9 needed_detections=person+ bicycle+ car+ motorcycle+ bus+ truck+ traffic_light detections=detections[needed_detections] detections=detections[detections.confidence >= 0.5] return detections def main(): cap_outside,model,zones,zone_annotators,box_annotators=object_detection_setting() CLASS_ID = [0,1,2,3,5,7,9] while True: _, frame = cap_outside.read() frame=cv2.flip(frame, 1) result = model(frame, agnostic_nms=True)[0] detections = sv.Detections.from_yolov8(result) detections=needed_detecteions_picker(detections) labels = [ f"{model.model.names[class_id]} {confidence:0.2f}" for _, confidence, class_id, _ in detections ] counter=0 regions=[0,0,0] for zone, zone_annotator, box_annotator in zip(zones, zone_annotators, box_annotators): mask = zone.trigger(detections=detections) detections_filtered = detections[mask] frame = box_annotator.annotate(scene=frame, detections=detections_filtered,labels=labels) frame = zone_annotator.annotate(scene=frame) regions[counter]=int(zone.current_count) counter+=1 cv2.imshow("yolov8", frame) if (cv2.waitKey(1) == 1): break if __name__ == "__main__": main() ``` Anyway with or without object tracking parts, I have this problem when the number of objects are increased (sometimes when I use webcam or when the number of objects in the video are a few, I do not have this problem and it works properly). And the sample data (a short video) is attached. Thanks, Amir

On Sat, 10 Tir 1402 05:43 PM, Piotr Skalski ***@***.***> wrote:

I look at the code you included, and it makes no sense. I'd love to understand what is happening. Could you upload a video you use for inference? I'll try to debug it live (have your code and your data). —

Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

-- This email was Anti Virus checked by Security Gateway.

[SkalskiP]

@amir-sbg I don't see any video :/

[amir-sbg]

Can you see it now? 😅

videoplayback.-.Trim.mp4

[SkalskiP]

Hi @amir-sbg 👋🏻 ! I'll try to take care of that this week. I'm probably going to be super busy. But I'll do my best.

[amir-sbg]

Ok sir. Thank you.
To be more familiar with the problem, I attached two frames of a runtime. As you can see in first one, some of the bounding boxes are labeled with complete format of label :)) but the car with blue bounding box is labeled just with object number :((
In second one, all of the bounding boxes are labeled just with object numbers :((

first:

second:

[SkalskiP]

Hi, @amir-sbg 👋🏻! Sorry I made you wait so long... I was working on other things on Monday. But at least I have a fix for you 🔥

Fixed code

import ultralytics
import cv2
import argparse
from ultralytics import YOLO
import supervision as sv
import numpy as np


def parse_arguments() -> argparse.Namespace:
    parser = argparse.ArgumentParser(description="YOLOv8 live")
    parser.add_argument(
        "--webcam-resolution",
        default=[640, 360],
        nargs=2,
        type=int
    )
    args = parser.parse_args()
    return args


def object_detection_setting():
    args = parse_arguments()
    frame_width, frame_height = args.webcam_resolution

    # cap_outside = cv2.VideoCapture(0)
    cap_outside = cv2.VideoCapture("test.mp4")
    cap_outside.set(cv2.CAP_PROP_FRAME_WIDTH, frame_width)
    cap_outside.set(cv2.CAP_PROP_FRAME_HEIGHT, frame_height)

    model = YOLO("yolov8l.pt")

    ZONE_POLYGON = [
        np.array([
        [0, 0],
        [0.25, 0],
        [0.25, 1],
        [0, 1]
    ]),
        np.array([
        [0.25, 0],
        [0.75, 0],
        [0.75, 1],
        [0.25, 1]
    ]),
        np.array([
        [0.75, 0],
        [1, 0],
        [1, 1],
        [0.75, 1]
    ]),

    ]
    zone_polygons = (ZONE_POLYGON * np.array(args.webcam_resolution)).astype(int)

    zones = [
    sv.PolygonZone(
        polygon=polygon,
        frame_resolution_wh=tuple(args.webcam_resolution)
    )
    for polygon
    in zone_polygons]


    colors = sv.ColorPalette.default()
    zone_annotators = [
    sv.PolygonZoneAnnotator(
        zone=zone,
        color=colors.by_idx(index),
        thickness=2,
        text_thickness=1,
        text_scale=0.5
    )
    for index, zone
    in enumerate(zones)]

    box_annotators = [
    sv.BoxAnnotator(
        color=colors.by_idx(index),
        thickness=1,
        text_thickness=1,
        text_scale=0.3
        )
    for index
    in range(len(zone_polygons))]

    return cap_outside,model,zones,zone_annotators,box_annotators

def needed_detecteions_picker(detections):
        person        = detections.class_id == 0
        bicycle       = detections.class_id == 1
        car           = detections.class_id == 2
        motorcycle    = detections.class_id == 3
        bus           = detections.class_id == 5
        truck         = detections.class_id == 7
        traffic_light = detections.class_id == 9

        needed_detections=person+ bicycle+ car+ motorcycle+ bus+ truck+ traffic_light

        detections=detections[needed_detections]
        detections=detections[detections.confidence >= 0.5]

        return detections

def main():
    cap_outside,model,zones,zone_annotators,box_annotators=object_detection_setting()
    CLASS_ID = [0,1,2,3,5,7,9]

    while True:
        _, frame = cap_outside.read()
        frame=cv2.flip(frame, 1)

        result = model(frame, agnostic_nms=True)[0]
        detections = sv.Detections.from_yolov8(result)

        detections=needed_detecteions_picker(detections)

        # remove labels definition from here
        # labels = [
        #     f"{model.model.names[class_id]} {confidence:0.2f}"
        #     for _, _, confidence, class_id, _
        #     in detections
        # ]

        counter=0
        regions=[0,0,0]

        for zone, zone_annotator, box_annotator in zip(zones, zone_annotators, box_annotators):
            mask = zone.trigger(detections=detections)
            detections_filtered = detections[mask]

            # move labels definition here
            labels = [
                f"{model.model.names[class_id]} {confidence:0.2f}"
                for _, _, confidence, class_id, _
                in detections_filtered
            ]

            frame = box_annotator.annotate(scene=frame, detections=detections_filtered,labels=labels)
            frame = zone_annotator.annotate(scene=frame)
            regions[counter]=int(zone.current_count)
            counter+=1


        cv2.imshow("yolov8", frame)

        if (cv2.waitKey(1) == 1):
            break


if __name__ == "__main__":
    main()

Explenation

The problem was defining labels before filtering detections. You filter detections inside for loop - detections_filtered = detections[mask], and as a result, your labels and detections_filtered have different lengths.

Actions

This behavior is non-intuitive. It would be much better if the script broke and provided an explanation than work incorrectly. All because of this line inside BoxAnnotator:

text = (
    f"{class_id}"
    if (labels is None or len(detections) != len(labels))
    else labels[i]
)

As you can see, if len(detections) != len(labels) annotator defaults to returning just class_id.

@hardikdava what do you think? I'd add len(detections) != len(labels) to check higher in the logic and simply raise Exception.

[hardikdava]

@SkalskiP all data should be validated beore any annotation process. It is a good idea to check data prior to any operations.

[Rudrashish-Das]

How can I just show the boxes and no labels?

[NaufalGhifari]

As per Supervision documentation: https://supervision.roboflow.com/0.15.0/annotators/#__tabbed_1_1

The default annotate() function does not include labels, only outlines. Code snippet:

import supervision as sv

image = ...
detections = sv.Detections(...)

# define annotator
bounding_box_annotator = sv.BoundingBoxAnnotator()

# call annotate() function
annotated_frame = bounding_box_annotator.annotate(
    scene=image.copy(),
    detections=detections
    )

Edit: formatting