Alpaca 

Taxonomy
LiDAR Cuboid | IN PROGRESS
Contents

 Annotation Rules
 Edge Cases
 Critical vs. Non-Critical Errors
 Labels Description & Examples

Taxonomy Updates:

6/16 - The minimum LiDAR points has changed for the project, it is now as follows:

 For stationary objects: a minimum of 1 lidar points are needed. Please see
examples here
 For moving objects: the number of LiDAR points needed for the beginning
and end of an object's path is now 1 LiDAR point.
 Please see examples here

5/10 - Please take note of the following updates:

1. New Object Motion attribute applied to ‘Car or Truck’, ‘Motorcycle’, ‘Bicycle’,

‘Emergency Vehicle’

1. 3 attributes: ‘Moving’, ‘Stopped’, ‘Parked’

2. 4 new labels - edge cases included for each label

 1. Grocery Cart - no attributes
2. Scout Bot - no attributes
3. Pedestrian Crossing Light - 4 attributes

1. ‘Can Cross’, ‘Stop’, ‘Occluded’, ‘Unclear’

4. Traffic Light - no attributes

3. New Traffic light Annotation Rules

4. Include pedestrians close to bot even if there are <5 lidar points (use lidar shadow to
annotate to best of ability)

4/16 - Please take note of the following updates:

1. Mild, Moderate, & Severe Echo - Please label scenes with mild and moderate echo. For
scenes with severe echo, please report the issue.
2. Edge Cases - (1) Garbage truck picking up a trash can, (2) Utility Pole Wires, (3) Speed
Bumps (do not label), & (4) Lidar points disappear and reappear

3/30 - Standard sizing for a pedestrian or a vehicle has been added for reference within Labels
Descriptions & Examples.
3/17 - Please take note of reflection and pedestrian updates. Lidar points from any reflections,
regardless of the source, should NOT be included. Please include the entire pedestrian when labeling,
even if there are missing lidar points for the upper body. Be sure to include the entire pedestrian by
referencing the camera view.
3/4 - Please take note that pitch and roll is enabled for the project. Any vehicle that is on a hill (incline or
decline) should be adjusted using pitch and roll functions.

Instructions
In this task you will draw 3D LiDAR cuboids on the following objects:
Labels

Pedestrians Stroller Trash Can

Car or Truck Wheelchair Pole

Motorcycle Animal Construction Sign

Bicycle Generic Obstacle Basketball Hoop Base

Emergency Vehicle Fire Hydrant Mailbox

Scout Bot Grocery Cart Pedestrian Crossing Light

Traffic Light

In this task you will also label the following attributes:

Attribute Attribute Options Description

Object Motion 1. Moving “Moving” - vehicle is operational and in motion

Attributes apply for the 2. Stopped “Stopped” - vehicle is operational and NOT in motion
following labels: 3. Parked
Car or Truck  Example: vehicle waiting at a stop sign or
Motorcycle traffic light (in motion for some frames, and
Bicycle not in motion in other frames)
Emergency Vehicle
“Parked” - vehicle is NOT operational

Pedestrian Crossing Light 1. Can cross “Can cross” - signaling pedestrians can proceed to the
attributes 2. Stop other side of the street
3. Occluded “Stop” - signaling pedestrians cannot proceed to the other
4. Unclear side of the street
“Occluded” - the crossing light is blocked and cannot be
seen. Examples of a blockage can include but not limited to
an object, event, or weather
“Unclear” - the crossing light is NOT blocked but the signal
cannot be determined to do poor visibility.
Back to top

Annotation Rules

Rule Description Example

Minimum Only label target objects Stationary objects:

LiDAR that have at least 5 A stationary cone is seen in camera 2 with LiDAR points from frames 1 to 45.
point LiDAR points at the
beginning and end of
an object’s path.
Stationary Objects:
Non-moving objects
must have 1
LiDAR point at least in
1 frame of the entire
scene. The objects
must be seen in camera
view and within 40m of
the ego.
Moving Objects:
The minimum LiDAR
points at the beginning
and end of an object’s
path has changed to 1
LiDAR point. The
objects must be seen in
camera view and within
40m of the ego.
Cuboid Do NOT mark any
Status object as stationary. All
cuboids should be
dynamic and objects
should only be labeled
when they meet the
lidar point rule.

Cuboid There should be no

Sizing more than +/- 10 cm of
space between the
cuboid border and the
closest point on the
object

Extremities Label all points that

belong to the target
object

  

Distant Do not label anything n/a

Objects greater than 40 meters
away

Echo Include instances of

echo/splitting points in
the cuboid.
Adjust the cuboid so the
front of the object
includes the echo/split.
 
 
Any task with mild echo Mild Echo (2 examples):
should be labeled. Echo
is considered mild if it is
limited to a few objects
and frames, and lidar
points are still visible
Please report Severe
Echo’ as a Reported
Issue
Severe Echo (flag for review)
Please report all cases of Echo to the team lead.
Noise Some noise may look
like echo/splitting
points. Do not include
that in the cuboid.
If you’re unsure whether
it is an echo or noise,
draw a cuboid around
the vehicle only.
 

Reflections DO NOT include LiDAR

points for
reflections regardless 
of source:
 
Some sources may
include the following but
can apply to any
source:
 
 Rain on
the road
 Shiny
fences or
walls
 Other
shiny
objects

                                  

ID IDs must be kept  n/a

Consistenc consistent for an object
y across all the frames
when it is visible.
 
Make sure IDs do not
switch between objects.
If an object appears
then disappears across
frames, that object must
keep the same ID.

Cuboids for Use camera view to

poles label the entire height of
the pole
Cuboid Any vehicle or object
pitch and that is on a hill (incline
roll or decline) should be
adjusted using pitch
and roll functions.
All Traffic Draw a tight cuboid
Lights around the traffic light
(includes box.
pedestrian Label traffic lights that
crossing are facing toward the
light) ego so that the bulb
colors and symbols can
be seen via camera
Please label ALL traffic
lights where bulb light is
visible, even if the traffic
light is not facing the
ego.
Also draw a ‘Pole’
cuboid around the
vertical part of the pole
holding the traffic light

Label this (green bulb can be seen):

Back to top

Edge Cases

Edge Case How to Handle It Example

Generic Only label generic
obstacles on obstacles that are
grass or soil on concrete
sidewalk,
ramp/driveway, or
road

Generic Label it as generic n/a

obstacle that is obstacle
on both grass
and sidewalk
Garbage truck Label both objects
picking up a separately. The
garbage can garbage can is
labeled as ‘Trash
Can’ and the
garbage truck is
labeled as ‘Car or
Truck’. The
cuboids will
overlap.
Utility pole Label these as
wires ‘Pole’ with 1
connecting to cuboid.
the ground at
an angle
Speed Bumps Do NOT label
speed bumps

Lidar points Only label target Example:

disappear and objects when at Frame 1: car has 4 lidar points
reappear least 5 LiDAR Frame 2-19: car has 5 lidar points
again points appear. Frame 20-25: car has 0 lidar points
The cuboid Frame 26-30: car has 5 lidar points
should remain Frame 31 to last frame: car has 4 lidar points
labeled until the Answer: Cuboid should appear throughout frames 2 and 30.
last frame in the
sequence that
 shows at least 5
LiDAR points.
Continue to keep
the cuboid, even if
LiDAR points
disappear in
middle frames.

Pedestrians Label pedestrian Example

close to ego close to ego
regardless of the
number of lidar
points. (Even if
there are less
than 5 lidar points
for the
pedestrian).
If pedestrians are
within the green
bulb of the ego,
please  label
them.
Frame 1:
Frame 2 (please keep annotated):
Back to top

Critical vs. Non-Critical Errors

Critical Errors Non-Critical Errors

1.   Object with at least 5 LiDAR points and within 40m is not labeled. 1. Object echo
Example: distance is >40m (DO NOT LABEL): and splitting
points are
not labeled

2. Object
noise is
labeled

3. Object
reflection is
  labeled
2.   Cuboids are mislabeled.
 
3.   More than +/- 10 cm of space exists between the cuboid and the closest point of the object.
 
Back to top

Labels Description & Examples

Labels/Attribut Description Example

es
Pedestrian Anyone who is walking,
standing, sitting, or is walking
a bike/motorcycle/scooter.
 
Include anything the
pedestrian is carrying (e.g.
shopping bags, umbrella etc.)
 
Any people riding vehicles
should not be labeled as
pedestrians. Instead, people
riding vehicles should be  
included as a part of that
vehicle.
Be sure to include the entire
pedestrian. Even if lidar
points are only available for
the lower body, adjust the
pedestrian height to include
the upper portion of the
pedestrian by referencing the
camera view.
Standard sizing for
a PEDESTRIAN should be .
5-1.3m x .4-1.3m x 1.6-1.9m
.
Please be sure to label all
pedestrians that are close
to the ego. See this edge
case detail for more
information
Car or Truck Any Car, Pickup Truck or
SUV.
 
Any moving or static vehicles,
including parked ones on
driveway.
 
Please note, boats and
trailers behind cars or trucks
should be labeled as generic
obstacles.
 
If a person is visible within
the vehicle, include that
person as a part of the  
vehicle.
Standard sizing for
a CAR (sedan/SUV) should
be 2-2.2m x 4-5m x 1.3-2m
Standard sizing for a CAR
(truck/minivan/van) should
be 1.9-2.1m x 4.5-5.5m x 1.8-
2.5m
Standard sizing for a TRUCK
(Semi-truck/box
truck) should be 2-3m x 5m+
x 1.8-2.5m
Added per-frame attributes; 3
options:

1. “Moving” -
vehicle is
operational
and in
motion
2. “Stopped” -
vehicle is
operational
and NOT in
 motion

1. Exa
mple
:
vehic
le
waiti
ng at
a
stop
sign
or
traffi
c
light
(in
moti
on
for
som
e
fram
es,
and
not
in
moti
on in
other
fram
es)

3. “Parked” -
vehicle is
NOT
operational
Motorcycle Use this label for any motor-
powered two wheeled vehicle
such as a motorcycle or
vespa.
 
If the vehicle is being ridden,
also include the rider.
 
Please include all the major
elements of the vehicle.
Include everything that's
visible such as the bicycle
frame, wheels, fork, seats,
pedals, crank, posts, and
handlebars.
Added per-frame attributes; 3
options:

1. “Moving” -
vehicle is
operational
 and in
motion
2. “Stopped” -
vehicle is
operational
and NOT in
motion

1. Exa
mple
:
vehic
le
waiti
ng at
a
stop
sign
or
traffi
c
light
(in
moti
on
for
som
e
fram
es,
and
not
in
moti
on in
other
fram
es)
3. “Parked” -
vehicle is
NOT
operational
Bicycle Use this label for any human-
powered two wheeled vehicle
such as bicycles and foot-
powered scooters.
 
If the vehicle is being ridden,
also include the rider.
 
 
Please outline all the major
elements of the bicycle.
Include everything that's
visible such as the bicycle
frame, wheels, fork, seats,
pedals, crank, posts, and
handlebars. If you encounter
elements attached to the
bicycle (like paniers) please
include those too.
 
 
Added per-frame attributes; 3
options:

1. “Moving” -
vehicle is
operational
and in
motion
2. “Stopped” -
vehicle is
operational
and NOT in
motion

1. Exa
mple
:
 vehic
le
waiti
ng at
a
stop
sign
or
traffi
c
light
(in
moti
on
for
som
e
fram
es,
and
not
in
moti
on in
other
fram
es)

3. “Parked” -
vehicle is
NOT
operational Per-frame attributes section:
Emergency Emergency vehicles.
Vehicle including all ambulance,
fire truck, and police vehicle.
Added per-frame attributes; 3
options:

1. “Moving” -
vehicle is
operational
and in
motion
2. “Stopped” -
vehicle is
operational
and NOT in
motion

1. Exa Per-frame attributes section:

mple
:
vehic
 le
waiti
ng at
a
stop
sign
or
traffi
c
light
(in
moti
on
for
som
e
fram
es,
and
not
in
moti
on in
other
fram
es)

3. “Parked” -
vehicle is
NOT
operational
  Only include the stroller and
  any potential child in the
  stroller.
Stroller  
Do not include the pedestrian
pushing the stroller. Instead,
please annotate the
pedestrian pushing the
stroller separately as a
pedestrian

Wheelchair Include mobility scooters and

wheelchairs.
 
If the wheelchair is being
ridden, include the
wheelchair AND the person
sitting in the wheelchair as   
one instance.
Animal Dogs, cats, bears, chickens
and other animals of various
shapes and sizes.
Generic Generic label for obstacles
Obstacle that don’t fit into any other
labels.
More precisely, a generic
obstacle is any 3D entity
(something which sticks out
of or sits on top of the
driveable surface) in scout's  
path which isn't included in
any of the other classes here
which might need bounding
boxes.
ONLY CONSIDER
OBSTACLES THAT ARE IN
THE PATH IN FRONT OF
THE ROBOT - ON THE
SIDEWALK, RAMP, OR
ROAD
 
Fire Hydrant Fire hydrants of varied colors
and shapes.

Trash Can Garbage containers of varied

sizes and shapes.
Pole Any vertical, cylindrical or
rectangular, metal or wooden
support greater than 3 feet
(0.9144m) tall should be
considered a pole.
 
If the pole has a horizontal
part like traffic sign, then only
include the vertical part as
pole.
 
 
Ignore fences. DO NOT label  
the pole under a mailbox as
pole.
Construction Label traffic cones as
Sign construction sign as well.

 
Basketball Basketball hoops consist of a
Hoop Base base, a supporting pole, and
a backboard with hoop.
 
Annotate the base as a base,
the supporting pole as a pole.
 
Do not annotate the
backboard with hoop.

 
Mailbox Free-standing mailboxes of
varied sizes and shapes. Do
not label mailboxes attached
to a house or building.

  
Scout Bot A six-wheeled robot with a
black base and a white or
blue top.
Include the small flag at the
top of the bot

Include small flag at the top of the bot within the cuboid
Grocery Cart Metal or plastic cart on Metal:
wheels for transporting
groceries.
Pedestrian Any light which indicates if a See Traffic Light annotation rules for more details
Crossing Light pedestrian should or should Example pedestrian crossing light:
not cross the street.
Pedestrian crossing lights
may contain symbols such as
a walking person, a hand, or
a countdown number, but
NOT arrows or single color
bulbs.
Do NOT label pedestrian
crossing lights where the
bulbs of the traffic light are
not visible by camera. This
can be due to:

1. Not captured
by the
camera
2. Traffic lights
facing away
from the ego

Includes the following per-

frame attributes:
-”Can cross”
-”stop”
-”Occluded”
-”Unclear”

Traffic Light Any light which indicates if a See Traffic Light annotation rules for more details
car should or should not Label bicycle light as traffic light:
drive. Traffic lights may
contain symbols such as
arrows or single color bulbs,
but NOT a walking person, a
hand, or a countdown
number. If the traffic light is
not facing you and therefore
you cannot see what color
the light is, then no need to
annotate it.
Includes bicycle lights
Label temporary construction
traffic lights that include a
large base as such:
 
-‘Generic Obstacle’ for the
base
-’Traffic Light’ for the traffic
light box

Temporary Construction Traffic Light Example:

Back to top