Map Data for Safer ADAS to HAD Solutions

Introduction: VSI Labs

VSI provides research to companies that design or develop active safety or automated
driving functions for passenger vehicles. VSI’s applied technical research pertains to the
design and development of active safety or automated driving functions. VSI Labs is one
of the few AV researchers who maintains its own vehicles for purposes of better
understanding the technical challenges associated with automated vehicle functions.
Our work is all about understanding how new methods or technologies improve the
performance and safety of automated functionality.

VSI performs its applied research on production vehicles that are owned and registered
by VSI. The vehicle is a 2018 Ford Fusion Hybrid that is equipped with various testing
computers that are interfaced to the vehicle CAN bus.

VSI has built up a number of AV stack configurations using a mix of open-source or

organically developed software modules. VSI regularly examines the latest hardware and
software components including leading softer development platforms, domain controllers
and sensors.

Many of the components that VSI researches are designed for use in active safety or low-
level automation functions found on production vehicles already. This largely includes
sensor-based system used for active safety or low-level automation.

1
Increase driver safety

Production vehicles on the roads are not fully utilizing the benefits of HD maps. Data
quality in HD maps is still improving and technologies in mapping software are still in
development to solve complicated tasks of road parsing problems that cannot be
completely solved by sensors, computer vision algorithms and AI-based perception
algorithms.

HD Maps assist with many aspects related to keeping the self-driving car safe while
achieving its functional goal in a variety of real scenarios.

VSI Labs performed a series of real-world tests by integrating HERE HD Live Map into a
variety of ADAS applications including 1) Map-enhanced Adaptive Cruise Control, 2)
Map-based Lane Keeping in Normal and Adverse Weather and 3) Map-based Lane
Changing.

The primary objective of the tests was increasing the safety of ADAS applications by
improving the functional performance of the vehicle, leveraging HERE HD Live Map while
assuming normal behavior of hardware/sensors in the vehicle.

Map-enhanced Adaptive Cruise Control (ACC)

Background
VSI Labs created algorithms for ACC and implemented low-speed ACC on our research
vehicle that is equipped with a Texas Instrument’s millimeter wave radar. In this project, we
added map data to our ACC system and examined how the map data can enhance the
functionalities of the ACC system.

Objective
Compare how our ACC system operates with and without the map data on different
roads and observe how the system operates based on curvature or location of the road.

Summary of findings related to increased safety with HERE HD map data

We performed the testing in four different road segments:
• curved state highway with high speed
• curved road with low speed
• highway exit ramp
• school zone
While testing, VSI observed the advised speed information of the HERE map data
fluctuated based on curvature and area of the road. This type of information would be

2
hard to obtain from perception sensors only. Also, VSI observed that our A CC system
operated properly with the map data in areas like a highway exit and school zone. This
has an advantage over regular ACC systems, whcih cannot handle these scenarios safely
or comfortably without the map data or input from a human driver.

Figure 1. These graphs show the relationship between the speed of the vehicle and the advised speed from the
map data over time at four different road segments, a. Curved road with high -speed, b. Curved road with low -
speed, c. Highway exit ramp, d. School zone, respectively.

Note: To read VSI’s full report on the Map-enhanced ACC testing, click here.

3
 Map-based Lane Keeping

Background
As an initial part of our AV build process, VSI Labs created a lane detection and lane
centering stack which implements computer vision only lane keeping on our research
vehicle. In this project, VSI addresses the application of HD map assets to improve the
safety and performance of automated vehicle features within the context of lane
keeping and trajectories. VSI is also validating this approach relative to other potential
approaches by visualizing computer vision (neural network) based lane detection while
running lane keeping with a HERE HD Live Map (HDLM) path following based approach in
adverse weather conditions (heavy snow and rain).

Objective
Apply HERE’s HD map data to a lane keeping application and examine the performance
of lane keeping with a map-based approach compared to a camera and computer
vision-based approach under normal and adverse weather conditions.

Summary of findings related to increased safety with HERE HD map data

Using a lane model from HERE’s HD Live Map (HDLM) can solve the common issues
involved in computer vision only lane keeping. Computer vision only lane keeping systems
tend to get confused and make errors in a vehicle’s trajectory when lane lines are out of
the ordinary or invisible.

VSI observed that our map-based lane keeping system provided a more reliable and
consistent performance over our computer vision only lane keeping system during the
tests. The map-based system operated properly in many complicated situations where
computer vision only system constantly failed. Specific testing use cases were lane
keeping in:
• lane lines expanding into a turn or exit lane
• an intersection without lane lines
• a widening in the lane
• adverse weather (heavy rainfall & snow-covered road)

4
 Figure 2. These images show a comparison of the two test results from the test location where lane lines
expand into a turn lane. The left picture shows the test result of the map-based lane keeping system by
using HERE’s HD map data. The right picture shows the test result of the computer vision only lane
keeping system. The map-based system (left) continued to stay in the center of its non-turn lane (desired
path) while the computer vision only system (right) veered into the turn lane requiring the human driver
to disengage the system and direct the vehicle back into the correct lane.

In adverse weather conditions, HDLM technologies compared to neural network-based

lane detection show some obvious advantages. Precipitation of various kinds can have
an impact on all sensors, but especially for cameras detecting lane markings. The
precipitation not only can obscure the camera through obfuscation of the lens, but also
through making the road more reflective and altogether changing its visual
characteristics. Such changes can be compensated for in neural network based
computer vision through proper training data which encompasses adverse weather
conditions, but similar to human visual perception there is a point where precipitation is so
bad (such as when the roads are covered with snow), that even comprehensive and well
labeled training data cannot allow a camera only lane detection system to function.

5
Figure 3. These images show a side by side visualization (for the video on the left visualizer, click here; for the
right, click here) on the left-hand side a video is overlaid with the output of a lane line detection neural
network attempting to inference and visualize the lanes on the snowy/rainy road the vehicle is traversing at
that moment. On the right-hand side is the 3D visualizer in the top down view seen earlier which is showing the
vehicle following a polyline (or centerline) on the HERE HDLM lane model. The two sides are corresponding, the
left-hand side exhibiting the real-world location featured on the 3D top down lane model on the right .

Note: To read VSI’s full report on the Map-based Lane Keeping testing, click here and on
the Map-based Lane Keeping in Adverse Weather testing, click here.

6
 Map-based Lane Changing

Background
In this project, we applied HERE’s HDLM data to an automated lane changing application
and examined the performance of lane changing with and without the map data.
Automated lane changing is one of the new ADAS features enabled in current production
vehicles as a result of the advancement in lane detection technology.

However, lane keeping and lane changing features have not yet lived up fully to their
potential. Software used in current production-level lane keeping systems still leads to
dangerous driving behavior. The current limitations in computer vision -based lane keeping
systems that rely solely on camera are centered around the difficulty found in imbuing
them with the higher-level perception and reasoning necessary for consistently safe
driving. The system needs to properly understand and act on an almost infinite variety of
conditions a driver encounters in everyday driving. This has shown itself clearly in some
systems that are in the market today, which have been well known to have difficulties in
some situations which can broadly be described as ‘boundary conditions.’

Objective
For this project, VSI developed our own lane changing systems, one using a computer
vision-based approach and the other using a map-based approach using HERE’s
HDLMWe compared the differences between the two approaches and describe test
results in normal and boundary conditions.

Summary of findings related to increased safety with HERE HD map data

When computer vision-based lane keeping systems have difficulties in detecting the lanes
in cases like boundary situations like lane merging, faded lane markings, inclement
weather, etc., hence our system constantly failed in lane changing, the HERE HD Map-
based lane change system performed the lane changing tasks in such cases successfully
and reliably.

One of the benefits of using an HDLM to perform lane keeping and changing is that the
HDLM’s lane model provides information about the boundaries of each lane. A lane
keeping controller could be implemented to use the lane boundaries and calculate a
trajectory to stay centered between the boundaries. In addition, HDLM also provides lane
centerline information, which makes it even more convenient to implement a lane
keeping controller that simply follows the centerline path of the lane.

After implementing map-based lane changing, we observed significant improvements in

performance. In contrast to the computer vision-based lane changes, the lane changes
with the HD map data were much more reliable and stable. This is mostly due to the fact

7
that the HD lane model has the precise lane geometry (width and curvature) of the
current lane and the lane it is changing into, whereas the computer vision-based lane
changes rely on lane detection which is occasionally not precise and can be sporadic.
Additionally, with map-based lane changing there is no concern of the lane detections
going missing in the middle of the lane change, which greatly increases confidence in the
system.

Figure 4. Unsuccessful computer-vision-based lane change: the lane to the right of the vehicle ends and is
forced to merge into the vehicle’s lane. Because the ego right lane markings disappear, the lane detection fails.
This causes the vehicle to start drifting to the right of the center of its lane while it does not have adequate
detections. Eventually the lane detection recovers, but by this point, the vehicle has already drifted away from
the center of the lane resulting in the safety driver engaging manually to re-center the vehicle safely.

Figure 5. Successful map-based lane changes with HERE HD Map: In the case of a lane merging from the
vehicle’s right-hand side into its current lane, the HERE map-based control algorithms safely anticipate and
tolerate the lane merging into the vehicle’s lane due to the clear distinction in the HERE map data between
different lane segments, even those in relative proximity.

Note: To read VSI’s full report on the Map-based Lane Changing testing, click here.

8
Enhance driver comfort

HD maps can greatly enhance driver comfort. There are a few production vehicles with
L2 highway pilot systems that can automate longitudinal and lateral controls at the
same time. Some of them rely primarily on the vision systems which are regularly
flummoxed by various roadway situations - including ping-ponging back & forth in its
lane instead of smoothly staying centered, shutting down for no apparent reason, and
missing a clearly marked curve and nearly running into the concrete barrier. On the
other hand, others leverage precision localization and 3D HD maps, which can be
greatly assisted by computer vision-based perception systems, not only to improve the
safety and performance of the system but also to enhance driver comfort significantly.

Automated Drive West (ADW): 2,000+ Mile Cross-country Highway Drive in

Autonomous Mode

Background
VSI used two main AV applications for this drive: HD map-based lane keeping and lane
changing and HD map-based ACC with radar. While VSI has done extensive research
on lane-keeping and ACC using HD maps, this research has always been conducted in
a geofenced area. In previous tests, VSI engineers downloaded and processed the
map data prior to testing. Because the ADW covered more than 2,000 miles of highway
roads, it was necessary to develop a method of downloading and processing map
data during the drive. In addition, VSI engineers had to adjust our AV system for map
distortion.

Objective
Test the benefit of using precision lane models and GNSS localization with real-time
kinematic (RTK) corrections to improve the performance and safety of highway
autonomous driving applications. Furthermore, VSI was seeking to better understand

9
how these technologies operate across varying terrains, weather, and driving
conditions throughout the project.

Also, the other important goal of this project was to assess the potential benefits of using
HD maps in L2 highway pilot systems and whether HD maps can greatly enhance driver
comfort without utilizing vision systems at all (no camera sensors used).

Summary of findings related to increased driver comfort with HERE HD map data
• The autonomous systems VSI developed and applied for this Drive West trip, ACC
and lane-keeping, both leveraged HERE HD Live Map and enabled 1,767 miles of
the drive out of 2, 000+ miles of the total trip.
• Adaptive Cruise Control: ACC is essentially a driver comfort feature, while map data
powered ACC can safely enhance the driver comfort feature. Although we have
not measured the level of comfort quantitatively, our 1st generation map-enhanced
ACC which leveraged HERE HDLM Road Model’s advised speed attributes and road
segment data enabled much smoother execution of the ACC algorithm. The
advised speed information changes in every few meters at exit ramps, which allows
the gradual change of the vehicle speed and provides a safer and more
comfortable driving experience for passengers. Likewise, our 2nd generation ACC
system, which was used in this project, significantly enhanced driver comfort
throughout the long stretch of cross-country highway by greatly reducing false
positives from radar object detection algorithms and staying in the lane while
keeping a comfortable distance with the proceeding vehicles. Our 2nd generation
ACC system utilized an inertial navigation system and HERE’s HDLM data to
determine the correct closest in-path vehicle. This approach helped improve the
performance and safety of an ACC system.
• Lane Keeping and Changing: Long stretches highway driving was automated in
large areas of coverage. This gave a great level of peace of mind to the driver
throughout the cross-country trip.

Conclusion

Map data is already used in many vehicle safety applications, even if they are for
human driving, such a location intelligence like speed limits, traffic and hazard
information improves driver’s awareness of the road environment where the driver is in.

VSI Labs has proved in real-world tests that HD map data can greatly increase driver
safety and significantly enhance driver comfort. HERE HD map data was used in our
various ADAS applications from ACC utilizing speed attributes and road segment data,
to map-based lane keeping and lane changing apps, and ACC powered by radar-
map data fusion. HD map data also showed the unique capabilities where computer
vision only systems frequently fail.

VSI’s Automated Drive West (ADW) project was the culmination of how these map-
based alternative solutions (to vision only based applications) can bring driver safety
and comfort together in a cross-country highway drive.

10
Feature Use Case Result w/o Map Data Result w/ Map Data
Tested
ACC Curved Constantly driving at 55mph, Changed to the advised
highway road dangerously passing through speed (50mph),
(posted speed curved area gradually increased to
limit: 55mph) 55mph after curved
area
Curved road Constantly driving at 35mph, Adjusted to 30 mph at
with low-speed dangerously passing through gentle curve, and to 20
(35mph) sharp curves mph at sharp curve
Highway exist No reduction of speed at the Gradual change of the
ramp (55mph) exit ramp vehicle speed as it was
approaching and
exiting at the exit ramp
School zone No changes of speed when a Vehicle speed was
(35mph vs. 25 reduced speed limit is in effect reduced from 35 mph to
mph) 25 mph based on
date/time for the zone
LKA Lane lines Veered into the turn lane, Continued to stay in the
expanding into requiring disengagement center of its non-turn
a turn or exit lane
lane
An intersection Gradually deviated and Stayed in the desired
w/o lane lines proceeded through the trajectory (driving
intersection but veered into straight & proceeding
the right (LKA disengaged) through intersection)
A widening in Followed the right lane line Stayed in the desired
the lane and slightly changed its path trajectory regardless of
to veer into the right (LKA the changes of lane
disengaged) markings
Highway in (in shadow mode only) lane Consistent and reliable
snow detection as inconsistent lane centering on the
snow-covered roads
Highway in rain (in shadow mode only) lane Consistent and reliable
detection as inconsistent lane centering in the
heavy rain fall
LCA Highway with Failed lane changing due to Safely anticipate and
boundary failed lane detection from tolerate the lane
situations poor/disappearing lane merging into vehicle’s
markings, when the lane lane and successful lane
widened near an exit lane changing regardless of
and when other lane split or lane visibility and
merged changes
Piloted Highway drive • Radar-based ACC: would • Map-based ACC:
Driving not-geofenced have had frequent false safely enhanced the
from driver comfort through

11
 Feature Use Case Result w/o Map Data Result w/ Map Data
Tested
Minneapolis to positives from radar object the long stretch of
Santa Clara detection algorithms cross-country highway
• Vision-based LKA and LCA: • Map-based LKA &
would have failed in many LCA: greater level of
cases as lane markings on peace of mind to the
the highways were not good safety driver
for much of the drive
Table 6. Summary of the Results from Features Tested by VSI

CONDITIONS OF THIS REPORT

This report was written by VSI Labs (aka. VSI, or Vision Systems Intelligence). VSI Labs is not liable for any
damages, losses, expenses, or costs arising in connection with the use of information contained in this
document. VSI Labs will not be liable for any omissions of fact or relevant information about the subject
matter presented in this document. Use of logos, brands, trademarks, and names may be registered
properties of their respective owners.

12