This section is divided into four subsections.
Section 5.1 presents the results of the characterization of the GPS errors and times of unavailability achieved from real GPS datasets. These situations of GPS unavailability were reproduced in simulations, and the proposed integrated DR aided CP was applied to reduce positioning errors and cover areas of GPS unavailability. Details about the simulation setup are presented in
Section 5.2. The positioning results are discussed in
Section 5.3, and results concerning network metrics are explained in
Section 5.4.
5.1. Characterization of GPS Problems from Real GPS Datasets
In order to identify the GPS unavailability problems, we investigated GPS data from real datasets in our two previous works [
28,
29]. These data were achieved from routes of taxis and buses of the cities of San Francisco, CA, USA [
30] and Rio de Janeiro, RJ, Brazil [
31]. Regions of interest near to the Rio 450 Years, Robert C. Levy, and Douglas McArthur tunnels from the cities of Rio de Janeiro and San Francisco, respectively, were selected. While the two first ones are urban tunnels, the last one is placed in a highway. Each tunnel has particular features, such as length, speed limit, number of lanes and so on. Photos of the tunnels are presented in
Figure 7, and detailed tunnel features are summarized in
Table 3.
Appendix A shows the results of the characterization process. For each outage point on the entrance of the tunnel, there is an another one at the exit, and vice-versa. It is visible that the outages occur not only at the exact entrance and exit of the tunnel, but some meters beforehand in both cases. This is justified by the delay in the process of outage and recovery on the GPS receiver. Additionally, there is a large area of unavailability in which the GPS does not work.
Figure 8 and
Figure 9 show the measured GPS errors and times of GPS unavailability through an Empirical Cumulative Distribution Function (ECDF) for each sense of the tunnels. The errors were measured considering the distance between the GPS point and its intersection with the most central lane of the road (at the perpendicular point). The computations were performed considering the Universal Transverse Mercator (UTM) coordinates system. This system of coordinates is used by the SUMO [
35] mobility simulator which obtains this information from the OpenstreetMaps
.osm files [
24]. Details about the algorithm to measure the errors can be found in our two previous works [
28,
29] and is outside the scope of this work. In
Figure 8, more frequent GPS errors vary between 5 and 10 m (The RIO450 tunnel is only one way (see
Table 3), and for this reason, its error curve does not appear on the exit entrance sense in
Figure 8b and
Figure 9b. Additionally, a map visualization is provided in
Appendix A Figure A3). These errors conform with the errors reported in Refs. [
1,
4] with a slight reduction. This is due the International GNSS Service (IGS) investments in the deployment of Satellite Based Augmentation Systems (SBAs) around the word [
36].
Moreover, considering only the portions with higher frequencies, the most converging times of GPS unavailability for all outages can be evaluated. As shown in
Figure 9 for the RIO450, RCLT, and DMAT tunnels, the most representative central values are respectively [98, 219], [52, 63], and [48, 72] s. The distance in outage is directly proportional to the time, and the time and distance are proportional to the tunnel length. These times of unavailability are not sufficient for several applications and services in VANETs and ITS [
1,
5].
This characterization aims to achieve a real estimation of the GPS error in the region of the tunnels and take information about time and distance that vehicles run without GPS availability. Moreover, with this data, we can reproduce the GPS behavior in simulation environments with better fidelity to the real world.
5.2. Simulation Setup
To evaluate the performance of the proposed solutions, we run simulations in the Veins version 4.5 [
37] framework that couples the SUMO [
35] simulator of mobility version 0.29 and the OMNET++ network simulator version 5.0. With both simulators, it is possible to define all parameters of the mobility of the vehicle and all parameters of the vehicular network following the IEEE 802.11p standard.
The Krauß Model has been configured with 50% driver imperfection [
20]. This means that vehicles will have different behaviors regarding acceleration, deceleration, and choice of lanes to drive in. Vehicles were set up to employ speeds varying from the mean and maximum values of the road. The gyroscope model parameters employed on the Dead Reckoning approach were obtained from an automotive gyroscope data-sheet developed by Bosch [
27]. The vehicle parameters are summarized in
Table 4.
The network simulation parameters are defined in
Table 5. The vehicles were configured with a power transmission of 20 mW, enabling a communication range of about 300 m. The nodes of the network disseminated beacons at a frequency of 10 Hz. All beacons were sent through the control channel
at a rate of 6 Mbits/s. The
log normal shadowing path loss model was configured in accordance with values of real experiments reported in Ref. [
21], under a sensitivity threshold of
dBm. In other words, beacons with RSSI values smaller than this threshold were not processed. Furthermore, only vehicles outside the tunnel disseminated beacons to cooperate with vehicles under GPS unavailability.
5.3. Positioning Results
All results presented in this section were obtained from 33 simulations following a Student’s T-Distribution with a 95% of confidence interval.
Figure 10,
Figure 11 and
Figure 12 show the results of the GPS outages reproduced in the simulation environment with the application of the proposed CP solution. Arbitrary outages were selected from each tunnel characterized in
Section 5.1. The main objective was to evaluate the impact of each scenario on the proposed CP solution. Results of trajectory and positioning error over time are presented.
Lines in red, green, magenta, and cyan represent, respectively, the trajectories and errors of the stand-alone GPS solution (GPS), GPS integrated with DR and their sensors (GPS + DR), GPS integrated with DR and CP (GPS + DR + CP) and the last solution with the Map Adjustment (GPS + DR + CP + MA). It is possible to observe that gyroscope random walk effect has a major contribution to the discrepancy in the DR estimative in the outage stage. In general, increasing the time of the use of DR without any source of reinitialization or calibration of sensors leads to a fast evolution of positioning error.
Furthermore, in
Figure 10a,
Figure 11a and
Figure 12a, the DR trajectories are far closer to the vehicle positioning due to the reinitialization/calibration of the sensors using the proposed CP positioning. In
Figure 12b of the Robert C. Levy Tunnel (RCLT), errors are under 10 m for 50% of the unavailability time and about 20 m for the other half of unavailability time. The error reaches the peak of 40 m at around 110 s of the trajectory, but it is corrected with the application of CP. In
Figure 12b, errors are also maintained below 10 m most of the time, and the error only reaches a peak of 20 m close to the end of the unavailability stage. This is due to the length of this tunnel. In
Figure 12b of the Douglas Mc Arthur Tunnel (DMAT), it is possible to see a large path of GPS unavailability where the CP reinitializes the sensors several times, keeping the error bellow 10 m in the first half of the unavailability stage. In the other half, the error fluctuates between 10 and 20 m. In general, we can observe an increase in the error close to the center of the trajectory during the unavailability stage.
After applying the Map Adjustment, the estimated trajectory is even closer to the vehicle trajectory. We can observe errors in the range of 5 to 7 m in the RIO 450 tunnel, and about 3 m in the RCLT and DMAT tunnels. However, note that the reliability of this adjustment depends of the accuracy of the digital map and quality of the estimated position under GPS unavailability. Aiming to obtain the overall behavior of the proposed integrated CP solution, we performed an exploratory analysis using the RMSE (Root Mean Square Error) metric (Equation (
24)). This metric is commonly used in the literature to evaluate the error of the localization solutions.
The variables and are the positions of the vehicle and the estimated positions of GPS, GPS + DR, GPS + DR + CP or GPS + DR + CP + MA. To show the impact of time on the error of the solution, we divide the outage times of each vehicle into levels of GPS unavailability times. Times ranging from to of the total GPS downtime were related to the RMSE.
In
Figure 13 the RMSE for all outages reproduced in the simulation environment are presented. Considering the first 10% of GPS unavailability time, the solutions for GPS, GPS + DR, GPS + DR + CP, GPS + DR + CP + MA achieve, respectively, RMSEs of 57, 13, 10, and 5 m for the RIO450 tunnel; 96, 12, 7, and 3 m for the RCLT tunnel; and 63, 9, 6, and 3 m for the DMAT tunnel. On the other levels (percentages) of unavailability, the GPS and DR solutions increase in accordance with the unavailability time. However, the GPS + DR + CP solution increases up to about 50% at the time of outage and decreases shortly thereafter. This is because only vehicles outside the tunnel disseminate beacons. Consequently, the error on the central region of GPS unavailability is higher than on the tunnel entrances. The maximum values of RMSEs are 17, 12 and 10 m for the RIO450, RCLT, and DMAT tunnels, respectively. The GPS + DR + CP + MA solution maintains the RMSE constant for all outages reproduced in the simulation environment, achieving RMSEs of 5, 3, and 3 m for the RIO450, RCLT, and DMAT tunnels.
To these results, we applied a metric named the average gain. This metric measures how better one solution is than the others in terms of RMSE. The average gain is defined in Equation (
25):
where
represents the GPS + DR + CP or GPS + DR + CP + MA solutions, and
represents the other compared solutions: GPS, GPS + DR and GPS + DR + CP. In summary, the gain represents the effective decrease in RMSE applying the integration of the developed solutions.
In detail, the results in
Table 6 show that the solutions covered
of the unavailable area. The GPS + DR + CP solution achieved average gains between
and
in relation to the GPS stand-alone and
e
in comparison with the GPS + DR solution. Moreover, after applying the Map Adjustment (GPS + DR + CP + MA), the gains were notably more effective: between
and
in comparison with GPS,
to
against GPS + DR, and
to
of RMSE reduction in relation to GPS + DR + CP.
Figure 14 shows the average RMSEs for different traffic regimes. Traffic varies from 240 vehicles per hour (veh/h) to 3600 veh/h which represents the sparsest to the densest network. A decrease in the RMSE is observed with an increase in density. Moreover, after adding a large number of vehicles a low variation in RMSE occurs. From this, we can confirm the fact that the geometry of selected nodes to perform CP (important part of the integrated solution) imposes more influence on the positioning accuracy than the increase in the number of nodes, as previously introduced in
Section 4.2. A higher density occurs in the major number of neighbors for one given vehicle. In this sense, a more accurate geometry with the lowest GDOP can be selected, and a better accuracy can be reached. The accuracy using Map Adjustment is the same for all densities, because the geometry of the road is applied to reduce positioning uncertainty.
5.4. Network Results
Figure 15 shows the average results of network metrics by vehicle in each scenario. These results were also obtained following the simulation setup explained in
Section 5.2.
Figure 15a shows the average number of received messages that increases as the density of vehicles increases. In general, the number of received messages also increases as the distance and time of the trip increases. This is because more messages are sent and consequently, received by the neighbor nodes. The DMAT tunnel has the lowest number of sent messages because its tunnel size is the smallest. The Rio450 tunnel has a major increasing rate as the density grows. However, the RCLT tunnel has the largest number of sent messages due to the large number of traffic lights in the tunnel region. As a result, vehicles spend more time stopped and consequently, send more beacons over time.
Figure 15b shows the number of lost messages that, for all tunnels, increases as the density increases. In the simulations, beacons are lost by three main factors that are reported by the MAC layer:
A packet was not received due to bit errors;
A packet was not received because a packet was being sent while receiving another packet;
A packet was dropped by the physical layer.
By comparing these results with the ones in
Figure 15a, we see that the number of lost messages floated between 1% to 7% for all scenarios. Moreover, despite the number of sent messages of tunnel RCLT being greater than the RIO450 Tunnel, the loss in the RIO450 Tunnel was higher. These details can be better visualized in
Figure 15c, which was computed using Equation (
26). This behavior occurs because the tunnel RIO450 has the largest extension which represents a major area of GPS unavailability and major effort of CP integrated with DR to compute positions using beacons in a multihop fashion. The DMAT has the lowest loss ratio because its has the smallest area of unavailability and consequently, requires a minor effort to perform the proposed CP solution.
Figure 15d presents the average delay per message received by vehicle. Is worth noticing that the delay decreases as the density increases. This is due to the fact that, as the density increases, so does the chance of more vehicles receive beacons [
2]. DMAT tunnel has the lowest delay, because the lowest size of tunnel, and consequently the minor vehicles inter-distances followed by RCLT and RIO450 tunnel (largest tunnel). These results can be also justified by the fact that for one same network path, as the density increases, one same receiver can be reached with a lower number of hops.