1. Introduction
The map matching of trajectory data is essential for many intelligent transportation applications, such as vehicle tracking, traffic flow analysis, detection of transport modes, and route planning [
1,
2,
3,
4,
5,
6]. Trajectory data consist of geolocation points recorded on the basis of global navigation satellite systems (GNSS) by a tracking application on a vehicle or any other object, and map-matching is applied to trajectory data if the movement is along a known mobility network [
7]. Map-matching algorithms determine the most likely segment of the mobility network for each GNSS point in the trajectory data [
5,
8,
9].
Existing map-matching algorithms consider numerous approaches to handle map-matching errors [
10,
11]. However, they are still prone to errors due to many reasons, such as the inevitable occurrence of systematic GNSS errors, including blocked signals and multipath effects (measurement error) [
4,
12,
13], low sample rate GNSS trajectories (sampling error) [
10], limitations in the efficiency of online map-matching algorithms, incomplete map data [
14], matching errors at junctions [
11], and matching to wrong mobility networks [
8,
10,
11,
12].
The above-mentioned applications require either online or offline map-matching algorithms [
6]. Online map-matching algorithms produce the most likely mobility network segment as soon as the GNSS point is recorded [
15,
16]. On the contrary, offline algorithms use an already recorded set of GNSS points to identify the path traversed by the moving object in hindsight [
5,
6]. This paper is concerned with offline map-matching. In this case, map-matching error identification would be straightforward in the presence of ground truth trajectory data (e.g., the mode and the route of the travel). However, the typical absence of ground-truth data makes it harder to identify and quantify map-matching errors. Currently, offline map-matching is open to visual inspection, which enables a visual process of validation by human reasoning. Using this process for identifying map-matching errors is a laborious task when large numbers of trajectories are processed.
Thus, for this case of the absence of ground truth, we first define a map-matching error: When a map-matched segment, incorporated in a map-matched route, reveals an unrealistic travel behavior, then matching this segment is considered to be an error. Consequently, map-matching errors can be quantified by counting the number of incorrectly (or unreasonably) matched segments. With these two definitions at hand, we propose the following research question: How can we automatically identify and quantify map matching errors when ground truth is not available?
We present a scalable automatic error identification technique that quantifies the map-matching error by counting the number of incorrectly (or unreasonably) matched segments. The automatic error identification is implemented based on an graph-theoretic unsupervised learning approach such that no human intervention is required. The proposed methodology is implemented as a component within a Python-based open-source tool that also allows one to interactively analyze visually the map-matched route of a trajectory point set (
Figure 1). Due to the lack of ground truth data, the visual analysis allows one to establish this ground truth, and to use this ground truth for a performance analysis of our novel technique.
The implemented tool sketched in
Figure 1 can function as a fully autonomous tool when the goal is to save human labor. However, a guided visual inspection to automatically identify map-matching errors can further improve the quality of the automatic process. The major contribution of this article, however, is the automatic algorithm for offline identifying errors in mapping matching. The algorithm is scalable because it does not require human intervention. Only for quality checks or performance measurements is a visual analytics component available.
The rest of the paper is structured as follows.
Section 2 describes current map-matching techniques.
Section 3 describes the relevant theory regarding errors in map-matching.
Section 4 presents our methodology for identifying incorrectly mapped GNSS points.
Section 5 describes the techniques and the experimental results to validate our methodology and measure its performance. Conclusions are provided in
Section 6.
3. Concepts of Map-Matching Error Identification
In this section, we first discuss how the residual map-matching errors can be characterized (
Section 3.1). This characterization allows then to formalize the quantitative error in a map-matched trajectory point series (
Section 3.2) and to introduce an edit distance for this quantification (
Section 3.3).
3.1. Qualitative Error in Map-Matching
Qualitative error in map-matching requires a specification of stationarity; e.g., is a car stopping at a red light interrupting its movement between two stationary activities? This requires consideration of the context of the movement, that is, the travel mode of the movement before and after a stop and the mode-specific thresholds of acceptable duration of these stops. Most frequently, these trajectories are sampled at regular time intervals, although sampling at regular distance intervals and irregular sampling is possible as well. Regular sampling strategies can also show gaps in their recordings. We have considered the following categories of unrealistic travel behavior to be reflected by qualitative errors in a map-matching algorithm:
Unrealistic travel behavior in a map-matched segment occurs due to three main reasons: (i) measurement error in collected GNSS points (since collected GNSS points are exposed to measurement error, true positions remain unknown, resulting in incorrectly matched segments in the road network [
27]); (ii) complex topology and incomplete map information in the road network [
11], leading to erroneous transition probability calculations in both statistics and distance-based approaches [
5]; and (iii) mismatch of travel mode and road network type (e.g., map-matching of a car route on a public transport network) [
12]. As a result of map-matching, in a map-matched route, there will be two types of map-matched segments: (i) correctly map-matched segments (reflects realistic travel behavior) and (ii) incorrectly map-matched segments (reflects unrealistic travel behavior).
3.2. Quantitative Error in Map-Matching
We propose quantifying qualitative mapping errors after a mapping algorithm produces the estimated route and map-matched segments. Let a
trajectory T contain a number
d of recorded GNSS data points,
, such that the
data point (
) contains the tuple
, where
and
are the coordinates of the location of the
jth point (e.g., longitude and latitude, respectively), and
is the timestamp of the location record. Let
E be the set of edges and
V be the set of vertices in the directed graph (digraph) of the road network
. Then:
Here, edges represent road segments on a map, a possible candidate segment of map-matching. We will refer to the term edge in the context of a graph and to a segment in the context of map data. The ground truth route of a trajectory data T is a sequence of connected road segments of the traveled route on the map. For this, we assume that the map is complete.
Let R be a sequence of the ground-truth map segments of the trajectory data T. Hence, R contains a sequence of connected road segments, i.e., map segments, which are expected to be matched correctly with GNSS points recorded on the same map segment. Let R have edges, and let be a subset of vertices of G such that there exists an induced connected subgraph whose vertex set is and whose edge set such that the vertex connectivity —i.e., is 1-vertex-connected. The vertex cut or separating set of contains at least one vertex whose removal renders disconnected. Thus, the ground truth route is a walk in the digraph .
Let the map-matched route for the trajectory T consist of a set of edges: . Thus, . Map matching ensures that for a set of GNSS data points given by coordinate there exists a matching set of road segments where and . The set of location points has a subjective functional relationship with the set of map-matched segments such that multiple points can be matched with one and the same road segment.
For a correctly map-matched route,
will have a finite sequence of map-matched segments such that any
map-matched segment
(with vertices
and
) will have an incident edge
where:
A map-matching error occurs if:
Equation (
3) implies that given ground truth
R, any difference of
can be identified and labeled as a map-matching error. Equation (
3) further refers to the fact that
follows
in a map-matching error.
3.3. Edit-Distance-Based Quantification
Edit distance is a string metric that measures the minimum number of operations required to transform one string into the other [
28]. The identified map-matching errors can be quantified by calculating the edit distance between
R and
, i.e., by quantifying the single-segment edits (insertions, deletions, or substitutions) between the sequences in
R and
. We apply the Levenshtein distance, among various edit distances, because it caters for these three edit operations [
28].
The Levenshtein distance between two
walks R and
of lengths
and
, respectively, can be given by
, where
where the tail of a walk is the
walk shortened by its first edge. The first edge, or head element, of a walk
R is
. Thus, the Levenshtein distance evaluates the number of mismatches between ground truth and the map-matched route.
We categorize four common types of error in map-matching algorithms (
Figure 2,
Figure 3,
Figure 4 and
Figure 5), all of which reveal unrealistic travel behavior in a map-matched segment. In the figures, the recorded GNSS points are shown as blue dots, and map-matched segments are shown in green lines. Each matched segment has a unique segment identification number in the travel sequence, shown in black. The identified common categories of map matching errors are the following:
Cat-I error: This type of error occurs when the incorrectly matched segments are unidirectional or hanging from the actual route (
Figure 2). Referring to Equation (
2),
Cat-I error occurs in the
ith map-matched segment
of
when the following conditions are true:
Such map-matched segments suggest the traveler needs to jump from the end of the segment (
) back to its start (
) in order to come back to the original route. An illustrative example of
Cat-I error is shown in
Figure 2.
Cat-II error: This type of error occurs when isolated lanes are matched with the GNSS points (
Figure 3).
Cat-II error occurs in the
ith map-matched segment
of
when the following conditions are true:
Such map-matched segments suggest that the traveler needs to have infinite velocity at that isolated segment. One illustrative example of a
Cat-II error is shown in
Figure 3. Note that the underlying movement behavior is perfectly legal and physically possible.
Cat-III error: This type of error occurs when there are discontinuities in the matched route due to sparseness of the recorded data points (
Figure 4).
Cat-III error occurs in the
ith map-matched segment
of
when the following conditions are true:
Such map-matched segments suggest that the traveler needs to have infinite velocity while transiting from one segment to the very next disconnected segment. The illustration of
Cat-III error is shown in
Figure 4.
Cat-IV error: Sometimes, map matched segments are ambiguously selected by the algorithm due to measurement error in the recorded points (
Figure 5). These are special cases of
Cat-II and
Cat-III errors:
Equation (
8) is independent of condition
or
. Such map-matched segments suggest the traveler needs to be in both the segments at the same time as if there is a simultaneous presence at two different lanes. This type of error is a
Cat-IV error, as shown in
Figure 5.
However, in the absence of a ground truth map-matched route (
R), map-matching errors (e.g.,
Cat-I–IV errors) can only be identified by visual analysis of
based on common-sense reasoning. With knowledge of
R and
, the Levenshtein distance can be calculated using Equation (
4).
4. Computing the Map-Matching Error Estimation
We propose an autonomous two-step methodology to estimate whether a map-matched segment is likely to be erroneous or not. We label each map-matched segment in as either correctly matched or incorrectly matched. Step 1: estimating the effect of measurement noise in the recorded GNSS points on map-matched segments. Step 2: identifying unrealistic travel behavior due to unreasonable sequences in the map-matched route. Combining both Step 1 and Step 2, we label each map-matched segment either as correctly matched or incorrectly matched. We utilize the Levenshtein edit distance as a quantitative measure of this error.
4.1. Step 1: Identification of Map-Matched Segments Affected by Noisy GNSS Points
In Step 1, our objective is to detect whether a segment matched with the map is influenced by the measurement error at the GNSS points. All recorded GNSS points have some measurement error. For some points, the measurement error is within an acceptable quality for map-matching. The rest of the points are not within the granularity of map-matching. The threshold value for the acceptability of the induced measurement error is unknown for classifying the GNSS points. Hence, we developed a new unsupervised classification technique to classify the recorded GNSS points into two categories: (i) acceptable erroneous points and (ii) unacceptable erroneous points (or decidedly erroneous point). We perform a binary labeling of each GNSS point followed by classification. Then, we measure the likelihood of a segment to be influenced by each labeled GNSS point.
Figure 6 presents a flow diagram to identify map-matching errors and the responsible erroneous GNSS points.
For the figure, a database of raw GNSS points and a road network graph G were used by a map-matching algorithm to generate the sequences of matched segments . We define the relevant features that are obtained using the recorded data point and the corresponding map-matched segment . These feature values will be used for unsupervised binary classifications of GNSS points to filter out unacceptable erroneous points from less erroneous points. Each GNSS point has a corresponding map-matched segment, and there exists a surjection mapping between the set of GNSS points and the set of map-matched segments: multiple GNSS points may be matched to a single segment. We performed a binary labeling of each map-matched segment on the route depending on the statistical mode of the labeled GNSS points matched to that segment. Thus, we achieve Step 1 of identifying incorrectly matched segments.
4.1.1. Feature Engineering
In the HMM-based model, the transition and emission probabilities of each map-matched segment depend on (i) the orthogonal distance of a GNSS point from the set of probable segments and (ii) the probability of measurement error in each GNSS point. Hence, the categorization of each GNSS point is achieved based on two measures (or features): (i) how far a GNSS point is away from the map-matched segment and (ii) how likely the GNSS point is to be corrupted. Features are individual measurable properties of an event being observed, here in the process of map-matching. Each feature is derived from a GNSS point (in T) and the corresponding mapped segment. Therefore, each feature of the jth point is a function of where . We have defined two features for the classifications discussed in the following subsections: (i) orthogonal distance and (ii) estimated noise.
Orthogonal Distance
We can write a straight line passing through the points
as:
, where bold means a vector and
K is a constant. Then, the orthogonal distance of the
jth GNSS point from
can be calculated as:
Estimated Noise
We use a Kalman filter [
29] to estimate the noise of the recorded GNSS points induced by measurement error. Noise can be determined if we can estimate the true position of a recorded GNSS point. A typical Kalman filter estimates the next true position based on the current true position and the current estimated position, using two processes, a prediction process, and a correction process [
29]. The prediction process at the
jth timestamp is governed by the following equations.
where
is the state vector;
the estimated state vector;
the variance-co-variance matrix for
jth state;
the initial variance-co-variance matrix;
the process co-variance, i.e., the Gaussian noise in prediction
, where
q is the standard deviation of the process error [
30]; and
is the time transition matrix. If
, then:
The correction process at the
jth state is controlled by the following set of equations.
where
is the Kalman gain matrix in the
jth state,
the measurement matrix for the observation process,
the observations at the
jth state, and
the co-variance matrix of the measurement error, that is, the Gaussian noise in the measurement
, where
is the standard deviation of the measurement error [
14]. We will get the estimated position
from
. Thus, we derived our fourth feature as the estimated noise in
as:
4.1.2. Unsupervised-Learning-Based Classification
Once we extracted the feature values of each GNSS point in the trajectory data
T, we prepared a training dataset
for the classification of
J points.
has 2 columns and
J rows. Each
jth row of the training dataset
has feature values
and is unlabeled (that is, we do not know the acceptable error in GNSS points). Hence, we used unsupervised learning to classify the GNSS points in
using a Gaussian mixture model [
31]. Gaussian mixture models (GMMs) are probabilistic models, where each cluster corresponds to a normal multivariate probability distribution. The GMM clustering algorithms define soft boundaries. The aim of the proposed GMM is to perform binary clustering of GNSS points based on some features to separate unacceptable erroneous GNSS points from acceptable erroneous GNSS points. The cluster centers will reveal the binary labels of the recorded GNSS points, 1 for no acceptable erroneous points and 0 for acceptable erroneous points. We identified the GNSS points associated with each map-matched segment. A map-matched segment will then be assigned a label based on a voting-based approach: the labels with the highest majority win. Each segment in
were labeled either 0 (correctly matched) or 1 (incorrectly matched), based on the label of the majority of the GNSS points associated with the segment.
4.2. Step 2: Identification of Unrealistic Travel Behavior Due to Topological Irregularities
This step aims to estimate the ground truth
R from
. A correctly map-matched route (when
) has the properties of a walk in a digraph
G. Therefore,
cannot be disconnected, cannot contain multiple components, and cannot have a unidirectional segment (Equation (
5)) or a unidirectional claw in
G. In graph theory, a claw
is a star with three edges.
is a tree with one internal node and 2 leaves, i.e., a complete bipartite graph
. The presence of multiple components and/or unidirectional claws on a map-matched route
is addressed as topological irregularities in a travel route. Hence, in
Step 2, we aim to detect whether a map-matched segment is causing unrealistic travel behavior due to the self-generated topological irregularities in the map-matched route. We propose labeling segments erroneously matched by the following methods:
4.2.1. Connected Components Analysis
A connected component is a maximally connected subgraph of a graph. We looked for the number of connected components in the map-matched route . In this way, we can determine any discontinuities in the map-matched route and the disconnected map-matched segments responsible for Cat-II, Cat-III, and Cat-IV error.
4.2.2. Claw Detection
In graph theory, a star with three edges is called a claw. Since unidirectional hanging segments are responsible for Cat-I errors, the ground truth R must be claw-free. After completing Step 2, each segment in will be labeled 0 (correctly matched) or 1 (incorrectly matched). We create a new database to store the output of Step 2 after labeling.
5. Performance of the Automatic Map-Matching Error Identification
In this section, we discuss the validation of our method and its performance using GeoLife trajectory data [
32]. Since GeoLife data are in principle multi-modal, but labeled with the transportation mode, we selected only the datasets labeled
drive/car to not confuse separate map-matching challenges. Our experimental dataset
T from GeoLife contains 23 datasets with in total
GNSS points. Additionally, we used the state-of-the-art HMM-based map matching [
5], which produced 1296 map-matched segments.
5.1. Establishing Ground Truth from Visual Inspection
Validation and performance assessment of the automatic error identification requires ground truth data for the map-matched segments. Since ground truth of map-matched segment data is not available with large-trajectory datasets [
14,
32], we apply visual inspection of the map-matched segments, applying the criteria of reasonable sequencing described above.
For example, in
Figure 7, part of the trajectory data of Dataset 4 is presented, where we can visually identify incorrectly map-matched segments from human reasoning. The discontinuities between Segments 8 and 9 suggest a Cat-III error, and those between Segments 5 and 6 suggest a Cat-IV error, according to Equations (
7) and (
8). A person would need infinite velocity to travel from Segment 5 to 6, and from Segment 8 to 9.
Once an error is visually identified, we label each erroneously map-matched segment . Thus, we create a ground truth dataset , where each map-matched segment of T is manually labeled as either 0 (if correctly matched) or 1 (if incorrectly matched).
To realize the visual inspection, we have developed an interactive, map-based tool (
Figure 8). The recorded GNSS points are shown as red dots, and map-matched segments are shown as blue markers with popup numbers. The tool tracks user interactions such as in the figure, where the map-matched segment 6388 has been clicked as a ground truth error by a human inspector.
5.2. Using the Ground Truth for Validation and Performance Assessment
Our proposed method labels the map-matched segments as correctly or incorrectly map-matched, and creates . We compared with the ground truth dataset and performed an error analysis of our method. For this analysis, we applied the usual error measures:
True positive (TP): The proposed method correctly identifies an erroneous segment according to ground truth.
False positive (FP): The proposed method indicates an error when there is no error in the map-matching.
False negative (FN): The proposed method fails to indicate an observed error in the map-matching.
True negative (TN): The proposed method correctly labeled a segment as non-erroneous.
Say the ground truth route
R has
n segments and the map-matched route
has
m segments, of those,
(
) segments are correctly matched. Let the observed (ground truth) error in map-matching occur in
of segments. Then:
We have defined two error measures to evaluate the accuracy of a map-matching algorithm: (i) the error percentage of the map-matching algorithm
and (ii) the error percentage of the model outcome
. Let
be the number of correctly labeled segments. Then:
In
Figure 7, Dataset 4 has five map-matched segments; hence,
. The
Cat-III error between Segments 8 and 9, and the
Cat-IV error between Segments 5 and 6 equate to
. The automatic method identified the
Cat-III error correctly, but the
Cat-IV error went undetected. Therefore,
, with one true positive and one false positive. Thus,
We will conclude our validation by comparing sensitivity analysis (true positive rate) and specificity analysis (true negative rate).
5.3. Performance Results of Automatic Error Identification
Illustrations of identified errors are presented in
Figure 9,
Figure 10,
Figure 11 and
Figure 12. All blue dots indicate recorded GNSS points. The incorrectly map-matched segments as estimation results are shown as red lines. The correctly estimated map-matched segments are shown with green lines.
In
Figure 9, a
Cat-I error has been identified and highlighted by a red line with Segments 2, 22, 20, and 18.
In
Figure 10, a
Cat-II error has been identified and highlighted by a red line with Segment 22.
In
Figure 11, a
Cat-III error has been identified and highlighted by red lines with Segments 16 and 15 and a green line 14, as it was not detected by our method.
In
Figure 12, a
Cat-IV error has been identified and highlighted by the red line with Segments 11 and 10.
In this way, we have compared the 23 drive-only map-matched trajectories from GeoLife by counting the total ground truth segments (
n), the observed errors in map-matching (
), correctly estimated errors (
), false positives (incorrectly estimated errors), and false negatives (not estimated errors) in the proposed method. Then, we have calculated
and
based on Equations (
21) and (
22). The results are presented in
Table 1, sorted according to the values of
in ascending order.
We have derived a two-dimensional matching matrix (
Table 1) to visualise the performance of the proposed methodology. Each row of the matching matrix represents observed instances, that is, the ground truth label of the map-matched segments: (i) without error and (ii) with error. Each column of the matching matrix represents the estimated instances, that is, the estimated labels of the map-matched segments using the proposed method: (i) segments estimated without error and (ii) segments estimated with error. Thus, a matching matrix evaluates a quantitative measure of mislabeled segments on the map using the proposed methodology. A matching matrix generated for the proposed methodology is presented in
Table 2, where
P and
N in parentheses indicate positive and negative instances, respectively.
Our proposed method was evaluated based on the following scores, obtained after evaluating the proposed matching matrix:
AUC (area under the curve)—
ROC (receiver operating characteristic), true positive rate (
TPR), true negative rate (
TNR), positive predictive value (
PPV), false omission rate (
FOR),
F1 score, and overall accuracy (
ACC), as defined in Equations (
25)–(
31):
6. Conclusions
Identifying errors in online and offline map-matching processes is important for related intelligent transportation applications, including driving behavior and traffic flow analyses, identifying changes in the street network, and planning of transport routes. In this paper, we have proposed a methodology to identify those map-matching errors in offline processes. We have discussed the theory of map-matching error identification. We then introduced and formalized a typology of four common types of map-matching errors arising from measurement errors and complex map topology. We proceeded to develop the theory to identify erroneous map-matched segments within map-matched trajectory data based on an unsupervised classification followed by a graph-theoretic approach to spot unrealistic travel behavior. Thus, our research question can be answered positively: We can automatically identify and quantify map matching errors when ground truth is not available. Beyond identification, we can also classify these errors using the proposed classification.
We have validated our methodology using real-world data collected in the GeoLife project. The validation results show that our method has a good accuracy score when map-matching errors are notably present: The method achieves an average accuracy of 91% in automatically identifying map-matching errors. This result indicates that our method can efficiently help the analysts to reduce human efforts in map-matching quality control by a large margin. By the choice of the unimodal trajectory subset of GeoLife, we evaluated data that should already work better with map-matching algorithms (i.e., present a hard baseline), and yet we demonstrated the presence and the ability of our method to identify of significant errors of current map-matching methods. The amount of potential errors that should be detectable using the proposed method is likely to be significantly higher for multimodal trajectories, observed in multimodal mobility networks. The proposed method and error categorization method remain valid for these trajectories and errors.
As a limitation of our method, we detected erroneously map-matched segments without correction. Although our method contributes a theoretical and practical step forward towards map-matching error identification, further investigations into feature engineering and classification algorithms could overcome current limitations. In future work, it will be interesting to investigate the performance measures of our model on Cat-I to Cat-IV errors separately. Although the proposed method is applied with offline map-matching, can also be applied in online map-matching (e.g., using a real-time simulation-based approach with Geolife data).