Pedestrian Dead Reckoning Based On Walking Pattern Recognition and Online Magnetic Fingerprint Trajectory Calibration
Pedestrian Dead Reckoning Based On Walking Pattern Recognition and Online Magnetic Fingerprint Trajectory Calibration
Pedestrian Dead Reckoning Based On Walking Pattern Recognition and Online Magnetic Fingerprint Trajectory Calibration
net/publication/343620390
CITATIONS READS
11 453
7 authors, including:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Qu Wang on 15 August 2020.
This paragraph of the first footnote will contain the date on which you Q. Wang, A. Men are with the School of Information and Communication
submitted your paper for review. This work was supported in part by the Engineering, Beijing University of Posts and Telecommunications, Beijing
National Key Research and Development Program under Grant 100876, China (e-mail: [email protected]; [email protected])
2018YFB0505200, the Action Plan Project of the Beijing University of Posts H. Luo is with the Beijing Key Laboratory of Mobile Computing and
and Telecommunications supported by the Fundamental Research Funds for the Pervasive Device, Institute of Computing Technology Chinese Academy of
Central Universities under Grant 2019XD-A06, the Special Project for Youth Sciences, Beijing 100190, China (e-mail: [email protected])
Research and Innovation, Beijing University of Posts and Telecommunications, H. Xiong and F. Zhao are with the School of Software Engineering, Beijing
the Fundamental Research Funds for the Central Universities under Grant University of Posts and Telecommunications, Beijing 100876, China (e-mail:
2019PTB-011, the National Science Foundation of China under Grant [email protected]; [email protected])
61872046, Grant 61671264 and Grant 61671077 , the Joint Research Fund for M. Xia is with the School of Electronic and Information Engineering,
Beijing Natural Science Foundation and Haidian Original Innovation under Beihang University, Beijing 100191, China (e-mail: [email protected])
Grant L192004, the Key Research and Development Project from Hebei C. Ou is with the School of Computer Science and Engineering, Nanyang
Province under Grant 19210404D, the BUPT Excellent Ph.D. Students Technological University, Singapore 639798, Singapore (e-mail:
Foundation under Grant CX2020306 and the Open Project of the Beijing Key [email protected])
Laboratory of Mobile Computing and Pervasive Device. (Corresponding author: Copyright (c) 2020 IEEE. Personal use of this material is permitted.
Haiyong Luo and Aidong Men) However, permission to use this material for any other purposes must be
obtained from the IEEE by sending a request to [email protected].
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 2
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 3
A. Pedestrian Motion Modes Recognition summarized as biomechanical models [27], linear models [28],
Pedestrian motion modes recognition is an essential topic in nonlinear models [29] and inverted pendulum model [30].
the health, computer-vision, and positioning and navigation Additionally, some researchers leveraged the regression-based
field because it has many applications, including human activity method [31],[32] and the deep learning-based method [25], [33],
recognition, prediction of human health status, sports training [34] to predict the walking distance of pedestrian.
and human-computer interaction. Generally, pedestrian motion Based on the fact that acceleration and magnetic field provide
modes recognition methods are classified into vision-based absolute heading whereas angular rates produce relative
methods and sensor-based methods. Vision-based motion heading by integrating iteratively, most smartphones-based
modes recognition obtains high recognition accuracy, but it heading estimation methods estimate pedestrian heading by
causes high power consumption and privacy problems. This fusing acceleration data, angular rates data and magnetic field
study focuses on pedestrian motion recognition using data. To estimate heading, Attitude and Heading Reference
smartphone embedded sensors (accelerometer, gyroscope, or Systems (AHRS) [35] leveraged gradient-descent algorithm
barometer) to recognize pedestrian motion modes. and complementary filter to combine inertial sensors and
Plenty of studies on pedestrian motion modes recognition magnetometer observations. Wang et al. [36] built a weighted
using the IMUs have been developed. Pei et al. [14] extracted model to fuse the IMUs outputs for heading estimation. To
the twenty-seven features by employing the raw observations obtain more accurate heading, numerous studies utilized
from the accelerometer and magnetometer embedded in extended Kalman filtering and complementary filters to fuse
smartphone to recognize eight motions. Ronao et al. [15] accelerometers, gyroscopes and magnetometer readings, and
proposed an efficient and effective pedestrian motion modes reduce sensor noise. Because of the inconsistency of pedestrian
recognition algorithm using sensors built-in smartphone, in coordinate frames and device coordinate frames, different
which a deep convolutional neural network (CNN) was smartphone carrying modes require different methods to
performed to exploit the inherent characteristics of motions and estimate pedestrian walking heading. To address this issue,
one-dimensional time-series signals. Zhang et al. [16] some studies [37]–[41] focus on heading estimation under
employed multi-head convolution neural networks integrated different smartphone carrying modes, and leverage the
with attention to recognize various motions, including walking, Principal Component Analysis (PCA) or enhanced PCA
standing, sitting, jogging, upstairs, and downstairs. Gao et al. algorithm to extract the largest variations of the horizontal
[17] recognized human activities based on Stacking Denoising accelerations for heading estimation [42]. In our previous work
Autoencoder and LightGBM. To address the imbalanced [43], we leverage the Spatial Transformer Networks to align the
distribution issue of labeled data over classes, Chen et al. [18] navigation coordinate system (NCS) and the body coordinate
proposed a pattern-balanced semi-supervised framework based system (BCS), then designed a hierarchical LSTM-based
on recurrent convolutional attention networks to extract Seq2Seq heading model to estimate the walking heading of the
features and recognize diverse latent motion patterns. Saeed et pedestrian. Despite its importance, heading estimation is still an
al. [19] proposed multi-task self-supervised learning to open issue in PDR.
recognize human activities. AROMA [20] leveraged deep With the three components above, the position of pedestrian
multi-task learning to recognize simple and complex activities. is estimated through accumulating displacements. However,
smartphone-embedded low-cost sensors lack factory
B. Pedestrian Dead Reckoning calibration, so that the measurements are easily contaminated
Extensive studies on PDR have been developed. A typical by hard or soft iron error, bias error, and scale factor errors. The
PDR system contains three components: step detection, step contaminated measurements result in significant errors in each
length estimation and heading estimation. of the three components in PDR, which causes accumulative
The first step of PDR is step detection, which is used to errors increasing with walking distance and time. Moreover,
determine the data interval required to estimate step-length and most PDR algorithms require that pedestrian hold smartphone
heading of pedestrian at each step. An ocean of studies on step horizontally and points to the walking heading, which limits the
detection and step counting have been done, categorizing into practicability of PDR. Therefore, providing an accurate and
the following groups: threshold setting [21] and peak detection robust location-based service just employing a smartphone in
method [22], correlation method [23], spectral analysis method intricate walking pattern is still an open challenge and needs to
and machine learning [24]. However, these algorithms’ be resolved urgently.
performance is highly sensitive to pedestrian walking patterns.
C. Trajectory Calibration
Methods of estimating pedestrian step length based on
inertial sensors are summarized as three categories: (1) double Numerous improved methods have been developed to
integration method of acceleration; (2) models or assumptions mitigate the accumulative errors of PDR systems. A low-pass
methods; (3) deep learning methods[25]. Step length estimation filter is applied to reduce the random noise of sensor
based on double integration does not rely on any individual measurements [44], [36]. Landmark calibration is a common
information [26]. However, the non-negligible drift and noise and effective way to eliminate accumulative errors. Magnago et
of inertial-sensors resulted in the motion distance estimation al. [45] leveraged the camera to capture the optimal image
error accumulating boundless with time [26]. Considerable step landmarks. However, image feature extraction and matching
length estimation studies based on model or assumption are caused high power consumption and privacy problems. Shang
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 4
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 5
Gyroscope
Head 2
Attention 2
Conv 2D M2
Assuming that the measurements of acceleration
Concatenate
Concatenate
Head 3
Maxpooling
Dense layer
gravity and magnetometer are represented as follows,
Softmax
Conv 2D
Attention 3
respectively. M3
Head 4
z T
Ab = abx aby a (2) Conv 2D
Accelerometer
b
Head 5 M4
z T
Gb = g g g
x
b
y
b b (3) Conv 2D
Head 6
T Attention n M5
M b = mbx mby mbz (4) Conv 2D
𝑦
where 𝑎𝑏𝑥 , 𝑎𝑏 , and 𝑎𝑏𝑧 represent acceleration measurements on
𝑦 Fig. 4. Walking pattern recognition based on multi-head convolutional
the respective axes at the BCS. 𝑔𝑏𝑥 , 𝑔𝑏 , and 𝑔𝑏𝑧 represent attention.
𝑦
gravity measurements on the respective axes at BCS. 𝑚𝑏𝑥 , 𝑚𝑏 ,
TABLE I
and 𝑚𝑏𝑧 represent magnetometer measurements on the EXPLANATION OF PEDESTRIAN WALKING PATTERNS.
respective axes at BCS. ID Walking pattern explanation
We import the sensor data of magnetometer, accelerometer M1 Forward walking
and gyroscope into AHRS to estimate the smartphone attitude M2 Backward walking
relative to the NCS. Then, we build the rotation matrix 𝐶𝑏𝑛 M3 Left lateral walking
according to the quaternion vector updated from AHRS. M4 Right lateral walking
q12 − q22 − q32 + q42 2q1q2 − 2q3 q4 2q1q3 + 2q2 q4 M5 Running
n
Cb = 2q1q2 + 2q3 q4 −q1 + q2 − q3 + q4
2 2 2 2
2q2 q3 − 2q1q4 (5)
2q1q3 − 2q2 q4 2q2 q3 + 2q1q4 −q12 − q22 + q32 + q42 to allocate different feature learning polices to different
components of the input signals, which is a promising facet for
We utilize the rotation matrix 𝐶𝑏𝑛 to project the acceleration feature extraction in multichannel time series signals received
and magnetometer measurements into NCS by left multiplying from inertial sensors. More details of the multi-head CNNs can
the transform matrix. be found in [58], and more details of the multi-head attention
T T
An = anx any anz = Cbn Ab =Cbn abx aby abz (6) mechanism can be found in [57].
T T
The inertial sensor measurements in NCS over a fixed sliding
M n = mnx m ny m nz = Cbn M b =Cbn mbx mby mbz (7) window are expressed as follows:
𝐴𝑖𝑥 = [𝑎𝑛,𝑡
𝑥 𝑥
, 𝑎𝑛,𝑡+1 𝑥
, ⋯ , 𝑎𝑛,𝑡+𝐾−1 ] (8)
D. Walking Pattern Recognition based on Multi-Head 𝑦 𝑦 𝑦 𝑦
𝐴𝑖 = [𝑎𝑛,𝑡 , 𝑎𝑛,𝑡+1 , ⋯ , 𝑎𝑛,𝑡+𝐾−1 ] (9)
Convolutional Attention Mechanism 𝐴𝑖𝑧 = [𝑎𝑛,𝑡
𝑧 𝑧
, 𝑎𝑛,𝑡+1 𝑧
, ⋯ , 𝑎𝑛,𝑡+𝐾−1 ] (10)
Various walking patterns bring challenge for PDR especially 𝐺𝑖𝑥 = [𝑔𝑡𝑥 , 𝑔𝑡+1
𝑥 𝑥
, ⋯ , 𝑔𝑡+𝐾−1 ] (11)
for step detection and step length estimation. In this study, we 𝑦 𝑦 𝑦 𝑦
𝐺𝑖 = [𝑔𝑡 , 𝑔𝑡+1 , ⋯ , 𝑔𝑡+𝐾−1 ] (12)
consider more detailed walking patterns and define five 𝑧 𝑧 𝑧 𝑧
𝐺𝑖 = [𝑔𝑡 , 𝑔𝑡+1 , ⋯ , 𝑔𝑡+𝐾−1 ] (13)
elemental walking patterns, as presented in TABLE I. These 𝑦 𝑦
𝐷𝑖 = [𝐴𝑖𝑥 , 𝐴𝑖 , 𝐴𝑖𝑧 , 𝐺𝑖𝑥 , 𝐺𝑖 , 𝐺𝑖𝑧 ] (14)
walking patterns are usually performed during the process of
where 𝐷𝑖 represents the input vector at time, K represents
pedestrian positioning and navigation.
Convolutional neural networks (CNNs) have proven the size of the sliding window. 𝑎 𝑥 , 𝑎 𝑦 and 𝑎 𝑧 indicate the
extremely effective in extracting informative representations of tri-axis accelerometer in NCS. 𝑔 𝑥 , 𝑔 𝑦 and 𝑔 𝑧 indicate the
data [56] and multi-head attention mechanism enables the tri-axis gyroscope in NCS.
presented approach to ignore the irrelevant features and to focus Sensor feature extraction is a key step in recognizing walking
on a subset of pertinent features [57]. To ensure more accurate patterns. To extract discriminative features, we design a 6-head
walking pattern recognition, we incorporated multi-head CNNs CNN to process the sensor vector. The feature maps of the
[58] and multi-head attention mechanism [57] for better feature previous layer are convolved with pre-trained convolutional
extraction and selection, as shown in Fig. 4. Multi-head CNNs kernels. The 𝑗𝑡ℎ feature map at the 𝑖 𝑡ℎ layer of 𝑐 𝑡ℎ head is a
is used in the feature extraction process. The multi-head matrix, and 𝑣𝑖𝑗𝑥,𝑐 denotes the value at the 𝑥 𝑡ℎ row, which is
attention mechanism supports feature selection. Multi-head computed as follows:
CNNs can have different filter banks and different processing 𝑥+𝑝
𝑣𝑖𝑗𝑥,𝑐 = 𝑓𝑅𝑒𝐿𝑈 (𝑓𝑐𝑜𝑛𝑣
𝑐
2𝑑 (𝑣𝑖−1 )) , ∀𝑐 = 1,2 ⋯ 6 (15)
layers in each head. For example, one head can have a 4×4
filters, and another head can have a 7×4 filters. Additionally, where 𝑓𝑅𝑒𝐿𝑈 is the activation function. The convolution
𝑐 𝑡ℎ
for a certain head, we can even choose whether to have dropout function 𝑓𝑐𝑜𝑛𝑣 2𝑑 of the 𝑐 head in the multi-head CNN is
or pooling layers. The multi-head CNNs has the unique ability presented as follows:
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 6
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 7
PCA
Initial Heading
Local Heading
Heading_pca_mag
(Global Heading) Heading_gyr Heading_last_step
Fig. 6. Step counting error rate of different walking patterns.
Heading Fusion
achieve accurate step counting in different walking modes, we
designed an auto-correlation pedometer algorithm based on the
walking pattern. The algorithm trains different thresholds Step_Heading
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 8
where Let ω𝑛𝑧 is the angular rate around the z-axis at NCS, and C2
𝑧
𝑏𝑔𝑦𝑟 is the bias errors around the z-axis. A x
The relative heading is obtained by the integral operation of
0
the compensated angular rate.
tf
Fig. 9. Pedestrian position update.
Gyr (tf) = Gyr (ts) + nz (t)dt (25) heading of the next step, and both two difference values beyond
ts
where [ts, tf] is the time interval, 𝜃𝑡𝑓 − 𝜃𝑡𝑠 is heading angle related threshold values, the heading estimation result of the
changes in radians at time interval[ts, tf]. Note that it a heading current step is regarded as an outlier. Then, we calibrate the
changes relative to the initial orientation. We calculate the heading of pedestrian by averaging the heading of the previous
absolute heading estimation of the gyroscope by adding the and next steps.
initial heading from the magnetometer with the relative heading G. Position Update According to Walking Patterns
from the gyroscope.
For forward walking situations, traditional PDR effectively
3) Heading Fusion
estimates the real-time heading of pedestrian. However, for
Gyroscope provides short-term accurate heading estimation,
backward walking or lateral walking situation, the heading
while magnetometer provides long-term coarse heading
angle or pedestrian remains forward, thus resulting in the
estimation without accumulative errors. Based on the above
reckoning trajectory will continue updating forward that is
characteristics, to obtain a more accurate and stable heading
perpendicular or opposite to the actual trajectory. As shown in
estimation, we fuse previous step heading, magnetometer
Fig. 9, in case of backward walking, pedestrian walks from
heading and gyroscope heading according to the magnetometer
starting point A to point B, and then back to starting point A.
variation as well as the correlation between gyroscope and
The final pedestrian trajectory reckoned by the traditional PDR
magnetometer. The pedestrian heading is fused as follows:
method is 𝐴 → 𝐵 → 𝐷, which is opposite to pedestrian truth
k −1 ,c c , ,m m trajectory. In the case of either left or right lateral walking,
k −1
+ Gyr ,c c , ,m m
k pedestrian walks from starting point A to point B, and then
k= k laterally walks from B to point 𝐶1 (𝐶2 ). The final pedestrian
Mag + Gyr , c c , ,m m
k
(26) trajectory reckoned by the traditional PDR method is still 𝐵 →
k −1
+ Mag + Gyr , c c , ,m m
k k
𝐷 , which is perpendicular to pedestrian truth trajectory.
Therefore, backward walking or lateral walking will cause an
,c = Mag
k
-Gyr
k
, ,m = Mag
k
- Mag
k -1
enormous cumulative error.
𝑘
where 𝜃𝑀𝑎𝑔 𝑘
and 𝜃𝐺𝑦𝑟 represent the heading acquired by the To address this issue, we define the Pedestrian positioning
equation as follows:
magnetometer and gyroscope for the 𝑘 𝑡ℎ step, respectively. α
β and γ represent the weighting coefficients of the previous step xk = xk −1 + k lk sin k + k lk cos k
(27)
yk = yk −1 + k lk cos k − k lk sin k
𝑘
heading 𝜃 𝑘−1 , magnetometer heading 𝜃𝑀𝑎𝑔 , and gyroscope
𝑘
heading 𝜃𝐺𝑦𝑟 , respectively. The threshold 𝜃𝑐 and 𝜃𝑚 are used
where ( xk , yk ) represents the pedestrian locations in the NCS
to determine the correlation 𝜃∆,𝑐 between magnetometer and
at the 𝑘 𝑡ℎ step. lk and k represent the step length and heading
gyroscope, and magnetometer variation𝜃∆,𝑚 .
4) Heading Outlier Filtering of pedestrian, respectively. k and k are the flag of detected
There should be no significant change in the heading walking patterns. In the case of forwards walking, k is set to
estimation results of neighbor steps during normal walking 1 and k is set to 0. In the case of backward walking, k is set
without significant turning [41]. However, unexpectedly
rotating or shaking a smartphone arouses marked heading to -1 and k is set to 0. In the case of left lateral walking, k is
fluctuation, but no turning event. To restrain the heading set to 0 and k is set to -1. In the case of left lateral walking,
fluctuation caused by unexpected hand movements, we k is set to 0 and k is set to 1. Compared with the traditional
describe a postprocessing outlier filtering algorithm by
PDR method, the proposed position updating method based on
analyzing the heading different values between adjacent steps.
walking pattern recognition effectively decrease the positioning
If the current step heading is significantly larger (smaller) than
error caused by backward walking or lateral walking.
the heading of the previous step and smaller (larger) than the
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 9
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 10
Fig. 11. The flowchart of online trajectory calibration based on forward and from InvenSense (ICM-20690) and a three-axis magnetometer
backward magnetic fingerprint trajectories. (amk09911)), with 8 core 2.4 GHz processor. The experimental
data are sampled at a frequency of 100 Hz.
A foot-mounted inertial navigation system (INS) aided by the
zero velocity update (ZUPT), zero angular rate update (ZARU)
and magnetic angular rate update (MARU) technologies has the
ability to constraint the accumulative errors and achieves a 0.3%
horizontal positioning accuracy on the traveled distance [62].
Additionally, INS can provide more high frequency. Therefore,
foot-mounted INS is the best way to evaluate the positioning
performance of PDR system. As illustrated in Fig. 13, the
performance evaluation system consists of an Android
smartphone and a foot-mounted INS module (MPU9250 [63]
(a) (b)
Fig. 12. Improving PDR performance with online magnetic fingerprint from Invensense Co., Ltd, San Jose, CA, USA). The precise
trajectory calibration. (a) magnetic loop closures. (b) trajectory calibration. pedestrian position from the foot-mounted INS module is sent
to the smartphone via Bluetooth, and synchronizes with the
measurements of smartphone embedded MEMS sensors.
IV. EXPERIMENTATION AND EVALUATION
We use the estimated total traveled distance (TTD), step
In this section, we thoroughly evaluate the proposed method. detection rate (SDR), step length error (SLE) and circular error
The experimental setup is first described. Then various probability (CEP) to quantify the performance of PDR. We
experiments are conducted in indoor and outdoor scenarios, and invited a group of heterogeneous volunteers to evaluate the
the accuracy of the proposed PDR method is verified by proposed method. TABLE II details the volunteers’ information.
comparing it with the ground truth from a foot-mounted IMUs
module. Section B and Section C evaluate the impact of B. Improving PDR Performance with Walking Pattern
walking pattern and online magnetic fingerprint trajectory Recognition
calibration on the localization accuracy, respectively. We carried out a long-distance walking experiment to verify
Positioning accuracy of the proposed PDR method under how much performance improvement the walking pattern
typical PDR scenarios is analyzed in Section D. Section E recognition brings to the proposed PDR method. We started
compares the proposed method with state-of-the-art methods. walking from an indoor office. We walked downstairs, left the
office and walked along the pre-planned path to my dormitory.
A. Experimental Setup
Fig. 14 illustrates the entire paths. Volunteers made a 1569 s
To evaluate the proposed PDR method based on walking (26.15 min) walk covering office, indoor stair, street, pedestrian
pattern recognition and online magnetic fingerprint trajectory skyway and subway station. To facilitate the evaluation, we
calibration, we carried out experiments in an indoor office, randomly divided the pre-planned paths into twelve segments
shopping malls, subway station, streets and pedestrian skyway. (see Fig. 14) set mark points at each split point. The
Mate 9 from Huawei Inc. is used for experiments, which built- corresponding walking pattern is detailed in TABLE III. During
in nine-DoF sensors (a three-axis accelerometer and gyroscope
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 11
TABLE II
DESCRIPTION OF VOLUNTEERS
Smartphone Subject Gender Age Height (cm) Weight (kg) Speed
S1 M 27 171 68 Fast/slow
S2 M 26 183 82 Fast
S3 F 23 155 45 Normal
S4 M 35 192 85 Fast/normal
S5 F 25 161 54 Fast/slow
S6 F 27 158 57 Fast/normal
S7 M 31 175 70 Fast
S6 F 57 162 49 Slow/normal
z S7 M 71 167 67 Slow
Slow (around 0.83 m/s), normal (around 1.4 m/s), fast (around 2.5 m/s)
0 x
IMU
D C
Pedestrain skyway Street
10 9 (a) (b)
D C 8 Fig. 15. Performance comparison of PDR with walking pattern recognition
End point
11 12 and no walking pattern recognition. (a) Office. (b) Shopping mall. Positioning
Accuracy in Typical Scenarios.
E C 7
E and the SDR, SLE, CEP (50%), CEP (75%) and CEP (95%) are
98.42%, 4.6 cm, 3.6 m, 6.72 m and 11.87 m, respectively.
Compared to PDR without walking pattern recognition, the
performance of PDR with walking pattern recognition was
Subway station
2 B 6
enhanced by 24.42%, 4.8 cm, 116.97 m, 252.75 m and 466.79
1 3 4 5
m in SDR, SLE, CEP (50%), CEP (75%) and CEP (95%). The
Start point
A
proposed PDR method enhances the localization accuracy by
adaptively adjusting the parameters of step detection, step
length model, heading model and position update according to
A B the walking pattern recognition results.
Office Stair
To further verify the robustness of the walking pattern
recognition algorithm, we supplement the experiment of
Fig. 14. The pre-planned paths description. The volunteer walked along the frequently switching walking pattern in a small-scale space.
highlighted curve with the corresponding walking pattern.
Volunteers change walking patterns every 8-10 meters (10-15
the experiment, volunteers precisely walked along the entire steps). The average walking pattern recognition accuracy drops
pre-planned paths with the corresponding walking pattern. to 91.8%. The main reason of recognition accuracy descension
Volunteers only change their walking pattern at each split point. is that the test window near the walk pattern switching point
A total number of 2400 steps are taken in the pre-planned paths. contains sensor data on two different walking patterns, which
The proposed PDR with walking pattern recognition bring more challenges for walking patterns recognition.
successfully detected 2438 steps with 98.42% accuracy, while Fortunately, in practical application, it is very rare to switch
the proposed PDR without walking pattern recognition walking patterns frequently in small-scale spaces.
successfully detected 1776 steps with 74% accuracy. The C. Improving PDR Performance with Online Magnetic
ground truth walking distance is 1615.18 m. When reached the Fingerprint Trajectory Calibration
endpoint, the final position error of proposed PDR with walking
To evaluate the online magnetic fingerprint matching
pattern recognition is 56.34 m, 3.49% over ground truth
calibration performance, we experimented on planned round-
walking distance, while the final position error of proposed
trip routes in office and shopping mall. The office is
PDR without walking pattern recognition is 595.57 m, 36.87%
characterized by narrow and regular corridors, while the
over ground truth walking distance.
shopping mall is characterized by wider and irregular walking
TABLE III summarizes the recognition accuracy of each
paths.
segment walking pattern. The performance comparison of PDR
The performance comparison of PDR with online trajectory
with walking pattern recognition and without walking pattern
calibration and without online trajectory calibration is detailed
recognition is detailed in TABLE IV. As expected, the walking
in TABLE V and Fig. 15. Fig. 15 shows the cumulative
pattern recognition significantly improves PDR performance,
distribution function (CDF) of our proposed PDR method in
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 12
TABLE III
ACCURACY OF WALKING PATTERN RECOGNITION IN DIFFERENT SEGMENTS.
Scenarios Forward walking Backward walking Left lateral walking Right lateral walking Running
Segment 1 5 8 11 2 9 3 7 4 12 6 10
Accuracy (%) 99.2 99.6 97.3 97.5 98.7
TABLE IV
PERFORMANCE COMPARISON OF PDR WITH WALKING PATTERN RECOGNITION AND NO WALKING PATTERN RECOGNITION.
Methods TTD SDR SLE CEP (50%) CEP (75%) CEP (95%)
Proposed PDR with walking pattern recognition 1583.64 m 98.42% 4.6 cm 3.64 m 6.72 m 11.87 m
Proposed PDR without walking pattern recognition 1163.38 m 74% 9.4cm 120.61 m 259.47 m 478.66 m
TABLE V
PERFORMANCE COMPARISON OF PDR WITH ONLINE TRAJECTORY CALIBRATION AND NO ONLINE TRAJECTORY CALIBRATION.
Scenarios Methods SDR SLE CEP (50%) CEP (75%) CEP (95%)
Proposed PDR with online magnetic fingerprint trajectory calibration 99.21 % 4.3 cm 0.93 m 1.50 m 2.50 m
Office
Proposed PDR without online magnetic fingerprint trajectory calibration 99.21 % 4.3 cm 1.43 m 2.45 m 4.55 m
Proposed PDR with online magnetic fingerprint trajectory calibration 98.67 % 4.5 cm 0.89 m 1.58 m 2.69 m
Shopping mall
Proposed PDR without online magnetic fingerprint trajectory calibration 98.67 % 4.5 cm 1.48 m 2.48 m 5.60 m
TABLE VI
EXPERIMENTAL RESULTS OF THREE TYPICAL SCENARIOS.
Scenarios SDR SLE CEP (50%) CEP (75%) CEP (95%)
Closed rectangular 99.19% 4.3 cm 0.66 m 1.06 m 1.71 m
Outdoor stadium 98.83% 4.4 cm 0.59 m 1.08 m 2.31 m
Intricate path 98.17% 4.5 cm 0.74 m 1.22 m 2.53 m
Mean 98.73% 4.4 cm 0.66 m 1.13 m 2.18 m
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 13
that the mean of the SDR, SLE, CEP (50%), CEP (75%) and scenarios. Since PCA-based heading estimation, RMPCA and
CEP (95%) of the proposed method in typical scenarios are Wang are more robust against different walking patterns than
98.73%, 4.4 cm, 0.66 m, 1.13 m, 2.18 m, respectively. the traditional PDR and SmartPDR. However, RMPCA and
Wang cannot adaptively adjust parameters of step counting and
E. Comparing with Other Methods
step length estimation for each walking pattern. The cumulative
To justify the superiority of our proposed PDR based on error of all compared methods is not well suppressed. Due to
walking patterns-recognition and online magnetic fingerprint accurate heading estimation for arbitrary attitude, parameters
trajectory calibration, we compared the proposed method with adaptive adjustment of step detection and step length model,
the following PDR methods in terms of CDF. and online opportunistic online magnetic fingerprint trajectory
• Traditional PDR, utilizes the step detection based on zero- calibration, the proposed method achieves promising
crossing, step length estimation based on the relationship positioning accuracy.
of acceleration variance and step frequency, and the yaw
angle from Android’s compass to reckon the locations of V. CONCLUSION
pedestrian.
The main issue of PDR system is that pedestrian usually
• SmartPDR [36], proposes a hybrid heading estimation
walks in a variety of walking patterns. Additionally, the heading
method that fuses previous heading estimation, gyroscope
offset between walking heading and pointing of smartphone
heading and magnetic heading based on the variation of
also brings challenge. In this paper, we propose a novel PDR
magnetometer measurements as well as the correlation
system based on walking pattern recognition and the online
between gyroscope measurements and magnetometer
magnetic fingerprint trajectory calibration for pedestrian
measurements. This PDR solution requires pedestrian
navigation and positioning using a smartphone. Compared with
keeping smartphone on hand all the time.
other PDR methods, the proposed PDR method does not force
• RMPCA [40], utilizes rotation matrix and PCA to pedestrian to walk forward and hold smartphone in a fixed
accurately estimate pedestrian heading, getting rid of the posture, as well as effectively restrains accumulative errors. We
constraints of fixed smartphone attitude. have thoroughly evaluated our proposed PDR method under a
• Wang [37], utilizes the PCA on global accelerations (PCA- variety of experimental scenarios. Experiments have proved
GA) to estimate pedestrian heading, as well as recognizes that both the walking pattern recognition and online magnetic
four motion states (walking, upstairs, downstairs and fingerprint trajectory calibration enhance localization accuracy,
running) to adaptive adjust parameters of step detection and the combination of them yields the best performance. We
and step length estimation. have enhanced positioning accuracy of the PDR method itself,
We compared the overall localization accuracy of the rather than utilizing the assistance of other auxiliary devices or
proposed method and that of mentioned above methods in historical training data. Therefore, the proposed PDR algorithm
different walking patterns. To make a fair comparison, we is not only available for emergency scenarios but also easy to
maintain the same walking path, walking pattern and holding integrate with other positioning technologies to improve
smartphone attitude for each comparison to validate the positioning accuracy.
performance of different methods. Fig. 17 shows the Humans are flexible structures, thus resulting in complex and
cumulative error distribution of different methods in variable movement modes. However, the proposed PDR
corresponding walking patterns. With respect to the forward method only considered five walking pattens (forward walking,
walking, 75th percentile positioning accuracy of the proposed backward walking, right lateral walking, left lateral walking,
PDR method is 2.28 m, while those of traditional PDR, and running). To obtain more accurate location-based services
SmartPDR, RMPCA and Wang are 4.49 m, 3.15 m, 3.21 m and more walking patterns, such as jumping, jogging, sprinting,
2.76 m, respectively. Although pointing of smartphone are ascending and descending stairs, crouch walking, and ladder-
consistent with walking heading under running pattern, the climbing, will be considered in our future work.
performance of all methods is degraded, compared to the
forward walking. As shown in Fig. 17, the traditional PDR and ACKNOWLEDGMENT
SmartPDR exist much larger errors, which are not suitable for
backward walking, left lateral walking or right lateral walking We would like to thank the editors and the anonymous
reviewers for their valuable comments, which greatly improved
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 14
the quality of this manuscript. Many thanks to the Beijing Key Method Using Wearable Sensors ” Proc. ACM Interactive, Mobile,
Wearable Ubiquitous Technol., vol. 2, no. 2, pp. 1–16, Jul. 2018.
Laboratory of Mobile Computing and Pervasive Device, [21] Hemin Zhang, Weizheng Yuan, Qiang Shen, Tai Li, and Honglong Chang,
Institute of Computing Technology Chinese Academy of “A Handheld Inertial Pedestrian Navigation System With Accurate Step
Sciences for the support in our research. Modes and Device Poses Recognition ” IEEE Sens. J., vol. 15, no. 3, pp.
1421–1429, Mar. 2015.
[22] Jiuchao Qian Jiabin Ma Rendong Ying Peilin Liu and Ling Pei “An
REFERENCES improved indoor localization method using smartphone inertial sensors ”
[1] grandviewresearch “Global Indoor Location-based Services (LBS) in International Conference on Indoor Positioning and Indoor Navigation,
Market Analysis Report ” 2019. . 2013, pp. 1–7.
[2] J. Lai et al. “TagSort : Accurate Relative Localization Exploring RFID [23] J. Santos A. Costa and M. J. Nicolau “Autocorrelation analysis of
Phase Spectrum Matching for Internet of Things ” IEEE Internet Things accelerometer signal to detect and count steps of smartphone users ” in
J., vol. 7, no. 1, pp. 389–399, Jan. 2020. 2019 International Conference on Indoor Positioning and Indoor
[3] K. Yu K. Wen Y. Li S. Zhang and K. Zhang “A Novel NLOS Navigation (IPIN), 2019, pp. 1–7.
Mitigation Algorithm for UWB Localization in Harsh Indoor [24] W. Shao, H. Luo, F. Zhao, C. Wang, A. Crivello, and M. Z. Tunio,
Environments ” IEEE Trans. Veh. Technol., vol. 68, no. 1, pp. 686–699, “DePedo: Anti Periodic Negative-Step Movement Pedometer with Deep
Jan. 2019. Convolutional Neural Networks ” in 2018 IEEE International Conference
[4] N. Yu X. Zhan S. Zhao Y. Wu and R. Feng “A Precise Dead Reckoning on Communications (ICC), 2018, pp. 1–6.
Algorithm Based on Bluetooth and Multiple Sensors ” IEEE Internet [25] Q. Wang et al. “Personalized Stride-Length Estimation based on Active
Things J., vol. 5, no. 1, pp. 336–351, Feb. 2018. Online Learning ” IEEE Internet Things J., vol. 4662, no. c, pp. 1–1, 2020.
[5] R. Wang Z. Li H. Luo F. Zhao W. Shao and Q. Wang “A Robust Wi- [26] L. E. Diez, A. Bahillo, J. Otegui, and T. Otim “Step Length Estimation
Fi Fingerprint Positioning Algorithm Using Stacked Denoising Methods Based on Inertial Sensors: A Review ” IEEE Sens. J., vol. 18, no.
Autoencoder and Multi-Layer Perceptron ” Remote Sens., vol. 11, no. 11, 17, pp. 6908–6926, Sep. 2018.
p. 1293, May 2019. [27] S. Miyazaki “Long-term unrestrained measurement of stride length and
[6] H. Zou Y. Zhou J. Yang and C. J. Spanos “Unsupervised WiFi-Enabled walking velocity utilizing a piezoelectric gyroscope. ” IEEE Trans.
IoT Device-User Association for Personalized Location-Based Service ” Biomed. Eng., vol. 44, no. 8, pp. 753–9, Aug. 1997.
IEEE Internet Things J., vol. 6, no. 1, pp. 1238–1245, Feb. 2019. [28] Q. Ladetto, B. M. Terrier T. Philippe and Y. Schutz “On foot navigation :
[7] Q. Wang, H. Luo, A. Men F. Zhao and Y. Huang “An Infrastructure- continuous step calibration using both complementary recursive prediction
Free Indoor Localization Algorithm for Smartphones ” Sensors, vol. 18, and adaptive Kalman filtering ” J. Navig., vol. 53, no. 02, pp. 279–285,
no. 10, p. 3317, Oct. 2018. 2000.
[8] Q. Wang et al. “Light positioning: A high-accuracy visible light indoor [29] H. Weinberg “Using the ADXL202 in pedometer and personal navigation
positioning system based on attitude identification and propagation model ” applications ” Analog devices AN-602 Appl. notes, vol. 2, pp. 1–6, 2002.
Int. J. Distrib. Sens. Networks, vol. 14, no. 2, p. 155014771875826, Feb. [30] T.-N. Do, R. Liu, C. Yuen, M. Zhang, and U.-X. Tan “Personal Dead
2018. Reckoning Using IMU Mounted on Upper Torso and Inverted Pendulum
[9] Y. Shu C. Bo G. Shen C. Zhao L. Li and F. Zhao “Magicol: Indoor Model ” IEEE Sens. J., vol. 16, no. 21, pp. 7600–7608, Nov. 2016.
Localization Using Pervasive Magnetic Field and Opportunistic WiFi [31] Q. Wang L. Ye H. Luo A. Men F. Zhao and C. Ou “Pedestrian Walking
Sensing ” IEEE J. Sel. Areas Commun., vol. 33, no. 7, pp. 1443–1457, Jul. Distance Estimation Based on Smartphone Mode Recognition ” Remote
2015. Sens., vol. 11, no. 9, p. 1140, May 2019.
[10] Q. Wang H. Luo F. Zhao and W. Shao “An indoor self-localization [32] D. Bousdar Ahmed E. M. Diaz and J. J. García Domínguez “Automatic
algorithm using the calibration of the online magnetic fingerprints and Calibration of the Step Length Model of a Pocket INS by Means of a Foot
indoor landmarks ” in 2016 International Conference on Indoor Inertial Sensor ” Sensors, vol. 20, no. 7, p. 2083, Apr. 2020.
Positioning and Indoor Navigation (IPIN), 2016, no. 978, pp. 1–8. [33] Q. Wang L. Ye H. Luo A. Men F. Zhao and Y. Huang “Pedestrian
[11] W. Shao et al. “Location Fingerprint Extraction for Magnetic Field Stride-Length Estimation Based on LSTM and Denoising Autoencoders ”
Magnitude Based Indoor Positioning ” J. Sensors, vol. 2016, no. June 2017, Sensors, vol. 19, no. 4, p. 840, Feb. 2019.
pp. 1–16, 2016. [34] S. Vandermeeren, H. Bruneel, and H. Steendam “Feature Selection for
[12] D. Liu S. Guo Y. Yang Y. Shi and M. Chen “Geomagnetism-Based Machine Learning Based Step Length Estimation Algorithms ” Sensors,
Indoor Navigation by Offloading Strategy in NB-IoT ” IEEE Internet vol. 20, no. 3, p. 778, Jan. 2020.
Things J., vol. 6, no. 3, pp. 4074–4084, Jun. 2019. [35] S. O. H. Madgwick A. J. L. Harrison and R. Vaidyanathan “Estimation
[13] L. Zhao, H. Huang, X. Li, S. Ding, H. Zhao, and Z. Han “An Accurate of IMU and MARG orientation using a gradient descent algorithm ” in
and Robust Approach of Device-Free Localization With Convolutional 2011 IEEE International Conference on Rehabilitation Robotics, 2011,
Autoencoder ” IEEE Internet Things J., vol. 6, no. 3, pp. 5825–5840, Jun. vol. 2011, pp. 1–7.
2019. [36] W. Kang and Y. Han “SmartPDR: Smartphone-Based Pedestrian Dead
[14] L. Pei J. Liu R. Guinness Y. Chen H. Kuusniemi and R. Chen “Using Reckoning for Indoor Localization ” IEEE Sens. J., vol. 15, no. 5, pp.
LS-SVM Based Motion Recognition for Smartphone Indoor Wireless 2906–2916, May 2015.
Positioning ” Sensors, vol. 12, no. 5, pp. 6155–6175, May 2012. [37] B. Wang X. Liu B. Yu R. Jia and X. Gan “Pedestrian Dead Reckoning
[15] C. A. Ronao and S.-B. Cho “Human activity recognition with smartphone Based on Motion Mode Recognition Using a Smartphone ” Sensors, vol.
sensors using deep learning neural networks ” Expert Syst. Appl., vol. 59, 18, no. 6, p. 1811, Jun. 2018.
pp. 235–244, Oct. 2016. [38] M. Elhoushi, J. Georgy, A. Noureldin, and M. J. Korenberg “Motion
[16] H. Zhang, Z. Xiao, J. Wang, F. Li, and E. Szczerbicki “A Novel IoT- mode recognition for indoor pedestrian navigation using portable devices ”
Perceptive Human Activity Recognition (HAR) Approach Using IEEE Trans. Instrum. Meas., vol. 65, no. 1, pp. 208–221, 2016.
Multihead Convolutional Attention ” IEEE Internet Things J., vol. 7, no. [39] J. Qian L. Pei J. Ma R. Ying and P. Liu “Vector Graph Assisted
2, pp. 1072–1080, Feb. 2020. Pedestrian Dead Reckoning Using an Unconstrained Smartphone ”
[17] X. Gao H. Luo Q. Wang F. Zhao L. Ye and Y. Zhang “A Human Sensors, vol. 15, no. 3, pp. 5032–5057, Mar. 2015.
Activity Recognition Algorithm Based on Stacking Denoising [40] Z.-A. Deng G. Wang Y. Hu and D. Wu “Heading Estimation for Indoor
Autoencoder and LightGBM ” Sensors, vol. 19, no. 4, p. 947, Feb. 2019. Pedestrian Navigation Using a Smartphone in the Pocket ” Sensors, vol.
[18] K. Chen, L. Yao, D. Zhang, X. Wang, X. Chang and F. Nie “A 15, no. 9, pp. 21518–21536, Aug. 2015.
Semisupervised Recurrent Convolutional Attention Model for Human [41] Z. Deng, X. Liu Z. Qu C. Hou and W. Si “Robust Heading Estimation
Activity Recognition ” IEEE Trans. Neural Networks Learn. Syst., pp. 1– for Indoor Pedestrian Navigation Using Unconstrained Smartphones ”
10, 2019. Wirel. Commun. Mob. Comput., vol. 2018, pp. 1–11, Jul. 2018.
[19] A. Saeed T. Ozcelebi and J. Lukkien “Multi-task Self-Supervised [42] D. Wu L. Xia and J. Geng “Heading Estimation for Pedestrian Dead
Learning for Human Activity Detection ” Proc. ACM Interactive, Mobile, Reckoning Based on Robust Adaptive Kalman Filtering ” Sensors, vol. 18,
Wearable Ubiquitous Technol., vol. 3, no. 2, pp. 1–30, Jun. 2019. no. 6, p. 1970, Jun. 2018.
[20] L. Peng L. Chen Z. Ye and Y. Zhang “AROMA: A Deep Multi-Task [43] Q. Wang et al. “Pedestrian Heading Estimation Based on Spatial
Learning Based Simple and Complex Human Activity Recognition Transformer Networks and Hierarchical LSTM ” IEEE Access, vol. 7, pp.
162309–162322, 2019.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 15
[44] Z. Chen Q. Zhu and Y. C. Soh “Smartphone Inertial Sensor-Based location-based services, pervasive computing, computer vision
Indoor Localization and Tracking With iBeacon Corrections ” IEEE Trans.
Ind. Informatics, vol. 12, no. 4, pp. 1540–1549, Aug. 2016.
and machine learning.
[45] V. Magnago, L. Palopoli, R. Passerone D. Fontanelli and D. Macii “A
nearly optimal landmark deployment for indoor localisation with limited HAIYONG LUO received the B.S. degree
sensing ” in 2017 International Conference on Indoor Positioning and
from the Department of Electronics and
Indoor Navigation (IPIN), 2017, pp. 1–8.
[46] J. Shang, F. Gu, X. Hu, and A. Kealy “APFiLoc: An Infrastructure-Free Information Engineering, Huazhong
Indoor Localization Method Fusing Smartphone Inertial Sensors, University of Science and Technology,
Landmarks and Map Information ” Sensors, vol. 15, no. 10, pp. 27251– Wuhan, China, in 1989, the M.S. degree from
27272, Oct. 2015.
[47] J.-H. Kim and B.-S. Kim “Autonomous Landmark Calibration Method for the School of Information and
Indoor Localization ” Sensors, vol. 17, no. 9, p. 1952, Aug. 2017. Communication Engineering, Beijing
[48] T. Liu X. Zhang Q. Li and Z. Fang “Modeling of structure landmark for University of Posts and Telecommunication,
indoor pedestrian localization ” IEEE Access, vol. 7, pp. 1–1, 2019.
[49] B. Zhou, Q. Li, Q. Mao, W. Tu X. Zhang and L. Chen “ALIMC: Activity China, in 2002, and the Ph.D. degree in computer science from
Landmark-Based Indoor Mapping via Crowdsourcing ” IEEE Trans. Intell. the University of Chines Academy of Sciences, Beijing, China,
Transp. Syst., vol. 16, no. 5, pp. 2774–2785, Oct. 2015. in 2008. He is currently an Associate Professor with the
[50] F. Hölzke J. P. Wolff F. Golatowski and C. Haubelt “Low-complexity
online correction and calibration of pedestrian dead reckoning using map Institute of Computer Technology, Chinese Academy of
matching and GPS ” Geo-Spatial Inf. Sci., vol. 22, no. 2, pp. 114–127, Science, China. His main research interests are location-based
2019. services, pervasive computing, mobile computing, and Internet
[51] L. Lin Shen and W. Wong Shung Hui “Improved Pedestrian Dead-
Reckoning-Based Indoor Positioning by RSSI-Based Heading Correction ”
of Things.
IEEE Sens. J., vol. 16, no. 21, pp. 7762–7773, Nov. 2016.
[52] X. Wang M. Jiang Z. Guo N. Hu Z. Sun and J. Liu “An Indoor HAO XIONG received the B.S. degree in the
Positioning Method for Smartphones Using Landmarks and PDR ”
School of Software Engineering, Beijing
Sensors, vol. 16, no. 12, p. 2135, Dec. 2016.
[53] H. Xia J. Zuo S. Liu and Y. Qiao “Indoor Localization on Smartphones University of Posts and Telecommunications,
Using Built-In Sensors and Map Constraints ” IEEE Trans. Instrum. Meas., Beijing, China, in 2019. HAO XIONG is
vol. 68, no. 4, pp. 1189–1198, Apr. 2019. currently pursuing the M.S. degree with the
[54] Y. Li, Y. Zhuang, P. Zhang, H. Lan, X. Niu, and N. El-Sheimy “An
improved inertial/wifi/magnetic fusion structure for indoor navigation ” School of Software Engineering, Beijing
Inf. Fusion, vol. 34, pp. 101–119, Mar. 2017. University of Posts and Telecommunications,
[55] Jiancheng Fang, Hongwei Sun, Juanjuan Cao, Xiao Zhang, and Ye Tao, Beijing, China. His current main interests
“A Novel Calibration Method of Magnetic Compass Based on Ellipsoid
Fitting ” IEEE Trans. Instrum. Meas., vol. 60, no. 6, pp. 2053–2061, Jun. include Location-based services, pervasive computing and
2011. machine learning.
[56] Y. LeCun Y. Bengio and G. Hinton “Deep learning ” Nature, vol. 521,
no. 7553, pp. 436–444, May 2015.
[57] A. Vaswani et al. “Attention Is All You Need ” Adv. Neural Inf. Process. AIDONG MEN received the B.S., M.S.,
Syst., vol. 2017-Decem, no. Nips, pp. 5999–6009, Jun. 2017. and Ph.D. degrees from the Department of
[58] M. Canizo I. Triguero A. Conde and E. Onieva “Multi-head CNN–RNN Radio Engineering, Beijing University of
for multi-time series anomaly detection: An industrial case study ”
Neurocomputing, vol. 363, no. July, pp. 246–260, Oct. 2019.
Posts and Telecommunications, Beijing, in
[59] J. Ruppelt N. Kronenwett and G. F. Trommer “A novel finite state 1987, 1990 and 1994. From 1994 to 2000,
machine based step detection technique for pedestrian navigation systems ” he was an Associate Professor with the
in 2015 International Conference on Indoor Positioning and Indoor
Department of Radio Engineering, Beijing
Navigation (IPIN), 2015, pp. 1–7.
[60] J. W. Kim, H. J. Jang, D.-H. Hwang and C. Park “A Step Stride and University of Posts and
Heading Determination for the Pedestrian Navigation System ” J. Glob. Telecommunications. Since 2000, he has been a Professor with
Position. Syst., vol. 3, no. 1&2, pp. 273–279, Dec. 2004. Telecom Engineering College, Beijing University of Posts and
[61] M. Müller “Dynamic Time Warping ” in Information Retrieval for Music
and Motion, 2007. Telecommunications. He is a fellow of the Chinese Institute of
[62] Y. Gu Q. Song Y. Li and M. Ma “Foot-mounted Pedestrian Navigation Electronics and China Institute of Communications. He is also
based on Particle Filter with an Adaptive Weight Updating Strategy ” J. an Invited Fellow of the Science and Technology Committee of
Navig., vol. 68, no. 01, pp. 23–38, Jan. 2015.
[63] “MPU9250.” [Online]. Available: https://invensense.tdk.com/products/ State Administration of Radio, Film and Television. His
motion-tracking/9-axis/mpu-9250/. [Accessed: 20-Apr-2020]. research interests include multimedia communication, digital
TV, images and speech signal processing and transmission, and
QU WANG received the B.S. degree from so on. He has published over 100 papers in journals and
the School of Software Engineering, Beijing international conference.
University of Posts and Telecommunication,
China, in 2013, the M.S. degree from the FANG ZHAO received the B.S. degree
University of Chinese Academy of Sciences, from the School of Computer Science and
Beijing, China, in 2016. QU WANG is Technology, Huazhong University of
currently pursuing the Ph.D. degree with the Science and Technology, Wuhan, China, in
School of Information and Communication 1990, the M.S. and Ph.D. degrees in
Engineering, Beijing University of Posts and computer science and technology from the
Telecommunications, China. His current main interests include Beijing University of Posts and
Telecommunication, Beijing, China, in 2004
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/JIOT.2020.3016146, IEEE Internet of
Things Journal
> REPLACE THIS LINE WITH YOUR PAPER IDENTIFICATION NUMBER (DOUBLE-CLICK HERE TO EDIT) < 16
and 2009, respectively. She is currently a Professor with the CHANGHAI OU was born in Tongren,
School of Software Engineering, Beijing University of Posts Guizhou Province of China, in 1989. In 2013,
and Telecommunication. Her research interests include mobile he received his bachelor's degree in
computing, location-based services, and computer networks. Computer Science and Technology from
School of Computer and Information
MING XIA was born in Yiyang, China, in Technology, Beijing Jiaotong University,
1987. He is currently pursuing the Ph.D China. He received his Ph.D degree in Cyber
degree with the School of Electronic and Security from Institute of Information
Information Engineering, Beihang Engineering, Chinese Academy of Sciences (i.e. School of
University at Beijing. His research interests
Cyber Security, University of Chinese Academy of Sciences)
include motion recognition, pedestrian
in July, 2018. He is now a Research Fellow in Hardware &
inertial positioning, wearable sensor-based
Embedded Systems Lab (HESL), School of Computer Science
positioning, and their applications in
location-based service applications, especially in the fire- and Engineering, Nanyang Technological University,
fighting field. Singapore. His current research interests include cryptographic
hardware and embedded system security, side channel attacks,
machine learning and privacy preserving.
2327-4662 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed
View publication stats use limited to: BEIJING UNIVERSITY OF POST AND TELECOM. Downloaded on August 14,2020 at 00:53:24 UTC from IEEE Xplore. Restrictions apply.