AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Tsinghua Science and Technology Article
PDF (577.9 KB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Inertial Motion Tracking on Mobile and Wearable Devices: Recent Advancements and Challenges

School of Software, Tsinghua University, Beijing 100084, China
Department of Computer Science and Engineering, Michigan State University, Michigan, MI 48824, USA
Department of Computer Science, City University of Hong Kong, Hong Kong 999077, China
Department of Automation and the Global Innovation eXchange Institute (GIX), Tsinghua University, Beijing 100084, China
Show Author Information

Abstract

Motion tracking via Inertial Measurement Units (IMUs) on mobile and wearable devices has attracted significant interest in recent years. High-accuracy IMU-tracking can be applied in various applications, such as indoor navigation, gesture recognition, text input, etc. Many efforts have been devoted to improving IMU-based motion tracking in the last two decades, from early calibration techniques on ships or airplanes, to recent arm motion models used on wearable smart devices. In this paper, we present a comprehensive survey on IMU-tracking techniques on mobile and wearable devices. We also reveal the key challenges in IMU-based motion tracking on mobile and wearable devices and possible directions to address these challenges.

Keywords

Inertial Measurement Units (IMUs)motion trackingaccelerometergyroscopemagnetometersensor fusion

References

[1]
A. Altenbuchner, S. Haug, R. Kretschmer, and K. Weber, How to measure physical motion and the impact of individualized feedback in the field of rehabilitation of geriatric trauma patients, Stud. Health Technol. Inform., vol. 248, pp. 226–232, 2018.
Google Scholar
[2]
F. Alemuda and F. J. Lin, Gesture-based control in a smart home environment, in Proc. IEEE Int. Conf. Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Exeter, UK, 2017, pp. 784–791.
[3]
Y. Shen, B. Du, W. Xu, C. Luo, B. Wei, L. Cui, and H. Wen, Securing cyber-physical social interactions on wrist-worn devices, ACM Transactions on Sensor Networks, vol. 16, no. 2, pp. 1–22, 2020.
Google Scholar
[4]
Z. M. Zhou, C. S. Wu, Z. Yang, and Y. H. Liu, Sensorless sensing with WiFi, Tsinghua Science and Technology, vol. 20, no. 1, pp. 1–6, 2015.
Google Scholar
[5]
Q. Shi, S. H. Zhao, X. W. Cui, M. Q. Lu, and M. D. Jia, Anchor self-localization algorithm based on UWB ranging and inertial measurements, Tsinghua Science and Technology, vol. 24, no. 6, pp. 728–737, 2019.
Google Scholar
[6]
S. Jiang, Z. Li, and P. Zhou. Memento: An emotion-driven lifelogging system with wearables, ACM Transactions on Sensor Networks, vol. 15, no. 1, pp. 1–23, 2019.
Google Scholar
[7]
Z. Zhang, X. Cong, W. Feng, H. P. Zhang, G. D. Fu, and J. Y. Chen, WAEAS: An optimization scheme of EAS scheduler for wearable applications, Tsinghua Science and Technology, vol. 26, no. 1, pp. 72–84, 2021.
Google Scholar
[8]
M. Kok, J. D. Hol, and T. B. Schön, Using inertial sensors for position and orientation estimation, arXiv preprint arXiv: 1704.06053, 2017.
Google Scholar
[9]
Z. Yang, C. S. Wu, Z. M. Zhou, X. L. Zhang, X. Wang, and Y. H. Liu, Mobility increases localizability: A survey on wireless indoor localization using inertial sensors, ACM Comput. Surv., vol. 47, no. 3, p. 54, 2015.
Google Scholar
[10]
H. Liu, R. Li, S. C. Liu, S. B. Tian, and J. Z. Du, Smartcare: Energy-efficient long-term physical activity tracking using smartphones, Tsinghua Science and Technology, vol. 20, no. 4, pp. 348–363, 2015.
Google Scholar
[11]
S. Shen, M. Gowda, and R. R. Choudhury, Closing the gaps in inertial motion tracking, in Proc. 24th Annu. Int. Conf. Mobile Computing and Networking (MobiCom), New Delhi, India, 2018, pp. 429–444.
[12]
O. J. Woodman, An introduction to inertial navigation, Technical Report UCAM-CL-TR-696, Computer Laboratory, University of Cambridge, https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-696.html, 2007.
[13]
G. Laput, R. Xiao, and C. Harrison, ViBand: High-fidelity bio-acoustic sensing using commodity smartwatch accelerometers, in Proc. 29th Annu. Symp. User Interface Software and Technology (UIST), New York, NY, USA, 2016, pp. 321–333.
[14]
V. T. Van Hees, L. Gorzelniak, E. C. D. León, M. Eder, M. Pias, S. Taherian, U. Ekelund, F. Renström, P. W. Franks, A. Horsch, et al., Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity, PLoS One, vol. 8, no. 4, p. e61691, 2013.
Google Scholar
[15]
Android developer, Motion sensors, https://developer.android.com/guide/topics/sensors/sensors_motion, 2021.
[16]
P. F. Zhou, M. Li, and G. B. Shen, Use it free: Instantly knowing your phone attitude, in Proc. 20th Annu. Int. Conf. Mobile Computing and Networking (MobiCom), Maui, HI, USA, 2014, pp. 605–616.
[17]
S. Poddar, V. Kumar, and A. Kumar, A comprehensive overview of inertial sensor calibration techniques, J. Dyn. Syst. Meas. Control, vol. 139, no. 1, p. 011006, 2016.
Google Scholar
[18]
P. D. Groves, Principles of GNSS, Inertial, and Multisensor Integrated Navigation Systems. 2nd ed. London, UK: Artech House, 2013.
[19]
M. J. Caruso, Applications of magnetic sensors for low cost compass systems, in Proc. IEEE 2000 Position Location and Navigation Symp., San Diego, CA, USA, 2000, pp. 177–184.
[20]
Y. F. Yin, L. Xie, T. Gu, Y. J. Lu, and S. L. Lu, Aircontour: Building contour-based model for in-air writing gesture recognition, ACM Trans. Sensor Net., vol. 15, no. 4, p. 44, 2019.
Google Scholar
[21]
P. Batista, C. Silvestre, P. Oliveira, and B. Cardeira, Accelerometer calibration and dynamic bias and gravity estimation: Analysis, design, and experimental evaluation, IEEE Trans. Control Syst. Technol., vol. 19, no. 5, pp. 1128–1137, 2011.
Google Scholar
[22]
E. H. Shin and E. S. Naser, Accuracy improvement of low cost INS/GPS for land applications, in Proc. 2002 National Technical Meeting of the Institute of Navigation, San Diego, CA, USA, 2002, pp. 146–157.
[23]
E. H. Shin and N. El-Sheimy, A new calibration method for strapdown inertial navigation systems, Zfv, vol. 127, pp. 41–50, 2002.
Google Scholar
[24]
X. J. Niu, Y. Li, H. P. Zhang, Q. J. Wang, and Y. L. Ban, Fast thermal calibration of low-grade inertial sensors and inertial measurement units, Sensors, vol. 13, no. 9, pp. 12192–12217, 2013.
Google Scholar
[25]
J. L. Crassidis, F. L. Markley, and Y. Cheng, Survey of nonlinear attitude estimation methods, J. Guid. Control Dyn., vol. 30, no. 1, pp. 12–28, 2007.
Google Scholar
[26]
N. H. Q. Phuong, H. J. Kang, Y. S. Suh, and Y. S. Ro, A DCM based orientation estimation algorithm with an inertial measurement unit and a magnetic compass, J. Univ. Comp. Sci., vol. 15, no. 4, pp. 859–876, 2009.
Google Scholar
[27]
E. Edwan, J. Y. Zhang, J. C. Zhou, and O. Loffeld, Reduced DCM based attitude estimation using low-cost IMU and magnetometer triad, in Proc. 2011 8th Workshop on Positioning, Navigation and Communication, Dresden, Germany, 2011, pp. 1–6.
[28]
S. Yean, B. S. Lee, C. K. Yeo, and C. H. Vun, Algorithm for 3D orientation estimation based on Kalman filter and gradient descent, in Proc. 2016 IEEE 7th Annu. Information Technology, Electronics and Mobile Communication Conf. (IEMCON), Vancouver, Canada, 2016, pp. 1–6.
[29]
L. Lou, X. Xu, J. Cao, Z. L. Chen, and Y. Xu, Sensor fusion-based attitude estimation using low-cost MEMS-IMU for mobile robot navigation, in Proc. 2011 6th IEEE Joint Int. Information Technology and Artificial Intelligence Conf., Chongqing, China, 2011, pp. 465–468.
[30]
W. Li and J. L. Wang, Effective adaptive Kalman filter for MEMS-IMU/magnetometers integrated attitude and heading reference systems, J. Navigat., vol. 66, no. 1, pp. 99–113, 2013.
Google Scholar
[31]
R. Zhu, D. Sun, Z. Y. Zhou, and D. Q. Wang, A linear fusion algorithm for attitude determination using low cost mems-based sensors, Measurement, vol. 40, no. 3, pp. 322–328, 2007.
Google Scholar
[32]
D. Jurman, M. Jankovec, R. Kamnik, and M. Topic, Calibration and data fusion solution for the miniature attitude and heading reference system, Sens. Actuators A: Phys., vol. 138, no. 2, pp. 411–420, 2007.
Google Scholar
[33]
Z. Ercan, V. Sezer, H. Heceoglu, C. Dikilitas, M. Gokasan, A. Mugan, and S. Bogosyan, Multi-sensor data fusion of DCM based orientation estimation for land vehicles, in Proc. 2011 IEEE Int. Conf. Mechatronics, Istanbul, Turkey, 2011, pp. 672–677.
[34]
N. Shantha and T. Jann, Estimation of attitudes from a low-cost miniaturized inertial platform using kalman filter-based sensor fusion algorithm, Sadhana, vol. 29, no. 2, pp. 217–235, 2004.
Google Scholar
[35]
H. F. Grip, T. I. Fossen, T. A. Johansen, and A. Saberi, Globally exponentially stable attitude and gyro bias estimation with application to GNSS/INS integration, Automatica, vol. 51, pp. 158–166, 2015.
Google Scholar
[36]
F. Olsson, M. Kok, K. Halvorsen, and T. B. Schön, Accelerometer calibration using sensor fusion with a gyroscope, in Proc. 2016 IEEE Statistical Signal Proc. Workshop (SSP), Palma de Mallorca, Spain, 2016, pp. 1–5.
[37]
E. Foxlin, Inertial head-tracker sensor fusion by a complementary separate-bias Kalman filter, in Proc. IEEE 1996 Virtual Reality Annu. Int. Symp., Santa Clara, CA, USA, 1996, pp. 185–194.
[38]
R. Munguia and A. Grau, Attitude and heading system based on EKF total state configuration, in Proc. 2011 IEEE Int. Symp. Industrial Electronics, Gdansk, Poland, 2011, pp. 2147–2152.
[39]
M. Romanovas, L. Klingbeil, M. Trachtler, and Y. Manoli, Efficient orientation estimation algorithm for low cost inertial and magnetic sensor systems, in Proc. 2009 IEEE/SP 15th Workshop on Statistical Signal Proc., Cardiff, UK, 2009, pp. 586–589.
[40]
H. G. de Marina, F. J. Pereda, J. M. Giron-Sierra, and F. Espinosa, UAV attitude estimation using unscented Kalman filter and TRIAD, IEEE Trans. Indust. Electron., vol. 59, no. 11, pp. 4465–4474, 2012.
Google Scholar
[41]
J. L. Marins, X. P. Yun, E. R. Bachmann, R. B. McGhee, and M. J. Zyda, An extended Kalman filter for quaternion-based orientation estimation using MARG sensors, in Proc. 2001 IEEE/RSJ Int. Conf. Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the Next Millennium (Cat. No. 01CH37180), Maui, HI, USA, 2002, pp. 2003–2011.
[42]
A. M. Sabatini, Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing, IEEE Trans. Biomed. Eng., vol. 53, no. 7, pp. 1346–1356, 2006.
Google Scholar
[43]
J. Goslinski, M. Nowicki, and P. Skrzypczynski, Performance comparison of EKF-based algorithms for orientation estimation on android platform, IEEE Sens.J., vol. 15, no. 7, pp. 3781–3792, 2015.
Google Scholar
[44]
B. Huyghe, J. Doutreloigne, and J. Vanfleteren, 3D orientation tracking based on unscented Kalman filtering of accelerometer and magnetometer data, in Proc. 2009 IEEE Sensors Applications Symp, New Orleans, LA, USA, 2009, pp. 148–152.
[45]
R. Mahony, T. Hamel, and J. M. Pflimlin, Nonlinear complementary filters on the special orthogonal group, IEEE Trans. Automat. Control, vol. 53, no. 5, pp. 1203–1218, 2008.
Google Scholar
[46]
M. Nowicki, J. Wietrzykowski, and P. Skrzypczynski, Simplicity or flexibility? complementary filter vs. EKF for orientation estimation on mobile devices, in Proc. 2015 IEEE 2nd Int. Conf. Cybernetics (CYBCONF), Gdynia, Poland, 2015, pp. 166–171.
[47]
S. O. H. Madgwick, An Efficient Orientation Filter for Inertial and Inertial/Magnetic Sensor Arrays. UK: University of Bristol, 2010.
[48]
S. O. H. Madgwick, A. J. L. Harrison, and R. Vaidyanathan, Estimation of IMU and MARG orientation using a gradient descent algorithm, in Proc. 2011 IEEE Int. Conf. Rehabilitation Robotics, Zurich, Switzerland, 2011, pp. 1–7.
[49]
M. El-Gohary, S. Pearson, and J. McNames, Joint angle tracking with inertial sensors, Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 208, pp. 1068–1071, 2008.
Google Scholar
[50]
M. El-Gohary and J. McNames, Shoulder and elbow joint angle tracking with inertial sensors, IEEE Trans. Biomed. Eng., vol. 59, no. 9, pp. 2635–2641, 2012.
Google Scholar
[51]
S. Shen, H. Wang, and R. R. Choudhury, I am a smartwatch and I can track my user’s arm, in Proc. 14th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’16, Singapore, 2016, pp. 85–96.
[52]
Y. Liu, C. D. Lin, Z. J. Li, Z. D. Liu, and K. S. Wu, When wearable sensing meets arm tracking (poster), in Proc. 17th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’19, Seoul, Republic of Korea, 2019, pp. 518–519.
[53]
B. Fang, X. Wei, F. C. Sun, H. M. Huang, Y. L. Yu, and H. P. Liu, Skill learning for human-robot interaction using wearable device, Tsinghua Sci. Technol., vol. 24, no. 6, pp. 654–662, 2019.
Google Scholar
[54]
P. del Moral, Nonlinear filtering: Interacting particle solution, Markov Process. Relat. Fields, vol. 2, no. 4, pp. 555–580, 1996.
Google Scholar
[55]
X. Y. Liu, Z. Zhou, W. R. Diao, Z. Li, and K. H. Zhang, When good becomes evil: Keystroke inference with smartwatch, in Proc. 22nd ACM SIGSAC Conf. Computer and Communications Security, Denver, CO, USA, 2015, pp. 1273–1285.
[56]
C. Xu, P. H. Pathak, and P. Mohapatra, Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch, in Proc.16th Int. Workshop on Mobile Computing Systems and Applications, Santa Fe, NM, USA, 2015, pp. 9–14.
[57]
Z. J. Ba, T. H. Zheng, X. Y. Zhang, Z. Qin, B. C. Li, X. Liu, and K. Ren, Learning-based practical smartphone eavesdropping with built-in accelerometer, in Proc. Network and Distributed Systems Security (NDSS) Symp., San Diego, CA, USA, 2020, pp. 23–26.
[58]
J. H. Hou, X. Y. Li, P. D. Zhu, Z. F. Wang, Y. Wang, J. W. Qian, and P. L. Yang, SignSpeaker: A real-time, high-precision smartwatch-based sign language translator, in Proc. 25th Annu. Int. Conf. Mobile Computing and Networking, MobiCom’19, Los Cabos, Mexico, 2019, pp. 24:1–24:15.
[59]
A. Parate, M. C. Chiu, C. Chadowitz, D. Ganesan, and E. Kalogerakis, RisQ: Recognizing smoking gestures with inertial sensors on a wristband, in Proc. 12th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’14, Bretton Woods, NH, USA, 2014, pp. 149–161.
[60]
C. Amma, M. Georgi, and T. Schultz, Airwriting: A wearable handwriting recognition system, Pers. Ubiquit. Comput., vol. 18, no. 1, pp. 191–203, 2014.
Google Scholar
[61]
S. D. Choi, A. S. Lee, and S. Y. Lee, On-line handwritten character recognition with 3D accelerometer, in Proc. 2006 IEEE Int. Conf. Information Acquisition, Veihai, China, 2006, pp. 845–850.
[62]
J. H. Wu, G. Pan, D. Q. Zhang, G. D. Qi, and S. J. Li, Gesture recognition with a 3-D accelerometer, in Proc. 6th Int. Conf. Ubiquitous Intelligence and Computing (UIC) 2009, Heidelberg, Germany, 2009, pp. 25–38.
[63]
L. Sun, D. Q. Zhang, B. Li, B. Guo, and S. J. Li, Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations, in Proc. 7th Int. Conf. Ubiquitous Intelligence and Computing (UIC), Berlin, Germany, 2010, pp. 548–562.
[64]
Xia Q. X., Hong F., Feng Y., and Guo Z. W., Motionhacker: Motion sensor-based eavesdropping on handwriting via smartwatch, in Proc. IEEE INFOCOM 2018-IEEE Conf. Computer Communications Workshops (INFOCOM WKSHPS), Honolulu, HI, USA, 2018, pp. 468–473.
[65]
S. Sen, K. Grover, V. Subbaraju, and A. Misra, Inferring smartphone keypress via smartwatch inertial sensing, in Proc. 2017 IEEE Int. Conf. Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA, 2017, pp. 685–690.
[66]
C. Zhang, J. R.Yang, C. Southern, T. E. Starner, and G. D. Abowd, Watchout: Extending interactions on a smartwatch with inertial sensing, in Proc. ACM Int. Symp. Wearable Computers, Kobe, Japan, 2016, pp. 136–143.
[67]
T. H. Vu, A. Misra, Q. Roy, K. C. T. Wei, and Y. Lee, Smartwatch-based early gesture detection 8 trajectory tracking for interactive gesture-driven applications, Proc. ACM Interact. Mobile Wearab. Ubiquit. Technol., vol. 2, no. 1, p. 39, 2018.
Google Scholar
[68]
H. Wang, T. T. T. Lai, and R. R. Choudhury, MoLe: Motion leaks through smartwatch sensors, in Proc. 21stAnnu. Int. Conf. Mobile Computing and Networking, MobiCom’15, Paris, France, 2015, pp. 155–166.
[69]
S. Agrawal, I. Constandache, S. Gaonkar, R. R. Choudhury, K. Caves, and F. DeRuyter, Using mobile phones to write in air, in Proc. 9th Int. Conf. Mobile Systems, Applications, and Services, MobiSys’11, Bethesda, MD, USA, 2011, pp. 15–28.
[70]
K. Katsuragawa, J. R. Wallace, and E. Lank, Gestural text input using a smartwatch, in Proc. Working Conf. Advanced Visual Interfaces, Bari, Italy, 2016, pp. 220–223.
[71]
L. Ardüser, P. Bissig, P. Brandes, and R. Wattenhofer, Recognizing text using motion data from a smartwatch, in Proc. 2016 IEEE Int. Conf. Pervasive Computing and Communication Workshops (PerCom Workshops), Sydney, Australia, 2016, pp. 1–6.
[72]
T. Deselaers, D. Keysers, J. Hosang, and H. A. Rowley, Gyropen: Gyroscopes for pen-input with mobile phones, IEEE Trans. Human Mach. Syst., vol. 45, no. 2, pp. 263–271, 2015.
Google Scholar
Tsinghua Science and Technology
Volume 26 Issue 5,
October 2021
Pages 692-705
DOI: 10.26599/TST.2021.9010017
Cite this article:
Song Z, Cao Z, Li Z, et al. Inertial Motion Tracking on Mobile and Wearable Devices: Recent Advancements and Challenges. Tsinghua Science and Technology, 2021, 26(5): 692-705. https://doi.org/10.26599/TST.2021.9010017

1391

Views

96

Downloads

23

Crossref

22

Web of Science

30

Scopus

1

CSCD

Altmetrics
Received: 07 February 2021
Accepted: 26 February 2021
Published: 20 April 2021
© The author(s) 2021

© The author(s) 2021. The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).
Return