A Deep Model for Solving Physical Constraints in Human Motions Captured by Sparse Sensors

Ou Chengbin; Zhong Yingchun; Zhang Gang

doi:10.12052/gdutxb.250127

English Version Journal Login

您当前的位置：

首页 >

文章列表页 >

A Deep Model for Solving Physical Constraints in Human Motions Captured by Sparse Sensors

更新时间：2026-08-04

- A Deep Model for Solving Physical Constraints in Human Motions Captured by Sparse Sensors
- Vol. 43, Issue 4, Pages: 67-79(2026)
- 作者机构：
  
  广东工业大学自动化学院,广东,广州,510006
- 作者简介：
- 基金信息：
- DOI：10.12052/gdutxb.250127
  CLC： TP391.4
- Published：2026
- 稿件说明：
移动端阅览
欧诚彬, 钟映春, 张钢. 稀疏传感器动作捕捉物理约束的深度模型求解方法[J]. 广东工业大学学报, 2026, 43(4): 67-79.
欧诚彬, 钟映春, 张钢. 稀疏传感器动作捕捉物理约束的深度模型求解方法[J]. 广东工业大学学报, 2026, 43(4): 67-79. DOI： 10.12052/gdutxb.250127.

DOI： 10.12052/gdutxb.250127

摘要

采用稀疏传感器捕捉人体动作，并驱动计算机中的虚拟人复现该动作，是虚拟现实等领域的关键技术之一；其中，复现动作的相关计算需同时满足运动约束与物理约束。目前，主要采用非线性优化方法解决此类计算的物理约束问题，该方法存在运算时间长、计算复杂度高、需要设计专用优化器等不足。为此，本文提出了一种基于深度神经网络模型求解稀疏传感器动作捕捉物理约束优化问题的方法。首先，将多模态特征融合网络和多路精炼网络有效结合，形成后段融合多层级结构的基础模型。其次，通过渐进式融合的方法建立基础模型层间的反向投影连接，实现基础模型的迭代，使深层融合的信息能够被浅层使用。最后，提出了一种包含物理约束的组合加权损失函数，适用于隐式和显式物理约束情况下对深度模型的微调。实验结果表明，本文提出的方法不仅可行性好，而且计算效率提高了20个百分点；与其他常用优化方法相比，本文方法在4种主流评价指标上均展现良好性能；在其他数据集上运行本文方法，同样得到良好效果，证明本文方法具有较好的泛化性。本文方法为形成端到端的稀疏传感器动作捕捉深度模型提供了一种新的思路。

Abstract

Using sparse sensors to capture human movements and driving virtual humans in computers to reproduce these movements is one of the key technologies in fields such as virtual reality

among which

the relevant calculations for reproducing movements must simultaneously satisfy motion constraints and physical constraints. Currently

nonlinear optimization methods are primarily employed to address the physical constraints in such computations. However

these methods suffer from several drawbacks

including long computation time

high computational complexity

and the necessity of designing dedicated optimizers. To address these issues

a method based on a deep neural network model for solving the physical constraints optimization in sparse sensor motion capture is proposed. Firstly

the model effectively combines the multi-modal feature fusion network and the multi-path refinement network to form a post-fusion multi-level structure

which serves as the basic model of this research. Secondly

through a progressive fusion method

back-projection connections between the layers of the basic model are established

enabling the iteration of the basic model. This allows the information fused at deeper layers to be utilized by the shallower layers. Thirdly

a loss function in the form of combined weighting that incorporates physical constraints is proposed

which is suitable for fine-tuning the deep model in the presence of both implicit and explicit physical constraints. The experimental results demonstrate that the proposed method not only exhibits good feasibility but also improves computational efficiency by approximately 20 percentage points. Compared with other commonly used optimization methods

the proposed method performs better in terms of four mainstream evaluation indicators. Additionally

the method yields favorable results when applied to other datasets

demonstrating its strong generalization capability. This method provides a new perspective for the formation of an end-to-end deep model for sparse sensor motion capture.

关键词

Keywords

references

ZHANG J H, HE B Y, YANG X S, et al . A review on wearable inertial sensor based human motion tracking[J ] . Acta Automatica Sinica, 2019, 45(8): 1439-1454.

CEREAITI A, CROCE U D, SABATINI A M. Three-dimensional human kinematic estimation using magneto-inertial measurement units[M]. MÜLLER B, WOLF S I. Handbook of Human Motion. Cham: Springer, 2017: 221-244.

LOPER M, MAHMOOD N, ROMERO J, et al . SMPL: a skinned multi-person linear model[J ] . ACM Transactions on Graphics, 2015, 34(6): 1-16.

IONESCU C, PAPAVA D, OLARU V, et al . Human3. 6m: large scale datasets and predictive methods for 3D human sensing in natural environments[J ] . IEEE Transactions On Pattern Analysis and Machine Intellig ence, 2013, 36(7): 1325-1339.

RIAZ Q, TAO G, KRÜGER B, et al . Motion reconstruction using very few accelerometers and ground contacts[J ] . Graphical Models, 2015, 79: 23-38.

TAO G, SUN S, HUANG S, et al . Human modeling and real-time motion reconstruction for micro-sensor motion capture[C ] //2011 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems Proceedings. Vancouver: IEEE, 2011: 1-5.

HUANG Y, KAUFMANN M, AKSAN E, et al . Deep inertial poser: learning to reconstruct human pose from sparse inertial measurements in real time[J ] . ACM Transactions on Graphics (TOG) , 2018, 37(6): 1-15.

YI X, ZHOU Y, XU F. Transpose: real-time 3D human translation and pose estimation with six inertial sensors[J]. ACM Transactions On Graphics (TOG) , 2021, 40(4): 1-13.

FEATHERSTONE R. Rigid body dynamics algorithms[M]. Boston, MA: Springer US, 2008.

DU X, VASUDEVAN R, JOHNSON-ROBERSON M. Bio-lstm: a biomechanically inspired recurrent neural network for 3-d pedestrian pose and gait prediction[J]. IEEE Robotics and Automation Letters, 2019, 4(2): 1501-1508.

ZOU Y, YANG J, CEYLAN D, et al . Reducing footskate in human motion reconstruction with ground contact constraints[C ] //Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Snowmass Village: IEEE, 2020: 459-468.

ZANFIR A, MARINOIU E, SMINCHISESCU C. Monocular 3d pose and shape estimation of multiple people in natural scenes—the importance of multiple scene constraints[C]// 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 2148-2157.

LI Z, SEDLAR J, CARPENTIER J, et al . Estimating 3D motion and forces of person-object interactions from monocular video[C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019). Long Beach: IEEE/CVF, 2019: 8640-8649.

SHIMADA S, GOLYANIK V, XU W, et al . Physcap: physically plausible monocular 3D motion capture in real time[J ] . ACM Transactions on Graphics (ToG) , 2020, 39(6): 1-16.

YI X, ZHOU Y, HABERMANN M, et al . Physical inertial poser (pip): physics-aware real-time human motion tracking from sparse inertial sensors[C ] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE/CVF, 2022: 13167-13178.

RAISSI M, PERDIKARIS P, KARNIADAKIS G E. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations[J]. Journal of Computational physics, 2019, 378: 686-707.

SONG Y, YUAN X, YUE H. The ADMM-PINNs algorithmic framework for nonsmooth PDE-constrained optimization: a deep learning approach[J]. SIAM Journal on Scientific Computing, 2024, 46(6): C659-C687.

PARK S J, HONG K S, LEE S. Rdfnet: RGB-D multi-level residual feature fusion for indoor semantic segmentation[C]// 2017 IEEE International Conference on Computer Vision (ICCV 2017). Venice: IEEE, 2017: 4980-4989.

LIN G, MILAN A, SHEN C, et al . RefineNet: multi-path refinement networks for high-resolution semantic segmentation[C ] // 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). Honolulu: IEEE, 2017: 1925-1934.

HE K, ZHANG X, REN S, et al . Identity mappings in deep residual networks[C ] //2016 European Conference on Computer Vision (ECCV 2016). Amsterdam: Springer, 2016: 630-645.

VARMA R. Picking loss functions-a comparison between mse, cross entropy, and hinge loss[J]. Machine Learning Journal, 2023, 30(2): 200-220.

XIE K, WANG T, IQBAL U, et al . Physics-based human motion estimation and synthesis from videos[C ] //2021 IEEE/CVF International Conference on Computer Vision (ICCV 2021). Montreal: IEEE/CVF, 2021: 11532-11541.

VON MARCARD T, ROSENHAHN B, BLACK M J, et al . Sparse inertial poser: Automatic 3d human pose estimation from sparse imus[C ] //2017 Eurographics Conference on Computer Graphics and Interactive Techniques (Eurographics 2017). Geneva: Eurographics Association, 2017: 349-360.

深圳市维特智能科技有限公司. WT901WiFi 型 9 轴 WiFi 姿态传感器产品规范: GB/T191SJ 20873-2016 [S]. 深圳: 深圳市维特智能科技有限公司, 2019.

CHEN Z, BADRINARAYANAN V, LEE C Y, et al . Gradnorm: gradient normalization for adaptive loss balancing in deep multitask networks[C ] //International Conference on Machine Learning (ICML). Stockholm: PMLR, 2018: 794-803.

KENDALL A, GAL Y, CIPOLLA R. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C] //Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 7482-7491.

SUN Q, SUN Y. Optimization technology of BP neural network based on sparrow search algorithm[J]. Journal of Shanghai Dianji University, 2022, 25(1): 12-16.

YANG J, WANG L, ZHANG B, et al . Remaining useful life prediction of vehicle-oriented PEMFC systems based on IGWO-BP neural network under real-world traffic conditions[J ] . Energy, 2024, 291: 130334.

WEI C, FAN Y C, ZHOU Y Q, et al . Multi-output physics-informed neural networks model based on the Runge-Kutta method[J ] . Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(10): 2405-2416.

CASTELLANO G, KESSOUS L, CARIDAKIS G. Emotion recognition through multiple modalities: face, body gesture, speech[M]. PETERS C. Affect and Emotion in Human-Computer Interaction: From Theory to Applications. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008: 92-103.

RAMIREZ G A, BALTRUŠAITIS T, MORENCY L P. Modeling latent discriminative dynamics of multi-dimensional affective signals[C] //2011 International Conference on Affective Computing and Intelligent Interaction (ACII 2011). Memphis, TN: Springer Berlin Heidelberg, 2011: 396-406.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Graph Non-negative Matrix Factorization Based on Self-representation Learning Update for Clustering

A Basic-element Transformation Algorithm Based on Decision Trees

A Method for Generating Test Cases for Component-based Industrial Software Systems Based on Large Language Models

Structure of the Global Virtual Meat Protein Trade Network

Preparation and Performance Analysis of SMA-grafted Aminated Carbon Nanotube Antistatic Agent

Related Author

No data

Related Institution

No data

AI问答

⁰