Skip to main content

Research Repository

Advanced Search

NeFT-Net: N-window extended frequency transformer for rhythmic motion prediction

Ademola, Adeyemi; Sinclair, David; Koniaris, Babis; Hannah, Samantha; Mitchell, Kenny

Authors

David Sinclair

Babis Koniaris

Samantha Hannah



Abstract

Advancements in prediction of human motion sequences are critical for enabling online virtual reality (VR) users to dance and move in ways that accurately mirror real-world actions, delivering a more immersive and connected experience. However, latency in networked motion tracking remains a significant challenge, disrupting engagement and necessitating predictive solutions to achieve real-time synchronization of remote motions. To address this issue, we propose a novel approach leveraging a synthetically generated dataset based on supervised foot anchor placement timings for rhythmic motions, ensuring periodicity and reducing prediction errors. Our model integrates a discrete cosine transform (DCT) to encode motion, refine high-frequency components, and smooth motion sequences, mitigating jittery artifacts. Additionally, we introduce a feed-forward attention mechanism designed to learn from N-window pairs of 3D key-point pose histories for precise future motion prediction. Quantitative and qualitative evaluations on the Human3.6M dataset highlight significant improvements in mean per joint position error (MPJPE) metrics, demonstrating the superiority of our technique over state-of-the-art approaches. We further introduce novel result pose visualizations through the use of generative AI methods.

Citation

Ademola, A., Sinclair, D., Koniaris, B., Hannah, S., & Mitchell, K. (2025). NeFT-Net: N-window extended frequency transformer for rhythmic motion prediction. Computers and Graphics, 129, Article 104244. https://doi.org/10.1016/j.cag.2025.104244

Journal Article Type Article
Acceptance Date May 2, 2025
Online Publication Date May 17, 2025
Publication Date 2025-06
Deposit Date Jan 15, 2025
Publicly Available Date Mar 31, 2025
Journal Elsevier Computers and Graphics Special Issue on CGVC
Print ISSN 0097-8493
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 129
Article Number 104244
DOI https://doi.org/10.1016/j.cag.2025.104244
Keywords Machine learning, Motion processing, Rendering, Virtual reality
Public URL http://researchrepository.napier.ac.uk/Output/4050770
This output contributes to the following UN Sustainable Development Goals:

SDG 3 - Good Health and Well-Being

Ensure healthy lives and promote well-being for all at all ages

Files








You might also like



Downloadable Citations