Adeyemi Ademola A.Ademola@napier.ac.uk
Research Student
DeFT-Net: Dual-Window Extended Frequency Transformer for Rhythmic Motion Prediction
Ademola, Adeyemi; Sinclair, David; Koniaris, Babis; Hannah, Samantha; Mitchell, Kenny
Authors
David Sinclair D.Sinclair@napier.ac.uk
Research Assistant
Dr Babis Koniaris B.Koniaris@napier.ac.uk
Lecturer
Samantha Hannah
Prof Kenny Mitchell K.Mitchell2@napier.ac.uk
Professor
Abstract
Enabling online virtual reality (VR) users to dance and move in a way that mirrors the real-world necessitates improvements in the accuracy of predicting human motion sequences paving way for an immersive and connected experience. However, the drawbacks of latency in networked motion tracking present a critical detriment in creating a sense of complete engagement, requiring prediction for online synchronization of remote motions. To address this challenge, we propose a novel approach that leverages a synthetically generated dataset based on supervised foot anchor placement timings of rhythmic motions to ensure periodicity resulting in reduced prediction error. Specifically, our model compromises a discrete cosine transform (DCT) to encode motion, refine high frequencies and smooth motion sequences and prevent jittery motions. We introduce a feed-forward attention mechanism to learn based on dual-window pairs of 3D key points pose histories to predict future motions. Quantitative and qualitative experiments validating on the Human3.6m dataset result in observed improvements in the MPJPE evaluation metrics protocol compared with prior state-of-the-art.
Citation
Ademola, A., Sinclair, D., Koniaris, B., Hannah, S., & Mitchell, K. (2024, September). DeFT-Net: Dual-Window Extended Frequency Transformer for Rhythmic Motion Prediction. Presented at EG UK Computer Graphics & Visual Computing (2024), London
Presentation Conference Type | Conference Paper (published) |
---|---|
Conference Name | EG UK Computer Graphics & Visual Computing (2024) |
Start Date | Sep 12, 2024 |
End Date | Sep 13, 2024 |
Acceptance Date | Jul 24, 2024 |
Deposit Date | Aug 20, 2024 |
Publisher | Eurographics Association |
Peer Reviewed | Peer Reviewed |
Keywords | Computing methodologies → Machine Learning; Motion Processing; Virtual Reality |
Publisher URL | https://diglib.eg.org/communities/20818d8d-d709-42e3-9adc-c29719de3e99 |
Ensure healthy lives and promote well-being for all at all ages
This file is under embargo due to copyright reasons.
Contact repository@napier.ac.uk to request a copy for personal use.
Related Outputs
DanceGraph: A Complementary Architecture for Synchronous Dancing Online
(2023)
Presentation / Conference Contribution
You might also like
DanceGraph: A Complementary Architecture for Synchronous Dancing Online
(2023)
Presentation / Conference Contribution
Design Considerations of Voice Articulated Generative AI Virtual Reality Dance Environments
(2024)
Presentation / Conference Contribution
DanceMark: An open telemetry framework for latency sensitive real-time networked immersive experiences
(2024)
Presentation / Conference Contribution
Pixel history linear models for real-time temporal filtering.
(2016)
Journal Article
IRIDiuM: immersive rendered interactive deep media
(2016)
Presentation / Conference Contribution
Downloadable Citations
About Edinburgh Napier Research Repository
Administrator e-mail: repository@napier.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search