Self-Regulated Egocentric Activity Anticipation with Uncertainty-Boosted Temporal Modeling
Keywords:
Egocentric Vision, Activity Anticipation, Uncertainty Estimation, Temporal ModelingAbstract
Anticipating future activities from egocentric video streams represents a critical challenge in computer vision, holding significant potential for advancements in human-robot interaction, wearable computing, and augmented reality. Unlike third-person activity recognition, egocentric anticipation requires models to infer unobserved future actions from a dynamic, inherently unstable first-person viewpoint. Current approaches predominantly rely on deterministic temporal modeling, which frequently struggles to account for the inherent ambiguity and multi-modality of future events. To address these limitations, this paper introduces a novel framework termed Self-Regulated Egocentric Activity Anticipation with Uncertainty-Boosted Temporal Modeling. Our approach fundamentally departs from traditional deterministic paradigms by explicitly quantifying and integrating predictive uncertainty into the temporal feature extraction process. We propose a self-regulated mechanism that dynamically adjusts the learning objective based on the estimated epistemic and aleatoric uncertainties, thereby preventing the model from becoming overconfident in highly ambiguous contexts. Furthermore, the uncertainty-boosted temporal modeling component effectively selectively emphasizes temporal dependencies that exhibit high predictive confidence while attenuating noisy or irrelevant historical cues. Through extensive theoretical analysis and systematic empirical evaluation on large-scale egocentric datasets, we demonstrate that our proposed framework not only provides robust uncertainty estimates but also significantly improves anticipation accuracy across varying time horizons. The findings underscore the critical role of uncertainty quantification in sequential decision-making tasks and establish a new methodological baseline for future research in first-person vision systems.
Downloads
References
1. Zhao, H., Gu, J., Wang, S., Lu, T., Zhang, X., Wu, Z., ... & Jiang, Y. G. (2026). LSTD: Long Short-Term Temporal Diffusion for Video Generation. IEEE Transactions on Multimedia.
2. Zhang, P., Liu, H., Ge, Z., Wang, C., & Lam, E. Y. (2024). Neuromorphic imaging with joint image deblurring and event denoising. IEEE Transactions on Image Processing, 33, 2318-2333.
3. Zhang, W., Zhang, C., Gu, C., Kou, J., Yuan, H., Fang, X., ... & Fang, Y. (2024, October). Hallucination in Large Language Models: From Mechanistic Understanding to Novel Control Frameworks. In 2024 7th International Conference on Universal Village (UV) (pp. 1-36). IEEE.
4. Anticipation, E. V. A. Self-Regulated Learning for Egocentric Video Activity Anticipation.
5. Qi, Z., Yuan, Y., Ruan, X., Wang, S., Zhang, W., & Huang, Q. (2024). Collaborative debias strategy for temporal sentence grounding in video. IEEE Transactions on Circuits and Systems for Video Technology, 34(11), 10972-10986.
6. Wang, W., Mi, L., Cen, S., Dai, H., Li, Y., Fu, X., & Liu, Y. (2025). Region-based Content Enhancement for {Efficient} Video Analytics at the Edge. In 22nd USENIX Symposium on Networked Systems Design and Implementation (NSDI 25) (pp. 613-633).
7. Tang, Y., Zhang, G., Liu, J. K., & Qin, R. (2025). Weakly supervised land-cover classification of high-resolution images with low-resolution labels through optimized label refinement. International Journal of Remote Sensing, 46(5), 1913-1937.
8. Dong, J., Qu, X., Zhang, C., Rong, S. Q., Thai, N. D., Pan, W., ... & Ong, Y. S. (2026). Tug-of-war no more: Harmonizing accuracy and robustness in vision-language models via stability-aware task vector merging. In The Fourteenth International Conference on Learning Representations.
9. Liu, Y., Liu, M., Huang, S., & Lv, J. (2025, April). Asymmetric visual semantic embedding framework for efficient vision-language alignment. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 6, pp. 5676-5684).
10. Zhang, P., Ge, Z., Song, L., & Lam, E. Y. (2023). Neuromorphic imaging with density-based spatiotemporal denoising. IEEE Transactions on Computational Imaging, 9, 530-541.
11. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of CVPR.
12. Tu, P., Xie, X., Ai, G., Li, Y., Huang, Y., & Zheng, Y. (2023). FemtoDet: An object detection baseline for energy versus performance tradeoffs. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 13318-13327).
13. Dai, S., Wu, Y., Chen, S., Huang, R., & Dannenberg, R. B. (2023, November). SingStyle111: A Multilingual Singing Dataset With Style Transfer. In ISMIR (pp. 765-773).
14. Xu, W., Huang, H., Cheng, M., Yu, L., Wu, Q., & Zhang, J. (2023, July). Masked cross-image encoding for few-shot segmentation. In 2023 IEEE international conference on multimedia and expo (ICME) (pp. 744-749). IEEE.
15. Li, Y., Yang, J., Yang, Z., Li, B., Han, L., He, H., ... & Tang, R. (2026, March). Make lvlms focus: Context-aware attention modulation for better multimodal in-context learning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 8, pp. 6610-6618).
16. Dong, J., Wang, Y., Lai, J., & Xie, X. (2023). Restricted black-box adversarial attack against deepfake face swapping. IEEE Transactions on Information Forensics and Security, 18, 2596-2608.
17. Tang, J., Qiao, S., Cui, B., Ma, Y., Zhang, S., & Kanoulas, D. (2022, October). You can even annotate text with voice: Transcription-only-supervised text spotting. In Proceedings of the 30th ACM international conference on multimedia (pp. 4154-4163).
18. Huang, J., Tao, A., Marco, R., Bogdanovic, M., Kelly, J., & Shkurti, F. (2025, May). Automated planning domain inference for task and motion planning. In 2025 IEEE International Conference on Robotics and Automation (ICRA) (pp. 12534-12540). IEEE.
19. Dong, J., Zhang, C., Qu, X., Ma, Z., Koniusz, P., & Ong, Y. S. (2026). Robust superalignment: Weak-to-strong robustness generalization for vision-language models. Advances in Neural Information Processing Systems, 38, 18345-18377.
20. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024). A conditional denoising diffusion probabilistic model for point cloud upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20786-20795).
21. Zhu, Guoying, et al. "Enabling MoE on the Edge via Importance-Driven Expert Scheduling." arXiv preprint arXiv:2508.18983 (2025).
22. Lv, Qi, et al. "F1: A vision-language-action model bridging understanding and generation to actions." arXiv preprint arXiv:2509.06951 (2025).
23. Kong, R., Li, Y., Feng, Q., Wang, W., Ye, X., Ouyang, Y., ... & Liu, Y. (2024, August). SwapMoE: Serving off-the-shelf MoE-based large language models with tunable memory budget. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 6710-6720).
24. Cai, Y., Jing, T., Liu, C., Lian, Z., Huang, S. S., & Huang, H. (2025, June). Spatio-Temporally Consistent Depth Estimation for Dynamic Scenes using 3D Scene Flows. In 2025 IEEE International Conference on Multimedia and Expo (ICME) (pp. 1-6). IEEE.
25. Lv, Qi, et al. "Spatial-temporal graph diffusion policy with kinematic modeling for bimanual robotic manipulation." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
26. Tu, P., Huang, Y., Zheng, F., He, Z., Cao, L., & Shao, L. (2022, June). Guidedmix-net: Semi-supervised semantic segmentation by using labeled images as reference. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 2, pp. 2379-2387).
27. Huang, Z., Xu, M., & Perry, S. (2025, November). RLD-GS: Reinforcement Learning-Driven Gaussian Splatting for High-Fidelity Neural Rendering. In International Conference on Neural Information Processing (pp. 268-282). Singapore: Springer Nature Singapore.
28. Xie, C., Zhu, D., Wang, Z., Zhang, H., & Wei, Z. (2026). Compliance-Aware Discharge Agent for Auditable ICU Discharge Planning: A Pilot Feasibility Study Using Structured eICU Records. Available at SSRN 6429758.
29. Zhao, H., Wang, Q., Zhan, G., Min, W., Zou, Y., & Cui, S. (2022). Need only one more point (NOOMP): Perspective adaptation crowd counting in complex scenes. IEEE Transactions on Multimedia, 25, 1414-1426.
30. Wang, Siheng, et al. "C3-owd: A curriculum cross-modal contrastive learning framework for open-world detection." arXiv preprint arXiv:2509.23316 (2025).
31. Zhang, S., Yang, S., Zhang, W., Xiong, Y., & Yao, S. (2026). Hybrid Beamforming for Subarray-Level Movable Antenna Enhanced MU-MIMO Communications. IEEE Wireless Communications Letters, 15, 2559-2563.
32. Mi, L., Wang, W., Tu, W., He, Q., Kong, R., Fang, X., ... & Liu, Y. (2025, March). Empower vision applications with LoRA LMM. In Proceedings of the Twentieth European Conference on Computer Systems (pp. 261-277).
33. Qi, Z., Wang, S., Zhang, W., & Huang, Q. (2024). Uncertainty-boosted robust video activity anticipation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 7775-7792.
34. Wang J, Fan L, Li B, et al. A Dynamic Factor Gating Architecture with Market Regime Awareness for Stock Return Forecasting[J]. 2026.
35. Liu, Y., Liu, H., Wang, H., Meng, F., & Liu, M. (2023). BCAN: Bidirectional correct attention network for cross-modal retrieval. IEEE Transactions on Neural Networks and Learning Systems, 35(10), 14247-14258.
36. Guo, Hanzhong, et al. "Leveraging verifier-based reinforcement learning in image editing." arXiv preprint arXiv:2604.27505 (2026).
37. Song, S., Tang, Y., & Qin, R. (2025). Synthetic Data Matters: Re-training with Geo-typical Synthetic Labels for Building Detection. IEEE Transactions on Geoscience and Remote Sensing.
38. Liu, Y., & Kwon, H. (2025). Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6252-6261)
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Lucia Gallo, Mathilde Laurent, Astrid Zimmermann
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This article is licensed under the Creative Commons Attribution 4.0 International License. This license permits use, sharing, and adaptation of the article for non-commercial purposes, provided the original authors and source are properly credited.
Alternatively, you may choose other Creative Commons licenses based on your preferences:
-
CC Attribution-NonCommercial-NoDerivatives 4.0: This license allows others to download the works and share them with others as long as they credit the author, but they can’t change them in any way or use them commercially.
-
CC Attribution-NonCommercial 4.0: This license allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author, but they cannot use the material for commercial purposes.
-
CC Attribution-NonCommercial-ShareAlike 4.0: This allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author and license their new creations under identical terms.
These terms ensure that your article is freely accessible and usable, while respecting the rights of the original authors.
