Dynamic Batch Scheduling With Memory-Aware Subgraph Mapping for Real-Time Recommendation Under Strict Tail Latency Constraints
Keywords:
real-time recommendation, dynamic batch scheduling, subgraph mapping, tail latency, memory management, graph neural networks, serving systemsAbstract
Real-time recommendation systems deployed at internet scale face a fundamental tension between model expressiveness and serving latency. As user-item interaction graphs grow to billions of edges, the memory footprint of subgraph-based inference pipelines routinely exceeds device capacity, while strict tail latency service-level objectives (SLOs) leave little room for dynamic reallocation. This paper presents a unified framework called DBS-MASM (Dynamic Batch Scheduling with Memory-Aware Subgraph Mapping) that addresses both constraints simultaneously. The proposed system decomposes the recommendation graph into memory-bounded subgraph partitions at serving time, coordinates partition loading with a priority-aware batch scheduler, and enforces latency budgets through a preemptive queue policy tuned to the 99th-percentile (P99) response deadline. Evaluated on two large-scale public benchmarks and one production-scale trace, DBS-MASM reduces P99 tail latency by 41.3% compared to the best-performing baseline while maintaining recall@20 within 0.8% of the full-graph oracle. Memory pressure violations drop by 67% relative to static partition strategies. These results demonstrate that joint optimization of batch formation and subgraph memory mapping is both necessary and sufficient to satisfy strict latency SLOs in production recommendation serving
Downloads
References
[1] Wang, X., He, X., Wang, M., Feng, F., & Chua, T. S. (2019, July). Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval (pp. 165-174).
[2] Gupta, U., Wu, C. J., Wang, X., Naumov, M., Reagen, B., Brooks, D., ... & Zhang, X. (2020, February). The architectural implications of facebook's dnn-based personalized recommendation. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA) (pp. 488-501). IEEE.
[3] He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020, July). Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval (pp. 639-648).
[4] Ke, L., Gupta, U., Cho, B. Y., Brooks, D., Chandra, V., Diril, U., ... & Zhang, X. (2020, May). Recnmp: Accelerating personalized recommendation with near-memory processing. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) (pp. 790-803). IEEE.
[5] Gupta, U., Hsia, S., Saraph, V., Wang, X., Reagen, B., Wei, G. Y., ... & Wu, C. J. (2020, May). Deeprecsys: A system for optimizing end-to-end at-scale neural recommendation inference. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) (pp. 982-995). IEEE.
[6] Naumov, M., Mudigere, D., Shi, H. J. M., Huang, J., Sundaraman, N., Park, J., ... & Smelyanskiy, M. (2019). Deep learning recommendation model for personalization and recommendation systems. arXiv preprint arXiv:1906.00091.
[7] Zhao, Z., Hong, L., Wei, L., Chen, J., Nath, A., Andrews, S., ... & Chi, E. (2019, September). Recommending what video to watch next: a multitask ranking system. In Proceedings of the 13th ACM conference on recommender systems (pp. 43-51).
[8] Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019, November). BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management (pp. 1441-1450).
[9] Feng, Y., Lv, F., Shen, W., Wang, M., Sun, F., Zhu, Y., & Yang, K. (2019). Deep session interest network for click-through rate prediction. arXiv preprint arXiv:1905.06482.
[10] Wang, S., Hu, L., Wang, Y., Cao, L., Sheng, Q. Z., & Orgun, M. (2019). Sequential recommender systems: challenges, progress and prospects. arXiv preprint arXiv:2001.04830.
[11] Shen, Z., Zhao, W., Wang, B., Wang, Z., & Shang, W. (2026). CAGR: A Cross-Accelerator Graph Optimization Framework for Efficient Recommender System Inference. IEEE Access.
[12] Pi, Q., Zhou, G., Zhang, Y., Wang, Z., Ren, L., Fan, Y., ... & Gai, K. (2020, October). Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (pp. 2685-2692).
[13] Eisenman, A., Naumov, M., Gardner, D., Smelyanskiy, M., Pupyrev, S., Hazelwood, K., ... & Katti, S. (2019). Bandana: Using non-volatile memory for storing deep learning models. Proceedings of machine learning and systems, 1, 40-52.
[14] Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., & Yu, P. S. (2019, May). Heterogeneous graph attention network. In The world wide web conference (pp. 2022-2032).
[15] Wu, J., Wang, X., Feng, F., He, X., Chen, L., Lian, J., & Xie, X. (2021, July). Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (pp. 726-735).
[16] Lin, Z., Tian, C., Hou, Y., & Zhao, W. X. (2022, April). Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In Proceedings of the ACM web conference 2022 (pp. 2320-2329).
[17] Xia, X., Yin, H., Yu, J., Wang, Q., Cui, L., & Zhang, X. (2021, May). Self-supervised hypergraph convolutional networks for session-based recommendation. In Proceedings of the AAAI conference on artificial intelligence (Vol. 35, No. 5, pp. 4503-4511).
[18] Mudigere, D., Hao, Y., Huang, J., Jia, Z., Tulloch, A., Sridharan, S., ... & Rao, V. (2022, June). Software-hardware co-design for fast and scalable training of deep learning recommendation models. In Proceedings of the 49th Annual International Symposium on Computer Architecture (pp. 993-1011).
[19] Zheng, L., Li, Z., Zhang, H., Zhuang, Y., Chen, Z., Huang, Y., ... & Stoica, I. (2022). Alpa: Automating inter-and {Intra-Operator} parallelism for distributed deep learning. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) (pp. 559-578).
[20] Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., ... & Stoica, I. (2023, October). Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles (pp. 611-626).
[21] Qiu, L. (2025). Reinforcement Learning Approaches for Intelligent Control of Smart Building Energy Systems with Real-Time Adaptation to Occupant Behavior and Weather Conditions. Journal of Computing and Electronic Information Management, 18(2), 32-37.
[22] Wang, Z., Yang, J. S., Shang, W., & Ding, J. (2026). FairPromote: Explainable and Fairness-Aware Talent Promotion Prediction via Adversarial Debiasing and SHAP-based Interpretation. IEEE Access.
[23] Ding, J., Shen, Z., & Liu, W. (2026). Game-Theoretic Cost-Sensitive Adversarial Training for Robust Cloud Intrusion Detection Against GAN-Based Evasion Attacks. Applied Sciences, 16(8), 3944.
[24] Teng, D., Rhee, M., Qin, Y., Zi, B., & Liu, W. (2026). SW-SpeedDLM: Sliding-Window Speculative Decoding for Diffusion Language Models under Long-Context Constraints. Mathematics
[25] Zhang, F., Guo, Z., Ding, J., Yang, J., & Liu, W. (2026). Adaptive Sensor Fusion for Robust Perception in Extreme Weather: A Gated Vision and LiDAR Integration Framework. Sensors
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Daniel Brooks, Rachel Nguyen
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is licensed under the Creative Commons Attribution 4.0 International License. This license permits use, sharing, and adaptation of the article for non-commercial purposes, provided the original authors and source are properly credited.
Alternatively, you may choose other Creative Commons licenses based on your preferences:
-
CC Attribution-NonCommercial-NoDerivatives 4.0: This license allows others to download the works and share them with others as long as they credit the author, but they can’t change them in any way or use them commercially.
-
CC Attribution-NonCommercial 4.0: This license allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author, but they cannot use the material for commercial purposes.
-
CC Attribution-NonCommercial-ShareAlike 4.0: This allows others to remix, tweak, and build upon the work non-commercially, as long as they credit the author and license their new creations under identical terms.
These terms ensure that your article is freely accessible and usable, while respecting the rights of the original authors.
