Abstract
Abstract
To manufacture a large CPU cost-effectively, the industry has begun exploiting emerging packaging technologies that integrate multiple chiplets—each comprising a subset of cores and/or memory and I/O subsystems—into a single package. However, such a CPU experiences longer memory access latency with more pronounced variance, especially when its cores in one chiplet access LLC slices or DRAM controllers in other chiplets. This creates unique challenges in μs-scale networking, which is highly sensitive to memory access latency. In this work, we start by proposing exploiting a little-known mode, known as Sub-NUMA Clustering (SNC), in the latest chiplet-based CPUs. As it restricts receiving and processing packets to a particular chiplet unless explicitly specified otherwise, it offers shorter memory access latency and, consequently, lower networking latency than the default mode (non-SNC). Nonetheless, when receiving long bursts of packets, SNC incurs higher networking latency than non-SNC, as it provides less LLC capacity for CPU cores processing the packets, making Direct Cache Access (DCA)—a commonly used CPU feature to reduce memory access latency for packet processing—ineffective. To address this drawback, we propose, a tiered network buffer architecture consisting of an enhanced NIC and networking stack, which opportunistically uses LLC slices in other chiplets for DCA only when receiving long bursts of packets. On average, reduces the mean (tail) latency by 25% (18%) and 28% (22%), compared to SNC and non-SNC, respectively, across diverse network applications and traces.
AI Summary
AI-Generated Summary (Experimental)
This summary was generated using automated tools and was not authored or reviewed by the article's author(s). It is provided to support discovery, help readers assess relevance, and assist readers from adjacent research areas in understanding the work. It is intended to complement the author-supplied abstract, which remains the primary summary of the paper. The full article remains the authoritative version of record. Click here to learn more.
Click here to comment on the accuracy, clarity, and usefulness of this summary. Doing so will help inform refinements and future regenerated versions.
To view this AI-generated plain language summary, you must have Premium access.
Formats available
You can view the full content in the following formats:
References
[1]
Saksham Agarwal, Arvind Krishnamurthy, and Rachit Agarwal. 2023. Host Congestion Control. In Proceedings of the ACM SIGCOMM 2023 Conference (New York, NY, USA) (ACM SIGCOMM '23). Association for Computing Machinery, New York, NY, USA, 275–287. https://doi.org/10.1145/3603269.3604878
[2]
Mohammad Alian, Siddharth Agarwal, Jongmin Shin, Neel Patel, Yifan Yuan, Daehoon Kim, Ren Wang, and Nam Sung Kim. 2023. IDIO: Network-Driven, Inbound Network Data Orchestration on Server Processors. In Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture (Chicago, Illinois, USA) (MICRO '22). IEEE Press, 480–493. https://doi.org/10.1109/MICRO56248.2022.00042
[3]
Mohammad Alizadeh, Shuang Yang, Milad Sharif, Sachin Katti, Nick McKeown, Balaji Prabhakar, and Scott Shenker. 2013. pFabric: minimal near-optimal datacenter transport. In Proceedings of the ACM SIGCOMM 2013 Conference on SIGCOMM (Hong Kong, China) (SIGCOMM '13). Association for Computing Machinery, New York, NY, USA, 435–446. https://doi.org/10.1145/2486001.2486031
[4]
AMD. Accessed in 2025. Smart Data Cache Injection (SDCI) White Paper. https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/white-papers/58725.pdf.
[5]
Tom Barbette, Georgios P. Katsikas, Gerald Q. Maguire, and Dejan Kosti'c. 2019. RSS: load and state-aware receive side scaling. In Proceedings of the 15th International Conference on Emerging Networking Experiments And Technologies (Orlando, Florida) (CoNEXT '19). Association for Computing Machinery, New York, NY, USA, 318–333. https://doi.org/10.1145/3359989.3365412
[6]
Theophilus Benson, Ashok Anand, Aditya Akella, and Ming Zhang. 2010. Understanding data center traffic characteristics. SIGCOMM Comput. Commun. Rev., Vol. 40, 1 (Jan. 2010), 92–99. https://doi.org/10.1145/1672308.1672325
[7]
Anat Bremler-Barr, Yotam Harchol, and David Hay. 2016. OpenBox: A Software-Defined Framework for Developing, Deploying, and Managing Network Functions. In Proceedings of the 2016 ACM SIGCOMM Conference (Florianopolis, Brazil) (SIGCOMM '16). Association for Computing Machinery, New York, NY, USA, 511–524. https://doi.org/10.1145/2934872.2934875
[8]
Alireza Farshin, Amir Roozbeh, Gerald Q. Maguire, and Dejan Kosti'c. 2019. Make the Most out of Last Level Cache in Intel Processors. In Proceedings of the Fourteenth EuroSys Conference 2019 (Dresden, Germany) (EuroSys '19). Association for Computing Machinery, New York, NY, USA, Article 8, 17 pages. https://doi.org/10.1145/3302424.3303977
[9]
Alireza Farshin, Amir Roozbeh, Gerald Q Maguire Jr, and Dejan Kosti?. 2020. Reexamining direct cache access to optimize intensive applications for multi-hundred-gigabit networks. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, Virtual, 673-689. https://doi.org/10.5555/3489146.3489192
[10]
Alex Forencich, Alex C. Snoeren, George Porter, and George Papen. 2020. Corundum: An Open-Source 100-Gbps Nic. In 2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 38-46. https://doi.org/10.1109/FCCM48280.2020.00015
[11]
Peter X. Gao, Akshay Narayan, Gautam Kumar, Rachit Agarwal, Sylvia Ratnasamy, and Scott Shenker. 2015. pHost: distributed near-optimal datacenter transport over commodity network fabric. In Proceedings of the 11th ACM Conference on Emerging Networking Experiments and Technologies (Heidelberg, Germany) (CoNEXT '15). Association for Computing Machinery, New York, NY, USA, Article 1, 12 pages. https://doi.org/10.1145/2716281.2836086
[12]
Hossein Golestani, Amirhossein Mirhosseini, and Thomas F. Wenisch. 2019. Software Data Planes: You Can't Always Spin to Win. In Proceedings of the ACM Symposium on Cloud Computing (Santa Cruz, CA, USA) (SoCC '19). Association for Computing Machinery, New York, NY, USA, 337–350. https://doi.org/10.1145/3357223.3362737
[13]
Pankaj Gupta, Steven Lin, and Nick McKeown. 1998. Routing lookups in hardware at memory access speeds. In Proceedings. IEEE INFOCOM'98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No. 98, Vol. 3. IEEE, IEEE, San Francisco, CA, USA, 1240-1247. https://doi.org/10.1109/INFCOM.1998.662938
[14]
Andrew Herdrich, Edwin Verplanke, Priya Autee, Ramesh Illikkal, Chris Gianos, Ronak Singhal, and Ravi Iyer. 2016. Cache QoS: From concept to reality in the Intel® Xeon® processor E5-2600 v3 product family. In 2016 IEEE International Symposium on High Performance Computer Architecture (HPCA). 657-668. https://doi.org/10.1109/HPCA.2016.7446102
[15]
Ram Huggahalli, Ravi Iyer, and Scott Tetrick. 2005. Direct Cache Access for High Bandwidth Network I/O. In Proceedings of the 32nd Annual International Symposium on Computer Architecture (ISCA '05). IEEE Computer Society, USA, 50–59. https://doi.org/10.1109/ISCA.2005.23
[16]
Intel. 2009. An Introduction to the Intel® QuickPath Interconnect -- intel.com. https://www.intel.com/content/www/us/en/io/quickpath-technology/quick-path-interconnect-introduction-paper.html
[17]
Intel. Accessed in 2025. 4th Gen Intel Xeon Processor Scalable Family, Sapphire Rapids. https://www.intel.com/content/www/us/en/developer/articles/technical/fourth-generation-xeon-scalable-family-overview.html
[18]
Intel. Accessed in 2025 a. Compute Express Link Optimize Memory Usage in Multithreaded Data Plane Development Kit (DPDK) Applications. https://www.intel.com/content/www/us/en/developer/articles/technical/optimize-memory-usage-in-multi-threaded-data-plane-development-kit-dpdk-applications.html.
[19]
Intel. Accessed in 2025, b. Data Plane Development Kit (DPDK). https://www.dpdk.org.
[20]
Intel. Accessed in 2025. Intel® Data Direct I/O Technology (Intel® DDIO): Technology Brief. https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology-brief.html.
[21]
Srikanth Kandula, Sudipta Sengupta, Albert Greenberg, Parveen Patel, and Ronnie Chaiken. 2009. The nature of data center traffic: measurements & analysis. In Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement (Chicago, Illinois, USA) (IMC '09). Association for Computing Machinery, New York, NY, USA, 202–208. https://doi.org/10.1145/1644893.1644918
[22]
Svilen Kanev, Juan Pablo Darago, Kim Hazelwood, Parthasarathy Ranganathan, Tipp Moseley, Gu-Yeon Wei, and David Brooks. 2015. Profiling a warehouse-scale computer. In Proceedings of the 42nd Annual International Symposium on Computer Architecture (Portland, Oregon) (ISCA '15). Association for Computing Machinery, New York, NY, USA, 158–169. https://doi.org/10.1145/2749469.2750392
[23]
Georgios P. Katsikas, Tom Barbette, Dejan Kosti?, Rebecca Steinert, and Gerald Q. Maguire Jr., 2018. Metron: NFV Service Chains at the True Speed of the Underlying Hardware. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton, WA, 171-186. https://doi.org/10.5555/3307441.3307457
[24]
Hyeontaek Lim, Bin Fan, David G. Andersen, and Michael Kaminsky. 2011. SILT: a memory-efficient, high-performance key-value store. In Proceedings of the Twenty-Third ACM Symposium on Operating Systems Principles (Cascais, Portugal) (SOSP '11). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/2043556.2043558
[25]
Pejman Lotfi-Kamran, Boris Grot, Michael Ferdman, Stavros Volos, Onur Kocberber, Javier Picorel, Almutaz Adileh, Djordje Jevdjic, Sachin Idgunji, Emre Ozer, and Babak Falsafi. 2012. Scale-out processors. In Proceedings of the 39th Annual International Symposium on Computer Architecture (Portland, Oregon) (ISCA '12). IEEE Computer Society, USA, 500–511.
[26]
ARM Ltd. Accessed in 2025. Arm DynamIQ Shared Unit Technical Reference Manual. https://developer.arm.com/documentation/100453/latest/.
[27]
Clémentine Maurice, Nicolas Scouarnec, Christoph Neumann, Olivier Heen, and Aurélien Francillon. 2015. Reverse Engineering Intel Last-Level Cache Complex Addressing Using Performance Counters. In Proceedings of the 18th International Symposium on Research in Attacks, Intrusions, and Defenses - Volume 9404 (Kyoto, Japan) (RAID 2015). Springer-Verlag, Berlin, Heidelberg, 48–65. https://doi.org/10.1007/978-3-319-26362-5_3
[28]
Behnam Montazeri, Yilong Li, Mohammad Alizadeh, and John Ousterhout. 2018. Homa: a receiver-driven low-latency transport protocol using network priorities. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (Budapest, Hungary) (SIGCOMM '18). Association for Computing Machinery, New York, NY, USA, 221–235. https://doi.org/10.1145/3230543.3230564
[29]
Samuel Naffziger, Kevin Lepak, Milam Paraschou, and Mahesh Subramony. 2020. 2.2 AMD chiplet architecture for high-performance server and desktop products. In 2020 IEEE International Solid-State Circuits Conference-(ISSCC). IEEE, IEEE, SAN FRANCISCO MARRIOTT MARQUIS HOTEL, 44-45. https://doi.org/10.1109/ISSCC19947.2020.9063103
[30]
Nevine Nassif, Ashley O. Munch, Carleton L. Molnar, Gerald Pasdast, Sitaraman V. Lyer, Zibing Yang, Oscar Mendoza, Mark Huddart, Srikrishnan Venkataraman, Sireesha Kandula, Rafi Marom, Alexandra M. Kern, Bill Bowhill, David R. Mulvihill, Srikanth Nimmagadda, Varma Kalidindi, Jonathan Krause, Mohammad M. Haq, Roopali Sharma, and Kevin Duda. 2022. Sapphire Rapids: The Next-Generation Intel Xeon Scalable Processor. In 2022 IEEE International Solid- State Circuits Conference (ISSCC), Vol. 65. IEEE, Online due to COVID-19, 44-46. https://doi.org/10.1109/ISSCC42614.2022.9731107
[31]
DPDK Project. Accessed in 2025. DPDK Programmer's Guide - Service Cores. https://doc.dpdk.org/guides/prog_guide/service_cores.html
[32]
Ravindra P Saraf, Rahul Pal, and Ashok Jagannathan. U.S. Patent 8862828B2 Oct. 2014. Sub-NUMA clustering.
[33]
Sarabjeet Singh and Manu Awasthi. 2019. Memory Centric Characterization and Analysis of SPEC CPU2017 Suite. In Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (Mumbai, India) (ICPE '19). Association for Computing Machinery, New York, NY, USA, 285–292. https://doi.org/10.1145/3297663.3310311
[34]
Igor Smolyar, Alex Markuze, Boris Pismenny, Haggai Eran, Gerd Zellweger, Austin Bolen, Liran Liss, Adam Morrison, and Dan Tsafrir. 2020. IOctopus: Outsmarting Nonuniform DMA. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Lausanne, Switzerland) (ASPLOS '20). Association for Computing Machinery, New York, NY, USA, 101–115. https://doi.org/10.1145/3373376.3378509
[35]
Wei Su, Abhishek Dhanotia, Carlos Torres, Jayneel Gandhi, Neha Gholkar, Shobhit Kanaujia, Maxim Naumov, Kalyan Subramanian, Valentin Andrei, Yifan Yuan, and Chunqiang Tang. 2025. DCPerf: An Open-Source, Battle-Tested Performance Benchmark Suite for Datacenter Workloads. In Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA '25). Association for Computing Machinery, New York, NY, USA, 1717–1730. https://doi.org/10.1145/3695053.3731411
[36]
The OpenSSL Project. Accessed in 2025. OpenSSL: The Open Source toolkit for SSL/TLS. www.openssl.org.
[37]
Amin Tootoonchian, Aurojit Panda, Chang Lan, Melvin Walls, Katerina Argyraki, Sylvia Ratnasamy, and Scott Shenker. 2018. : Enabling in network function virtualization. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18). USENIX Association, Renton WA USA, 283-297. https://doi.org/10.5555/3307441.3307466
[38]
Marina Vemmou, Albert Cho, and Alexandros Daglis. 2023. Patching up Network Data Leaks with Sweeper. In Proceedings of the 55th Annual IEEE/ACM International Symposium on Microarchitecture (Chicago, Illinois, USA) (MICRO '22). IEEE Press, 464–479. https://doi.org/10.1109/MICRO56248.2022.00041
[39]
Krishnaswamy Viswanathan. 2024. Intel® Memory Latency Checker v3.11. https://www.intel.com/content/www/us/en/developer/articles/tool/intelr-memory-latency-checker.html
[40]
Midhul Vuppalapati, Saksham Agarwal, Henry Schuh, Baris Kasikci, Arvind Krishnamurthy, and Rachit Agarwal. 2024. Understanding the Host Network. In Proceedings of the ACM SIGCOMM 2024 Conference (Sydney, NSW, Australia) (ACM SIGCOMM '24). Association for Computing Machinery, New York, NY, USA, 581–594. https://doi.org/10.1145/3651890.3672271
[41]
Minhu Wang, Mingwei Xu, and Jianping Wu. 2022. Understanding I/O Direct Cache Access Performance for End Host Networking. Proc. ACM Meas. Anal. Comput. Syst., Vol. 6, 1, Article 22 (Feb. 2022), 37 pages. https://doi.org/10.1145/3508042
[42]
Mengjia Yan, Read Sprabery, Bhargava Gopireddy, Christopher Fletcher, Roy Campbell, and Josep Torrellas. 2019. Attack directories, not caches: Side channel attacks in a non-inclusive world. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE, IEEE, San Francisco, CA, USA, 888-904. https://doi.org/10.1109/SP.2019.00004
[43]
Qiao Zhang, Vincent Liu, Hongyi Zeng, and Arvind Krishnamurthy. 2017. High-resolution measurement of data center microbursts. In Proceedings of the 2017 Internet Measurement Conference (London, United Kingdom) (IMC '17). Association for Computing Machinery, New York, NY, USA, 78–85. https://doi.org/10.1145/3131365.3131375