Author
At NSDI ’22, Waleed presented our RedN paper that shows a suprising result, namely that Remote Direct Memory Access (RDMA), as implemented in widely deployed RDMA Network Interface Cards, is Turing Complete. We leverage this finding to reduce the tail latency of services running on busy servers by 35x! Full Abstract is below, and the video is on the USENIX Youtube channel. This is joint work with Waleed Reda, Marco Canini (KAUST), Dejan Kostić, and Simon Peter (UW).
It is becoming increasingly popular for distributed systems to exploit offload to reduce load on the CPU. Remote Direct Memory Access (RDMA) offload, in particular, has become popular. However, RDMA still requires CPU intervention for complex offloads that go beyond simple remote memory access. As such, the offload potential is limited and RDMA-based systems usually have to work around such limitations.
We present RedN, a principled, practical approach to implementing complex RDMA offloads, without requiring any hardware modifications. Using self-modifying RDMA chains, we lift the existing RDMA verbs interface to a Turing complete set of programming abstractions. We explore what is possible in terms of offload complexity and performance with a commodity RDMA NIC. We show how to integrate these RDMA chains into applications, such as the Memcached key-value store, allowing us to offload complex tasks such as key lookups. RedN can reduce the latency of key-value get operations by up to 2.6× compared to state-of-the-art KV designs that use one-sided RDMA primitives (e.g., FaRM-KV), as well as traditional RPC-over-RDMA approaches. Moreover, compared to these baselines, RedN provides performance isolation and, in the presence of contention, can reduce latency by up to 35× while providing applications with failure resiliency to OS and process crashes.
Author
We are happy to announce that Amir Roozbeh successfully defended his PhD thesis! Prof. Gerald Q. Maguire Jr. has as usual done a stellar job as a co-advisor. Prof. Jonathan M. Smith was a superb opponent at the defense seminar. Amir’s thesis is available online:
Realizing Next-Generation Data Centers via Software-Defined “Hardware” Infrastructures and Resource Disaggregation: Exploiting your cache
This is the second year in which we couldn’t take the obligatory hallway shot, so here is the fake gift giving over Zoom!
Author
Two project proposals led by Marco Chiesa and Dejan Kostić have recently been funded by Vetenskaprådet (VR, Swedish Research Council in English).
The first project is a Starting Grant with a single PI (Marco Chiesa), titled “ResoNet: Resilient Optimized Network Synthesis”, and funded with 4M SEK and running between 2022 and 2025. The project aims at developing new network synthesis methods that guarantee performance and robustness requirements.
The second project is a Project Grant with Dejan Kostić as PI, titled “Scalable Federated Learning”, and funded with 3.8M SEK. This project is a collaboration with three more co-PIs: Magnus Boman (KTH), Marco Chiesa (KTH), and Sabine Koch (KI). The project will allow our group to explore a new research direction and, more specifically, we aim to develop a highly scalable, flexible, extensible, distributed federated machine learning approach that can directly benefit public health and wellness.
See a list of all funded VR projects here.
Author
Our upcoming NSDI 2022 paper Packet Order Matters shows a surprising result: deliberately delaying packets can improve the performance of backend servers by up to about a factor of 2 (e.g., those used for Network Function Virtualization)! This applies to both throughput and latency (including the time spent in our Reframer). We show three different scenarios in which Reframer can be deployed. Source code is available here.
This is joint work with:
Hamid Ghasemirahni, Tom Barbette, Georgios P. Katsikas, Alireza Farshin, Amir Roozbeh, Massimo Girondi, Marco Chiesa, Gerald Q. Maguire Jr., and Dejan Kostić.
Full abstract is below:
Data centers increasingly deploy commodity servers with high-speed network interfaces to enable low-latency communication. However, achieving low latency at high data rates crucially depends on how the incoming traffic interacts with the system’s caches. When packets that need to be processed in the same way are consecutive, i.e., exhibit high temporal and spatial locality, caches deliver great benefits.
In this paper, we systematically study the impact of temporal and spatial traffic locality on the performance of commodity servers equipped with high-speed network interfaces. Our results show that (i) the performance of a variety of widely deployed applications degrade substantially with even the slightest lack of traffic locality, and (ii) a traffic trace from our organization reveals poor traffic locality as networking protocols, drivers, and the underlying switching/routing fabric spread packets out in time (reducing locality). To address these issues, we built Reframer, a software solution that deliberately delays packets and reorders them to increase traffic locality. Despite introducing μs-scale delays of some packets, we show that Reframer increases the throughput of a network service chain by up to 84% and reduces the flow completion time of a web server by 11% while improving its throughput by 20%.
Author
We are very happy to announce that our LineFS paper was among the three papers that won the Best Paper Award at SOSP 2021!
LineFS builds upon our previous work on Assise [OSDI ’20] by offloading CPU-intensive tasks to a SmartNIC (BlueField-1 in our case) for about 80% performance improvement across the board.
Jongyul’s presentation is already available:
This is joint work with
Jongyul Kim (KAIST), Insu Jang (University of Michigan), Waleed Reda (KTH Royal Institute of Technology / Université catholique de Louvain), Jaeseong Im (KAIST), Marco Canini (KAUST), Dejan Kostić (KTH Royal Institute of Technology), Youngjin Kwon (KAIST), Simon Peter (The University of Texas at Austin), and Emmett Witchel (The University of Texas at Austin / Katana Graph).
Full abstract is as follows:
In multi-tenant systems, the CPU overhead of distributed file systems (DFSes) is increasingly a burden to application performance. CPU and memory interference cause degraded and unstable application and storage performance, in particular for operation latency. Recent client-local DFSes for persistent memory (PM) accelerate this trend. DFS offload to SmartNICs is a promising solution to these problems, but it is challenging to fit the complex demands of a DFS onto simple SmartNIC processors located across PCIe.
We present LineFS, a SmartNIC-offloaded, high-performance DFS with support for client-local PM. To fully leverage the SmartNIC architecture, we decompose DFS operations into execution stages that can be offloaded to a parallel data-path execution pipeline on the SmartNIC. LineFS offloads CPU-intensive DFS tasks, like replication, compression, data publication, index and consistency management to a SmartNIC.
We implement LineFS on the Mellanox BlueField SmartNIC and compare it to Assise, a state-of-the-art PM DFS. LineFS improves latency in LevelDB up to 80% and throughput in Filebench up to 79%, while providing extended DFS availability during host system failures.
