Agamemnon Kyriazis, Panagiotis Miliadis, Dimitris Theodoropoulos, Nectarios Koziris, Dionisios Pnevmatikatos
6 min
Abstract
As modern Coarse Grain Reconfigurable Arrays (CGRAs) grow in size, efficient utilization of the available fabric by a single application becomes increasingly difficult. Existing CGRA mappers either fail to utilize the available fabric or rely on rigid static code transformations with limited adaptability. Multi-tenant CGRAs have emerged as a promising solution to increase hardware utilization, but current attempts fail to address key challenges such as fabric fragmentation and live migration. To address this gap, we present Mestra, an end-to-end system for CGRA multi-tenancy that supports dynamic scheduling and resource allocation in a shared environment. Mestra addresses fabric fragmentation caused by kernels completing out of order by supporting both stateless and stateful live kernel migration as a de-fragmentation mechanism. We assess our solution on an Alveo-U280 data-center-grade FPGA card, reporting area, frequency, and power. Performance is evaluated using routines from the PolyBench benchmark suite and kernels derived from common machine learning operators. Results show that spatial sharing of the available fabric across multiple users improves workload makespan by up to 70.48%, while live kernel migration reduces tail latency on fragmented layouts by up to 29.60%. The custom tightly coupled controller and read-back paths required for virtualization and stateful migration introduce a LUT cost of 0.13% per region. Our evaluation reveals that multi-tenancy is important for efficient CGRA utilization, and live kernel migration can further improve performance by recovering fragmented space with minimal hardware cost.
Alex: So it's copying specific machine states like positions and half-done calculations. How does the system decide when to trigger a move?
Sam: The hypervisor checks a map of the grid when a new kernel can't fit despite enough free space. It simulates shuffling on a virtual copy first, using a greedy method that pulls running kernels toward one corner to pack them tighter. If that works, it applies the changes: halting kernels, snapshotting for stateful moves, reconfiguring regions, restoring state, and resuming. This reactive approach cuts tail latency—the wait for slowest jobs—by nearly 30% on messy layouts.
Alex: So de-fragmentation only kicks in when needed, testing virtually first. Unlike earlier systems, this migrates live kernels.
Sam: Precisely. Prior work handled partitioning but skipped live moves. The tiled setup—with load-store and compute elements on a 2D mesh, plus per-region controllers—makes this feasible on real hardware.
Alex: That seems like a meaningful step for packing these grids efficiently.
Alex: With migrations in place, how do they measure if the setup pays off?
Sam: They tested on real hardware emulating a 4 by 4 grid of processor blocks, using workloads from standard benchmarks like PolyBench and machine learning tasks in batches of 64 jobs. Makespan measures total time from first job arriving until the last finishes. Compared to single-job runs, the new system cuts average wait times to under a tenth of before.
Alex: Run times go up from shared memory access, but overall jobs finish much faster since fewer sit idle.
Sam: Yes. Execution time increases about three times due to congestion, but wait time drops over tenfold. Turnaround—from arrival to done—improves substantially.
Alex: Between stateless and stateful, when does each make sense?
Sam: Stateless suits kernels far from finishing; it restarts but discards done work. They check progress—estimated by current loop versus total—and skip if under 10% left. Stateful snapshots exact states, adding about 30% more time than basic setup but preserves everything for non-restartable jobs.
Alex: So the threshold keeps stateless cheap for early shuffles, while stateful ensures no lost effort. That balance seems key.
Sam: Exactly. Their tests show reactive compaction yields a clear tail latency drop, validating the low hardware cost. Overall, the evidence points to reliable gains for data center workloads.
Alex: Huh. No more jobs stuck on scattered holes. But with a simple greedy packer, aren't there times it overdoes shuffling?
Sam: Fair point. The greedy method can rack up overhead if many move at once. It lacks fancier rules, so costs rise under extreme scatter. The study notes this as a limitation, trading simplicity for proof-of-concept gains.
Alex: Makes sense. For data centers, this lets grids handle multi-tenant loads reliably, reclaiming idle space without big hardware hits.
Sam: Precisely. The evidence suggests a solid path forward for efficient sharing of these specialized chips.
Alex: Well said. Thanks for breaking it down, Sam. Thanks for listening to ResearchPod.