NIFA: NonlinearIMC enhanced FPGA for efficient ML inference

Modern FPGA architectures have begun integrating analog in-memory computing (IMC) to accelerate deep learning (DL) inference. While these IMC blocks excel at static-weight vector-matrix multiplication (VMM), they struggle with Transformer-based models that require frequent nonlinear operations and dynamic matrix-matrix multiplication (DIMM). Furthermore, traditional IMC designs rely on power-hungry analog-to-digital converters (ADCs) that consume the majority of the block's area. This paper asks: can a novel FPGA architecture using ADC-free, nonlinear-capable IMC blocks improve inference efficiency for both CNNs and Transformers?

The authors propose NIFA, an FPGA architecture that replaces conventional ADC-based IMC blocks with Analog Content-Addressable Memory (ACAM) units. These ACAMs perform nonlinear activation functions (like ReLU or tanh) natively within the IMC block. To optimize this, the authors conduct a two-round FPGA-aware design-space exploration (DSE): first, they determine optimal crossbar dimensions to balance area and performance; second, they sweep the area budget to find the Pareto-optimal trade-off between DL throughput and general-purpose FPGA flexibility. Additionally, they develop a mapping strategy that performs Attention computation in the log domain, allowing the ACAMs to accelerate DIMM operations that would otherwise fall back to slow, energy-intensive soft logic.

NIFA significantly outperforms existing IMC-enhanced FPGA designs. For CNN workloads, it achieves up to 40x higher energy efficiency and 4.1x higher area efficiency. For Transformer-based workloads, which are typically bottlenecked by nonlinear operations and DIMM, the architecture provides 1.9x higher energy efficiency and 2.5x higher area efficiency. The integration of ACAMs allows the system to sustain these gains even as input sequence lengths increase, effectively addressing the primary limitations of prior IMC-FPGA integration strategies.