On Approximate 8-bit Floating-Point Operations Using Integer Operations | Theodor Lindberg, Oscar Gustafsson | ResearchPod

Transcript: On Approximate 8-bit Floating-Point Operations Using Integer Operations

Alex: Welcome to another episode of ResearchPod.

Sam: Today we're looking at a paper about a specific challenge in chip design — how to do a particular kind of math on tiny, low-power computer chips without the expensive hardware that math normally requires.

Alex: That sounds like it has real practical stakes. What's the math problem, exactly?

Sam: So, computers handle numbers in different ways. For rough counting — like how many items are in a list — they use what's called integer math, which is simple and cheap. But for precise calculations involving decimals, like the kind needed in AI, they use something called floating-point math. Think of it like scientific notation in school: instead of writing 0.000045, you write 4.5 times ten to the negative five. It's a compact, flexible way to represent very large or very small numbers. That flexibility is powerful, but it comes at a cost.

Alex: What kind of cost?

Sam: A hardware cost. To do floating-point math properly, a chip needs a dedicated circuit — a specialized unit built just for that purpose. It has to align decimal points, multiply parts separately, and reformat the result. That's a lot of steps, and it takes up physical space on the chip and draws significant power. For a large processor in a laptop or server, that's fine. But for a tiny sensor that runs on a small battery — the kind you'd find in a smart home device or an industrial monitor — it's often simply not an option.

Alex: So these small devices are stuck doing simpler math?

Sam: Historically, yes. And that's a problem as we try to run AI models on those devices. AI calculations are almost entirely floating-point operations. So this paper asks: what if we could fake the expensive floating-point math using only the cheap integer tools these small chips already have?

Alex: How do you fake it convincingly enough that the results still hold up?

Sam: The key insight comes from a property of logarithms. Here's the idea: if you want to multiply two numbers, you can instead convert them to logarithms, add those logarithms together, and then convert back. Addition is far simpler for a chip than multiplication. There's a classic tool called a slide rule that worked exactly this way — engineers used it for decades before electronic calculators existed. The authors use a similar trick, called Mitchell's approximation, to treat floating-point numbers as if they were already in logarithmic form. That turns multiplication and division into addition and subtraction.

Alex: That's a clever shortcut. But shortcuts usually introduce errors. How do they keep the results accurate?

That's the second piece of the puzzle. The approximation on its own isn't quite precise enough — it gets close, but not close enough to meet the reliability standards that computers are expected to follow. So the authors add a small correction at the very end of the calculation. It's a single, simple logic element called a carry-in — think of it as a tiny nudge that pushes the result from "close enough" to "correct." For most cases, that nudge is sufficient to meet the standard.

On Approximate 8-bit Floating-Point Operations Using Integer Operations

Theodor Lindberg, Oscar Gustafsson

Jun 26, 2024·5:56·arxiv·Original Paper

FP8 (Eight-bit floating-point)Integer Arithmetic ApproximationLogarithmic Number System (LNS)Rounding ModesError Compensation

Listen

6 min

Summary

Research Question

As FP8 formats (such as E5M2 and E4M3) gain traction in machine learning and edge computing, the lack of native hardware support on many microcontrollers presents a bottleneck. This paper investigates whether approximate FP8 arithmetic can be efficiently implemented using standard integer operations, potentially enabling high-performance floating-point computation on hardware lacking dedicated floating-point units (FPUs).

Approach

The authors utilize Mitchell’s approximation to map floating-point numbers into the logarithmic domain, where multiplication and division simplify to integer addition and subtraction. To address the precision loss inherent in this approximation, they derive specific carry-in terms—boolean expressions based on the mantissa bits of the operands—to compensate for rounding errors. They evaluate these methods across seven different rounding modes, including IEEE 754-compliant modes and faithful rounding, for both E5M2 and E4M3 formats.

Main Findings

The study shows that for most operations, correctly rounded results can be achieved by adding a conditional carry-in term to the integer operation. For the E5M2 format, most rounding modes are supported directly or via simple carry-in logic. For E4M3, while the logic is more complex, it remains highly efficient for FPGA implementation because the boolean expressions fit within single Look-Up Tables (LUTs). Hardware synthesis results indicate that this integer-based approach offers significant area and speed improvements over traditional floating-point multipliers, especially on FPGA platforms.

Why It Matters

This work provides a practical pathway for deploying energy-efficient, low-latency AI models on resource-constrained edge devices. By bypassing the need for complex, power-hungry FPUs and leveraging existing integer-based SIMD instructions or FPGA logic, developers can implement FP8 arithmetic with minimal overhead, facilitating the integration of advanced neural networks into IoT and 6G applications.

Key Terms and Definitions

FP8 — An 8-bit floating-point format, such as E5M2 or E4M3, used primarily in error-resilient applications like machine learning.
Mitchell’s Approximation — A method for approximating floating-point multiplication and division by converting numbers into the logarithmic domain using simple integer arithmetic.
Faithful Rounding — A rounding property where the result is guaranteed to be one of the two floating-point numbers surrounding the exact mathematical result.
Carry-in — A boolean-derived value added to an integer operation to compensate for the approximation error and achieve a specific rounding mode.
E5M2/E4M3 — Specific FP8 formats where the numbers indicate the bit-width allocation for the exponent and mantissa fields, respectively.

Abstract

In this work, approximate eight-bit floating-point operations performed using simple integer operations is discussed. For two-bit mantissa formats, faithful rounding can always be obtained for the considered operations. For all operations, correctly rounded results can be obtained for different rounding modes, either directly or by adding a conditional carry in. For three-bit mantissa formats, faithful rounding can be sometimes be obtained directly, while for other operations a conditional carry in must be added. Correctly rounded results can be obtained for most operations and rounding modes using slightly more complicated expressions for the carry in. Hardware implementation results for multiplication using both standard cell and FPGA technology are presented illustrating the potential benefit of integer computation. Especially for FPGA, significant resource savings are obtained.

On Approximate 8-bit Floating-Point Operations Using Integer Operations: Research Paper Podcast