---
title: "Floating Point (IEEE 754)"
description: "Sign, exponent, mantissa visualized"
source: https://vizlearn.app/cs/architecture/floating-point
---

# Floating Point (IEEE 754)

A floating point number does not store the number you asked for. It stores the nearest value its format can represent, and it makes the substitution silently. The format itself is three fields — a sign bit, an exponent, and a fraction — packed into a fixed-width word, with normal values read back as ±1.f × 2ᵉ.

IEEE 754 fixes the widths and the rounding rule that picks which value you actually get.

The analogy worth carrying through this page is a ladder with uneven rungs. Floats are not a fine grid laid over the number line. They are a ladder whose rungs sit tightly packed near zero and drift apart as you climb, doubling their spacing every time the exponent goes up; only the subnormals nearest zero share one fixed spacing. Ask for an in-range number that is not on a rung and nothing complains; you get the nearest rung.

The widths are the only thing that changes between formats: 1 + 5 + 10 for the 16-bit binary16 this page animates, 1 + 8 + 23 for a 32-bit `float`, 1 + 11 + 52 for the `double` your language almost certainly uses by default. Wider fields put the rungs closer together. None of them changes the fact that you land on a rung.

## How Floating Point Works

Take the number apart the way scientific notation does. Every nonzero value can be written as a sign, a significand between 1 and 2, and a power of two: 0.1 is 1.6 × 2⁻⁴, and 4096 is 1.0 × 2¹². The three fields store exactly those three things, and the whole design falls out of squeezing them into a fixed number of bits.

The **sign** is one bit, and it does nothing else. Sign-magnitude is fine here precisely because floats are never added by a plain integer adder anyway.

The **exponent** field holds `e + bias`, not `e`. In binary16 the bias is 15, so an exponent of −4 is stored as 11. The offset is not decoration: it keeps the field unsigned and monotonically increasing, which preserves ordering for nonnegative, non-NaN encodings. Two positive finite floats compare correctly when you compare their raw bit patterns as unsigned integers, and hardware exploits that constantly. Store the exponent as a signed two's-complement number instead and that property dies immediately — every tiny value would sort above every huge one.

The **fraction** field holds the bits after the leading `1.`, and it does not store the 1. Every normalized value has one, so spending a bit on it would be spending a bit to say something you already know. Binary16 gets eleven bits of precision out of ten stored bits for free. The trick costs one thing: zero has no leading 1, so it cannot be written in this form at all, which is why the all-zero exponent field is reserved.

Now the part that bites. Ten fraction bits means a normal significand is one of 1024 evenly spaced values between 1 and 2, and multiplying by 2ᵉ stretches that spacing by 2ᵉ. So the distance between neighbouring normal floats — one **ulp**, one unit in the last place — is 2^(e − 10). Near 0.1 that gap is 2⁻¹⁴, about 0.000061. Near 4096 the same format has a gap of 4. Precision is relative, never absolute, and a number is only "close enough" relative to its own magnitude.

When your value falls between two rungs, IEEE 754's default rule is round-to-nearest, ties-to-even: take the closer rung, and on an exact tie take the one whose significand is even. Rounding to nearest keeps the error at or below half an ulp. The tie-break looks fussy until you sum a million values and notice that a rule which always broke ties upward would drift the total in one direction, while ties-to-even lets the errors cancel.

## Floating Point Step by Step

The default run stores `0.1` in **binary16** — 1 sign bit, 5 exponent bits, 10 fraction bits. Half precision is small enough to write out by hand, and it reproduces every problem the wider formats have.

1. **The word starts empty.** Sixteen slots, in three named fields. There is no decimal point anywhere in the hardware; the point's position has to come out of the exponent.
2. **Write the sign.** 0.1 is positive, so bit 15 is `0`. One bit spent, and it will not come up again.
3. **Find the exponent.** 0.1 = 1.6 × 2⁻⁴. Every nonzero value sits on or above exactly one power of two, and that power is e. All that is left to write is the 1.6.
4. **Write it in binary.** 1.6 is `1.1001100110011…` — the block `1001` repeats forever. This is the whole problem in one line. One tenth is a terminating decimal because 10 factors into 2 and 5; in base 2 the 5 has nowhere to go, so the expansion never closes. No finite field can hold this number, and no amount of precision will change that.
5. **Bias the exponent.** e = −4, bias = 15, so the exponent field holds −4 + 15 = 11 = `01011`.
6. **Copy the fraction.** The first ten bits after the point are `1001100110`, and they go into the fraction field. The bits after those ten — `011001…` — sit outside the rail with nowhere to go.
7. **Round.** The discarded tail is worth about 0.4 of one step, which is under half, so the stored bits stay as written. Nothing is added; the tail is simply gone.
8. **Read it back.** The bits now say (1 + 614/1024) × 2⁻⁴, which is **0.0999755859375**.
9. **Measure the ladder.** Rungs here are 0.000061 apart, and stay that far apart everywhere from 0.0625 up to 0.125 — one exponent's worth of values. Cross into the next exponent and the gap doubles to 0.00012, because it is 2^(e − 10) and nothing else. The animation draws both spans at the same width, and the second one fits half as many rungs. Precision is relative to the magnitude, never absolute.
10. **Measure the damage.** You asked for 0.1 and got 0.0999755859375, an error of about −2.4 × 10⁻⁵, or 0.4 of a step. Every decimal within half a step — everything from about 0.09995 to 0.10001 — collapses onto these exact same sixteen bits. Your 0.1 was never stored. A representative of it was.

On the canvas the significand workbench sits above the encoded word, split by a vertical rail: everything left of the rail is bits the format keeps, everything right of it is bits about to be discarded. The bottom row is the word that actually gets stored, and the ladder strip in the panel shows 0.1 sitting between two rungs rather than on one.

Two notes on the trace model. Encoding ends once the three fields are fixed; reading the value back out is the lesson interpreting those fields, not another encoding step. Round-to-nearest-even is the only lossy operation; every other transformation is exact bookkeeping.

## Why 0.1 + 0.2 Is Not 0.3

Switch to the **add** variant and the lesson stops being about storage and starts being about what storage does to arithmetic. The default is the famous one, `0.1 + 0.2`, and in binary16 it goes like this.

Both operands are already wrong before the addition starts: 0.1 is stored as 0.0999755859375 and 0.2 as 0.199951171875. Their exponents differ by one, so the smaller one shifts right a column to line the columns up — here that shift is clean and loses nothing. The two significands then go through a single integer addition, which produces `10.0110011001` × 2⁻³. That leading `10.` means the sum has outgrown the range a significand is allowed to live in, so it shifts right once more and pays an exponent, becoming `1.0011001100|1` × 2⁻².

That last bit, the one now sitting past the rail, is worth **exactly** half a step. This is a true tie, and ties go to the even significand, so the result keeps `…1100` rather than rounding up to `…1101`. The answer is 0.2998046875. Meanwhile binary16 stores 0.3 as 0.300048828125 — one rung higher. `0.1 + 0.2 == 0.3` is false, and the thing that decided it was a tie-break rule, not a bug.

Now load the **Same sum, binary32** scenario and watch the verdict flip. Identical code, identical decimals, 23 fraction bits instead of 10, and `0.1 + 0.2 == 0.3` comes out **true**. In binary64 — what Python, JavaScript, Java doubles, and C `double` all use by default — it is false again, which is why `0.1 + 0.2` famously prints `0.30000000000000004` in a Python REPL.

That flip is the lesson. There is no rule of the form "floats break on tenths." Two roundings happen in every addition — once when each operand was stored, once when the result is squeezed back into the format — and whether the two paths land on the same rung depends on precisely where those roundings fall. You cannot predict it by reading the decimals, so equality of computed floats needs a numerical contract rather than an assumption that algebraic equality survives rounding. The result is deterministic, not random.

The other scenarios push on the same machinery from different sides. **Absorbed: 4096 + 1** is the gap rule turning lethal: rungs at 4096 are 4 apart, so 1 is a quarter of a step, falls off the end during alignment, and `a + b == a`. **Overflow: 65504 + 100** takes two ordinary finite numbers past the largest value binary16 has and lands on infinity. IEEE 754 does raise its overflow and inexact flags there — it just raises them into a status register nobody reads, so in a non-trapping environment the next line can run on an infinity; language and library settings can report or trap it. **Subnormal: 0.00004** goes the other way: the exponent field bottoms out at 0, the implicit leading 1 is given up, and the value keeps shrinking toward zero while losing a bit of precision at every step.

## A Note on Simplification

This lesson animates binary16 and binary32 because sixteen and thirty-two columns fit on a screen; binary64's 52 fraction bits do not. The rules are identical at every width, and each scenario states the numbers it is using.

The addition model here is the textbook one: align on the larger exponent, add once, renormalize, round once at the end using the discarded tail. Real hardware does exactly this, but with guard, round, and sticky bits engineered for the corner cases, and often with fused multiply-add that rounds once across two operations. Several things are deliberately out of frame: subtraction and the catastrophic cancellation it causes, NaN and its propagation rules, the three rounding modes other than round-to-nearest-even, the exception flags the standard raises but nothing here animates, and x87's 80-bit extended registers. Treat this as the mental model, not as a specification — when correctness matters, the standard and your platform's docs are the authority.

## Edge Cases

- **Zero is a reserved pattern.** The implicit leading 1 makes zero unwritable as 1.f × 2ᵉ, so the all-zero exponent and fraction is spent on it. The sign bit still applies, which is why `-0.0` exists; it compares equal to `0.0` but has different bits, and IEEE non-trapping division gives opposite infinities for `1/0.0` and `1/-0.0`. A language can instead raise an exception, as Python does for float division by zero.
- **Subnormals trade precision for range.** Below the smallest normal (2⁻¹⁴ in binary16), the exponent stops moving and the leading bit becomes 0. Values keep going down to 2⁻²⁴, but with fewer significant bits each step — a gentle underflow instead of a cliff.
- **Infinity follows arithmetic rules of its own.** The all-ones exponent with a zero fraction means infinity. Adding a finite value with the same sign leaves it at infinity; IEEE status flags may be set without a language-level exception. Infinity is not a universal absorber: `inf - inf`, `inf * 0`, and `inf / inf` produce NaN, the all-ones exponent with a nonzero fraction, which compares unequal to everything including itself. Two infinities only produce NaN in an indeterminate form such as `inf - inf`; `inf + inf` is still infinity.
- **Rounding can move the exponent.** Store 2047.9 in binary16 and the round-up carries out of the fraction field entirely, incrementing the exponent and landing on exactly 2048. A rounding is not always a small nudge to the last bit.
- **Some decimals are exact.** 0.375 is 0.011 in binary, a short sum of powers of two, so it lands on a rung with zero error. A finite binary fraction is exact when its significand and exponent fit the format — which is why 0.5, 0.25, and 2.75 never show up in bug reports.

## Common Mistakes

- **Comparing computed values with `==`.** Two expressions that are algebraically identical can land on different rungs. Use a tolerance with both terms — `abs(a - b) <= max(rel * max(abs(a), abs(b)), abs_tol)` — because a purely relative test collapses to zero near zero and a fixed epsilon is absurd at 10⁶. Equality itself is not banned: integers of magnitude at most 2⁵³ in binary64, values you stored and never did arithmetic on, and sentinel constants all compare exactly. It is the computed ones that need the tolerance.
- **Assuming more precision fixes it.** Binary64 has the same disease with more bits: 0.1 is still not representable, `0.1 + 0.2` is still not 0.3. Widening the format moves the error, it does not remove it.
- **Expecting addition to be associative.** `(a + b) + c` and `a + (b + c)` round at different points and genuinely disagree. Sum a large value and many small ones in the wrong order and the small ones get absorbed one at a time; sum smallest-first, or use a compensated sum, and they survive.
- **Using floats for money.** Cents are hundredths, and hundredths are not powers of two. Store integer cents or use a decimal type; a rounding error of half an ulp is fine for physics and unacceptable in a ledger.
- **Believing the printed value.** Most languages print the shortest decimal that round-trips, so `0.1` displays as `0.1` even though the stored value is not 0.1. The display is lying politely on the format's behalf; the bits are the truth.
- **Testing overflow after the fact.** Once a sum reaches infinity the operands are gone, and the flag the standard set for you lives somewhere you are not looking — C reaches it through `<fenv.h>`, NumPy through `np.errstate`, and overflowing float addition in plain Python or JavaScript can return infinity. Other operations and APIs may raise exceptions. Check the range before the operation, or check `isfinite` after it.