From 3662bd79407aebc7d07558df3ecfc4fc2b3e69ec Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 27 May 2026 15:09:30 +0100 Subject: [PATCH 01/10] bf16 rfc --- text/3983-bf16.md | 521 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 521 insertions(+) create mode 100644 text/3983-bf16.md diff --git a/text/3983-bf16.md b/text/3983-bf16.md new file mode 100644 index 00000000000..7618f776d0a --- /dev/null +++ b/text/3983-bf16.md @@ -0,0 +1,521 @@ +# `bf16` primitive floating-point type + +## Summary + +Add a primitive type, `bf16`, to Rust for the 16-bit **Brain Floating Point** format. + +`bf16` is a 16-bit floating-point format derived from `f32`. It keeps the same 1-bit sign and 8-bit exponent layout as `f32`, but stores only the top 7 fraction bits. This gives it approximately the same dynamic range as `f32`, but much lower precision. The primary role of `bf16` is representing bfloat16 values in memory, in APIs, in FFI signatures where the target ABI supports bfloat16, and as an element type for SIMD and matrix operations. It is expected to be uncommon for a `bf16` to be used as a general-purpose arithmetic type in the same way as `f32` or `f64`. + +`bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for conversions and scalar arithmetic. These are specified as as widening to `f32`, performing the corresponding `f32` operation, and narrowing back to `bf16` using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. + +Hardware support is present in x86_64, AArch64 and RISC-V to name a few. `bf16` support and implementation varies across architectures, and even similar vector or matrix instructions on the same architecture can differ in their handling of rounding, subnormals, NaNs, and accumulation. Target-specific vendor SIMD and matrix intrinsics expose the semantics of the underlying hardware and are defined as target-dependent in this RFC. + +### Acknowledgements + +Thanks to [Folkert de Vries](https://github.com/folkertdev), [Trevor Gross](https://github.com/tgross35), [Tyler Mandry](https://github.com/tmandry), [Ralf Jung](https://github.com/RalfJung) and [Sayantan Chakraborty](https://github.com/sayantn) for providing early feedback on a draft of this RFC. + +## Motivation + +`bf16` is useful where memory bandwidth, storage size, and vector throughput matter more than decimal precision. The main use case is machine learning and AI workloads, where values are often stored or multiplied in `bf16` while accumulation happens in wider precision. + +Increasingly `bf16` is found in hardware across architectures to support these workloads. The below is not an exhaustive list but illustrative of `bf16`'s prevalence in hardware. + +On AArch64, `bf16` is supported by Armv8.6-A and later, implementations that include the `bf16` architectural extension, include platforms based on [Neoverse V1][Neoverse V1], [Neoverse N2][Neoverse N2], and [Neoverse V2][Neoverse V2]. On x86_64, `bf16` is supported through `AVX512_BF16` on platforms such as [Intel Cooper Lake][Intel Cooper Lake], [Intel Sapphire Rapids][Intel Sapphire Rapids] and later platforms with `AMX_BF16`, and [AMD Zen 4][AMD Zen 4] and later platforms with `AVX-512 BF16` support. On [RISC-V][RISC_V], `bf16` is supported through the ratified `Zfbfmin`, `Zvfbfmin`, and `Zvfbfwma` extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively. + +Consequently, `bf16` is not limited to accelerator hardware; it is a natively supported scalar/vector format on modern CPUs, with availability discoverable through target-feature mechanisms. + +A primitive `bf16` type lets Rust APIs use the intended numeric format directly, while still allowing target-specific operations to expose the behaviour of the underlying hardware. For example x86_64 has instructions like [`VADDBF16`][intel-latest] which prove tricky to make in stdarch without a `bf16` type. As more `bf16` instructions are introduced Rust's support will fall behind. + +Beyond raw hardware support, `bf16` is already supported in the surrounding ecosystem. A `bf16` a type is provided by [`stdfloat` in C++23][CPP_float_types] with `std::bfloat16_t`, a vendor extension in GCC and Clang (`__bf16`), and a [first-class scalar type in LLVM IR][llvm_bfloat] (`bfloat`). Rust today cannot name this type directly, so FFI signatures, SIMD element types, and codegen all route around the gap with `u16`, wrapper types, and ad-hoc lowering rules (see Current Rust support). A driving motivation for a primitive `bf16` is to close that interop gap with a `bf16` type, in a similar spirit that [RFC 3892][RFC-3892] proposes `core::num::Complex` to match C99 `_Complex` at [the FFI boundary][RFC-3892-motivation]. + +## Guide-level explanation + +### Naming the 16-bit Brain floating-point `bf16` + +The proposed name for the Brain Floating Point type is `bf16`. See rationale as to why. + +### Usage + +This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. + +```rust +// Simple `bf16` scalar operations followed by printing + +let a: bf16 = 6.7; +let b: bf16 = 42.69; +let c = a + b; +println!("{}", c); +``` + +### Intro to Floating Point bit representation + +Floating-point types represent real numbers approximately using a fixed number of bits. These bits are divided into three fields: a sign bit, which determines whether the value is positive or negative. Exponent bits, which determine the scale or range of the value and fraction bits, also commonly called mantissa or significand bits, which determine the precision of the value within that range. Below is the bit representation of an `f32`. + +```text + 31 30 23 22 0 + +---+-----------------+----------------------------------------+ + | S | Exponent | Mantissa | + +---+-----------------+----------------------------------------+ + 1 8 bits 23 bits +``` + +### Trade-offs between representations + +Increasing the number of exponent bits allows a format to represent much larger and much smaller values. Increasing the number of fraction bits allows it to represent values more precisely. Different floating-point formats therefore make different trade-offs between range, precision, storage size, and hardware efficiency. + +`bf16` is a 16-bit floating-point format intended to preserve the dynamic range of `f32` while using half the storage. It does this by using the same number of exponent bits as `f32`, but fewer mantissa bits. This makes `bf16` useful in machine learning and numerical workloads where range is often more important than precision, and where data movement, memory bandwidth, and hardware throughput matter. + +Compared with IEEE-754 binary16 / `f16`, `bf16` trades precision for range: + +| data type | sign bits | exponent bits | fraction bits | +| --------- | --------- | ------------- | ------------- | +| `f16` | 1 | 5 | 10 | +| `bf16` | 1 | 8 | 7 | +| `f32` | 1 | 8 | 23 | + +That makes the essence of `bf16` feel more like a compact storage/compute companion to `f32`, rather than a smaller-range replacement for it. `bf16` will generate the `bfloat` type in LLVM IR. It is also equivalent to C++ `std::bfloat16_t`, and GCC's and clang's `__bf16` vendor extension. + +The `bf16` layout is conceptually similar to taking the high 16 bits of an IEEE-754 binary32 / `f32` value: + +```text + f32, 32 bits: + + 31 30 23 22 0 + +---+-----------------+----------------------------------------+ + | S | Exponent | Mantissa | + +---+-----------------+----------------------------------------+ + 1 8 bits 23 bits + + + bf16, 16 bits: + + 15 14 7 6 0 + +---+-----------------+---------------+ + | S | Exponent | Mantissa | + +---+-----------------+---------------+ + 1 8 bits 7 bits + + + Relationship: + + f32: + +---+-----------------+---------------+--------------------------+ + | S | E E E E E E E E | M M M M M M M | discarded lower f32 frac | + +---+-----------------+---------------+--------------------------+ + \___________________ _______________/ + kept as bf16 high 16 bits + + bf16: + +---+-----------------+---------------+ + | S | E E E E E E E E | M M M M M M M | + +---+-----------------+---------------+ +``` + +### Vector operations + +A first-class `bf16` type lets Rust SIMD, dot-product, and matrix APIs use the natural element type for `bf16` data instead of `u16`, wrapper types, or architecture-specific encodings. This improves clarity, and portability of API surfaces. + +Target-specific vector and matrix operations may still have target-specific numerical behaviour. APIs exposing hardware instructions should be understood as exposing those instructions, not as promising bit-exact results across architectures. + +For more details, see reference-level explanation. + +### Scalar arithmetic + +The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. + +Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. + +For more details, see reference-level explanation. + +## Reference-level explanation + +### ABI + +This RFC does not propose a new platform ABI for `bf16`. + +On AArch64, [the AAPCS64 specification defines `__bf16` as the Brain floating-point format with byte size 2 and natural alignment 2][aapcs64-bf16]. On x86_64, `bf16` ABI compatibility should likewise follow the platform psABI and toolchain support for `__bf16`, which is stated as having ["the same size, alignment, parameter passing and return rules as _Float16"][x86_64-bf16]. + +Within Rust, `bf16` shall have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding described above. These are properties of the Rust type itself, not a new foreign calling convention. + +For `extern "C"` and other foreign ABIs, Rust should not independently choose how `bf16` is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ `__bf16` type, Rust's `bf16` should be ABI-compatible with that type. On platforms where the ABI is undefined, see [Unresolved Questions](#unresolved-questions). + +### Vector operations + +A first-class `bf16` type is primarily useful for target-specific SIMD and Matrix APIs. These APIs commonly operate on vectors or tiles of `bf16` values and accumulate into `f32`, reflecting the way `bf16` is typically used in machine learning and numerical code. + +For example, Arm exposes instructions that multiply vectors of `bf16` values and accumulate into `f32` results. x86_64 platforms expose similar functionality through features such as `AVX512_BF16` and, on some Intel platforms, `AMX_BF16`. + +With a primitive `bf16` type, Rust APIs can model these operations using the natural element type of the source operands: + +```rust +// Example Neon intrinsic that would be added to `std::arch::aarch64` +fn vbfdot_lane_f32(r: float32x2_t, a: bfloat16x4_t, b: bfloat16x4_t) -> float32x2_t; +``` + +Without a scalar `bf16` type, these APIs must instead represent `bf16` data using `u16`, wrapper types, or backend-specific encodings. Using `u16` preserves the raw bits, but it loses the meaning of the value. It also cannot use the usual floating-point surface of the language, such as casts, literals, constants, formatting, arithmetic traits, or generic bounds over floating-point types. That makes the APIs less clear, less portable, and less directly connected to the hardware operations they expose. If adopting a wrapper approach like `core::arch::x86_64::bf16` Rust would need a `bf16` type per architecture, this could cause confusion especially if names for the type diverged. + +For operations that are hardware-specific, some `bf16` instructions can have different numerical semantics across architectures. For example, Arm [`BFDOT`][ARM_ARM] and related `bf16` dot-product instructions operate on `bf16` inputs and accumulate into `f32` results, but their pseudocode specifies architecture-defined intermediate rounding behaviour, and this behaviour can differ further when enhanced `bf16` support is enabled. + +By contrast, x86_64's [`VDPBF16PS`][intel_a64] specifies `bf16` dot products accumulated into `f32` using round-to-nearest ties to even for the FMA accumulation, while also treating input denormals as zero, flushing output denormals to zero, and ignoring `MXCSR`. + +Therefore, target-specific SIMD, dot-product, and matrix APIs should be understood as exposing the hardware operation, not as promising target-independent bit-exact results. Portable Rust APIs may use `bf16` as the element type for clarity and type safety, but portability of the source-level API should not be read as portability of every numerical detail of the underlying target-specific instruction. This is true today with intrinsics for "packed/vector float min" for example; + +``` +x86_64: _mm_min_ps(-0.0, +0.0) -> +0.0 bits 0x00000000 +AArch64: vminq_f32(-0.0, +0.0) -> -0.0 bits 0x80000000 +``` + +While ostensibly similar they produce different results. + +### Scalar Arithmetic + +While this section is detailed, the primary use case of `bf16` is **not** expected to be scalar arithmetic but SIMD/matrix support. Moreover a compelling argument can be made for **not** making scalar arithmetic portable which is laid out in the [rationale and alternatives](#scalar-arithmetic-alternatives) section. So too has a case for not including scalar arithmetic at all. Nevertheless this RFC proposes portable semantics. + +For narrowing an `f32` to `bf16`, the conversion shall use the IEEE-754-2019 [default rounding-direction attribute for binary floating-point results, namely `roundTiesToEven`][ieee-754]. For subnormals the default delivers a rounded result that may or may not be zero. In other words subnormals are not flushed to zero by default. + +This is a semantic specification, not an implementation requirement. An implementation may use any instruction sequence, including native `bf16` instructions, only when the observable result is identical to the specified model for all non-NaN inputs. The result of scalar `bf16` arithmetic must not depend on whether the target has native `bf16` arithmetic, on optimisation level, or on whether the expression is evaluated at compile time or run time. + +Scalar arithmetic for `bf16` where no native scalar instruction are present are specified as if both operands are first converted exactly to `f32`, the corresponding Rust `f32` operation is performed, and the resulting `f32` value is then narrowed to `bf16` using round-to-nearest, ties-to-even (RNE). This keeps things portable between targets and the rounding mode the same as the IEEE-754 specification. + +This looks as follows (Widen -> `f32` operations -> narrow using RNE): + +```text +bf16 + bf16 + -> widen both operands exactly to f32 + -> perform Rust f32 addition + -> narrow the f32 result to bf16 using RNE +``` + +In code, this looks as follows; + +```rust +let a = 1.1bf16; +let b = 2.2bf16; + +let c = a + b; +``` + +Which is semantically equivalent to: + +```rust +let c = (f32::from(a) + f32::from(b)) as bf16; +``` + +Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. + +As a direct consequence: + +- Native `bf16` arithmetic operations that round directly to `bf16`, rather than producing the same result as the specified through the above sequence, are **not** legal lowerings for scalar `bf16` arithmetic. They produce different result bits for some inputs. Such operations remain available through explicit target-specific intrinsics, where the user has opted in to native semantics. +- The narrowing step always rounds to nearest, ties-to-even. Truncation is not a conforming narrowing. +- The arithmetic performs two IEEE rounding steps: the inner operation rounds to `f32` precision, and the narrow then rounds that result to `bf16` precision. + +The two-step rounding may produce results that differ by one [ULP][ULP_WIKI] from a hypothetical single-rounded native `bf16` operation for a small fraction of inputs. This is a deliberate trade-off in favour of portable, bit-exact semantics; targets without native `bf16` arithmetic could not otherwise be supported uniformly. Code that requires single-rounded native semantics should use target-specific intrinsics. + +As with Rust's existing floating-point types, this specification does not strengthen Rust's guarantees about NaN payload preservation; NaN payloads remain unspecified within the limits Rust already allows for `f32` and `f64`. + +Please see [footnotes on `bf16` conversion](#bf16-conversions) for a more indepth exploration into hardware considerations and support. + +### Display + +`Display` for `bf16` should follow the same round-trip principle as Rust's existing primitive floats: it should produce a decimal representation that parses back to the same `bf16` value via `FromStr`, preferably using the shortest such representation. + +The maximum number of decimal digits needed to uniquely round-trip any `bf16` value is given by the standard formula `ceil(1 + p × log10(2))` from [IEEE-754, 2019][ieee-754], where `p = 8` for `bf16`, yielding `4`. In practice, shortest-round-trip formatters use fewer digits where the value permits, so most `bf16` values will display with 1-3 significant digits. + +Special values follow the existing primitive-float conventions: `NaN`, `inf`, `-inf`. `Debug` should be consistent with `Display` for ordinary values; its exact format is not specified here. + +As with `f32` and `f64`, displayed values may differ from the source literal when the literal is not exactly representable in the format. This effect is more visible for `bf16` because of its lower precision. + +### Constants + +`bf16` should expose the same style of constants as other primitive float types: + +```rust +bf16::BITS +bf16::DIGITS // approximately 2 decimal digits +bf16::EPSILON +bf16::INFINITY +bf16::MANTISSA_DIGITS // 8, including the implicit leading bit +bf16::MAX +bf16::MAX_10_EXP +bf16::MAX_EXP +bf16::MIN +bf16::MIN_10_EXP +bf16::MIN_EXP +bf16::MIN_POSITIVE +bf16::NAN +bf16::NEG_INFINITY +bf16::RADIX // 2 +``` + +The `half` crate documents `bf16::MANTISSA_DIGITS` as 8, including the implicit bit, and `DIGITS` as 2 decimal digits. However, a crate-defined type cannot fully substitute for a primitive type. Though its use is demonstrative of users' need for a `bf16` type. + + +### Literal suffix + +`bf16` supports a literal suffix. + +```rust +let a = 0.64545bf16; +``` + +### Conversions + +Conversions between `bf16` and integer types are numeric conversions. + +#### `From` Conversions + +Conversions from `bf16` to a wider floating-point type shall preserve the represented value by expanding the `bf16` bit pattern into the destination floating-point format. + +At minimum: + +```rust +impl From for f32 { /* exact widening */ } +impl From for f64 { /* exact widening */ } + +impl From for bf16 { /* ... */ } +impl From for bf16 { /* ... */ } + +let x: bf16 = 1.0bf16; +let y: f32 = f32::from(x); + +println!("Size of bf16 in bytes: {}", core::mem::size_of::()); // 2 +``` +#### No `From` + +`From for bf16` and vice versa should not exist in either direction as neither direction is value preserving. `f16` has 11 bits of precision while `bf16` has 8. A `bf16`'s range vastly exceeds that of an `f16` and `From impl`'s are for lossless conversions. Ostensibly the naming including `16` makes them look similar when reading the type however how they use those 16 bits differs wildly. + +#### Lossy Conversions + +Lossy conversions to `bf16` should be available through casts, performing down casting using round-to-nearest, ties to even: + +```rust +let x: bf16 = 3.1415927f32 as bf16; +let y: bf16 = 3.141592653589793f64 as bf16; +``` + +Raw bit conversion: + +```rust +impl bf16 { + pub const fn from_bits(bits: u16) -> bf16; + pub const fn to_bits(self) -> u16; +} +``` +#### Traits + +`bf16` follows the same general pattern as the other primitive floats, supporting: + +- `Add`, `Sub`, `Mul`, `Div`, `Rem` +- `Neg` +- `PartialEq`, `PartialOrd` +- `Copy`, `Clone`, `Default` +- `Debug`, `Display`, and the other formatting traits +- `Sum`, `Product` +- `FromStr` + +These scalar operations must be implemented in conformance to the widen -> `f32` operations -> narrow using RNE as specified above. + +## Drawbacks + +- As with [RFC 3453][RFC 3453], support for this type has a fairly specific usecase as opposed to general purpose programming. `bf16` is important for machine learning, numerical code, storage, conversion, and target-specific SIMD or matrix operations, but most Rust code will not use it directly. + +- Many targets do not provide native scalar `bf16` arithmetic. On those targets, scalar operations must be lowered using a wider floating-point type or helper routines, which may make scalar `bf16` arithmetic slower than users expect. This seems acceptable because scalar arithmetic is not expected to be the primary use case for `bf16`. + +- `bf16` is not simply a smaller `f32` or an alias for IEEE binary16 / `f16`. It has its own layout, precision, rounding behaviour, NaN behaviour, and conversion rules. Adding it as a primitive type therefore requires Rust to specify these semantics explicitly. + +- The most significant drawback is that native hardware behaviour and portable emulated behaviour may not be identical for all arithmetic operations. This RFC specifies `bf16` scalar arithmetic in terms of widening operands to `f32`, performing the operation in `f32`, and narrowing the result back to `bf16`, so that `bf16` arithmetic has portable Rust semantics rather than target-specific native semantics. + +## Rationale and alternatives + +### Placement: global namespace versus `core::num` + +Primitive types;`f16/f32/f64/f128`, `bool`, `char` etc... lives in the global namespace and are always in scope without an import. This RFC proposes the primitive `bf16` would by default do the same, reserving the bare name `bf16` in every Rust program. + +An alternative is to make `bf16` a compiler-known primitive that is reached by path, as `core::num::bf16`, and is not added to the prelude. This mirrors the placement proposed for `Complex` in [RFC 3892][RFC-3892], which positions a numeric interchange type in `core::num` rather than the global namespace. + +#### The Tradeoffs + +A path only `core::num::bf16` does not consume a globally reserved name which signals that this is a specialist interchange type rather than a general-purpose arithmetic type which arguably a `bf16` could be. This could also set a precedent for future numeric formats without committing prelude real estate each time. If we want `bf16` to be something that is only used as an exchange format then `use core::num::bf16;` could be a more natural fit. + +However a global namespace is consistent with `f16/f128` and the rest of the primitives, requires no import, and is what users would need if they are wanting to suffix floating point numbers with `bf16` to define `bf16`'s. Though whether this is desirable needs to be ironed out. + +This decision is reversible across an edition. Prelude membership is edition-scoped, so `bf16` can ship path-only and later be added to a future edition's prelude (or the reverse) without breaking existing code. This RFC proposes a global namespace first. + +### Scalar Arithmetic Alternatives + +Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic for a `bf16` means that the rather complicated and, depending on support, possibly inefficient scalar operations could be omitted for `bf16` which makes the type simpler and prevents a user being confused with the resultant code of widening to `f32`, performing the operations and narrowing to `bf16` along with preventing misuse of the type. It seems reasonable a user could utilise the `From` and `as` casts themselves, perform scalar arithmetic in an `f32` and convert back to `bf16` without relying on built in support. Moreover vendor specific intrinsics could be used, where available, to perform scalar operations. + +Rust does already accept target-dependent behaviour in a primitive: `usize` differs in size between 32-bit and 64-bit targets, so identical source an overflow on one target and not another. This could be seen as a precedent for letting `bf16` scalar arithmetic follow whatever the hardware supports, thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which the portable specification would forbid. Ostensibly this should, theoretically, be more performant that converting to `f32` doing the operation and narrowing back to `bf16`. + +The `usize` precedent is weaker than first appears as `usize` is deterministic per target: given a target triple its behaviour is fully specified and stable. Its variation is structural 32-bits vs 64-bits and the type system prevents it being mixed with fixed sized integers. However `bf16` would vary on the same target triple depending on what target features have been made available. This problem is described in the [Prior Art](#c-compiler-support). + +Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type. + +### Naming `bf16` + +The name `bf16` maps directly to hardware-provider naming conventions, is a simple abbreviation of the official name [`bfloat16`][bfloat16_name], is similar to the [`__bf16` extension name in C++][__bf16_extension_name] and is immediately distinguishable from the similarly named `f16` by the `b` prefix, meaning `brain` and `f` denoting `float`. This abbreviation also follows Rust's naming conventions for the currently supported numerical types where `float` is abbreviated with `f` or `integer` with an `i`. + +### `bf16` as a library type + +Adding a new float primitive _could_ invite a slippery slope of further including special float types like `f8`, `f4` or any other so called ["minifloats"][minifloats]. Conceivably `bf16` could be a library type as opposed to a primitive. However in Rust today there is no possible way for a user to define custom `as` casts, literal suffixes or a way to lower to a specified LLVM primitive. Until these exist a `bf16` library type cannot be the target of a built-in `as` cast, cannot define a `1.0bf16` suffix, cannot fix foreign-ABI passing rules and cannot lower to LLVM's `bfloat`. Having `bf16` defined as a primitive scalar addresses all of these limitations. Thus the RFC proposes `bf16` as a primitive for now while acknowledging a library type could be possible in the future. + +## Prior Art + +### C++ compiler support + +The `C++23` standard includes `std::bfloat16_t` in `` as part of the extended floating-point types introduced by [P1467][P1467]. This was motivated by the prevalence of availability of `bf16` on hardware, library support through class wrappers being cumbersome to use and a desire for more portable code. [(for the section on adding extended floating point types see here)][cpp_bf16_motivation] + +GCC 13 implements the C++23 [P1467][P1467] extended floating-point types, including `std::bfloat16_t` and the `bf16` literal suffix. Clang's support for extended floating-point types, at the time of writing, is more limited and varies by feature; see the [Clang C++ status page][clang_status_page] (then search the page for `P1467`). Both GCC and Clang nevertheless support `bfloat16` through the `__bf16` type. + +For arithmetic operations the standard states; + +> Trivial implementations of the math functions for extended floating-point types that are no bigger than long double can be done by casting the arguments to a standard floating-point that is at least as big as the extended floating-point type, doing the calculations with the standard floating-point type, then casting the result back down to the extended floating-point type + +Of which the proposal here for up-casting to `f32`, doing the binary operation then downcasting to `bf16` via round to nearest, ties even would be compatible with the C++ standard. + +Further more the standard states: + +> An implementation may also provide additional types that represent floating-point values and define them (and cv-qualified versions thereof) to be extended floating-point types. The standard and extended floating-point types are collectively termed floating-point types. [...] Except as specified in [basic.extended.fp], the object and value representations and accuracy of operations of floating-point types is implementation-defined. + +As such, during the time of writing, whether compiling for `x86_64` using emulated or hardware specific instructions, subnormals will be handled differently. +### Current Rust support + +Rust already has an experimental `core::arch::x86::bf16` / `core::arch::x86_64::bf16` [type for AVX-512 BF16 intrinsics][rust_AVX-512_BF16]. This type is nightly-only, x86-specific, and documented as "the BFloat16 type used in AVX-512 intrinsics" rather than as a general-purpose primitive type. Its API is limited to raw `from_bits` and `to_bits` operations, and the stdarch tracking issue describes it as "just a wrapper around `u16`" intended to make intrinsic signatures more readable. + +That approach is useful for stdarch, but it does not solve the general language problem. A `u16` wrapper can represent the raw bits of a bfloat16 value, and such a wrapper could be made available across architectures. However, it would still make `bf16` a library-defined convention rather than a primitive floating-point type known to the language and compiler. + +This distinction matters for operations that are built into the language or depend on compiler lowering. A wrapper cannot be the target of built-in float casts or literal suffixes, cannot by itself define how `bf16` values are passed and returned for foreign ABIs, and does not give Rust a scalar type that corresponds directly to LLVM's `bfloat` type. It also risks fragmenting the ecosystem into multiple incompatible wrapper types for the same 16-bit format. + +A primitive `bf16` gives Rust one common type for scalar values, memory layout, conversions, FFI where the target ABI supports bfloat16, and target-specific SIMD or matrix intrinsics. + +There is precedent for adding new primitive floating-point types to Rust with `f16` and `f128` [RFC 3453][RFC 3453] improving the ergonomics and capabilities of the language. + +There is also a code generation reason not to treat this as merely an x86 stdarch convention. Some LLVM intrinsics use LLVM's first-class `bf16` / `bf16xN` types in their signatures, but Rust cannot currently name those types directly. As a result, Rust intrinsic signatures have to describe bfloat16 operands using integer-like types such as `i16`, `u16`, or vectors of those types, and rustc then needs special-case lowering rules to connect those signatures to LLVM intrinsics that actually expect `bf16` or `bf16xN`. This is an ad hoc workaround: it applies only to selected intrinsics and does not give users or libraries a general Rust type for the format. + +PR [#140763][pr_140763] is an example of the current workaround: it adds temporary codegen bypasses so Rust signatures using `i16` / `i16xN` can call LLVM intrinsics whose real signatures use `bf16` / `bf16xN`. The need for that bypass illustrates the gap this RFC is intended to close. + +This RFC has been split, and expanded upon, from an [RFC proposing `bf16`, `f64f64` and `f80` types][RFC 3456]. This RFC focuses specifically on `bf16` and addresses feedback about naming the type, floating-point semantics, especially the behaviour of scalar arithmetic and conversions to and from other floating-point types. + +This RFC proposes that `bf16` should be that first-class Rust type, rather than requiring each architecture-specific module to invent its own wrapper and each backend to add ad hoc lowering rules from integer-like Rust types to LLVM's `bf16` type. A primitive `bf16` gives Rust a single semantic type for the format, while still allowing target-specific APIs such as x86 `AVX-512 BF16`, `Intel AMX BF16`, and `Arm BF16` intrinsics to expose the underlying hardware operations using `bf16` as the operand type. + +## Unresolved Questions + +### ABI on targets without a defined `bf16` calling convention + +The reference ABI section above focuses on platforms where a specified ABI exists. That leaves open what Rust should do on targets whose ABI does not (yet) specify how `bf16` is passed and returned. `f16` [faces similar problems][llvm-f16-support] and several questions carry over: + +- Should `bf16` be rejected in `extern "C"` signatures on such targets, via an error, or should Rust pick a provisional lowering? +- If provisional then should Rust follow Clang's behaviour on that target? However Clang and GCC have historically differed on how `_Float16`' are passed on some targets so "match the C toolchain" is not a single answer and a provisional choice may risk a silent ABI break of the platform standardises something different. +- Most issues stem from `bf16` being available on a target without the required FP/SIMD registers being available. This problem is not exclusive to `bf16`. [`f32` 32-bit targets without SSE][f32-no-sse] have unsound floating point behaviour, as mentioned `f16` has similar issues without SSE registers on a 32-bit x86 target. + +Where it is not possible to fix these issues a reasonable compromise is suggested in [RFC-3514 for floating point semantics][RFC-3514] where compatibility is documented for the target in a proposed "support page". Presently platform support information is documented in [platform-support.md][platform-support] along with [platform support errata][platform-support-pr]. + +### Literal Suffix + +For a literal suffix `0bf16` may clash with the lexer's integer suffix `0b` for binary numbers thus; + +```rust +let a = 0bf16; +``` + +Would need to be written; +```rust +let a = 0.0bf16; +``` + +## Future possibilities + +### Portable `bf16` vector operations + +A first-class `bf16` primitive opens the door to portable vector operations on `bf16`, but specifying those operations is out of scope for this RFC. The reasons are the same divergences documented for narrowing in the cross-architecture conformance table: even when two architectures provide an instruction with the same source-level shape, their handling of rounding, subnormals, and intermediate precision can differ. Arm's `BFDOT` and x86_64's `VDPBF16PS` both compute a `bf16` dot product accumulating into `f32`, but they do not produce bit-identical results, and the difference is not a bug; it is a deliberate ISA-level choice. + +Specifying portable vector semantics therefore requires settling several questions that the scalar specification in this RFC does not answer: + +- Whether portable vector ops should follow a widen-to-f32 / op-in-f32 / narrow model (analogous to the scalar specification), accepting the performance cost on hardware whose native instructions take a different path, or whether some looser "approximately equivalent" semantics should be permitted for vector operations as a deliberate trade-off. +- How fused multiply-add and dot-product reductions should behave across targets, given that the intermediate accumulator width and rounding behaviour are architecture-defined for the native instructions. +- How subnormal handling should be specified when some targets flush by default and others preserve. +- Whether portable vector `bf16` should be part of `core::simd` (the portable SIMD effort) or a separate surface, and how it should interact with target-feature gating. + +### Unifying `bf16` usages + +For example in stdarch's `core::arch::x86_64::bf16` type should be transitioned away from being a wrapper to using the `bf16`. This path should, where possible, follow for other wrapper types that are aiming to use `bf16`. + +## Footnotes + +### `bf16` conversions + +The below explores some of the real world nuances and considerations for architecture support. + +#### Narrowing: with native conversion support + +Some architectures, including AArch64, x86_64, and RISC-V, provide dedicated instructions for converting to and from `bf16`. These instructions are not interchangeable: their behaviour can differ in rounding mode selection, subnormal handling, NaN handling, and interaction with floating-point control registers. + +The examples below cover AArch64, x86_64, and RISC-V, but the same considerations apply to any target with native `bf16` conversion instructions. The word "conforming" in this section is used to implicate following the semantics of RNE and not flushing subnormals to zero, following IEEE-754 as described in the previous section of this RFC. + +##### x86_64 + +On x86_64 with `AVX512_BF16`, [`VCVTNEPS2BF16`][intel_a64] performs RNE and does not respect or update `MXCSR`. This makes it independent of the dynamic `MXCSR.RC` rounding mode. For subnormals, however, the instruction flushes to zero. Thus, `VCVTNEPS2BF16` under what is proposed in this RFC is not a conforming lowering for an `f32` -> `bf16` narrowing operation that is required to preserve representable `bf16` subnormal results. If this behaviour is required for a program one can use the relevant platform specific intrinsic. Thus this instruction, under the behaviour proposed in this RFC, could not be used. + +##### AArch64 + +On AArch64 with the `bf16` architectural extension, narrowing from `f32` to `bf16` can be performed by [`BFCVT`][BFCVT]. In the normal FPCR mode (`FPCR.AH == 0`), the BFCVT-family instructions use the rounding mode selected by `FPCR.RMode` of which`RMode = 0b00` selects RNE. If `FPCR.AH == 1`, the BFCVT-family instructions use RNE regardless of `RMode`. Therefore, for an unconditional RNE specification, `BFCVT` is a conforming lowering only when the compilation/runtime model guarantees that the relevant execution observes RNE, for example because `FPCR.RMode == 0b00` or because `FPCR.AH == 1` gives BFCVT-family instructions RNE semantics regardless of `RMode`. Subnormal preservation is a separate requirement from the rounding mode. AArch64 `BFCVT` is a conforming lowering for this specification only when the execution environment both selects RNE for the conversion and does not enable a mode that flushes the relevant denormal/subnormal results to zero. In the normal FPCR-zero state, this means `FPCR.RMode == 0b00` and `FPCR.FZ == 0`. + +On Linux/arm64, a newly exec'd task starts with zeroed user FPSIMD state. The kernel's arm64 FPSIMD code stores FPSR and FPCR, then the exec/reset path initializes this user state to zero. Consequently, at process entry after `execve()`, the user FPCR is zero: `FPCR.RMode[23:22] == 0b00`, which is Round to Nearest, i.e. nearest-even. This is an initial process state, not a guarantee that FPCR cannot later be changed by user code or libraries. [See source][linux]. + +Rust assumes `FPCR` remains in a conforming state across the lifetime of a process; code that mutates `FPCR.RMode` or `FPCR.FZ` may observe non-conforming `bf16` results, analogous to existing expectations about `MXCSR` on x86_64 or effects the FPCR register has on `f32`. Thus `BFCVT` conforms to the behaviour proposed in the RFC. + +##### RISC-V + +On RISC-V with `Zfbfmin`, narrowing from `f32` to `bf16` is performed by [`FCVT.BF16.S`][RISC_V], which is a standard narrowing floating-point-to-floating-point conversion that rounds according to the `frm` field (either the static rounding mode encoded in the instruction or the dynamic mode in the `frm CSR`). Setting `frm` to `RNE (0b000)` selects round-to-nearest, ties-to-even, making `FCVT.BF16.S` a conforming lowering under this specification when the execution environment selects RNE rounding for the conversion. Static encoding of the rounding mode in the instruction itself avoids dependence on the dynamic `frm CSR` and is the preferred lowering where the compiler can emit it directly. + +The three architectures above each provide a hardware narrow from `f32` to `bf16`, but no two are conforming lowerings for this RFC's specification under the same conditions: + +| Architecture | Instruction | Rounding | Subnormal behaviour | Conforming as default lowering? | +| ---------------------- | --------------- | ------------------------------------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------- | +| AArch64 (`bf16` ext.) | `BFCVT` | Per `FPCR.RMode` when `FPCR.AH == 0`; RNE when `FPCR.AH == 1` | Per `FPCR.FZ` | Only when `FPCR.RMode == 0b00` and `FPCR.FZ == 0` (or `FPCR.AH == 1` with `FPCR.FZ == 0`) | +| x86_64 (`AVX512_BF16`) | `VCVTNEPS2BF16` | RNE, independent of `MXCSR.RC` | Flushes subnormal outputs to zero, ignores `MXCSR` | No, fails to preserve subnormals | +| RISC-V (`Zfbfmin`) | `FCVT.BF16.S` | Per instruction-encoded RM, or `frm` CSR | Preserved | Yes, when RM/`frm` selects RNE | + +#### Narrowing: without native conversion support or where hardware does not provide a portable solution + +Where no hardware instruction performs an RNE narrow from `f32` to `bf16` or hardware does not provide a portable solution, the narrowing must be emulated. A straightforward RNE implementation inspects bit 15 of the `f32` significand and the sticky bits below it to decide whether to round up, with the usual ties-to-even tiebreak; this is a handful of integer ops. This is the same kind of operation normally provided by soft-float conversion helpers such as `__truncsfbf2`. + +[Neoverse V1]: https://developer.arm.com/Processors/Neoverse%20V1 +[Neoverse N2]: https://developer.arm.com/documentation/102099/0003/The-Neoverse-N2--core/Supported-standards-and-specifications +[Neoverse V2]: https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/arm-neoverse-v2-platform-best-in-class-cloud-and-ai-ml-performance +[Intel Cooper Lake]: https://en.wikipedia.org/wiki/Cooper_Lake_(microprocessor) +[Intel Sapphire Rapids]: https://edc.intel.com/content/www/us/en/products/performance/benchmarks/4th-generation-intel-xeon-scalable-processors/ +[AMD Zen 4]: https://www.amd.com/en/products/processors/server/epyc/4th-generation-architecture.html +[aapcs64-bf16]: https://github.com/ARM-software/abi-aa/blob/main/aapcs64/aapcs64.rst +[x86_64-bf16]: https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf +[RFC 3453]: https://rust-lang.github.io/rfcs/3453-f16-and-f128.html +[P1467]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html +[clang_status_page]: https://clang.llvm.org/cxx_status +[rust_AVX-512_BF16]: https://doc.rust-lang.org/core/arch/x86/struct.bf16.html +[pr_140763]: https://github.com/rust-lang/rust/pull/140763 +[RFC 3456]: https://github.com/rust-lang/rfcs/pull/3456 +[BFCVT]: https://developer.arm.com/documentation/ddi0596/2021-03/SIMD-FP-Instructions/BFCVT--Floating-point-convert-from-single-precision-to-BFloat16-format--scalar-- +[arm_fpcr]: https://developer.arm.com/documentation/ddi0601/2024-12/AArch64-Registers/FPCR--Floating-point-Control-Register +[intel_a64]: https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-combined-volumes-1-2a-2b-2c-2d-3a-3b-3c-3d-and-4 +[cpp_bf16_motivation]: https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p1467r9.html#motivation +[bfloat16_name]: https://docs.cloud.google.com/tpu/docs/bfloat16 +[__bf16_extension_name]: https://clang.llvm.org/docs/LanguageExtensions.html#half-precision-floating-point +[round_to_nearest_ties_to_even]: https://en.wikipedia.org/wiki/Rounding#Rounding_half_to_even +[ieee-754]: https://ieeexplore.ieee.org/document/8766229 +[linux]: https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux/%2B/4333a9b0b67bb4e8bcd91bdd80da80b0ec151162/arch/arm64/kernel/fpsimd.c +[ARM_ARM]: https://developer.arm.com/-/cdn-downloads/permalink/Exploration-Tools-Arm-Architecture-Features/AARCHMRS/AARCHMRS_A_profile-2026-03_96.tar.gz +[RISC_V]: https://docs.riscv.org/reference/isa/unpriv/bfloat16.html#insns-fcvt.s.bf16 +[ULP_WIKI]: https://en.wikipedia.org/wiki/Unit_in_the_last_place +[mm_min_ps]: https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html#text=_mm_min_ps&ig_expand=4489 +[RFC-3892]: https://github.com/rust-lang/rfcs/blob/master/text/3892-complex-numbers.md +[RFC-3892-motivation]: https://github.com/rust-lang/rfcs/blob/master/text/3892-complex-numbers.md#motivation +[CPP_float_types]: https://en.cppreference.com/cpp/types/floating-point +[llvm_bfloat]: https://rocm.docs.amd.com/projects/llvm-project/en/latest/LLVM/llvm/html/LangRef.html#floating-point-types +[minifloats]: https://en.wikipedia.org/wiki/Minifloat +[intel-latest]: https://www.intel.com/content/www/us/en/content-details/919693/intel-advanced-vector-extensions-10-2-intel-avx10-2-architecture-specification.html +[usize-type]: https://doc.rust-lang.org/std/primitive.usize.html +[llvm-f16-support]: https://github.com/llvm/llvm-project/issues/112885#issuecomment-4798333405 +[f32-no-sse]: https://github.com/rust-lang/rust/issues/114479 +[RFC-3514]: https://rust-lang.github.io/rfcs/3514-float-semantics.html +[platform-support]: github.com/rust-lang/rust/blob/main/src/doc/rustc/src/platform-support.md +[platform-support-pr]: https://github.com/rust-lang/rust/pull/157288 From 0e1f5c4d79429158fc6cfe3a66f59de0b936d9a8 Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Tue, 21 Jul 2026 14:55:45 +0100 Subject: [PATCH 02/10] Addressed feedback and some verbosity issues --- text/3983-bf16.md | 340 ++++++++++++++++++++++------------------------ 1 file changed, 162 insertions(+), 178 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 7618f776d0a..1fb79c94c85 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -6,7 +6,7 @@ Add a primitive type, `bf16`, to Rust for the 16-bit **Brain Floating Point** fo `bf16` is a 16-bit floating-point format derived from `f32`. It keeps the same 1-bit sign and 8-bit exponent layout as `f32`, but stores only the top 7 fraction bits. This gives it approximately the same dynamic range as `f32`, but much lower precision. The primary role of `bf16` is representing bfloat16 values in memory, in APIs, in FFI signatures where the target ABI supports bfloat16, and as an element type for SIMD and matrix operations. It is expected to be uncommon for a `bf16` to be used as a general-purpose arithmetic type in the same way as `f32` or `f64`. -`bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for conversions and scalar arithmetic. These are specified as as widening to `f32`, performing the corresponding `f32` operation, and narrowing back to `bf16` using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. +`bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for conversions and scalar arithmetic. These are specified as widening to `f32`, performing the corresponding `f32` operation, and narrowing back to `bf16` using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. Hardware support is present in x86_64, AArch64 and RISC-V to name a few. `bf16` support and implementation varies across architectures, and even similar vector or matrix instructions on the same architecture can differ in their handling of rounding, subnormals, NaNs, and accumulation. Target-specific vendor SIMD and matrix intrinsics expose the semantics of the underlying hardware and are defined as target-dependent in this RFC. @@ -18,9 +18,7 @@ Thanks to [Folkert de Vries](https://github.com/folkertdev), [Trevor Gross](http `bf16` is useful where memory bandwidth, storage size, and vector throughput matter more than decimal precision. The main use case is machine learning and AI workloads, where values are often stored or multiplied in `bf16` while accumulation happens in wider precision. -Increasingly `bf16` is found in hardware across architectures to support these workloads. The below is not an exhaustive list but illustrative of `bf16`'s prevalence in hardware. - -On AArch64, `bf16` is supported by Armv8.6-A and later, implementations that include the `bf16` architectural extension, include platforms based on [Neoverse V1][Neoverse V1], [Neoverse N2][Neoverse N2], and [Neoverse V2][Neoverse V2]. On x86_64, `bf16` is supported through `AVX512_BF16` on platforms such as [Intel Cooper Lake][Intel Cooper Lake], [Intel Sapphire Rapids][Intel Sapphire Rapids] and later platforms with `AMX_BF16`, and [AMD Zen 4][AMD Zen 4] and later platforms with `AVX-512 BF16` support. On [RISC-V][RISC_V], `bf16` is supported through the ratified `Zfbfmin`, `Zvfbfmin`, and `Zvfbfwma` extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively. +Increasingly `bf16` is found in hardware across architectures (see appendix for a few examples) to support these workloads. The `half` crate adds an `f16` and `bf16` as a wrapper over a `u16`. However, a crate-defined type cannot fully substitute for a primitive type. Though its use is demonstrative of users' need for a `bf16` type. Consequently, `bf16` is not limited to accelerator hardware; it is a natively supported scalar/vector format on modern CPUs, with availability discoverable through target-feature mechanisms. @@ -32,11 +30,11 @@ Beyond raw hardware support, `bf16` is already supported in the surrounding ecos ### Naming the 16-bit Brain floating-point `bf16` -The proposed name for the Brain Floating Point type is `bf16`. See rationale as to why. +The proposed name for the Brain Floating Point type is `bf16`. See rationale for alternatives and to why. ### Usage -This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. +This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations (albeit with a non-standard binary format) and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. ```rust // Simple `bf16` scalar operations followed by printing @@ -47,97 +45,105 @@ let c = a + b; println!("{}", c); ``` -### Intro to Floating Point bit representation -Floating-point types represent real numbers approximately using a fixed number of bits. These bits are divided into three fields: a sign bit, which determines whether the value is positive or negative. Exponent bits, which determine the scale or range of the value and fraction bits, also commonly called mantissa or significand bits, which determine the precision of the value within that range. Below is the bit representation of an `f32`. +### Vector operations -```text - 31 30 23 22 0 - +---+-----------------+----------------------------------------+ - | S | Exponent | Mantissa | - +---+-----------------+----------------------------------------+ - 1 8 bits 23 bits -``` +A first-class `bf16` type lets Rust SIMD, dot-product, and matrix APIs use the natural element type for `bf16` data. Target-specific vector and matrix operations may still have target-specific numerical behaviour. APIs exposing hardware instructions should be understood as exposing those instructions, not as promising bit-exact results across architectures. -### Trade-offs between representations +For more details, see reference-level explanation. -Increasing the number of exponent bits allows a format to represent much larger and much smaller values. Increasing the number of fraction bits allows it to represent values more precisely. Different floating-point formats therefore make different trade-offs between range, precision, storage size, and hardware efficiency. +### Scalar arithmetic -`bf16` is a 16-bit floating-point format intended to preserve the dynamic range of `f32` while using half the storage. It does this by using the same number of exponent bits as `f32`, but fewer mantissa bits. This makes `bf16` useful in machine learning and numerical workloads where range is often more important than precision, and where data movement, memory bandwidth, and hardware throughput matter. +The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. -Compared with IEEE-754 binary16 / `f16`, `bf16` trades precision for range: +A compelling argument can be made for **not** making scalar arithmetic portable is laid out in the [rationale and alternatives](#scalar-arithmetic-alternatives) section as is a case for not implementing scalar arithmetic at all. Nevertheless this RFC proposes portable semantics. -| data type | sign bits | exponent bits | fraction bits | -| --------- | --------- | ------------- | ------------- | -| `f16` | 1 | 5 | 10 | -| `bf16` | 1 | 8 | 7 | -| `f32` | 1 | 8 | 23 | - -That makes the essence of `bf16` feel more like a compact storage/compute companion to `f32`, rather than a smaller-range replacement for it. `bf16` will generate the `bfloat` type in LLVM IR. It is also equivalent to C++ `std::bfloat16_t`, and GCC's and clang's `__bf16` vendor extension. +Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. -The `bf16` layout is conceptually similar to taking the high 16 bits of an IEEE-754 binary32 / `f32` value: +For more details, see reference-level explanation. -```text - f32, 32 bits: +### Conversions - 31 30 23 22 0 - +---+-----------------+----------------------------------------+ - | S | Exponent | Mantissa | - +---+-----------------+----------------------------------------+ - 1 8 bits 23 bits +Conversions between `bf16` and integer types are numeric conversions. +#### `From` Conversions - bf16, 16 bits: +Conversions from `bf16` to a wider floating-point type shall preserve the represented value by expanding the `bf16` bit pattern into the destination floating-point format. - 15 14 7 6 0 - +---+-----------------+---------------+ - | S | Exponent | Mantissa | - +---+-----------------+---------------+ - 1 8 bits 7 bits +At minimum: +```rust +impl From for f32 { /* exact widening */ } +impl From for f64 { /* exact widening */ } - Relationship: +impl From for bf16 { /* ... */ } +impl From for bf16 { /* ... */ } - f32: - +---+-----------------+---------------+--------------------------+ - | S | E E E E E E E E | M M M M M M M | discarded lower f32 frac | - +---+-----------------+---------------+--------------------------+ - \___________________ _______________/ - kept as bf16 high 16 bits +let x: bf16 = 1.0bf16; +let y: f32 = f32::from(x); - bf16: - +---+-----------------+---------------+ - | S | E E E E E E E E | M M M M M M M | - +---+-----------------+---------------+ +println!("Size of bf16 in bytes: {}", core::mem::size_of::()); // 2 ``` +#### No `From` -### Vector operations +`From for bf16` and vice versa should not exist in either direction as neither direction is value preserving. `f16` has 11 bits of precision while `bf16` has 8. A `bf16`'s range vastly exceeds that of an `f16` and `From impl`'s are for lossless conversions. Ostensibly the naming including `16` makes them look similar when reading the type however how they use those 16 bits differs wildly. + +#### Lossy Conversions -A first-class `bf16` type lets Rust SIMD, dot-product, and matrix APIs use the natural element type for `bf16` data instead of `u16`, wrapper types, or architecture-specific encodings. This improves clarity, and portability of API surfaces. +Lossy conversions to `bf16` should be available through casts, performing down casting using round-to-nearest, ties to even: -Target-specific vector and matrix operations may still have target-specific numerical behaviour. APIs exposing hardware instructions should be understood as exposing those instructions, not as promising bit-exact results across architectures. +```rust +let x: bf16 = 3.1415927f32 as bf16; +let y: bf16 = 3.141592653589793f64 as bf16; +``` -For more details, see reference-level explanation. +Raw bit conversion: -### Scalar arithmetic +```rust +impl bf16 { + pub const fn from_bits(bits: u16) -> bf16; + pub const fn to_bits(self) -> u16; +} +``` -The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. +### Traits -Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. +`bf16` follows the same general pattern as the other primitive floats, supporting: -For more details, see reference-level explanation. +- `Add`, `Sub`, `Mul`, `Div`, `Rem` +- `Neg` +- `PartialEq`, `PartialOrd` +- `Copy`, `Clone`, `Default` +- `Debug`, `Display`, and the other formatting traits +- `Sum`, `Product` +- `FromStr` + +These scalar operations must be implemented in conformance to the widen -> `f32` operations -> narrow using RNE as specified above. ## Reference-level explanation -### ABI +### Scalar Arithmetic -This RFC does not propose a new platform ABI for `bf16`. +Scalar `bf16` arithmetic without native scalar instructions are defined by exactly converting both operands to `f32`, performing the corresponding Rust `f32` operation, and narrowing the result to `bf16` using [IEEE-754-2019][ieee-754] `roundTiesToEven`, without flushing subnormals to zero. Native `bf16` operations that round directly to `bf16` are legal only if they produce the same result as this sequence, ensuring consistent and portable behaviour across targets. -On AArch64, [the AAPCS64 specification defines `__bf16` as the Brain floating-point format with byte size 2 and natural alignment 2][aapcs64-bf16]. On x86_64, `bf16` ABI compatibility should likewise follow the platform psABI and toolchain support for `__bf16`, which is stated as having ["the same size, alignment, parameter passing and return rules as _Float16"][x86_64-bf16]. +In other words, this code: + +```rust +let a = 1.1bf16; +let b = 2.2bf16; + +let c = a + b; +``` + +Must be semantically equivalent to: + +```rust +let c = (f32::from(a) + f32::from(b)) as bf16; +``` -Within Rust, `bf16` shall have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding described above. These are properties of the Rust type itself, not a new foreign calling convention. +Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. -For `extern "C"` and other foreign ABIs, Rust should not independently choose how `bf16` is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ `__bf16` type, Rust's `bf16` should be ABI-compatible with that type. On platforms where the ABI is undefined, see [Unresolved Questions](#unresolved-questions). +Please see [footnotes on `bf16` conversion](#bf16-conversions) for a more indepth exploration into hardware considerations and support. ### Vector operations @@ -152,9 +158,7 @@ With a primitive `bf16` type, Rust APIs can model these operations using the nat fn vbfdot_lane_f32(r: float32x2_t, a: bfloat16x4_t, b: bfloat16x4_t) -> float32x2_t; ``` -Without a scalar `bf16` type, these APIs must instead represent `bf16` data using `u16`, wrapper types, or backend-specific encodings. Using `u16` preserves the raw bits, but it loses the meaning of the value. It also cannot use the usual floating-point surface of the language, such as casts, literals, constants, formatting, arithmetic traits, or generic bounds over floating-point types. That makes the APIs less clear, less portable, and less directly connected to the hardware operations they expose. If adopting a wrapper approach like `core::arch::x86_64::bf16` Rust would need a `bf16` type per architecture, this could cause confusion especially if names for the type diverged. - -For operations that are hardware-specific, some `bf16` instructions can have different numerical semantics across architectures. For example, Arm [`BFDOT`][ARM_ARM] and related `bf16` dot-product instructions operate on `bf16` inputs and accumulate into `f32` results, but their pseudocode specifies architecture-defined intermediate rounding behaviour, and this behaviour can differ further when enhanced `bf16` support is enabled. +For intrinsics that live in stdarch that are hardware-specific, some `bf16` instructions can have different numerical semantics across architectures. For example, Arm [`BFDOT`][ARM_ARM] and related `bf16` dot-product instructions operate on `bf16` inputs and accumulate into `f32` results, but their pseudocode specifies architecture-defined intermediate rounding behaviour, and this behaviour can differ further when enhanced `bf16` support is enabled. By contrast, x86_64's [`VDPBF16PS`][intel_a64] specifies `bf16` dot products accumulated into `f32` using round-to-nearest ties to even for the FMA accumulation, while also treating input denormals as zero, flushing output denormals to zero, and ignoring `MXCSR`. @@ -167,54 +171,6 @@ AArch64: vminq_f32(-0.0, +0.0) -> -0.0 bits 0x80000000 While ostensibly similar they produce different results. -### Scalar Arithmetic - -While this section is detailed, the primary use case of `bf16` is **not** expected to be scalar arithmetic but SIMD/matrix support. Moreover a compelling argument can be made for **not** making scalar arithmetic portable which is laid out in the [rationale and alternatives](#scalar-arithmetic-alternatives) section. So too has a case for not including scalar arithmetic at all. Nevertheless this RFC proposes portable semantics. - -For narrowing an `f32` to `bf16`, the conversion shall use the IEEE-754-2019 [default rounding-direction attribute for binary floating-point results, namely `roundTiesToEven`][ieee-754]. For subnormals the default delivers a rounded result that may or may not be zero. In other words subnormals are not flushed to zero by default. - -This is a semantic specification, not an implementation requirement. An implementation may use any instruction sequence, including native `bf16` instructions, only when the observable result is identical to the specified model for all non-NaN inputs. The result of scalar `bf16` arithmetic must not depend on whether the target has native `bf16` arithmetic, on optimisation level, or on whether the expression is evaluated at compile time or run time. - -Scalar arithmetic for `bf16` where no native scalar instruction are present are specified as if both operands are first converted exactly to `f32`, the corresponding Rust `f32` operation is performed, and the resulting `f32` value is then narrowed to `bf16` using round-to-nearest, ties-to-even (RNE). This keeps things portable between targets and the rounding mode the same as the IEEE-754 specification. - -This looks as follows (Widen -> `f32` operations -> narrow using RNE): - -```text -bf16 + bf16 - -> widen both operands exactly to f32 - -> perform Rust f32 addition - -> narrow the f32 result to bf16 using RNE -``` - -In code, this looks as follows; - -```rust -let a = 1.1bf16; -let b = 2.2bf16; - -let c = a + b; -``` - -Which is semantically equivalent to: - -```rust -let c = (f32::from(a) + f32::from(b)) as bf16; -``` - -Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. - -As a direct consequence: - -- Native `bf16` arithmetic operations that round directly to `bf16`, rather than producing the same result as the specified through the above sequence, are **not** legal lowerings for scalar `bf16` arithmetic. They produce different result bits for some inputs. Such operations remain available through explicit target-specific intrinsics, where the user has opted in to native semantics. -- The narrowing step always rounds to nearest, ties-to-even. Truncation is not a conforming narrowing. -- The arithmetic performs two IEEE rounding steps: the inner operation rounds to `f32` precision, and the narrow then rounds that result to `bf16` precision. - -The two-step rounding may produce results that differ by one [ULP][ULP_WIKI] from a hypothetical single-rounded native `bf16` operation for a small fraction of inputs. This is a deliberate trade-off in favour of portable, bit-exact semantics; targets without native `bf16` arithmetic could not otherwise be supported uniformly. Code that requires single-rounded native semantics should use target-specific intrinsics. - -As with Rust's existing floating-point types, this specification does not strengthen Rust's guarantees about NaN payload preservation; NaN payloads remain unspecified within the limits Rust already allows for `f32` and `f64`. - -Please see [footnotes on `bf16` conversion](#bf16-conversions) for a more indepth exploration into hardware considerations and support. - ### Display `Display` for `bf16` should follow the same round-trip principle as Rust's existing primitive floats: it should produce a decimal representation that parses back to the same `bf16` value via `FromStr`, preferably using the shortest such representation. @@ -247,73 +203,23 @@ bf16::NEG_INFINITY bf16::RADIX // 2 ``` -The `half` crate documents `bf16::MANTISSA_DIGITS` as 8, including the implicit bit, and `DIGITS` as 2 decimal digits. However, a crate-defined type cannot fully substitute for a primitive type. Though its use is demonstrative of users' need for a `bf16` type. - - -### Literal suffix - -`bf16` supports a literal suffix. - -```rust -let a = 0.64545bf16; -``` - -### Conversions - -Conversions between `bf16` and integer types are numeric conversions. - -#### `From` Conversions - -Conversions from `bf16` to a wider floating-point type shall preserve the represented value by expanding the `bf16` bit pattern into the destination floating-point format. - -At minimum: - -```rust -impl From for f32 { /* exact widening */ } -impl From for f64 { /* exact widening */ } - -impl From for bf16 { /* ... */ } -impl From for bf16 { /* ... */ } - -let x: bf16 = 1.0bf16; -let y: f32 = f32::from(x); +### ABI -println!("Size of bf16 in bytes: {}", core::mem::size_of::()); // 2 -``` -#### No `From` +This RFC does not propose a new platform ABI for `bf16`. -`From for bf16` and vice versa should not exist in either direction as neither direction is value preserving. `f16` has 11 bits of precision while `bf16` has 8. A `bf16`'s range vastly exceeds that of an `f16` and `From impl`'s are for lossless conversions. Ostensibly the naming including `16` makes them look similar when reading the type however how they use those 16 bits differs wildly. +On AArch64, [the AAPCS64 specification defines `__bf16` as the Brain floating-point format with byte size 2 and natural alignment 2][aapcs64-bf16]. On x86_64, `bf16` ABI compatibility should likewise follow the platform psABI and toolchain support for `__bf16`, which is stated as having ["the same size, alignment, parameter passing and return rules as _Float16"][x86_64-bf16]. -#### Lossy Conversions +Within Rust, `bf16` should usually have a size 2 and alignment 2, all bit patterns are valid values, and its memory representation is the 16-bit Brain floating-point encoding described above. -Lossy conversions to `bf16` should be available through casts, performing down casting using round-to-nearest, ties to even: +For `extern "C"` and other foreign ABIs, Rust should not independently choose how `bf16` is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ `__bf16` type, Rust's `bf16` should be ABI-compatible with that type. On platforms where the ABI is undefined, see [Unresolved Questions](#unresolved-questions). -```rust -let x: bf16 = 3.1415927f32 as bf16; -let y: bf16 = 3.141592653589793f64 as bf16; -``` +### Literal suffix -Raw bit conversion: +`bf16` supports a literal suffix. ```rust -impl bf16 { - pub const fn from_bits(bits: u16) -> bf16; - pub const fn to_bits(self) -> u16; -} +let a = 0.64545bf16; ``` -#### Traits - -`bf16` follows the same general pattern as the other primitive floats, supporting: - -- `Add`, `Sub`, `Mul`, `Div`, `Rem` -- `Neg` -- `PartialEq`, `PartialOrd` -- `Copy`, `Clone`, `Default` -- `Debug`, `Display`, and the other formatting traits -- `Sum`, `Product` -- `FromStr` - -These scalar operations must be implemented in conformance to the widen -> `f32` operations -> narrow using RNE as specified above. ## Drawbacks @@ -327,11 +233,11 @@ These scalar operations must be implemented in conformance to the widen -> `f32` ## Rationale and alternatives -### Placement: global namespace versus `core::num` +### Global namespace with literal suffix versus `core::num` -Primitive types;`f16/f32/f64/f128`, `bool`, `char` etc... lives in the global namespace and are always in scope without an import. This RFC proposes the primitive `bf16` would by default do the same, reserving the bare name `bf16` in every Rust program. +Primitive types;`f16/f32/f64/f128`, `bool`, `char` etc... lives in the global namespace and are always in scope without an import. This RFC proposes the primitive `bf16` would by default do the same, reserving the bare name `bf16` in every Rust program and allowing a literal `bf16` suffix. -An alternative is to make `bf16` a compiler-known primitive that is reached by path, as `core::num::bf16`, and is not added to the prelude. This mirrors the placement proposed for `Complex` in [RFC 3892][RFC-3892], which positions a numeric interchange type in `core::num` rather than the global namespace. +An alternative is to make `bf16` a compiler-known primitive that is reached by path, as `core::num::bf16` or `core::aarch:bf16`, and is not added to the prelude nor allowing a `bf16` suffix. This mirrors the placement proposed for `Complex` in [RFC 3892][RFC-3892], which positions a numeric interchange type in `core::num` rather than the global namespace. #### The Tradeoffs @@ -343,7 +249,7 @@ This decision is reversible across an edition. Prelude membership is edition-sco ### Scalar Arithmetic Alternatives -Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic for a `bf16` means that the rather complicated and, depending on support, possibly inefficient scalar operations could be omitted for `bf16` which makes the type simpler and prevents a user being confused with the resultant code of widening to `f32`, performing the operations and narrowing to `bf16` along with preventing misuse of the type. It seems reasonable a user could utilise the `From` and `as` casts themselves, perform scalar arithmetic in an `f32` and convert back to `bf16` without relying on built in support. Moreover vendor specific intrinsics could be used, where available, to perform scalar operations. +Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic would keep `bf16` simpler and avoid potentially complex or inefficient implementations. Such operations may require widening the value to `f32`, performing the calculation, and then narrowing the result back to `bf16`, which could confuse users or encourage misuse of the type. Instead, users could explicitly convert values to `f32` using `From` or `as`, perform the scalar arithmetic, and then convert the result back to `bf16`. Where supported, vendor-specific intrinsics could also provide more efficient scalar operations. Rust does already accept target-dependent behaviour in a primitive: `usize` differs in size between 32-bit and 64-bit targets, so identical source an overflow on one target and not another. This could be seen as a precedent for letting `bf16` scalar arithmetic follow whatever the hardware supports, thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which the portable specification would forbid. Ostensibly this should, theoretically, be more performant that converting to `f32` doing the operation and narrowing back to `bf16`. @@ -353,8 +259,13 @@ Portable scalar arithmetic could prove useful if predictable behaviour is expect ### Naming `bf16` +Other such proposed names have been; `bf16`, `f16b`, `fb16`, `bfloat16`, and `__bf16`. + +This RFC prefers `bf16`. + The name `bf16` maps directly to hardware-provider naming conventions, is a simple abbreviation of the official name [`bfloat16`][bfloat16_name], is similar to the [`__bf16` extension name in C++][__bf16_extension_name] and is immediately distinguishable from the similarly named `f16` by the `b` prefix, meaning `brain` and `f` denoting `float`. This abbreviation also follows Rust's naming conventions for the currently supported numerical types where `float` is abbreviated with `f` or `integer` with an `i`. + ### `bf16` as a library type Adding a new float primitive _could_ invite a slippery slope of further including special float types like `f8`, `f4` or any other so called ["minifloats"][minifloats]. Conceivably `bf16` could be a library type as opposed to a primitive. However in Rust today there is no possible way for a user to define custom `as` casts, literal suffixes or a way to lower to a specified LLVM primitive. Until these exist a `bf16` library type cannot be the target of a built-in `as` cast, cannot define a `1.0bf16` suffix, cannot fix foreign-ABI passing rules and cannot lower to LLVM's `bfloat`. Having `bf16` defined as a primitive scalar addresses all of these limitations. Thus the RFC proposes `bf16` as a primitive for now while acknowledging a library type could be possible in the future. @@ -380,7 +291,7 @@ Further more the standard states: As such, during the time of writing, whether compiling for `x86_64` using emulated or hardware specific instructions, subnormals will be handled differently. ### Current Rust support -Rust already has an experimental `core::arch::x86::bf16` / `core::arch::x86_64::bf16` [type for AVX-512 BF16 intrinsics][rust_AVX-512_BF16]. This type is nightly-only, x86-specific, and documented as "the BFloat16 type used in AVX-512 intrinsics" rather than as a general-purpose primitive type. Its API is limited to raw `from_bits` and `to_bits` operations, and the stdarch tracking issue describes it as "just a wrapper around `u16`" intended to make intrinsic signatures more readable. +Rust already has an experimental `core::arch::x86::bf16` / `core::arch::x86_64::bf16` [type for AVX-512 BF16 intrinsics][rust_AVX-512_BF16]. This type is nightly-only, x86-specific, and documented as "the BFloat16 type used in AVX-512 intrinsics" rather than as a general-purpose primitive type. Its API is limited to raw `from_bits` and `to_bits` operations, and the stdarch tracking issue describes it as "just a wrapper around `u16`" intended to make intrinsic signatures more readable. If adopting a wrapper approach like `core::arch::x86_64::bf16` Rust would need a `bf16` type per architecture, this could cause confusion especially if names for the type diverged. That approach is useful for stdarch, but it does not solve the general language problem. A `u16` wrapper can represent the raw bits of a bfloat16 value, and such a wrapper could be made available across architectures. However, it would still make `bf16` a library-defined convention rather than a primitive floating-point type known to the language and compiler. @@ -480,6 +391,79 @@ The three architectures above each provide a hardware narrow from `f32` to `bf16 Where no hardware instruction performs an RNE narrow from `f32` to `bf16` or hardware does not provide a portable solution, the narrowing must be emulated. A straightforward RNE implementation inspects bit 15 of the `f32` significand and the sticky bits below it to decide whether to round up, with the usual ties-to-even tiebreak; this is a handful of integer ops. This is the same kind of operation normally provided by soft-float conversion helpers such as `__truncsfbf2`. +## Appendix + +### `bf16` hardware support +On AArch64, `bf16` is supported by Armv8.6-A and later, implementations that include the `bf16` architectural extension, include platforms based on [Neoverse V1][Neoverse V1], [Neoverse N2][Neoverse N2], and [Neoverse V2][Neoverse V2]. On x86_64, `bf16` is supported through `AVX512_BF16` on platforms such as [Intel Cooper Lake][Intel Cooper Lake], [Intel Sapphire Rapids][Intel Sapphire Rapids] and later platforms with `AMX_BF16`, and [AMD Zen 4][AMD Zen 4] and later platforms with `AVX-512 BF16` support. On [RISC-V][RISC_V], `bf16` is supported through the ratified `Zfbfmin`, `Zvfbfmin`, and `Zvfbfwma` extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively. + +### `bf16` compared to other floating point types +Below is a brief introduction as to how floats are represented and a comparison between the representation of a `bf16` in comparison to an `f32`. + +#### Intro to Floating Point bit representation + +Floating-point types represent real numbers approximately using a fixed number of bits. These bits are divided into three fields: a sign bit, which determines whether the value is positive or negative. Exponent bits, which determine the scale or range of the value and fraction bits, also commonly called mantissa or significand bits, which determine the precision of the value within that range. Below is the bit representation of an `f32`. + +```text + 31 30 23 22 0 + +---+-----------------+----------------------------------------+ + | S | Exponent | Mantissa | + +---+-----------------+----------------------------------------+ + 1 8 bits 23 bits +``` + +#### Trade-offs between representations + +Increasing the number of exponent bits allows a format to represent much larger and much smaller values. Increasing the number of fraction bits allows it to represent values more precisely. Different floating-point formats therefore make different trade-offs between range, precision, storage size, and hardware efficiency. + +`bf16` is a 16-bit floating-point format intended to preserve the dynamic range of `f32` while using half the storage. It does this by using the same number of exponent bits as `f32`, but fewer mantissa bits. This makes `bf16` useful in machine learning and numerical workloads where range is often more important than precision, and where data movement, memory bandwidth, and hardware throughput matter. + +Compared with IEEE-754 binary16 / `f16`, `bf16` trades precision for range: + +| data type | sign bits | exponent bits | fraction bits | +| --------- | --------- | ------------- | ------------- | +| `f16` | 1 | 5 | 10 | +| `bf16` | 1 | 8 | 7 | +| `f32` | 1 | 8 | 23 | + +That makes the essence of `bf16` feel more like a compact storage/compute companion to `f32`, rather than a smaller-range replacement for it. `bf16` will generate the `bfloat` type in LLVM IR. It is also equivalent to C++ `std::bfloat16_t`, and GCC's and clang's `__bf16` vendor extension. + +The `bf16` layout is conceptually similar to taking the high 16 bits of an IEEE-754 binary32 / `f32` value: + +```text + f32, 32 bits: + + 31 30 23 22 0 + +---+-----------------+----------------------------------------+ + | S | Exponent | Mantissa | + +---+-----------------+----------------------------------------+ + 1 8 bits 23 bits + + + bf16, 16 bits: + + 15 14 7 6 0 + +---+-----------------+---------------+ + | S | Exponent | Mantissa | + +---+-----------------+---------------+ + 1 8 bits 7 bits + + + Relationship: + + f32: + +---+-----------------+---------------+--------------------------+ + | S | E E E E E E E E | M M M M M M M | discarded lower f32 frac | + +---+-----------------+---------------+--------------------------+ + \___________________ _______________/ + kept as bf16 high 16 bits + + bf16: + +---+-----------------+---------------+ + | S | E E E E E E E E | M M M M M M M | + +---+-----------------+---------------+ +``` + + [Neoverse V1]: https://developer.arm.com/Processors/Neoverse%20V1 [Neoverse N2]: https://developer.arm.com/documentation/102099/0003/The-Neoverse-N2--core/Supported-standards-and-specifications [Neoverse V2]: https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/arm-neoverse-v2-platform-best-in-class-cloud-and-ai-ml-performance From 5321ed6256c1596766456a4ca4f4bec6563b6609 Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 10:53:05 +0100 Subject: [PATCH 03/10] Lift most of the `bf16` scalar arithmetic to the appendex, single out `FMA` as an op where a library call should be emitted --- text/3983-bf16.md | 49 +++++++++++++++++++++++++++-------------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 1fb79c94c85..58bedd324a0 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -124,26 +124,9 @@ These scalar operations must be implemented in conformance to the widen -> `f32` ### Scalar Arithmetic -Scalar `bf16` arithmetic without native scalar instructions are defined by exactly converting both operands to `f32`, performing the corresponding Rust `f32` operation, and narrowing the result to `bf16` using [IEEE-754-2019][ieee-754] `roundTiesToEven`, without flushing subnormals to zero. Native `bf16` operations that round directly to `bf16` are legal only if they produce the same result as this sequence, ensuring consistent and portable behaviour across targets. +`bf16` scalar operations behave according to the [IEEE-754-2019][ieee-754] specification. The scalar operation will return the value representable in the `bf16` format that is closest to the infinite-precision result with `roundTiesToEven` and preserving subnormals without flushing to zero. This will be true on targets woth with native scalar arithmetic instructions and those without. -In other words, this code: - -```rust -let a = 1.1bf16; -let b = 2.2bf16; - -let c = a + b; -``` - -Must be semantically equivalent to: - -```rust -let c = (f32::from(a) + f32::from(b)) as bf16; -``` - -Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. - -Please see [footnotes on `bf16` conversion](#bf16-conversions) for a more indepth exploration into hardware considerations and support. +Please see [footnotes on `bf16` scalar arithmetic](#bf16-scalar) for a more indepth exploration into hardware considerations and support. ### Vector operations @@ -353,9 +336,32 @@ For example in stdarch's `core::arch::x86_64::bf16` type should be transitioned ## Footnotes -### `bf16` conversions +### `bf16` scalar + +The below explores some of the real world nuances and considerations for architecture support. Typically, at the time of writing, hardware does not provide instructions for `bf16` scalar arithmetic. Instead, scalar operations would typically follow widening to `f32` then narrowing to `bf16` consider this example; + +```rust +let a: bf16 = 1.1bf16; +let b: bf16 = 2.2bf16; +let c: bf16 = 3.3bf16; + +let d = a + b + c; +``` + +Which could semantically equate to any of the following; + +```rust +// Widen to `f32` first +let d: bf16 = (f32::from(a) + f32::from(b) + f32::from(c)) as bf16; + +// bf16 ADD exists +let d: bf16 = a + b + c; + +// Each intermediary step is converted to `f32` and narrowed to `bf16` +let d: bf16 = (f32::from((f32::from(a) + f32::from(b)) as bf16) + f32::from(c)) as bf16; +``` -The below explores some of the real world nuances and considerations for architecture support. +Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. For an instance where the precision is not enough to perform the operation a suitable library call should be emitted. Such as with the case of `bf16`'s [`FMA` in LLVM][llvm-fma-bug] where casting to an `f64` does not have enough precision to perform a fused-multiply-add without double rounding becoming an issue. #### Narrowing: with native conversion support @@ -503,3 +509,4 @@ The `bf16` layout is conceptually similar to taking the high 16 bits of an IEEE- [RFC-3514]: https://rust-lang.github.io/rfcs/3514-float-semantics.html [platform-support]: github.com/rust-lang/rust/blob/main/src/doc/rustc/src/platform-support.md [platform-support-pr]: https://github.com/rust-lang/rust/pull/157288 +[llvm-fma-bug]: https://github.com/llvm/llvm-project/issues/131531 From 376dbcf8155033e6c7871df3101ef42765c72ddf Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 11:14:12 +0100 Subject: [PATCH 04/10] RFC Formatting --- text/3983-bf16.md | 57 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 53 insertions(+), 4 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 58bedd324a0..0a2184dc18d 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -1,6 +1,11 @@ -# `bf16` primitive floating-point type +- Feature Name: (fill me in with a unique ident, `bf16`) +- Start Date: (fill me in with today's date, 2026-07-15) +- RFC PR: [rust-lang/rfcs#3983](https://github.com/rust-lang/rfcs/pull/3983) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + ## Summary +[summary]: #summary Add a primitive type, `bf16`, to Rust for the 16-bit **Brain Floating Point** format. @@ -11,10 +16,12 @@ Add a primitive type, `bf16`, to Rust for the 16-bit **Brain Floating Point** fo Hardware support is present in x86_64, AArch64 and RISC-V to name a few. `bf16` support and implementation varies across architectures, and even similar vector or matrix instructions on the same architecture can differ in their handling of rounding, subnormals, NaNs, and accumulation. Target-specific vendor SIMD and matrix intrinsics expose the semantics of the underlying hardware and are defined as target-dependent in this RFC. ### Acknowledgements +[acknowledgements]: #acknowledgements Thanks to [Folkert de Vries](https://github.com/folkertdev), [Trevor Gross](https://github.com/tgross35), [Tyler Mandry](https://github.com/tmandry), [Ralf Jung](https://github.com/RalfJung) and [Sayantan Chakraborty](https://github.com/sayantn) for providing early feedback on a draft of this RFC. ## Motivation +[motivation]: #motivation `bf16` is useful where memory bandwidth, storage size, and vector throughput matter more than decimal precision. The main use case is machine learning and AI workloads, where values are often stored or multiplied in `bf16` while accumulation happens in wider precision. @@ -27,12 +34,15 @@ A primitive `bf16` type lets Rust APIs use the intended numeric format directly, Beyond raw hardware support, `bf16` is already supported in the surrounding ecosystem. A `bf16` a type is provided by [`stdfloat` in C++23][CPP_float_types] with `std::bfloat16_t`, a vendor extension in GCC and Clang (`__bf16`), and a [first-class scalar type in LLVM IR][llvm_bfloat] (`bfloat`). Rust today cannot name this type directly, so FFI signatures, SIMD element types, and codegen all route around the gap with `u16`, wrapper types, and ad-hoc lowering rules (see Current Rust support). A driving motivation for a primitive `bf16` is to close that interop gap with a `bf16` type, in a similar spirit that [RFC 3892][RFC-3892] proposes `core::num::Complex` to match C99 `_Complex` at [the FFI boundary][RFC-3892-motivation]. ## Guide-level explanation +[guide-level-explanation]: #guide-level-explanation ### Naming the 16-bit Brain floating-point `bf16` +[naming]: #naming The proposed name for the Brain Floating Point type is `bf16`. See rationale for alternatives and to why. ### Usage +[usage]: #usaage This RFC proposes the `bf16` type to behave like the other primitive floating-point types in Rust, with standard traits implemented. It can be used in bindings, passed to functions, stored in arrays and structs, converted to and from other numeric types, and used with the usual floating-point operations. It follows the IEEE-754 standard for floating point types for scalar operations (albeit with a non-standard binary format) and only diverge when using vendor intrinsics gated by a target feature. Details for scalar and vector operations are discussed in greater depth below. @@ -45,14 +55,15 @@ let c = a + b; println!("{}", c); ``` - ### Vector operations +[guide-vector-arithmetic]: #guide-vector-arithmetic A first-class `bf16` type lets Rust SIMD, dot-product, and matrix APIs use the natural element type for `bf16` data. Target-specific vector and matrix operations may still have target-specific numerical behaviour. APIs exposing hardware instructions should be understood as exposing those instructions, not as promising bit-exact results across architectures. For more details, see reference-level explanation. ### Scalar arithmetic +[guide-scalar-arithmetic]: #guide-scalar-arithmetic The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. @@ -60,13 +71,15 @@ A compelling argument can be made for **not** making scalar arithmetic portable Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. -For more details, see reference-level explanation. +For more details, see [reference-level explanation][reference-scalar-operations]. ### Conversions +[conversions]: #conversions Conversions between `bf16` and integer types are numeric conversions. -#### `From` Conversions +#### `From` Conversions +[from-conversions]: #from-conversions Conversions from `bf16` to a wider floating-point type shall preserve the represented value by expanding the `bf16` bit pattern into the destination floating-point format. @@ -85,10 +98,12 @@ let y: f32 = f32::from(x); println!("Size of bf16 in bytes: {}", core::mem::size_of::()); // 2 ``` #### No `From` +[no-from-bf16]: #bo-from-bf16 `From for bf16` and vice versa should not exist in either direction as neither direction is value preserving. `f16` has 11 bits of precision while `bf16` has 8. A `bf16`'s range vastly exceeds that of an `f16` and `From impl`'s are for lossless conversions. Ostensibly the naming including `16` makes them look similar when reading the type however how they use those 16 bits differs wildly. #### Lossy Conversions +[lossy-conversions]: #lossy-conversions Lossy conversions to `bf16` should be available through casts, performing down casting using round-to-nearest, ties to even: @@ -107,6 +122,7 @@ impl bf16 { ``` ### Traits +[traits]: #traits `bf16` follows the same general pattern as the other primitive floats, supporting: @@ -121,14 +137,17 @@ impl bf16 { These scalar operations must be implemented in conformance to the widen -> `f32` operations -> narrow using RNE as specified above. ## Reference-level explanation +[reference-level-explanation]: #reference-level-explanation ### Scalar Arithmetic +[reference-scalar-operations]: #reference-scalar-operations `bf16` scalar operations behave according to the [IEEE-754-2019][ieee-754] specification. The scalar operation will return the value representable in the `bf16` format that is closest to the infinite-precision result with `roundTiesToEven` and preserving subnormals without flushing to zero. This will be true on targets woth with native scalar arithmetic instructions and those without. Please see [footnotes on `bf16` scalar arithmetic](#bf16-scalar) for a more indepth exploration into hardware considerations and support. ### Vector operations +[reference-vector-operations]: #vector-operations A first-class `bf16` type is primarily useful for target-specific SIMD and Matrix APIs. These APIs commonly operate on vectors or tiles of `bf16` values and accumulate into `f32`, reflecting the way `bf16` is typically used in machine learning and numerical code. @@ -155,6 +174,7 @@ AArch64: vminq_f32(-0.0, +0.0) -> -0.0 bits 0x80000000 While ostensibly similar they produce different results. ### Display +[display]: #display `Display` for `bf16` should follow the same round-trip principle as Rust's existing primitive floats: it should produce a decimal representation that parses back to the same `bf16` value via `FromStr`, preferably using the shortest such representation. @@ -165,6 +185,7 @@ Special values follow the existing primitive-float conventions: `NaN`, `inf`, `- As with `f32` and `f64`, displayed values may differ from the source literal when the literal is not exactly representable in the format. This effect is more visible for `bf16` because of its lower precision. ### Constants +[constants]: #constants `bf16` should expose the same style of constants as other primitive float types: @@ -187,6 +208,7 @@ bf16::RADIX // 2 ``` ### ABI +[abi]: #abi This RFC does not propose a new platform ABI for `bf16`. @@ -197,6 +219,7 @@ Within Rust, `bf16` should usually have a size 2 and alignment 2, all bit patter For `extern "C"` and other foreign ABIs, Rust should not independently choose how `bf16` is passed or returned. Instead, on targets where the platform defines and toolchains implement an ABI for the corresponding C/C++ `__bf16` type, Rust's `bf16` should be ABI-compatible with that type. On platforms where the ABI is undefined, see [Unresolved Questions](#unresolved-questions). ### Literal suffix +[literal-suffix]: #literal-suffix `bf16` supports a literal suffix. @@ -205,6 +228,7 @@ let a = 0.64545bf16; ``` ## Drawbacks +[drawbacks]: #drawbacks - As with [RFC 3453][RFC 3453], support for this type has a fairly specific usecase as opposed to general purpose programming. `bf16` is important for machine learning, numerical code, storage, conversion, and target-specific SIMD or matrix operations, but most Rust code will not use it directly. @@ -215,14 +239,17 @@ let a = 0.64545bf16; - The most significant drawback is that native hardware behaviour and portable emulated behaviour may not be identical for all arithmetic operations. This RFC specifies `bf16` scalar arithmetic in terms of widening operands to `f32`, performing the operation in `f32`, and narrowing the result back to `bf16`, so that `bf16` arithmetic has portable Rust semantics rather than target-specific native semantics. ## Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives ### Global namespace with literal suffix versus `core::num` +[global-namespace-with-literal-suffix-versus-core::num]: #global-namespace-with-literal-suffix-versus-core::num Primitive types;`f16/f32/f64/f128`, `bool`, `char` etc... lives in the global namespace and are always in scope without an import. This RFC proposes the primitive `bf16` would by default do the same, reserving the bare name `bf16` in every Rust program and allowing a literal `bf16` suffix. An alternative is to make `bf16` a compiler-known primitive that is reached by path, as `core::num::bf16` or `core::aarch:bf16`, and is not added to the prelude nor allowing a `bf16` suffix. This mirrors the placement proposed for `Complex` in [RFC 3892][RFC-3892], which positions a numeric interchange type in `core::num` rather than the global namespace. #### The Tradeoffs +[the-tradeoffs]: #the-trade-offs A path only `core::num::bf16` does not consume a globally reserved name which signals that this is a specialist interchange type rather than a general-purpose arithmetic type which arguably a `bf16` could be. This could also set a precedent for future numeric formats without committing prelude real estate each time. If we want `bf16` to be something that is only used as an exchange format then `use core::num::bf16;` could be a more natural fit. @@ -231,6 +258,7 @@ However a global namespace is consistent with `f16/f128` and the rest of the pri This decision is reversible across an edition. Prelude membership is edition-scoped, so `bf16` can ship path-only and later be added to a future edition's prelude (or the reverse) without breaking existing code. This RFC proposes a global namespace first. ### Scalar Arithmetic Alternatives +[scalar-arithmetic-alternatives]: #scalar-arithmetic-alternatives Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic would keep `bf16` simpler and avoid potentially complex or inefficient implementations. Such operations may require widening the value to `f32`, performing the calculation, and then narrowing the result back to `bf16`, which could confuse users or encourage misuse of the type. Instead, users could explicitly convert values to `f32` using `From` or `as`, perform the scalar arithmetic, and then convert the result back to `bf16`. Where supported, vendor-specific intrinsics could also provide more efficient scalar operations. @@ -241,6 +269,7 @@ The `usize` precedent is weaker than first appears as `usize` is deterministic p Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type. ### Naming `bf16` +[naming-bf16]: naming-bf16 Other such proposed names have been; `bf16`, `f16b`, `fb16`, `bfloat16`, and `__bf16`. @@ -250,12 +279,15 @@ The name `bf16` maps directly to hardware-provider naming conventions, is a simp ### `bf16` as a library type +[bf16-as-a-library-type]: #bf16-as-a-library-type Adding a new float primitive _could_ invite a slippery slope of further including special float types like `f8`, `f4` or any other so called ["minifloats"][minifloats]. Conceivably `bf16` could be a library type as opposed to a primitive. However in Rust today there is no possible way for a user to define custom `as` casts, literal suffixes or a way to lower to a specified LLVM primitive. Until these exist a `bf16` library type cannot be the target of a built-in `as` cast, cannot define a `1.0bf16` suffix, cannot fix foreign-ABI passing rules and cannot lower to LLVM's `bfloat`. Having `bf16` defined as a primitive scalar addresses all of these limitations. Thus the RFC proposes `bf16` as a primitive for now while acknowledging a library type could be possible in the future. ## Prior Art +[prior-art]: #prior-art ### C++ compiler support +[cpp-compiler-support]: #cpp-compiler-support The `C++23` standard includes `std::bfloat16_t` in `` as part of the extended floating-point types introduced by [P1467][P1467]. This was motivated by the prevalence of availability of `bf16` on hardware, library support through class wrappers being cumbersome to use and a desire for more portable code. [(for the section on adding extended floating point types see here)][cpp_bf16_motivation] @@ -273,6 +305,7 @@ Further more the standard states: As such, during the time of writing, whether compiling for `x86_64` using emulated or hardware specific instructions, subnormals will be handled differently. ### Current Rust support +[current-rust-support]: #current-rust-support Rust already has an experimental `core::arch::x86::bf16` / `core::arch::x86_64::bf16` [type for AVX-512 BF16 intrinsics][rust_AVX-512_BF16]. This type is nightly-only, x86-specific, and documented as "the BFloat16 type used in AVX-512 intrinsics" rather than as a general-purpose primitive type. Its API is limited to raw `from_bits` and `to_bits` operations, and the stdarch tracking issue describes it as "just a wrapper around `u16`" intended to make intrinsic signatures more readable. If adopting a wrapper approach like `core::arch::x86_64::bf16` Rust would need a `bf16` type per architecture, this could cause confusion especially if names for the type diverged. @@ -293,6 +326,7 @@ This RFC has been split, and expanded upon, from an [RFC proposing `bf16`, `f64f This RFC proposes that `bf16` should be that first-class Rust type, rather than requiring each architecture-specific module to invent its own wrapper and each backend to add ad hoc lowering rules from integer-like Rust types to LLVM's `bf16` type. A primitive `bf16` gives Rust a single semantic type for the format, while still allowing target-specific APIs such as x86 `AVX-512 BF16`, `Intel AMX BF16`, and `Arm BF16` intrinsics to expose the underlying hardware operations using `bf16` as the operand type. ## Unresolved Questions +[unresolved-questions]: #unresolved-questions ### ABI on targets without a defined `bf16` calling convention @@ -318,8 +352,10 @@ let a = 0.0bf16; ``` ## Future possibilities +[future-possibilities]: #future-possibilities ### Portable `bf16` vector operations +[portable-bf16-vector-operations]: portable-bf16-vector-operations A first-class `bf16` primitive opens the door to portable vector operations on `bf16`, but specifying those operations is out of scope for this RFC. The reasons are the same divergences documented for narrowing in the cross-architecture conformance table: even when two architectures provide an instruction with the same source-level shape, their handling of rounding, subnormals, and intermediate precision can differ. Arm's `BFDOT` and x86_64's `VDPBF16PS` both compute a `bf16` dot product accumulating into `f32`, but they do not produce bit-identical results, and the difference is not a bug; it is a deliberate ISA-level choice. @@ -331,12 +367,15 @@ Specifying portable vector semantics therefore requires settling several questio - Whether portable vector `bf16` should be part of `core::simd` (the portable SIMD effort) or a separate surface, and how it should interact with target-feature gating. ### Unifying `bf16` usages +[unifying-bf16-usages]: #unifying-bf16-usages For example in stdarch's `core::arch::x86_64::bf16` type should be transitioned away from being a wrapper to using the `bf16`. This path should, where possible, follow for other wrapper types that are aiming to use `bf16`. ## Footnotes +[footnotes]: #footnotes ### `bf16` scalar +[bf16-scalar]: #bf16-scalar The below explores some of the real world nuances and considerations for architecture support. Typically, at the time of writing, hardware does not provide instructions for `bf16` scalar arithmetic. Instead, scalar operations would typically follow widening to `f32` then narrowing to `bf16` consider this example; @@ -364,16 +403,19 @@ let d: bf16 = (f32::from((f32::from(a) + f32::from(b)) as bf16) + f32::from(c)) Where the `as bf16` cast performs the narrowing under round-to-nearest, ties-to-even, as specified above. For an instance where the precision is not enough to perform the operation a suitable library call should be emitted. Such as with the case of `bf16`'s [`FMA` in LLVM][llvm-fma-bug] where casting to an `f64` does not have enough precision to perform a fused-multiply-add without double rounding becoming an issue. #### Narrowing: with native conversion support +[narrowing-with-native-support]: #narrowing-with-native-support Some architectures, including AArch64, x86_64, and RISC-V, provide dedicated instructions for converting to and from `bf16`. These instructions are not interchangeable: their behaviour can differ in rounding mode selection, subnormal handling, NaN handling, and interaction with floating-point control registers. The examples below cover AArch64, x86_64, and RISC-V, but the same considerations apply to any target with native `bf16` conversion instructions. The word "conforming" in this section is used to implicate following the semantics of RNE and not flushing subnormals to zero, following IEEE-754 as described in the previous section of this RFC. ##### x86_64 +[x86-64-narrowing]: #x86-64-narrowing On x86_64 with `AVX512_BF16`, [`VCVTNEPS2BF16`][intel_a64] performs RNE and does not respect or update `MXCSR`. This makes it independent of the dynamic `MXCSR.RC` rounding mode. For subnormals, however, the instruction flushes to zero. Thus, `VCVTNEPS2BF16` under what is proposed in this RFC is not a conforming lowering for an `f32` -> `bf16` narrowing operation that is required to preserve representable `bf16` subnormal results. If this behaviour is required for a program one can use the relevant platform specific intrinsic. Thus this instruction, under the behaviour proposed in this RFC, could not be used. ##### AArch64 +[aarch64-narrowing]: #aarch64-narrowing On AArch64 with the `bf16` architectural extension, narrowing from `f32` to `bf16` can be performed by [`BFCVT`][BFCVT]. In the normal FPCR mode (`FPCR.AH == 0`), the BFCVT-family instructions use the rounding mode selected by `FPCR.RMode` of which`RMode = 0b00` selects RNE. If `FPCR.AH == 1`, the BFCVT-family instructions use RNE regardless of `RMode`. Therefore, for an unconditional RNE specification, `BFCVT` is a conforming lowering only when the compilation/runtime model guarantees that the relevant execution observes RNE, for example because `FPCR.RMode == 0b00` or because `FPCR.AH == 1` gives BFCVT-family instructions RNE semantics regardless of `RMode`. Subnormal preservation is a separate requirement from the rounding mode. AArch64 `BFCVT` is a conforming lowering for this specification only when the execution environment both selects RNE for the conversion and does not enable a mode that flushes the relevant denormal/subnormal results to zero. In the normal FPCR-zero state, this means `FPCR.RMode == 0b00` and `FPCR.FZ == 0`. @@ -382,6 +424,7 @@ On Linux/arm64, a newly exec'd task starts with zeroed user FPSIMD state. The ke Rust assumes `FPCR` remains in a conforming state across the lifetime of a process; code that mutates `FPCR.RMode` or `FPCR.FZ` may observe non-conforming `bf16` results, analogous to existing expectations about `MXCSR` on x86_64 or effects the FPCR register has on `f32`. Thus `BFCVT` conforms to the behaviour proposed in the RFC. ##### RISC-V +[risc-v-narrowing]: #risc-v-narrowing On RISC-V with `Zfbfmin`, narrowing from `f32` to `bf16` is performed by [`FCVT.BF16.S`][RISC_V], which is a standard narrowing floating-point-to-floating-point conversion that rounds according to the `frm` field (either the static rounding mode encoded in the instruction or the dynamic mode in the `frm CSR`). Setting `frm` to `RNE (0b000)` selects round-to-nearest, ties-to-even, making `FCVT.BF16.S` a conforming lowering under this specification when the execution environment selects RNE rounding for the conversion. Static encoding of the rounding mode in the instruction itself avoids dependence on the dynamic `frm CSR` and is the preferred lowering where the compiler can emit it directly. @@ -394,18 +437,23 @@ The three architectures above each provide a hardware narrow from `f32` to `bf16 | RISC-V (`Zfbfmin`) | `FCVT.BF16.S` | Per instruction-encoded RM, or `frm` CSR | Preserved | Yes, when RM/`frm` selects RNE | #### Narrowing: without native conversion support or where hardware does not provide a portable solution +[narrowing-without-native-support]: #narrowing-without-native-support Where no hardware instruction performs an RNE narrow from `f32` to `bf16` or hardware does not provide a portable solution, the narrowing must be emulated. A straightforward RNE implementation inspects bit 15 of the `f32` significand and the sticky bits below it to decide whether to round up, with the usual ties-to-even tiebreak; this is a handful of integer ops. This is the same kind of operation normally provided by soft-float conversion helpers such as `__truncsfbf2`. ## Appendix +[appendix]: #appendix ### `bf16` hardware support +[bf16-hardware-support]: #bf16-hardware-support + On AArch64, `bf16` is supported by Armv8.6-A and later, implementations that include the `bf16` architectural extension, include platforms based on [Neoverse V1][Neoverse V1], [Neoverse N2][Neoverse N2], and [Neoverse V2][Neoverse V2]. On x86_64, `bf16` is supported through `AVX512_BF16` on platforms such as [Intel Cooper Lake][Intel Cooper Lake], [Intel Sapphire Rapids][Intel Sapphire Rapids] and later platforms with `AMX_BF16`, and [AMD Zen 4][AMD Zen 4] and later platforms with `AVX-512 BF16` support. On [RISC-V][RISC_V], `bf16` is supported through the ratified `Zfbfmin`, `Zvfbfmin`, and `Zvfbfwma` extensions, providing scalar conversion, vector conversion, and vector widening multiply-accumulate respectively. ### `bf16` compared to other floating point types Below is a brief introduction as to how floats are represented and a comparison between the representation of a `bf16` in comparison to an `f32`. #### Intro to Floating Point bit representation +[intro-to-floating-point-bit-representation]: #intro-to-floating-point-bit-representation Floating-point types represent real numbers approximately using a fixed number of bits. These bits are divided into three fields: a sign bit, which determines whether the value is positive or negative. Exponent bits, which determine the scale or range of the value and fraction bits, also commonly called mantissa or significand bits, which determine the precision of the value within that range. Below is the bit representation of an `f32`. @@ -418,6 +466,7 @@ Floating-point types represent real numbers approximately using a fixed number o ``` #### Trade-offs between representations +[trade-offs-between-representations]: #trade-offs-between-representations Increasing the number of exponent bits allows a format to represent much larger and much smaller values. Increasing the number of fraction bits allows it to represent values more precisely. Different floating-point formats therefore make different trade-offs between range, precision, storage size, and hardware efficiency. From e491009c23014da8a1770c13c6a0d6d02044d5ef Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 11:22:50 +0100 Subject: [PATCH 05/10] typo & corrected summary --- text/3983-bf16.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 0a2184dc18d..5a30a721e69 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -11,7 +11,7 @@ Add a primitive type, `bf16`, to Rust for the 16-bit **Brain Floating Point** fo `bf16` is a 16-bit floating-point format derived from `f32`. It keeps the same 1-bit sign and 8-bit exponent layout as `f32`, but stores only the top 7 fraction bits. This gives it approximately the same dynamic range as `f32`, but much lower precision. The primary role of `bf16` is representing bfloat16 values in memory, in APIs, in FFI signatures where the target ABI supports bfloat16, and as an element type for SIMD and matrix operations. It is expected to be uncommon for a `bf16` to be used as a general-purpose arithmetic type in the same way as `f32` or `f64`. -`bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for conversions and scalar arithmetic. These are specified as widening to `f32`, performing the corresponding `f32` operation, and narrowing back to `bf16` using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. +`bf16` is not an IEEE-754 binary floating-point format however this RFC specifies the proposed `bf16` type as following portable IEEE-754 semantics for scalar arithmetic and conversions using round-to-nearest, ties-to-even (RNE) without flushing subnormals to zero. As such, implementations may use native `bf16` instructions for scalar arithmetic and scalar conversion only where they produce identical results. Hardware support is present in x86_64, AArch64 and RISC-V to name a few. `bf16` support and implementation varies across architectures, and even similar vector or matrix instructions on the same architecture can differ in their handling of rounding, subnormals, NaNs, and accumulation. Target-specific vendor SIMD and matrix intrinsics expose the semantics of the underlying hardware and are defined as target-dependent in this RFC. @@ -142,7 +142,7 @@ These scalar operations must be implemented in conformance to the widen -> `f32` ### Scalar Arithmetic [reference-scalar-operations]: #reference-scalar-operations -`bf16` scalar operations behave according to the [IEEE-754-2019][ieee-754] specification. The scalar operation will return the value representable in the `bf16` format that is closest to the infinite-precision result with `roundTiesToEven` and preserving subnormals without flushing to zero. This will be true on targets woth with native scalar arithmetic instructions and those without. +`bf16` scalar operations behave according to the [IEEE-754-2019][ieee-754] specification. The scalar operation will return the value representable in the `bf16` format that is closest to the infinite-precision result with `roundTiesToEven` and preserving subnormals without flushing to zero. This will be true on targets with native scalar arithmetic instructions and those without. Please see [footnotes on `bf16` scalar arithmetic](#bf16-scalar) for a more indepth exploration into hardware considerations and support. From c55b204966189fbcb04b6a029aa297efb9d25c41 Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 11:50:27 +0100 Subject: [PATCH 06/10] Remove incorrect scalar semantics --- text/3983-bf16.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 5a30a721e69..30f5c3980c2 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -390,9 +390,6 @@ let d = a + b + c; Which could semantically equate to any of the following; ```rust -// Widen to `f32` first -let d: bf16 = (f32::from(a) + f32::from(b) + f32::from(c)) as bf16; - // bf16 ADD exists let d: bf16 = a + b + c; From 04a5572bb9bab8beeedc07357f19ed4e8d07a52b Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 13:11:25 +0100 Subject: [PATCH 07/10] Significantly trim Reference level vector operations text --- text/3983-bf16.md | 21 +++------------------ 1 file changed, 3 insertions(+), 18 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 30f5c3980c2..0a11fa05e22 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -147,32 +147,17 @@ These scalar operations must be implemented in conformance to the widen -> `f32` Please see [footnotes on `bf16` scalar arithmetic](#bf16-scalar) for a more indepth exploration into hardware considerations and support. ### Vector operations -[reference-vector-operations]: #vector-operations +[reference-vector-operations]: #reference-vector-operations -A first-class `bf16` type is primarily useful for target-specific SIMD and Matrix APIs. These APIs commonly operate on vectors or tiles of `bf16` values and accumulate into `f32`, reflecting the way `bf16` is typically used in machine learning and numerical code. +The types and operations for vectors will be provided by `core::arch`, for the moment. As is usual for these operations their semantics are architecture specific. Where similar instructions between plaftorms should not be expected to produce the same results. -For example, Arm exposes instructions that multiply vectors of `bf16` values and accumulate into `f32` results. x86_64 platforms expose similar functionality through features such as `AVX512_BF16` and, on some Intel platforms, `AMX_BF16`. - -With a primitive `bf16` type, Rust APIs can model these operations using the natural element type of the source operands: +With a primitive `bf16` type, Rust APIs can model vector operations using the natural element type of the source operands: ```rust // Example Neon intrinsic that would be added to `std::arch::aarch64` fn vbfdot_lane_f32(r: float32x2_t, a: bfloat16x4_t, b: bfloat16x4_t) -> float32x2_t; ``` -For intrinsics that live in stdarch that are hardware-specific, some `bf16` instructions can have different numerical semantics across architectures. For example, Arm [`BFDOT`][ARM_ARM] and related `bf16` dot-product instructions operate on `bf16` inputs and accumulate into `f32` results, but their pseudocode specifies architecture-defined intermediate rounding behaviour, and this behaviour can differ further when enhanced `bf16` support is enabled. - -By contrast, x86_64's [`VDPBF16PS`][intel_a64] specifies `bf16` dot products accumulated into `f32` using round-to-nearest ties to even for the FMA accumulation, while also treating input denormals as zero, flushing output denormals to zero, and ignoring `MXCSR`. - -Therefore, target-specific SIMD, dot-product, and matrix APIs should be understood as exposing the hardware operation, not as promising target-independent bit-exact results. Portable Rust APIs may use `bf16` as the element type for clarity and type safety, but portability of the source-level API should not be read as portability of every numerical detail of the underlying target-specific instruction. This is true today with intrinsics for "packed/vector float min" for example; - -``` -x86_64: _mm_min_ps(-0.0, +0.0) -> +0.0 bits 0x00000000 -AArch64: vminq_f32(-0.0, +0.0) -> -0.0 bits 0x80000000 -``` - -While ostensibly similar they produce different results. - ### Display [display]: #display From f756210d6c8c5e4f4fc530b713499197c8eb39d6 Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Wed, 22 Jul 2026 13:22:02 +0100 Subject: [PATCH 08/10] Gramma & spelling tweak --- text/3983-bf16.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 0a11fa05e22..db5c0a31369 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -149,7 +149,7 @@ Please see [footnotes on `bf16` scalar arithmetic](#bf16-scalar) for a more inde ### Vector operations [reference-vector-operations]: #reference-vector-operations -The types and operations for vectors will be provided by `core::arch`, for the moment. As is usual for these operations their semantics are architecture specific. Where similar instructions between plaftorms should not be expected to produce the same results. +The types and operations for vectors will be provided by `core::arch`, for the moment. As is usual for these operations, their semantics are architecture-specific. Where similar instructions between platforms should not be expected to produce the same results. With a primitive `bf16` type, Rust APIs can model vector operations using the natural element type of the source operands: From 07390f16d2c54260bbb00f4300ca17d4cd119b4e Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Thu, 23 Jul 2026 13:20:38 +0100 Subject: [PATCH 09/10] Remove segment on usize, trim guide-level scalar section, trim motivation --- text/3983-bf16.md | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index db5c0a31369..21bfd5ef5a9 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -25,11 +25,9 @@ Thanks to [Folkert de Vries](https://github.com/folkertdev), [Trevor Gross](http `bf16` is useful where memory bandwidth, storage size, and vector throughput matter more than decimal precision. The main use case is machine learning and AI workloads, where values are often stored or multiplied in `bf16` while accumulation happens in wider precision. -Increasingly `bf16` is found in hardware across architectures (see appendix for a few examples) to support these workloads. The `half` crate adds an `f16` and `bf16` as a wrapper over a `u16`. However, a crate-defined type cannot fully substitute for a primitive type. Though its use is demonstrative of users' need for a `bf16` type. +Increasingly `bf16` is found in [hardware across architectures][bf16-hardware-support] to support these workloads. A primitive `bf16` type lets Rust APIs use the intended numeric format directly, while still allowing target-specific operations to expose the behaviour of the underlying hardware. For example x86_64 has instructions like [`VADDBF16`][intel-latest] which prove tricky to make in stdarch without a `bf16` type. As more `bf16` instructions are introduced Rust's support will fall behind. -Consequently, `bf16` is not limited to accelerator hardware; it is a natively supported scalar/vector format on modern CPUs, with availability discoverable through target-feature mechanisms. - -A primitive `bf16` type lets Rust APIs use the intended numeric format directly, while still allowing target-specific operations to expose the behaviour of the underlying hardware. For example x86_64 has instructions like [`VADDBF16`][intel-latest] which prove tricky to make in stdarch without a `bf16` type. As more `bf16` instructions are introduced Rust's support will fall behind. +Some crates, such as `half`, add an `f16` and `bf16` as a wrapper over a `u16`. However, a crate-defined type cannot fully substitute for a primitive type. Though its use is demonstrative of users' need for a `bf16` type. Beyond raw hardware support, `bf16` is already supported in the surrounding ecosystem. A `bf16` a type is provided by [`stdfloat` in C++23][CPP_float_types] with `std::bfloat16_t`, a vendor extension in GCC and Clang (`__bf16`), and a [first-class scalar type in LLVM IR][llvm_bfloat] (`bfloat`). Rust today cannot name this type directly, so FFI signatures, SIMD element types, and codegen all route around the gap with `u16`, wrapper types, and ad-hoc lowering rules (see Current Rust support). A driving motivation for a primitive `bf16` is to close that interop gap with a `bf16` type, in a similar spirit that [RFC 3892][RFC-3892] proposes `core::num::Complex` to match C99 `_Complex` at [the FFI boundary][RFC-3892-motivation]. @@ -65,9 +63,7 @@ For more details, see reference-level explanation. ### Scalar arithmetic [guide-scalar-arithmetic]: #guide-scalar-arithmetic -The main motivation for `bf16` is SIMD and matrix support, not scalar arithmetic. Scalar `bf16` operations are specified portably as widening operands to `f32`, performing the corresponding Rust `f32` operation, then narrowing the result to `bf16` using round-to-nearest, ties-to-even (RNE). Subnormal results are not flushed to zero by default. - -A compelling argument can be made for **not** making scalar arithmetic portable is laid out in the [rationale and alternatives](#scalar-arithmetic-alternatives) section as is a case for not implementing scalar arithmetic at all. Nevertheless this RFC proposes portable semantics. +All the usual scalar arithmetic will be supported, these are specified as following IEEE-754 for providing portable arithmetic both with and without native hardware instructions. A compelling argument for target-dependent arithmetic is laid out in the [rationale and alternatives](#scalar-arithmetic-alternatives) section, so too is an argument for not implementing scalar arithmetic at all. Nevertheless this RFC proposes portable semantics. Implementations may use native `bf16` instructions only when they produce the same observable result. Instructions with different rounding, flushing, semantics remain available through target-specific intrinsics. NaN payload behaviour is not strengthened beyond Rust's existing floating-point guarantees from [RFC-3514][RFC-3514]. @@ -133,6 +129,7 @@ impl bf16 { - `Debug`, `Display`, and the other formatting traits - `Sum`, `Product` - `FromStr` +- `TryFrom` These scalar operations must be implemented in conformance to the widen -> `f32` operations -> narrow using RNE as specified above. @@ -247,9 +244,7 @@ This decision is reversible across an edition. Prelude membership is edition-sco Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic would keep `bf16` simpler and avoid potentially complex or inefficient implementations. Such operations may require widening the value to `f32`, performing the calculation, and then narrowing the result back to `bf16`, which could confuse users or encourage misuse of the type. Instead, users could explicitly convert values to `f32` using `From` or `as`, perform the scalar arithmetic, and then convert the result back to `bf16`. Where supported, vendor-specific intrinsics could also provide more efficient scalar operations. -Rust does already accept target-dependent behaviour in a primitive: `usize` differs in size between 32-bit and 64-bit targets, so identical source an overflow on one target and not another. This could be seen as a precedent for letting `bf16` scalar arithmetic follow whatever the hardware supports, thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which the portable specification would forbid. Ostensibly this should, theoretically, be more performant that converting to `f32` doing the operation and narrowing back to `bf16`. - -The `usize` precedent is weaker than first appears as `usize` is deterministic per target: given a target triple its behaviour is fully specified and stable. Its variation is structural 32-bits vs 64-bits and the type system prevents it being mixed with fixed sized integers. However `bf16` would vary on the same target triple depending on what target features have been made available. This problem is described in the [Prior Art](#c-compiler-support). +Another option would be target-dependent behaviour for scalar arithmetic thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which does not follow IEEE-754 and is not portable. Ostensibly being more performant that converting to `f32` doing the operation and narrowing back to `bf16`. Some portability issues are presently observable in clang when converting from an `f32` to `bf16` which are discussed in [Prior Art c++ compiler support](#c-compiler-support). Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type. @@ -500,7 +495,6 @@ The `bf16` layout is conceptually similar to taking the high 16 bits of an IEEE- +---+-----------------+---------------+ ``` - [Neoverse V1]: https://developer.arm.com/Processors/Neoverse%20V1 [Neoverse N2]: https://developer.arm.com/documentation/102099/0003/The-Neoverse-N2--core/Supported-standards-and-specifications [Neoverse V2]: https://developer.arm.com/community/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/arm-neoverse-v2-platform-best-in-class-cloud-and-ai-ml-performance From 25d6822dfea5c8b42475d466126e625792207824 Mon Sep 17 00:00:00 2001 From: James Barford-Evans Date: Thu, 23 Jul 2026 14:37:59 +0100 Subject: [PATCH 10/10] Add line about target-dependent behaviour remaining accessible through `core::arch` --- text/3983-bf16.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/3983-bf16.md b/text/3983-bf16.md index 21bfd5ef5a9..11f1767f7df 100644 --- a/text/3983-bf16.md +++ b/text/3983-bf16.md @@ -244,7 +244,7 @@ This decision is reversible across an edition. Prelude membership is edition-sco Having a `bf16` type in the global namespace could reasonably set the expectation of having scalar operations baked in. All other globally available primitive numerical types have scalar operations. However omitting scalar arithmetic would keep `bf16` simpler and avoid potentially complex or inefficient implementations. Such operations may require widening the value to `f32`, performing the calculation, and then narrowing the result back to `bf16`, which could confuse users or encourage misuse of the type. Instead, users could explicitly convert values to `f32` using `From` or `as`, perform the scalar arithmetic, and then convert the result back to `bf16`. Where supported, vendor-specific intrinsics could also provide more efficient scalar operations. -Another option would be target-dependent behaviour for scalar arithmetic thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which does not follow IEEE-754 and is not portable. Ostensibly being more performant that converting to `f32` doing the operation and narrowing back to `bf16`. Some portability issues are presently observable in clang when converting from an `f32` to `bf16` which are discussed in [Prior Art c++ compiler support](#c-compiler-support). +Another option would be target-dependent behaviour for scalar arithmetic thus producing plausibly faster code at the cost of portability. For instance the aforementioned `VADDBF16` instruction flushes subnormals to zero, which does not follow IEEE-754 and is not portable. Ostensibly being more performant that converting to `f32` doing the operation and narrowing back to `bf16`. Target-dependent instructions, which are not portable, would remain available through `core::arch`. Some portability issues are presently observable in clang when converting from an `f32` to `bf16` which are discussed in [Prior Art c++ compiler support](#c-compiler-support). Portable scalar arithmetic could prove useful if predictable behaviour is expected. Baking this into the language could lessen the possible footguns from the myriad of disparate, and somewhat confusing, hardware support available. Thus inefficiencies in using this type for scalar arithmetic is deemed an acceptable tradeoff favouring predictable behaviour for a feature that is not anticipated to be the main usecase of the type.