Module fpenv

Module fpenv 

Source
Expand description

IEEE 754 subnormals for the numerical work, whatever the caller’s floating-point mode. treams-rs extension.

The kernels assume gradual underflow. Some callers flush subnormals to zero: XLA sets flush-to-zero and denormals-are-zero on the thread that runs a computation, also while a jax.pure_callback runs, torch.set_flush_denormal(True) sets them on the calling thread, and Linux threads inherit the mode of their creator. Flushing breaks algorithms that scale through the subnormal range: faer’s complex reciprocal of an LU pivot z with max(|Re z|, |Im z|) <= 1 < |z| forms f64::MIN_POSITIVE / |z|^2, so a flushed scalar solve of 1 + 0.1i returns zero.

ieee runs work with subnormals on the calling thread and then restores the caller’s flushing bits; every Python entry point of the bindings runs its whole body in it. flushing does the opposite, for tests. What precedes the body stays in the caller’s mode: PyO3 reads a float32 subnormal as zero on a flushing thread, and float64 values that the Python layer computes from subnormal inputs flush before the call into Rust.

ieee guards the calling thread only. The workers of the thread pool (crate::threads) clear their flushing bits once, as each one starts (keep_subnormals_on_worker), so they keep subnormals whatever the mode of the thread that builds the pool; Linux threads would otherwise inherit their creator’s mode for good. treams-rs never uses Rayon’s global pool, whose workers would keep the mode of the thread that started it.

This is the only module that reads or writes the control register: MXCSR on x86-64 (flush-to-zero, bit 15, and denormals-are-zero, bit 6) and FPCR on AArch64 (FZ, bit 24, which XLA sets, and FIZ, bit 0, defined by FEAT_AFP). On other targets both functions only run their work.

§Soundness

Rust assumes the default floating-point environment everywhere, and its standard library documents changing flush-to-zero or denormals-are-zero as undefined behaviour even when the change is undone (_mm_setcsr in core::arch::x86_64): the compiler does not model the control register, so it may move floating-point arithmetic across the instructions that write it. A flushing caller has already left that assumption when it calls Rust code. ieee cannot formally remove the violation, but it keeps the work between its two register writes by data dependence rather than by code layout. Each write is an asm! block without nomem, readonly or pure, which the Rust Reference treats like a call to an unknown function that may read and write memory, and it takes a pointer operand:

  • The clearing write takes a pointer to the closure, so for the compiler it may read and replace every captured value. Arithmetic on the inputs can only start after it, whether they live in registers or in memory.
  • The restoring write takes a pointer to the result, so for the compiler it reads the result and may replace it. The result is complete before the write, and code after it reads the result back instead of recomputing it.
  • Accesses to memory visible outside the function, such as NumPy buffers, and calls with side effects stay on their side of either write.

This relies on the specified meaning of asm! operands, which no optimization may see through, not on core::hint::black_box, which is documented as best effort only. Operations whose operands are all compile-time constants are not anchored: LLVM usually folds them with IEEE rules, but one that it leaves to run time may run outside the window, so guarded work must not rely on subnormal results of constant expressions. guarded_register_arithmetic_stays_inside fails at opt-level 1 when either pointer operand is removed, and tests/bindings/test_float_environment.py compares every scalar binding and ufunc loop on flushing threads bit for bit with callers that keep subnormals.

Modules§

control 🔒

Structs§

Restore 🔒
Restores the flushing bits a thread had before with_flush changed them.

Functions§

ieee
Run work with IEEE 754 subnormals on the current thread.
keep_subnormals_on_worker 🔒
Clear the current thread’s flushing bits for the rest of its life.
swap_flush 🔒 ⚠
Set the current thread’s flushing bits to flush & FLUSH, keeping every other bit, and return the flushing bits it had. A write, when needed, may read and write *data for the compiler.
with_flush 🔒 ⚠
Run work with the flushing bits flush & FLUSH on the current thread, then restore the caller’s flushing bits, also while unwinding a panic.