Expand description
IEEE 754 subnormals for the numerical work, whatever the caller’s floating-point mode. treams-rs extension.
The kernels assume gradual underflow. Some callers flush subnormals to
zero: XLA sets flush-to-zero and denormals-are-zero on the thread that runs a
computation, also while a jax.pure_callback runs,
torch.set_flush_denormal(True) sets them on the calling thread, and Linux
threads inherit the mode of their creator. Flushing breaks algorithms that
scale through the subnormal range: faer’s complex reciprocal of an LU pivot
z with max(|Re z|, |Im z|) <= 1 < |z| forms f64::MIN_POSITIVE / |z|^2,
so a flushed scalar solve of 1 + 0.1i returns zero.
ieee runs work with subnormals on the calling thread and then restores
the caller’s flushing bits; every Python entry point of the bindings runs its
whole body in it. flushing does the opposite, for tests. What precedes
the body stays in the caller’s mode: PyO3 reads a float32 subnormal as
zero on a flushing thread, and float64 values that the Python layer
computes from subnormal inputs flush before the call into Rust.
ieee guards the calling thread only. The workers of the thread pool
(crate::threads) clear their flushing bits once, as each one starts
(keep_subnormals_on_worker), so they keep subnormals whatever the mode of
the thread that builds the pool; Linux threads would otherwise inherit their
creator’s mode for good. treams-rs never uses Rayon’s global pool, whose
workers would keep the mode of the thread that started it.
This is the only module that reads or writes the control register: MXCSR
on x86-64 (flush-to-zero, bit 15, and denormals-are-zero, bit 6) and FPCR on
AArch64 (FZ, bit 24, which XLA sets, and FIZ, bit 0, defined by
FEAT_AFP). On other targets both functions only run their work.
§Soundness
Rust assumes the default floating-point environment everywhere, and its
standard library documents changing flush-to-zero or denormals-are-zero as
undefined behaviour even when the change is undone (_mm_setcsr in
core::arch::x86_64): the compiler does not model the control register, so
it may move floating-point arithmetic across the instructions that write it.
A flushing caller has already left that assumption when it calls Rust
code. ieee cannot formally remove the violation, but it keeps the work
between its two register writes by data dependence rather than by code
layout. Each write is an asm! block without nomem, readonly or pure,
which the Rust Reference treats like a call to an unknown function that may
read and write memory, and it takes a pointer operand:
- The clearing write takes a pointer to the closure, so for the compiler it may read and replace every captured value. Arithmetic on the inputs can only start after it, whether they live in registers or in memory.
- The restoring write takes a pointer to the result, so for the compiler it reads the result and may replace it. The result is complete before the write, and code after it reads the result back instead of recomputing it.
- Accesses to memory visible outside the function, such as
NumPybuffers, and calls with side effects stay on their side of either write.
This relies on the specified meaning of asm! operands, which no
optimization may see through, not on core::hint::black_box, which is
documented as best effort only. Operations whose operands are all
compile-time constants are not anchored: LLVM usually folds them with IEEE
rules, but one that it leaves to run time may run outside the window, so
guarded work must not rely on subnormal results of constant expressions.
guarded_register_arithmetic_stays_inside fails at opt-level 1 when either
pointer operand is removed, and tests/bindings/test_float_environment.py
compares every scalar binding and ufunc loop on flushing threads bit for bit
with callers that keep subnormals.
Modules§
- control 🔒
Structs§
- Restore 🔒
- Restores the flushing bits a thread had before
with_flushchanged them.
Functions§
- ieee
- Run
workwith IEEE 754 subnormals on the current thread. - keep_
subnormals_ 🔒on_ worker - Clear the current thread’s flushing bits for the rest of its life.
- swap_
flush 🔒 ⚠ - Set the current thread’s flushing bits to
flush & FLUSH, keeping every other bit, and return the flushing bits it had. A write, when needed, may read and write*datafor the compiler. - with_
flush 🔒 ⚠ - Run
workwith the flushing bitsflush & FLUSHon the current thread, then restore the caller’s flushing bits, also while unwinding a panic.