[Cfp-interest 2711] Re: Floating point rounding modes

[Hans Boehm]

On Sun, Feb 26, 2023 at 9:39 PM Jim Thomas <jaswthomas at sbcglobal.net> wrote:

> This is an unfortunately late response to the discussion in the CFP
> thread, “Floating point rounding modes”.
>
> Below are thoughts about the 7 bullets in the Introduction in
> https://isocpp.org/files/papers/P2746R0.pdf, and other comments about the
> proposal and issues discussed in the thread.
>
> - Jim Thomas
>
> Introduction
>
> Currently floating point rounding modes are specified either dynamically,
> through the floating-point environment via fesetround() or, in C,
> statically, via #pragma STDC FENV_ROUND. As far as we can tell,
> fesetround(), in spite of its long history, has very few real use cases,
> and those are either not fully correct, or not portable across common
> implementations. There are many reasons for this:
>
>  1. Implementations can't be counted on to implement it correctly.
> Without the FENV_ACCESS pragma, compilers perform optimizations that do not
> preserve rounding behavior. *That fenv.h pragma is explicitly not
> required to be supported in C++, and hence C++ provides no guarantee that
> fesetround() behaves reasonably.*
>
>
> Some compilers also perform optimizations that don’t honor other language
> semantics (e.g. they reassociate, replace division by multiplication,
> distribute * over +, skip conversions). There’s a general problem of
> aligning optimizations and strict floating-point semantics.
>
>
> 2. In practice, it seems to be inconsistent, and somewhat unpredictable,
> what calls to standard math functions, and even more so, user-defined
> functions, do when invoked with non-standard rounding modes. (This is based
> on looking at some implementations. I didn't study this systematically)
>
>
> C (even with Annex F support for IEC 60559) does not require all standard
> math function to honor rounding directions. This reflects history and the
> state of the art. The correctly rounded math functions for which C reserves
> cr_ prefix names are required to honor rounding directions. Regarding
> user-defined functions, see 7.6 #4 (in e.g. C23 draft N3054).
>
>
> 3. If you really need to control rounding to guarantee specific properties
> of the result, rounding modes cannot just be specified for a region of
> code; they need to be carefully applied to each operation. (What does it
> mean to set the rounding mode for cos(a + b)?)
>
>
> This isn’t generally true. For example, an accumulated sum of squares can
> be computed with just one specification of the rounding direction.
>
Thanks! I'll tone this down in the paper. It does make sense when every
operation in the block is monotone increasing in every argument, or when
justified by a careful analysis. But those both strike me as somewhat rare
cases. It still seems like an odd design to me to effectively subset the
language inside the rounding scope

>
>
> 4. As Jim Thomas points out, you could try to get some idea of floating
> point errors by trying with different rounding modes. But, by the same
> argument as (3), that seems to be at best a little better than randomly
> perturbing the results. And the whole approach can break either because the
> program logic relies on rounding mode, or if the code itself sets the
> rounding mode.
>
>
> The technique I mentioned, and alternatives, are discussed at length by W
> Kahan in http://people.eecs.berkeley.edu/~wkahan/Mindless.pdf, which
> concludes
>
>
>
> “At present only the second of the five schemes explored in these notes
> offers an economical way to diagnose anomalies caused by roundoff in
> precompiled software: Rerun the suspected module with exactly the same
> input but with default roundings (those not already directed by the author
> of the module) redirected. Though far from foolproof, this scheme has
> worked on over-zealous optimization in §3, on intermediate iterates xn in
> §5, on T(§6), on MATLAB’s log2(§7), acosh(§8) and acos(§8), on Gaussian
> Elimination (§9), on step H(§10) , on subtended angle ψ(§11), on
> subspace(§12) and on innumerable other examples upon which nothing else so
> inexpensive could possibly have worked so well.”
>
>
> The quote is in the context of debuggers, but the preferred technique is
> available to anyone who has the code they are debugging, provided dynamic
> rounding modes are available.
>
Thanks for the reference. Added. AFAICT, this still fails completely for an
expression as simple as (x - y) - (y - x), which I think should produce the
same result with FE_UPWARD and FE_DOWNWARD, Which is of course consistent
with Kahan's statement.

>
> 5. Compilers seem to commonly ignore rounding modes for compile-time
> evaluations, making it very hard for the programmer to predict what they
> are actually getting. (This is my reading anyway, though the current C
> standard seems to say otherwise?)
>
>
> The intention in C (with Annex F) is that compile-time arithmetic is fully
> specified. Annex F supports IEC 60559. The general C specification for
> floating point gives great flexibility to the implementation, largely to
> avoid breaking existing (including non portable) code.
>
>
> 6. If we really wanted to use rounding modes to bound results, constants
> would also have to be rounded according to the current rounding mode. This
> is explicitly disallowed, even by C: "All floating constants of the same
> source form(79) shall convert to the same internal format with the same
> value.”
>
>
> This quote was a holdover from before static rounding modes (via rounding
> pragmas) were introduced for C23. It has been changed to reflect that
> static rounding modes do affect constants.
>
Yes, And that was already corrected in my presentation to WG21.

>
> 7. Especially in C++, it is unclear what the rounding modes mean for
> operations that are not correctly rounded to start with. Although IEEE
> requires correct rounding, basically no standard library implementations
> conform.
>
>
> Right. But I don’t think this is relevant. The same situation will exist
> unless or until practical correctly-rounded versions of all the math
> functions are implemented and adopted—regardless of how rounding modes are
> supported. Quality implementations endeavor to honor (in one sense or
> another) the direction of directed rounding modes, along with other
> properties not required by C, like monotonicity, symmetry, and nearly
> correct rounding.
>
Maybe. Except there seems to be little agreement on what properties matter.
A recent WG21 paper argued that the trig functions should be accurate for
an input value near the supplied argument, which is entirely different from
the IEEE standard. But I suspect a number of implementations have followed
that rule. And I have no clue what rounding mode means in this case, since
the acceptable answers for sin(x), for many x, are anything in [-1,1].

>
> General comments
>
>
> a. Would all the proposed functions raise floating-point exceptions (as
> required by IEC 60559)? Could the current C features for
> test/clear/get/set exceptions be used? That would seem to defeat the goal
> of context independence. How would an over-aggressive optimizer be
> prevented from moving the function calls across accesses to the exception
> flags? With the current approach, if FENV_ACCESS is not “on”, optimizers
> are free to move the calls and do other transformations without regard to
> the floating-point environment, and even use faster versions of the
> functions that don’t raise exception “correctly”. A functions approach
> would need to include a way of managing exceptions, or say exceptions are
> not supported.
>
I would assume that the behavior in C would be similar to the proposed cr_
functions. Since FENC_ACCESS isn't really supported in C++, I think that'd
officially be less of an issue there. But this clearly needs to be worked
out if we want cross-language support.

>
> b. The functions approach wouldn't satisfy the IEC 60559 requirement for
> static rounding attributes, which apply to a program block. An IEEE 754
> revision committee seems like an important forum for evaluating such a
> proposal. Note that CFP's role is to fit IEC 60559 as-is into C.
>
That's clearly a valid point, though I don't think one that would have much
weight in WG21. And I'm personally unconvinced it's the right approach., It
seems to me that IEC 60559s role should be to specify how the arithmetic
works. But how to specify rounding modes seems to me like it should be a
language decision.  Has IEC 60559 ever really considered the implications
here? It seems to me that historically this was more of a hardware-driven
decision?

>
> c. Another consideration is performance. It would depend on how rounding
> modes are handled by the underlying hardware. If only dynamic rounding is
> supported in hardware, then how would the functions be implemented?
> Presumably, at some level with save/set/restore dynamic rounding mode.
> There’d still the performance overhead of dynamic modes plus a function
> call, for every operation. Inlining might allow some optimization but the
> functions approach is conceived to avoid bad optimization which inlining
> might facilitate.
>
At least in the case of static rounding modes, compilers should be able to
basically generate the same code for equivalent source code, I think.

>
> d. Part of the email discussion was about how code might look when written
> with the proposed functions vs with C23 static modes. For another example, how
> would the functions approach look for
>
> float z, x;
> {
> #pragma STDC FENV_ROUND FE_TOWARDZERO
> z = x + 0.1;
> }
>
> This involves a translation-time evaluation of 0.1, a conversion of x to
> double, an add, and a conversion to float.
>
That's not fully clear yet. Probably something like

z = cr_add(x, cr_const("0.1", FE_TOWARD_ZERO), FE_TOWARD_ZERO);

I'm not convinced this expression actually does anything useful (consider x
== -0.2), and I think the double explicit rounding modes make that clearer.

>
> At this point, the proposal seems more for a direction than for a specific
> feature. It’s hard to really evaluate how it would be to write and read
> code without a more specific interface and details about what is or is not
> allowed.
>
> e. How would the functions approach fit with the various evaluation
> methods C allows? Would there be functions to evaluate to wider formats?  IEC
> 60559 recommends (block scoped) preferredWidthFormat attributes which
> specify the evaluation method (in C terminology). These are not supported
> in C23, but are in IEC TS 18661-5.
>
> f. In [Cfp-interest 2638] is
>
> The potential advantage of the functional notation is that it's much less
>
> intrusive on the implementation, though in C that argument may apply mostly
>
> to the version of the proposal that does not introduce a new type. We're
>
> potentially just adding some library code.
>
> AFAICT, a significant problem with the current and proposed facilities is
>
> that they are so rarely used that implementations commonly don't pay enough
>
> attention to them. I think many implementations don't properly handle
>
> FENV_ACCESS correctly, so I have relatively little confidence in
>
> FENV_ROUND, especially since I think we agree it still doesn't handle
>
> constants. I also have my doubts about FENV_ROUND in C++. I think there is
>
> currently not even a proposal to add it there?
>
>
> Yes, the functions are less intrusive on the compiler implementation,
> assuming no new types.
>
>
> Partly it’s a “Which comes first: the chicken or the egg?” problem.
> Implementations don’t invest in features that aren’t used, and users don’t
> use features that aren’t well implement. But I think there are other reason
> too, maybe for discussion at another time.
>
>
> As noted above, FENV_ROUND does handle constants.
>
>
> g. I think it’s difficult to assess how much a standard feature is used. There
> have been statements like, AFAICT feature x is not implement/used. Do these
> assessments cover the range of C compilers and users (proprietary,
> university, etc.)? The proprietary compiler I last worked with (as a
> numerics engineer) had an excellent implementation of the C features for
> the floating-point environment.
>
I think there is universal agreement that more data would be welcome. The
code base I have access to seems to make very little use of any of this.
But it's also almost all C++, and no C23.

Hans

>
> _______________________________________________
> Cfp-interest mailing list
> Cfp-interest at oakapple.net
> http://mailman.oakapple.net/mailman/listinfo/cfp-interest
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.oakapple.net/pipermail/cfp-interest/attachments/20230227/865df819/attachment-0001.htm>