Rounding mode

General discussion about LÖVE, Lua, game development, puns, and unicorns.
User avatar
pgimeno
Party member
Posts: 1942
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Rounding mode

Post by pgimeno » Mon Dec 16, 2019 12:59 pm

Is the rounding mode guaranteed to be round-to-nearest in all platforms?

If not, could the engine please set it explicitly to FE_TONEAREST?

User avatar
raidho36
Party member
Posts: 1992
Joined: Mon Jun 17, 2013 12:00 pm

Re: Rounding mode

Post by raidho36 » Mon Dec 16, 2019 2:32 pm

Are you talking about Lua or GL? In Lua there's no rounding, only flooring and ceiling. In OpenGL it's implementation-defined, but "roundEven" always rounds towards nearest even number on the boundary condition (it's slower though, normal rounding is allowed to go either way for performance reasons specifically).

Floating point math in general is not expected to produce any uniform results across different platforms so unless you pass a constant 0.5 to round function it can go either way just because of least significant bit differences.

User avatar
pgimeno
Party member
Posts: 1942
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Rounding mode

Post by pgimeno » Mon Dec 16, 2019 4:20 pm

raidho36 wrote:
Mon Dec 16, 2019 2:32 pm
Are you talking about Lua or GL? In Lua there's no rounding, only flooring and ceiling.
I'm talking about Lua. Lua floating-point operations are subject to rounding. See e.g. https://stackoverflow.com/questions/183 ... e#58411671

What I'm seeking is for Löve to guarantee that the FP rounding mode is round-to-nearest-or-even, instead of leaving it up to the OS. I don't think any supported OS defaults to a different rounding mode, but it'd be nice to have explicit support from the engine.

raidho36 wrote:
Mon Dec 16, 2019 2:32 pm
Floating point math in general is not expected to produce any uniform results across different platforms so unless you pass a constant 0.5 to round function it can go either way just because of least significant bit differences.
There should be no least significant bit differences at least in +, -, *, / and square root operations when working with normal (i.e. not denormal) numbers, if the rounding mode is the same in all platforms.

User avatar
raidho36
Party member
Posts: 1992
Joined: Mon Jun 17, 2013 12:00 pm

Re: Rounding mode

Post by raidho36 » Mon Dec 16, 2019 5:18 pm

Oh, that. Well AFAIK there's no way to do this from plain Lua because you need to make an OS call, and there's no way to make it fully portable because it's OS-specific, and LuaJIT basically has --ffast-math flag force-enabled so it will use floating point hardware that's not compliant with IEEE because it's faster.

You should be able to call fesetround on Windows and Linux (constants might be different values though), Android has some JAVA interface but the C call might work, no idea about Apple OSes.
pgimeno wrote:
Mon Dec 16, 2019 4:20 pm
There should be no least significant bit differences at least in +, -, *, / and square root operations when working with normal (i.e. not denormal) numbers, if the rounding mode is the same in all platforms.
"Should" is the keyword. There will be subtle differences and you will spend absolute forever hunting them down and you might never arrive at a fully operational solution. You deal with floating point math quirks by writing code to allow these differences, not by brute forcing it into producing exact results. Or by using fixed point math, that never fails.

User avatar
pgimeno
Party member
Posts: 1942
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Rounding mode

Post by pgimeno » Mon Dec 16, 2019 10:03 pm

I just want to be able to rely on the use of a certain rounding mode, nothing else. See the above SO post for an example of its usefulness. I don't want to change the rounding mode from Lua, I'm asking for the engine to enable it.

raidho36 wrote:
Mon Dec 16, 2019 5:18 pm
Oh, that. Well AFAIK there's no way to do this from plain Lua because you need to make an OS call, and there's no way to make it fully portable because it's OS-specific, and LuaJIT basically has --ffast-math flag force-enabled so it will use floating point hardware that's not compliant with IEEE because it's faster.
-ffast-math does many kinds of destructive transformations that LuaJIT does not do. Among other things, it assumes associativity of sums and products, and divides by a constant by pre-calculating the inverse. LuaJIT does neither of these. If addition was considered associative, in the rounding function from the SO post the compiler would probably cancel the constants, and the rounding would not work. Multiplication by the inverse does not always work well; see this thread: https://love2d.org/forums/viewtopic.php?f=5&t=85236 but LuaJIT does not optimize division by 255 to multiplication by the inverse of 255, so if you divide, there's nothing to worry about.

User avatar
raidho36
Party member
Posts: 1992
Joined: Mon Jun 17, 2013 12:00 pm

Re: Rounding mode

Post by raidho36 » Tue Dec 17, 2019 5:24 am

That's why I said "basically", it doesn't have such setting it doesn't do exactly the same thing. But it does uses fast non compliant hardware whenever available; even if you make an explicit os setting it might not honor it anyway.

You say "just rounding mode" but it's the boundary condition, and you might arrive at a different point entirely if the floating point engine works slightly differently, or even if the jit compiler produces different assembly for some random reason. It's not just rounding setting problem, it's a nasty can of worms about the whole floating point math concept.

If you need to mess with OS settings like this you're probably doing something wrong, in all likelihood it is pretty much used just to patch logical errors elsewhere in the code. Having such setting in the engine is probably a bad idea in many many ways; there's a reason plain Lua doesn't have such facilities.

Also if you just want rounding to nearest integer, you can do floor(value + 0.5), no need to screw around with exponents.

User avatar
pgimeno
Party member
Posts: 1942
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Rounding mode

Post by pgimeno » Thu Jan 09, 2020 4:14 pm

raidho36 wrote:
Tue Dec 17, 2019 5:24 am
If you need to mess with OS settings like this you're probably doing something wrong,
Calling a C++ standard function is not "messing with OS settings", it's just "ensuring we're using the correct rounding even if the OS does not have it as a default".

User avatar
wolfboyft
Prole
Posts: 17
Joined: Sat Oct 21, 2017 5:28 pm
Location: London
Contact:

Re: Rounding mode

Post by wolfboyft » Thu Jan 09, 2020 6:34 pm

Round to nearest, ties at even seems most common according to some tests I did with the community. I didn't get back any others. Even so, I would very much appreciate this feature being added. That is to say: bump.
Tachytaenius

User avatar
raidho36
Party member
Posts: 1992
Joined: Mon Jun 17, 2013 12:00 pm

Re: Rounding mode

Post by raidho36 » Thu Jan 09, 2020 6:38 pm

pgimeno wrote:
Thu Jan 09, 2020 4:14 pm
Calling a C++ standard function is not "messing with OS settings"
People who wrote printer drivers to change floating point OS settings had the same idea.

User avatar
pgimeno
Party member
Posts: 1942
Joined: Sun Oct 18, 2015 2:58 pm
Location: Valencia, ES

Re: Rounding mode

Post by pgimeno » Thu Jan 09, 2020 9:04 pm

We're not talking an OS- or driver-level setting, but an application-level [Edit: actually thread-level, see https://en.cppreference.com/w/cpp/numeric/fenv] setting mandated by the C++11 standard. The OS just provides a default; it's up to the application to change it. All I'm asking is that LÖVE does this to guarantee a sane rounding mode. It *might* not be necessary, because probably all modern OSes do this as well, but it's better to be able to rely on it regardless of the OS' default.

Edit2: Apparently, gcc builds already guarantee it: https://www.gnu.org/software/libc/manua ... nding.html

Post Reply

Who is online

Users browsing this forum: No registered users and 29 guests