Difference between revisions of "User talk:Strife Onizuka/Float Functions"

From Second Life Wiki
Jump to navigation Jump to search
(Three replies in one go)
Line 322: Line 322:


:::I start the denormalized range a power of two early but if it's failing all the way up to -104... that's a big problem. I'll have to play with it. Multiplying by a power of two shouldn't trash the mantissa. -- '''[[User:Strife_Onizuka|Strife]]''' <sup><small>([[User talk:Strife_Onizuka|talk]]|[[Special:Contributions/Strife_Onizuka|contribs]])</small></sup> 20:29, 11 February 2013 (PST)
:::I start the denormalized range a power of two early but if it's failing all the way up to -104... that's a big problem. I'll have to play with it. Multiplying by a power of two shouldn't trash the mantissa. -- '''[[User:Strife_Onizuka|Strife]]''' <sup><small>([[User talk:Strife_Onizuka|talk]]|[[Special:Contributions/Strife_Onizuka|contribs]])</small></sup> 20:29, 11 February 2013 (PST)
::::Maybe I misunderstood. If what you meant is that hex floating point format is unable to produce denormal numbers, then that's exactly the problem, yes. 0x1p-127 produces 0, even if the denormal range can represent numbers down to 0x1p-149. What I meant is that inputs to the function representing exponents of -104 or below failed (as these involved calculation of 0x1p-127 or below). Now some good news. The new function revised to use [[llPow]] works correctly for small exponents and seems to work correctly for the rest. Also, under Mono at least its performance is about 2.5 times that of the old (string operations are s-l-o-w). --[[User:Pedro Oval|Pedro Oval]] 06:24, 12 February 2013 (PST)


== Bug in fui ==
== Bug in fui ==
Line 383: Line 385:
}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.
}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.
</lsl>
</lsl>
::The above fails at every number from the range I tested (mantissa in the vicinity of 0x3FFFFF, exponent -2 and below). --[[User:Pedro Oval|Pedro Oval]] 06:24, 12 February 2013 (PST)


This uses a modified error correction code. I'm not sure if it will be better or worse. It has the potential to run out the high side of the stated bounds on <code>a</code> but I don't think this will be the case. We shall have to see.
This uses a modified error correction code. I'm not sure if it will be better or worse. It has the potential to run out the high side of the stated bounds on <code>a</code> but I don't think this will be the case. We shall have to see.
Line 416: Line 419:
}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.
}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.
</lsl>
</lsl>
::The above passed the test in the range where version 1 failed (vicinity of 0x7fffff for mantissa and exponents up to -2). Also, fui(iuf(i))==i produces no mismatch in a random sample of 2M integers (version 1 produced 2 mismatches in a random sample of 1M). Looks promising. I'm not testing it in LSO though - too slow :) --[[User:Pedro Oval|Pedro Oval]] 06:24, 12 February 2013 (PST)

Revision as of 07:24, 12 February 2013

Ever considered making a fui16?

I was just looking around for how I could union a float into a 16 bit integer, perhaps using the IEEE 754-2008 half-precision format. Yes, I know there would be losses, and I also know that SL olny has 32 bit integers. The final goal for my application is to pack two floats into a single integer, via bitshifting the first into the high range and then ORing with the second, a la: <lsl> // To pack: float a = PI; float b = 4.2; integer result = (fui16(a) << 0x10) | fui16(b);

// To unpack: float c = i16uf(result >> 0x10); // Might need to perform some further magic here to account for the fact that the sign bit doesn't shift. float d = i16uf(result & 0xFFFF); </lsl> Of course, in the general case one can't make assumptions about range and resolution. That said, in my application I can make such assumptions, and I'm just going to hack together something that works based on those assumptions. I'm just here feeding the fetish that resulted in these very-useful-to-me functions (fui, iuf) in the first place. >:D

—The preceding unsigned comment was added by Cron Stardust

Lets see, I could either work on Template:LSL Function et al as to split the notes section in half, or I could write fui16 a most clever function. Hmmm. Fun clever coding or Template drudge. Half precision doesn't have the same range, grrrr... -- Strife (talk|contribs) 20:57, 24 December 2012 (PST)
Upon second though, I really don't want to dig into floating point math at the moment. Mostly you just need to add clipping to fui, change the constants and add infinity support to iuf. With clipping implemented the error correcting code can be removed. Here is my first crack at updating the constants. I've not tested it. -- Strife (talk|contribs) 21:35, 24 December 2012 (PST)

<lsl> integer fui16(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union half-precision float to short integer

   if((a)){//is it non zero?
       integer b = (integer)(a < 0) << 15;//the sign, but later this variable is reused to store the shift
       if((a = llFabs(a)) < 0.0001220703125)//Denormalized range check & last stride of normalized range
           return b | (integer)(a * 16777216.0);//the math overlaps; saves cpu time.
       if(a > 65504.0)//Round up to infinity
           return b | 0x7C00;//Positive or negative infinity
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818)) + 14;//extremes will error towards extremes. following yuch corrects it.
       return (0x3FF & (integer)(a * (0x800 >> b))) | (((c + (b = (integer)(a *= (0.0000152587890625 * (0x40000000 >> c))))) << 10 ) | b);
   }//for grins, detect the sign on zero. it's not pretty but it works. the previous requires a lot of unwinding to understand it.
   return (integer)((string)a == (string)(-0.0)) << 15;

}

float i16uf(integer a) {//union short integer to half-precision float

   if((a & 0x7C00) == 0x7C00)
       return (1 | ((a << 16) >> 31)) * (float)"Infinity";
   return 0.000000059604644775390625 * (1 << (a - !!a)) * ((!!(a = (0x1f & (a >> 10))) << 10) | ((a & 0x3ff))) * (1 | ((a << 16) >> 31));

} </lsl>

<lsl> // To pack: float a = PI; float b = 4.2; integer result = (fui16(a) << 16) | fui16(b);

// To unpack: float c = i16uf(result >> 16); // No magic needed, sign bit is handled correctly. float d = i16uf(result); </lsl>

Just tested in SL with the following results: Had to place a closing paren in the first return statement of iuf16, I placed it just before the mult as I suspect that's what was intended. I wrote a simple ownersay of the existing test code which gave the following as the output: Cron Stardust 16:12, 25 December 2012 (PST)

<lsl>

 a = 3.141593 = 0xC90FDBp-22

= c = 3.140625 = 0xC9p-6

 b = 4.200000 = 0x433333p-20

= d = 4.199219 = 0x433p-8</lsl>

Further testing, looking for error cases, resulted in a lot of good: the functions seem to operate as they should. I tested using all the constants demonstrated on the Wikipedia article and only one failed to match perfectly: 0.0000000596046 doesn't encode to 0x0001, instead it results in 0x0000. All other values seem to work great. Cron Stardust 22:00, 25 December 2012 (PST)
Also tested that with a = -2.0 and b = -2.0 that everything works out: no issues happen with the sign bit status that I could detect - so I took the liberty of changing the comment in your code. Cron Stardust 22:34, 25 December 2012 (PST)
I'm willing to bet it's a lack of precision in their approximation. Try: 0.000000059604644775390625 -- Strife (talk|contribs) 00:36, 26 December 2012 (PST)
I'm not 100% sure of the "llFloor((llLog(a) / 0.6931471...)" line but I think in this range it's accurate and we don't need to do any correction. The only way to know for sure is to try every value in the range [0.0001220703125, 65504.0]. I know it's not going to overflow so I've removed the overflow protection code. If you want to do that testing, here is the return line you would use which assumes that "c" is accurate: return (0x3FF & (integer)((a / (float)("0x.004p"+(string)c)))) | (((c + 15) << 10 ) | b); It has the added bonus of making fui16 LSLEditor safe. This was also a good demonstration of how the layers of correction code were bolted on. -- Strife (talk|contribs) 00:50, 26 December 2012 (PST)

This is a better version of what I was talking about but it needs extensive testing. If c is not accurate this will not work. <lsl> integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818)) + 15;//extremes will error towards extremes. No corrections are attempted. return (0x3FF & (integer)((a * 0.03125 * (0x40000000 >> c)))) | ((c << 10) | b); </lsl> I've gone back and removed my string-float pow code since it can all be done with integers and float math. Faster this way. Less stressful for the sim. -- Strife (talk|contribs) 01:32, 26 December 2012 (PST)

Testing

Just run the following code and it will tell you if c is accurate for the entire half-precision range (this is one of the things iuf is really good for, iterating over every float value). -- Strife (talk|contribs) 02:55, 26 December 2012 (PST) <lsl> integer fui(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union float to integer

   if((a))
   {//is it non zero?
       integer b = (a < 0) << 31;//the sign, but later this variable is reused to store the shift
       if((a = llFabs(a)) < 2.3509887016445750159374730744445e-38)//Denormalized range check & last stride of normalized range
           return b | (integer)(a / 1.4012984643248170709237295832899e-45);//the math overlaps; saves cpu time.
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818));//extremes will error towards extremes. following yuch corrects it.
       return (0x7FFFFF & (integer)(a * (0x1000000 >> b))) | (((c + 126 + (b = ((integer)a - (3 <= (a /= (float)("0x1p"+(string)(c -= ((c >> 31) | 1)))))))) << 23 ) | b);
   }//for grins, detect the sign on zero. it's not pretty but it works. the previous requires a lot of unwinding to understand it.
   return ((string)a == (string)(-0.0)) << 31;

}

float iuf(integer a) {//union integer to float

   return ((float)("0x1p"+(string)((a | !a) - 150))) * ((!!(a = (0xff & (a >> 23))) << 23) | ((a & 0x7fffff))) * (1 | (a >> 31));

}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.

integer test(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union half-precision float to short integer

   if((a))
   {//is it non zero?
       if((a = llFabs(a)) < 0.0001220703125)//Denormalized range check & last stride of normalized range
           return FALSE;//the math overlaps; saves cpu time.
       if(a > 65504.0)//Round up to infinity
           return FALSE;//Positive or negative infinity
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818)) + 14;//extremes will error towards extremes. following yuch corrects it.
       return (integer)(a *= (0.0000152587890625 * (0x40000000 >> c))) - 1;
   }//for grins, detect the sign on zero. it's not pretty but it works. the previous requires a lot of unwinding to understand it.
   return FALSE;

}

float i16uf(integer a) {//union short integer to half-precision float

   if((a & 0x7C00) == 0x7C00)
       return (1 | ((a << 16) >> 31)) * (float)"Infinity";
   return 0.000000059604644775390625 * (1 << (a - !!a)) * ((!!(a = (0x1f & (a >> 10))) << 10) | ((a & 0x3ff))) * (1 | ((a << 16) >> 31));

}

list gBadBlockCounts;

default {

   state_entry()
   {
       llOwnerSay("beginning test");llResetTime();
       
       integer badCount = 0;
       integer badStart;
       
       integer start = fui(i16uf(1) / 2.0);//start just before min value, these will get clipped to zero.
       integer end = fui(65536.0);//stop just after infinity
       integer i;
       for(i = start;i < end; i++)
           if(test(iuf(i)))
           {
               llSetText("bad:  "+(string)i, <1.0, 0.0, 0.0>, 1.0);
               if (badCount++ <= 0)
               {
                   badStart = i;
                   llOwnerSay("Found bad number: " + (string) i);
               }
           }
           else
           {
               llSetText("good: "+(string)i, <0.0, 1.0, 0.0>, 1.0);
               if (badCount > 0)
               {
                   llOwnerSay("Total of " + (string) badCount + " numbers, ending with " + (string) (i-1));
                   gBadBlockCounts = [badStart, i - 1] + gBadBlockCounts;
                   badCount = 0;
               }
           }
       llSetText("test complete.\ntook " + (string) llGetTime() + " secs over " + (string) (end - start) + " values\nsaw " + (string) llGetListLength(gBadBlockCounts) + " bad blocks.\nTouch for list of bad.", <0.0, 1.0, 1.0>, 1.0);
       state printout;
   }
   

}

state printout {

   touch_end(integer index)
   {
       while (--index >= 0)
       {
           if (llGetListLength(gBadBlockCounts))
           {
               llRegionSayTo(llDetectedKey(index), 0,
                   "Bad blocks strided list, using stride pattern (block_start, block_end):"
               );
               llRegionSayTo(llDetectedKey(index), 0,
                   llDumpList2String(gBadBlockCounts, ", ")
               );
           }
           else
           {
               llRegionSayTo(llDetectedKey(index), 0,
                   "Nothing to report."
               );
           }
       }
   }

} </lsl>

Tested this morning. Went bad at values 864026624 through 900508790. Not sure if there were further values: I set it up to run and then wen to work for the day, when I came back it had been yelling for a very long time... :P Will see what I can do, however while I can understand the code, I've not yet been able to get my mind in deep enough to understand the algorithm. Cron Stardust 18:55, 26 December 2012 (PST) EDIT: Just noticed that there was a function named test in there that wasn't used: I'd pasted in the fui16 function from above to make the code compile and run. That may have invalidated the test, so I've going to retest usung the "test" function in place of the fui16 call in state_entry. I've edited the above state_entry to reflect my current code.
Just passed the 864026624 mark without errors, so yes: my previous test was invalid. Continuing testing - just like GLaDOS likes. Cron Stardust 20:59, 26 December 2012 (PST)
I should have thought of this sooner, we don't need to test every value, just the ones who's values surround where the value of "c" changes. We only need to test ummm 84 values... just the powers of two and the immediate lesser values. -- Strife (talk|contribs) 01:44, 27 December 2012 (PST)

<lsl> integer fui(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union float to integer

   if((a))
   {//is it non zero?
       integer b = (integer)(a < 0) << 31;//the sign, but later this variable is reused to store the shift
       if((a = llFabs(a)) < 2.3509887016445750159374730744445e-38)//Denormalized range check & last stride of normalized range
           return b | (integer)(a / 1.4012984643248170709237295832899e-45);//the math overlaps; saves cpu time.
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818));//extremes will error towards extremes. following yuch corrects it.
       a /= (float)("0x1p"+(string)(c -= ((c >> 31) | 1)));
       integer bb = ((integer)a - (3 <= a));
       return (0x7FFFFF & (integer)(a * (0x1000000 >> bb))) | (((c + 126 + bb) << 23 ) | b);
   }//for grins, detect the sign on zero. it's not pretty but it works. the previous requires a lot of unwinding to understand it.
   return (integer)((string)a == (string)(-0.0)) << 31;

}

float iuf(integer a) {//union integer to float

   integer aa = 0xff & (a >> 23);
   return ((float)("0x1p"+(string)((aa | !aa) - 150))) * ((!!aa << 23) | ((a & 0x7fffff))) * (1 | (a >> 31));

}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted.

integer get_log2(float a) {//we use the natural log because testing showed it's more accurate than log10.

   return llFloor((llLog(a) / 0.69314718055994530941723212145818));//must be divide, multiplying the reciprocal adds extra corruption.

}

integer test_log2(integer power) {

   float a = (float)("0x1p"+(string)power);
   float b = iuf(fui(a) - 1);//The underflow is in our favor and does what we want.
   integer c = get_log2(a);
   integer d = get_log2(b);
   integer e = (c == power) && ((d + 1) == power);

// llOwnerSay(llList2CSV([power, c, d, Float2Hex(a), Float2Hex(b), int2hexdword(fui(a)), int2hexdword(fui(b)), e]));

   return e;

}

default {

   state_entry()
   {
       integer start = -25;
       integer end = 17;
       integer i;
       for(i = start; i <= end; i++)
           if(!test_log2(i))
               llOwnerSay((string)i);
   }

} </lsl>

Ok, so executed - though I'm leaving the other script running, and it seems good thus far; will report when it completes. The results from this new test are that I saw values reported in the following ranges: [-25, -16], -14, [-12, -2], [3, 17] -- Cron Stardust 19:14, 27 December 2012 (PST)
Well that is bad. Guess c is not accurate so we can't remove the correction code (the overflow code can still be removed). I wonder what happens if you feed it NaN. Probably need a special case just for that. -- Strife (talk|contribs) 20:45, 27 December 2012 (PST)
I remembered how the correction code works and I fused one of the moving parts. I don't think it's necessary for this range but I'm adding it back in. Since "c" could be off by one in either direction, this gives us three possible outcomes when generating new-a.
  1. It's too small, the result is that new-a is in range [0.5, 1)
  2. C is correct, new-a is in range [1, 2)
  3. It's too big, the result is that new-a is in range [2,4)
Solution? Map these three outcomes to three consecutive values so that they can be used to adjust c. Truncating new-a gives us [0.5, 1)->0, [1, 2)->1, [2,4)->{2,3}. Yes they are consecutive but we need to map the final set to a single value. Hence the subtract by (3 <= a).
It's a shame I never documented which edge cases this code fixed. It would take a lot of the guess work out. -- Strife (talk|contribs) 21:22, 27 December 2012 (PST)
For what it's worth, the old test code finally completed! Took about 6-8 days continuous* calculation in my home region. (* script reported 521479.8 seconds, but I'd had to stop and fix a UI error after several hours of computation, and picked up where I'd left off.) Here's the report: Cron Stardust 18:14, 5 January 2013 (PST)
Bad blocks list, using stride pattern [block_start, block_end]:
[
 [1191182336, 1191182336], [1182793721, 1182793727], [1174405120, 1174405120], [1166016505, 1166016511], 
 [1157627901, 1157627903], [1149239293, 1149239295], [1140850685, 1140850687], [1132462077, 1132462079], 
 [1124073469, 1124073471], [1115684861, 1115684863], [1107296255, 1107296255], [1098907647, 1098907647], 
 [1090519039, 1090519039], [1048575999, 1048575999], [1040187390, 1040187391], [1031798782, 1031798783], 
 [1023410174, 1023410175], [1015021564, 1015021567], [1006632956, 1006632959], [ 998244348,  998244351], 
 [ 989855740,  989855743], [ 981467132,  981467135], [ 973078524,  973078527], [ 964689912,  964689919]
]

fui support for NaN and Inf

I must be one of the few people who have found an actual use to fui :) I'm using it for compact lossless transmission of floats over HTTP to a PHP server in base 64 (after removing the == and the trailing A's), and I decode it in the server using unpack() and some string manipulation. I would love to see fui support NaN, Inf and -Inf, but more for the sake of completion than because I have a real use (except possibly debugging). Maybe it's as straightforward as adding this on top? <lsl>

 if ((string)a == "Infinity") return 0x7F800000;
 if ((string)a == "-Infinity") return 0xFF800000;
 if ((string)a == "NaN") return 0x7FC00000; // to choose one

</lsl> I guess there's no way to extract the actual bits of a NaN, or is there? --Pedro Oval 05:09, 26 December 2012 (PST)

Oops, I've corrected "Inf" to read "Infinity" above. --Pedro Oval 07:58, 26 December 2012 (PST)

Sure, I don't see why not. I'll think about ways of doing it (I'm not sure if successive tests is really the best way of doing so). The reason Infinity was never handled was that LSO doesn't allow it. These functions were originally conceived before Mono was on the horizon (at the time I had wanted to build an LSO bytecode emulator in LSL). -- Strife (talk|contribs) 01:01, 27 December 2012 (PST)
It's done. -- Strife (talk|contribs) 22:43, 27 December 2012 (PST)
You will be pleased (or horrified) to see that Float2Hex and Float2Sci both support NaN and Infinities now. -- Strife (talk|contribs) 00:22, 29 December 2012 (PST)

Alternate code

I was pondering alternate ways to b >> (13 + ((b & 0x3FFF) == 0x2000)) because it's just so ugly.

  • b >> (((0x3FFF & b ^ 0x1FFF) + 0x34001) >> 14) - we flip the low bits, so if all of them are on, that is to say our condition has been met, adding one will cause a carry into the 14th bit.
  • b >> ((0x38000 - (0x3FFF & b ^ 0x2000)) >> 14) - similar to above, we turn the high bit off, so if it's zero it doesn't cause a carry from above.
  • b / ((((0x3FFF & b ^ 0x1FFF) + 0x4001) & 0xC000) >> 1)

It's a shame there isn't a way to further improve it. It's an ugly mess that only gets uglier. ~_~ -- Strife (talk|contribs) 23:22, 28 December 2012 (PST)

Bugs in iuf and fui

Bug in iuf

iuf does not support exponent parts of 2-104 or below. The reason is that in an intermediate step, it is trying to calculate 0x1p-127 or below, which yields 0. For exaple, iuf(0x0bffffff) returns 0, while iuf(0x0c00000) returns the correct result.

Rather than trying to find a fix, I created my own iuf that seems to work, even if it's unoptimized and not very fast (though it contains a small trick to reduce the total number of iterations to a maximum of 25 or so):

<lsl> float iuf(integer e) {

   float result = e & 0x7FFFFF;
   integer sign = e & 0x80000000;
   e = ((e >> 23) & 0xFF) + -150;
   // deal with Inf/NaN
   if (e == 105) { if (result) return (float)"NaN"; if (sign) return (float)"-Inf"; return (float)"Inf"; }
   if (e ^ -150)
   {
       result += 0x800000;
       while (e > 9) { result *= 512; e += -9; }
       while (e < -9) { result *= 0.001953125; e += 9; }
       while (e > 0) { result *= 2; --e; }
       while (e < 0) { result *= 0.5; ++e; }
       if (sign) return -result;
       return result;
   }
   if (sign) return -(result*1e-45);
   return result*1e-45;

} </lsl>

A fix might be along the lines of not applying the -23 offset in advance (as usage of -150 implies) but postmultiplying by 2^-23 instead. I haven't tried though. --Pedro Oval 09:37, 8 February 2013 (PST)


If the denormalized range isn't working then Float2Hex is likely broken too. hmmm -- Strife (talk|contribs) 11:11, 9 February 2013 (PST)
It's not just the denormal range, it affects the normal range too. 2-104 is small but not denormal; the denormal range starts at 2-127. --Pedro Oval 16:22, 9 February 2013 (PST)
I start the denormalized range a power of two early but if it's failing all the way up to -104... that's a big problem. I'll have to play with it. Multiplying by a power of two shouldn't trash the mantissa. -- Strife (talk|contribs) 20:29, 11 February 2013 (PST)
Maybe I misunderstood. If what you meant is that hex floating point format is unable to produce denormal numbers, then that's exactly the problem, yes. 0x1p-127 produces 0, even if the denormal range can represent numbers down to 0x1p-149. What I meant is that inputs to the function representing exponents of -104 or below failed (as these involved calculation of 0x1p-127 or below). Now some good news. The new function revised to use llPow works correctly for small exponents and seems to work correctly for the rest. Also, under Mono at least its performance is about 2.5 times that of the old (string operations are s-l-o-w). --Pedro Oval 06:24, 12 February 2013 (PST)

Bug in fui

Alas, iuf is not the only one with problems. For exponents of -2 or less, and mantissas in the vicinity of 0x7FFFFF, results are almost always wrong. Examples:

  • fui(iuf(0x017fffdd)) = 0x01ffffee (the first one failing with mantissa > 0x7FFF00, exponent is -124)
  • fui(iuf(0x017fffff)) = 0x01ffffff (the last one failing with that exponent; all in the middle also fail)
  • fui(iuf(0x01ffffc5)) = 0x027fffe2 (next one failing)
  • fui(iuf(0x01ffffff)) = 0x027fffff (last one with that exponent, all in the middle fail)
  • fui(iuf(0x037ffffd)) = 0x03fffffe (for this exponent the failure range is shorter - just 3 numbers)
  • fui(iuf(0x38ffffff)) = 0x38ffffff (for a few exponents there are no failures at all; this is an example)
  • fui(iuf(0x3e7fffff)) = 0x3effffff (last one failing)

I've manually checked some of the iuf conversions and they worked correctly with the code I posted above, so the problem is in fui. For example: if (iuf(0x25fffff6) == .44408894e-15) llOwnerSay("iuf ok"); says "iuf ok", but fui(.44408894e-15) gives 0x267ffffb, not 0x25fffff6. --Pedro Oval 11:48, 8 February 2013 (PST)

Are you doing these tests in Mono, LSO or both? -- Strife (talk|contribs) 10:48, 9 February 2013 (PST)
I tested the full range only in Mono, but a quick run seems to show that LSO is affected in exactly the same way. At least the range starts in the same number (0x017fffdd) and produces the same result (0x01ffffee). --Pedro Oval 16:31, 9 February 2013 (PST)
It will take me some time (maybe a few weeks) to get around to this, I'm very busy currently. -- Strife (talk|contribs) 11:14, 9 February 2013 (PST)
I admit to kicking off Pedro's investigation, as I tried to use iuf and fui in Mono and found that fui (iuf (x) ) gave zero for all values of x from 1 to 3 million, at which point when I gave up :) They seem to work a lot better in LSO from my briefer tests so far. Omei Qunhua 15:52, 9 February 2013 (PST)

Try these

I've swapped out the hack pow method for the real function. I'm hoping this will fix iuf.

I've also expanded the compressed lines so that they are easier to tinker with. I've enlarged the supported range of a in the error calculation, the range supported has changed from [0.5, 4) to [0.25, 4).

<lsl> integer fui(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union float to integer

   if((a)){//is it greater than or less than zero?
       integer b = (a < 0) << 31;//the sign, but later this variable is reused to store the shift
       if((a = llFabs(a)) < 2.3509887016445750159374730744445e-38)//Denormalized range check & last stride of normalized range
           return b | (integer)(a / 1.4012984643248170709237295832899e-45);//the math overlaps; saves cpu time.
       if(a > 3.4028234663852885981170418348452e+38)//Round up to infinity
           return b | 0x7F800000;//Positive or negative infinity
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818));//extremes will error towards extremes. following yuch corrects it.
       c -= (c >> 31) | 1;//this sets up an error towards the zero
       a /= llPow(2.0, c);//a should now be in the range [0.25, 4).
       integer d = (integer)a + (0.5 >= a) - (3 <= a);//This calculates the actual error. It does so by mapping the value of a to the error.
       //Specifically we take advantage of the where the normalization bit has been shifted.
       //[0.25, 0.5) == -2, [0.5, 1) == -1, [1,2) == 0, [2, 4) == 1 ~ note the actual values of d are off by two.
       llOwnerSay(llList2CSV([0x800000 == (0xF800000 & (integer)(a * (0x2000000 >> d))), c, d, a, (a < 0.25) || (a > 4)]));//first value tells you if you got d right or wrong.
       //The check works by seeing where the normalization bit has been shifted. The normalization bit gets chopped off with 0x7FFFFF.
       return (0x7FFFFF & (integer)(a * (0x2000000 >> d))) | (((c + 125 + d) << 23 ) | b);
   }//the previous requires a lot of unwinding to understand it.
   if(a == 0)//Just because it's not greater than or less than zero doesn't mean it's non-zero.
       return ((string)a == (string)(-0.0)) << 31;//for grins, detect the sign on zero. it's not pretty but it works.
   //Mono does not support indeterminates so I'm not going to worry about it.
   return 0x7FFFFFFF;//NaN time! We have no way to tell NaN's apart so lets just choose one.

}

float iuf(integer a) {//union integer to float

   if((a & 0x7F800000) == 0x7F800000)
       return (1 | (a >> 31)) * (float)llList2String(["NaN","Infinity"], !(a & 0x7FFFFF));
   integer b = 0xff & (a >> 23);
   return llPow(2.0, (b | !b) - 150) * (((!!b) << 23) | ((a & 0x7fffff))) * (1 | (a >> 31));

}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted. </lsl>

The above fails at every number from the range I tested (mantissa in the vicinity of 0x3FFFFF, exponent -2 and below). --Pedro Oval 06:24, 12 February 2013 (PST)

This uses a modified error correction code. I'm not sure if it will be better or worse. It has the potential to run out the high side of the stated bounds on a but I don't think this will be the case. We shall have to see.

<lsl> integer fui(float a)//Mono Safe, LSO Safe, Doubles Unsupported, LSLEditor Unsafe {//union float to integer

   if((a)){//is it greater than or less than zero?
       integer b = (a < 0) << 31;//the sign, but later this variable is reused to store the shift
       if((a = llFabs(a)) < 2.3509887016445750159374730744445e-38)//Denormalized range check & last stride of normalized range
           return b | (integer)(a / 1.4012984643248170709237295832899e-45);//the math overlaps; saves cpu time.
       if(a > 3.4028234663852885981170418348452e+38)//Round up to infinity
           return b | 0x7F800000;//Positive or negative infinity
       integer c = llFloor((llLog(a) / 0.69314718055994530941723212145818));//extremes will error towards extremes. following yuch corrects it.
       --c;//Make it error down. Hopefully this won't shift anyone into the [4, 8) range.
       a /= llPow(2.0, c);//a should now be in the range [0.5, 4), we assume no error is added.
       integer d = (integer)a - (3 <= a);//This calculates the actual error. As it turns out, the integer portion of a is {0,1,2,3} the errors in c are {-1,0,1,1}
       llOwnerSay(llList2CSV([0x800000 == (0xF800000 & (integer)(a * (0x2000000 >> d))), c, d, a]));
       return (0x7FFFFF & (integer)(a * (0x1000000 >> d))) | (((c + 126 + d) << 23 ) | b);
   }//the previous requires a lot of unwinding to understand it.
   if(a == 0)//Just because it's not greater than or less than zero doesn't mean it's non-zero.
       return ((string)a == (string)(-0.0)) << 31;//for grins, detect the sign on zero. it's not pretty but it works.
   //Mono does not support indeterminates so I'm not going to worry about it.
   return 0x7FFFFFFF;//NaN time! We have no way to tell NaN's apart so lets just choose one.

}

float iuf(integer a) {//union integer to float

   if((a & 0x7F800000) == 0x7F800000)
       return (1 | (a >> 31)) * (float)llList2String(["NaN","Infinity"], !(a & 0x7FFFFF));
   integer b = 0xff & (a >> 23);
   return llPow(2.0, (b | !b) - 150) * (((!!b) << 23) | ((a & 0x7fffff))) * (1 | (a >> 31));

}//will crash if the raw exponent == 0xff; reason for crash deviates from float standard; though a crash is warranted. </lsl>

The above passed the test in the range where version 1 failed (vicinity of 0x7fffff for mantissa and exponents up to -2). Also, fui(iuf(i))==i produces no mismatch in a random sample of 2M integers (version 1 produced 2 mismatches in a random sample of 1M). Looks promising. I'm not testing it in LSO though - too slow :) --Pedro Oval 06:24, 12 February 2013 (PST)