User:Pedro Oval/Float formatting functions

From Second Life Wiki
< User:Pedro Oval
Revision as of 21:39, 23 January 2015 by Pedro Oval (talk | contribs) (<lsl> to <source>)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

WARNING: WORK IN PROGRESS - NOT READY FOR USE YET

This is a collection of functions for formatting floats with rounding, carefully tuned to get the expected results in most cases, something that other widely available functions often fail to accomplish. None of them will work with more than 6 decimals though.

The variations are for slightly different purposes. Some can be a question of taste; others depend on functionality.

The variants with the dot removed are good for human-readable formatted output, because it's prettier to see "4" than to see "4."; if the output has to include the decimals mandatorily, "4.0" is usually considered prettier than "4.". The variants that don't remove the dot are good e.g. for machine-generated code where the formatting is not important but the type is. (4 is an integer; 4. is a float.) This may save an implicit type conversion at runtime where a float is expected but an integer is given, which just wastes memory and speed unnecessarily. For example, <1, 0, 0> adds 3 type conversions to the code, but <1., 0., 0.> doesn't add any extra burden.

Pick your poison:

Just trim zeros at the end

These functions take just one argument and return at most 6 decimals, with the zeros trimmed and rounded to the 6th decimal. For example, FormatFloat(4.31) returns "4.31" instead of "4.310000".

Dot removed if last character

E.g. FormatFloat(4.0) returns "4".

Faster:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0";
    if (llGetSubString(fs, -6, -1) == "000000" return llDeleteSubString(fs, -7, -1);
    if (llGetSubString(fs, -5, -1) == "00000" return llDeleteSubString(fs, -5, -1);
    if (llGetSubString(fs, -4, -1) == "0000" return llDeleteSubString(fs, -4, -1);
    if (llGetSubString(fs, -3, -1) == "000" return llDeleteSubString(fs, -3, -1);
    if (llGetSubString(fs, -2, -1) == "00" return llDeleteSubString(fs, -2, -1);
    if (llGetSubString(fs, -1, -1) == "0" return llDeleteSubString(fs, -1, -1);
    return fs;
}

Shorter:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0";
    integer i;
    string last;
    do ; while ((last = llGetSubString(fs, i=~-i, i)) == "0");
    return llGetSubString(fs, 0, i - (last == "."));
}

Dot not removed if last character

E.g. FormatFloat(4.0) returns "4.".

Faster:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0.";
    if (llGetSubString(fs, -6, -1) == "000000" return llDeleteSubString(fs, -6, -1);
    if (llGetSubString(fs, -5, -1) == "00000" return llDeleteSubString(fs, -5, -1);
    if (llGetSubString(fs, -4, -1) == "0000" return llDeleteSubString(fs, -4, -1);
    if (llGetSubString(fs, -3, -1) == "000" return llDeleteSubString(fs, -3, -1);
    if (llGetSubString(fs, -2, -1) == "00" return llDeleteSubString(fs, -2, -1);
    if (llGetSubString(fs, -1, -1) == "0" return llDeleteSubString(fs, -1, -1);
    return fs;
}

Shorter:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0.";
    integer i;
    do ; while (llGetSubString(fs, (i=~-i), i) == "0");
    return llGetSubString(fs, 0, i);
}

1 decimal minimum

E.g. FormatFloat(4.0) returns "4.0".

Faster:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0.0";
    if (llGetSubString(fs, -5, -1) == "00000" return llDeleteSubString(fs, -5, -1);
    if (llGetSubString(fs, -4, -1) == "0000" return llDeleteSubString(fs, -4, -1);
    if (llGetSubString(fs, -3, -1) == "000" return llDeleteSubString(fs, -3, -1);
    if (llGetSubString(fs, -2, -1) == "00" return llDeleteSubString(fs, -2, -1);
    if (llGetSubString(fs, -1, -1) == "0" return llDeleteSubString(fs, -1, -1);
    return fs;
}

Shorter:

string FormatFloat(float f)
{
    string fs = (string)f;
    if (fs == "-0.000000") return "0.0";
    integer i;
    do ; while (i > -6 && llGetSubString(fs, (i=~-i), i) == "0");
    return llGetSubString(fs, 0, i);
}

Fixed number of decimals

These functions take two arguments: the float to format and the number of decimals. They perform rounding in the last digit. For example, FormatFloat(4.361262, 4) returns "4.3613" and FormatFloat(4.36, 4) returns "4.3600".

Dot removed if zero decimals

E.g. FormatFloat(4.3, 0) returns "4"; FormatFloat(-4.6, 0) returns "-5".

string FormatFloat(float f, integer num_decimals)
{
    float rounding = (float)(".5e-" + (string)num_decimals) - 0.0000005;
    if (f < 0.) f -= rounding; else f += rounding;
    string ret = llGetSubString((string)f, 0, num_decimals - !num_decimals - 7);
    if ((float)ret == 0. && llGetSubString(ret, 0, 0) == "-")
        ret = llDeleteSubString(ret, 0, 0);
    return ret;
}

Dot not removed if zero decimals

E.g. FormatFloat(4.3, 0) returns "4.". This is the shortest version, recommended if you will never use zero decimals.

string FormatFloat(float f, integer num_decimals)
{
    float rounding = (float)(".5e-" + (string)num_decimals) - 0.0000005;
    if (f < 0.) f -= rounding; else f += rounding;
    string ret = llGetSubString((string)f, 0, num_decimals - 7);
    if ((float)ret == 0. && llGetSubString(ret, 0, 0) == "-")
        ret = llDeleteSubString(ret, 0, 0);
    return ret;
}

Add .0 if zero decimals

E.g. FormatFloat(4.3, 0) returns "4.0". This can be unexpected and surprising for many users, so use it only if you are sure you need something like that.

string FormatFloat(float f, integer num_decimals)
{
    float rounding = (float)(".5e-" + (string)num_decimals) - 0.0000005;
    if (f < 0.0) f -= rounding; else f += rounding;
    string ret = llGetSubString((string)f, 0, num_decimals - 7);
    if ((float)ret == 0. && llGetSubString(ret, 0, 0) == "-")
        ret = llDeleteSubString(ret, 0, 0);
    if (! num_decimals) ret += "0";
    return ret;
}

Fixed number of decimals, trimming trailing zeros

This is an hybrid between both kinds of functions above. It limits the number of decimals output to the given maximum, but also trims zeros at the end, so it's possible to get less decimals than requested (but not more). For example, FormatFloat(4.396, 2) returns "4.4" because 4.396 rounded to two decimals is 4.40 and then the trailing zero is trimmed.

Dot removed if zero decimals

Dot not removed if zero decimals

Add .0 if zero decimals

Test suite for all of the above

The program below exercises some corner cases I could come up with, to ensure that the resuls are as expected. Many other float formatting functions fail some of these tests.