Why does the query behave differently depending on number size

Question

question

Axel Kohonen asked Mar 8, '19 Axel Kohonen edited Mar 14, '19

Why does the query behave differently depending on number size

Hi,

If I enter a double type number into an array cell in a global table the formatting is different above a certain limit.I have not yet figured out what the limit is, but 500000000 is below it and when I multiply that by five, then it is above the limit.Above the limit a dot and following zeroes are added into the array. I guess it has something to do with how the number is stored after a certain limit.

The problem is that when I do a query on the table I have to include the dot and the following zeroes to find the value if it is above the limit, but when below the limit it does not matter if I include the dot and the zeroes. It seems weird that I would need to include the dot and following zeroes when querying for a number, but I guess it has to do with it that the query uses strings. So I need to do string.fromNum(number, 1) where one is the precision I need to find the value.

See the attached model for a showcase of the problem. The testing code is in the script window
numbersinarraychangehowtheyshowabovelimit.fsm

Kind regards,

Axel

Software Version:

FlexSim 18.2.3

query double string.fromnum

numbersinarraychangehowtheyshowabovelimit.fsm (17.9 KiB)

Answer 1 · 2019-03-12T09:28:16Z

martin.j answered Mar 12, '19 Axel Kohonen edited Mar 14, '19

Hi Axel,

This appears to have been resolved in Flexsim 19.02. Since I cant reproduce the result if update your model to that version.

If I open it in 18.2.3 you are correct. I would however prefer to construct my queries differently:

Rather than create a whole query string and add the "aboveLimit" value converted to string, I suggest that you use the $1 syntax to the query.

string queryString = "SELECT * FROM [GlobalTable1] WHERE $1 IN testColumn";
Table result = Table.query(queryString, AboveLimit);

I believe this is the intended way to construct queries with variables, and if you do it like this you issue wont occur.

The fact that the issue apperas when you convert your variable, suggests that issue lies in the query's ability to recognize that 1.00 = 1. This could also be a rounding error caused by the non infinite memory size of a floating point value.

Apparently that has been resolved in later versions of Flexsim, event though the very large numbers still have a .00 behind them.

Best regards
Martin
FlexSim Nordic

· 1

Axel Kohonen commented · Mar 12, 2019 at 09:32 AM

HI @martin.jensen

Thank you for the reply! Good to know. That makes sense. Might use this in the future. It just seemed an easy way to debug the query when one can see all of it in the string.

The above approach cannot be used with list.pull though, as the list.pull syntax does not allow for parameters. But it should work with queries e.g. in calculated tables like SELECT * FROM listName WHERE...

Axel

0 ·

question