question

Huajiang avatar image
0 Likes"
Huajiang asked Jason Lightfoot answered

user command precision problem

As shown in the model, after running the user command "aa" for many times, param (1) should be an integer multiple of 0.2, but it is not. Why1.png2.png函数精度问题_1.fsm

FlexSim 22.2.0
buguser commands
1.png (50.6 KiB)
2.png (48.2 KiB)
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

·
Jason Lightfoot avatar image
0 Likes"
Jason Lightfoot answered

Doubles only store up to 16 decimal digits accurately although in practice the last digit often suffers rounding inaccuracy. The local variable window is showing sixteen digits, so you're seeing the affect of the inaccuracy and some rounding on that last digit which accumulates over many calculations and starts affecting the 15th and earlier digits. Most applications don't need that level of precision on large numbers, and those that do probably use a multiple precision arithmetic library to overcome the machine level limitations.

5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.