Incorrect calculation values

Question

question

Daniel Pavan asked Jul 11 2016 at 6:35 PM Ralf Gruber commented Mar 20 2019 at 10:50 AM

Incorrect calculation values

Please run the following code in a script window with the output window open:

int test1 = 138;
 
int test2 = 50;
 
double test3 = test1/test2;
 
pr();pt(\"dp number testx\");pt(numtostring(test3, 54, 50));
 
return test3;

The answer provided in the script window appears to be correct (2.7600) but the answer shown in the output windows shows the calulated value(2.75999999999999980). Why is the actual calculated value slightly wrong?

Software Version:

FlexSim (other - please specify)

Software Version:

FlexSim (other)

calculation decimal precision comparison floating point accuracy

______

Cookie preferences

Your privacy is important to us and so is an optimal experience. To help us customize information and build applications, we collect data about your use of this site.

May we collect and use your data?

Learn more about the Third Party Services we use and our Privacy Statement.

Strictly necessary – required for our site to work and to provide services to you

These cookies allow us to record your preferences or login information, respond to your requests or fulfill items in your shopping cart.

YES

Improve your experience – allows us to show you what is relevant to you

These cookies enable us to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we use to deliver information and experiences tailored to you. If you do not allow these cookies, some or all of these services may not be available for you.

YES

NO

Customize your advertising – permits us to offer targeted advertising to you

These cookies collect data about you based on your activities and interests in order to show you relevant ads and to track effectiveness. By collecting this data, the ads you see will be more tailored to your interests. If you do not allow these cookies, you will experience less targeted advertising.

YES

NO

Are you sure you want a less customized experience?

We can access your data only if you select "yes" for the categories on the previous screen. This lets us tailor our marketing so that it's more relevant for you. You can change your settings at any time by visiting our privacy statement

Your experience. Your choice.

We care about your privacy. The data we collect helps us understand how you use our products, what information you might be interested in, and what we can improve to make your engagement with Autodesk more rewarding.

May we collect and use your data to tailor your experience?

Explore the benefits of a customized experience by managing your privacy settings for this site or visit our Privacy Statement to learn more about your options.

Answer 1 · 2016-07-11T18:37:08Z

Phil BoBo answered Jul 11 2016 at 6:37 PM Ralf Gruber commented Mar 20 2019 at 10:50 AM

Danny,

That is the nature of how computers handle decimal precision values, particularly how C++ handles a division operation. There are lots of complicated explanations and papers about why it is done this way.

A relatively short explanation can be found on wikipedia: http://en.wikipedia.org/wiki/Floating_point

There is a section entitled "Accuracy problems" near the bottom of that article that explains it a bit.

Usually it doesn't matter that the precision is slightly off, but sometimes it messes with your logic when doing comparison operations as shown below.

The main issue is when you think it is going to get into one of the simple cases (==, <=, >=, <, >, or !) but it doesn't. There are a few things you can do to see if it is "close enough" though to avoid such precision comparison issues. I've included some of those in the following code as well. It happens that your example actually does work for the == test although ofttimes such division calculations wouldn't. You have to be careful in C++ with == when you are comparing to a double value that has been assigned based on a division or multiplication operation.

int test1 = 138.0;
int test2 = 50.0;
double test3 = test1/test2;
 
pt("dp number testx");pt(numtostring(test3, 54, 50));
pr();
if(test3 == 2.76)
{
 pt("exactly 2.76");
 pr();
}
if(fabs(test3 - 2.76) < 0.0001)
{
 pt("close enough to equal 2.76");
 pr();
}
if(test3 <= 2.76)
{
 pt("<= 2.76");
 pr();
}
if(test3-0.0001 <= 2.76)
{
 pt("close enough to <= 2.76");
 pr();
}
if(test3 >= 2.76)
{
 pt(">= 2.76");
 pr();
}
if(test3+0.0001 >= 2.76)
{
 pt("close enough to >= 2.76");
 pr();
}
return test3;

I hope this helps. Good luck.

· 14

Mischa Spelt commented · Jul 15 2016 at 7:43 AM

fabs(test3)-2.76

Did you mean:

fabs(test3-2.76)

?

0 ·

Phil BoBo ♦♦ Mischa Spelt commented · Jul 15 2016 at 12:57 PM

Yes, thanks. I've updated it.

0 ·

christoph gruber commented · Mar 14 2019 at 3:15 PM

@phil.bobo

Need a solution for my problem with pulling values from internal list near 0.00001.

Before I pull a request number from a internal list I would like to ask for the labelvalue on the list.

For Gobal List I do this with a Tablequery for example:

Table T_result = Table.query("Select * From internalList WHERE Material == 'Tape'");

double value = T_result[1][x];

double request;

if ( request < 0.0000001)

{request = value;}

return request;

But how can I do this for internal lists because Tablequery is not possible?

Thanks for you help!

0 ·

Phil BoBo ♦♦ christoph gruber commented · Mar 14 2019 at 3:23 PM

Using Pull From List, pass 0 for the request number and require number if you want to query a list without removing any entries.

See Pull From List: Request Number and Require Number

"If the request number is zero, then the pull operation will become a mere querying operation. The pull will retrieve all objects on the list that meet the query filter, but it will not remove any values from the list."

"If the require number is zero then the activity will always finish immediately, returning the amount that it successfully pulled. In this case no back order will be created."

--

Also, the code you posted doesn't make sense. You aren't initializing the request variable so it will always be 0 and always be < 0.0000001. All that code snippet is doing is returning T_result[1][x].

0 ·

christoph gruber commented · Mar 14 2019 at 8:56 PM

c-problem-0-example.fsm

@phil.bobo

sorry, my english is not so good, I thing I have not described my problem in a good way.

enclosed you will find a examplemodel, maybe my problem will be a little bit clearly. With the global List the model runs good. If you change the global List with the internel List the Model stoped at the last pallet!

0 ·

c-problem-0-example.fsm (24.8 KiB)

christoph gruber commented · Mar 15 2019 at 5:48 AM

@phil.bobo

I do this in the second example (using Internal List without Tablequery). You will see that the token at the last run (number 5) stocks in the Pull from List activity and not in the End of Production activity. That´s my problem. I have a lot of internal list like this exampl in my real model (round about 350 pull activity in my model).

c-problem-0-example-intlist.fsm

0 ·

c-problem-0-example-intlist.fsm (24.6 KiB)

Ralf Gruber ♦ commented · Mar 18 2019 at 2:10 PM

@jordan.johnson

I think what @christoph gruber is actually looking for has already been asked for in this thread:

https://answers.flexsim.com/questions/57299/sql-query-on-internal-list.html

Looking at the Release Notes, this has not yet been addressed, is that correct?

Thanks

Ralf

0 ·

Jordan Johnson ♦♦ Ralf Gruber ♦ commented · Mar 18 2019 at 2:24 PM

That is correct, we have not added that feature yet.

0 ·

Show more comments

question