How to make sure error of different replication results are small enough?

Question

question

shanice.c asked Oct 04 2021 at 12:28 AM Ryan Clark commented Oct 05 2021 at 5:48 PM

How to make sure error of different replication results are small enough?

I've gone through the manual, the Experimenter is a tool that enables you to run the same simulation model multiple times, changing one or more parameters each time to see its effect on the performance measures.

But I'd like to know what if I don't need to change any variables(only 1 scenario), I simply want to run a model several times(ex: 10 replications) and then want to know the error between these 10 replications to make sure the error is small enough to prove the simulation result is strong enough to reflect the real world system? I'm having the question because generally or academically, we couldn't use the result of 1 replication to conclude to simulation result right? So I would like to know what tools or modules I could use to do I've said above in FlexSim. Thank you.

Software Version:

FlexSim 21.2.0

flexsim experimenter

· 1

______

Cookie preferences

Your privacy is important to us and so is an optimal experience. To help us customize information and build applications, we collect data about your use of this site.

May we collect and use your data?

Learn more about the Third Party Services we use and our Privacy Statement.

Strictly necessary – required for our site to work and to provide services to you

These cookies allow us to record your preferences or login information, respond to your requests or fulfill items in your shopping cart.

YES

Improve your experience – allows us to show you what is relevant to you

These cookies enable us to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we use to deliver information and experiences tailored to you. If you do not allow these cookies, some or all of these services may not be available for you.

YES

NO

Customize your advertising – permits us to offer targeted advertising to you

These cookies collect data about you based on your activities and interests in order to show you relevant ads and to track effectiveness. By collecting this data, the ads you see will be more tailored to your interests. If you do not allow these cookies, you will experience less targeted advertising.

YES

NO

Are you sure you want a less customized experience?

We can access your data only if you select "yes" for the categories on the previous screen. This lets us tailor our marketing so that it's more relevant for you. You can change your settings at any time by visiting our privacy statement

Your experience. Your choice.

We care about your privacy. The data we collect helps us understand how you use our products, what information you might be interested in, and what we can improve to make your engagement with Autodesk more rewarding.

May we collect and use your data to tailor your experience?

Explore the benefits of a customized experience by managing your privacy settings for this site or visit our Privacy Statement to learn more about your options.

Answer 1 · 2021-10-04T00:51:06Z

David Seo answered Oct 04 2021 at 12:51 AM

@Fiona C

What do you mean exactly about 'error' during run FlexSim (simulation in general)?

For example, the result of each simulation is different from each or the tiny error during running like warning message? The model suould not have any error message in the model logic.

Your 'result error' means 'distribution of the result value' after running?

If so, it means 'distribution of the result value'.

If your current model result is exactly same with each running, make 'repeat random streams' uncheck. The option is located in 'Statistics>Repeat Random Streams'.

The option makes the result value distributed.

2021-10-04-094943.jpg (27.6 KiB)

· 3

shanice.c commented · Oct 04 2021 at 7:27 AM

@David Seo If all processing time and flow item arrival time are constant, even if unchecked the"repeat random streams", the 10 replications result would have no difference?

0 ·

David Seo shanice.c commented · Oct 04 2021 at 9:13 AM

@Fiona C Yes. If you set all processing time and arrival time to be constant, the results of the repeated replications are same. No difference. If the input value is contant and then the result is constant not deviation.

0 ·

shanice.c David Seo commented · Oct 04 2021 at 9:42 AM

@David Seo Thanks for the reply. I'll keep this in mind.

0 ·

Answer 2 · 2021-10-04T00:44:23Z

Joerg Vogel answered Oct 04 2021 at 12:44 AM

Your thoughts are correct. And you can do it exactly as you describe it. You let run your model several times in the experimenter.

· 3

shanice.c commented · Oct 04 2021 at 7:24 AM

Hi,@Joerg Vogel I think what I want is the KPI like picture below(with average, standard deviation, Min, Max).

In order to gain this, is it necessary to use statistics collector before, so that I can use calculated table to receive KPI I want, and then I could use performance measurement, finally to get so socalled "data summary"?

I understand maybe these tools provide flexibility for users to collect whatever they want, but I'm a kind of unaware of where to start from. After reading the manual I feel like confused about using these tools. Is there before&after relationship using these tools? Actually I would like to collect very simple KPIs, such as throughput, Queue waiting time, Processor Utilization, AGV Utilization. Would you give me some more example of how use these tools?

0 ·

1633332095412.png (26.9 KiB)

Joerg Vogel shanice.c commented · Oct 04 2021 at 7:47 AM

What do you want to achieve depends on the distribution sequence of input and the process duration of your model. If you create all items at model run start the efficiency is poor. If you create only items if the level of availability of your processes is good the efficiency rises. If you want to compare different option setting you need to control parameters in different scenarios in the experimenter.

Maybe you want to evaluate your model after a warmup has been finished and all involved stations have something to do.
Perhaps you can work with the tutorials to gather data on a small and easy model of your own first. Then if you have problems with your model, you come here back and tell us about them and share your testing model with us.

0 ·

shanice.c Joerg Vogel commented · Oct 04 2021 at 9:41 AM

@Joerg Vogel Thanks for explanation. I'll go through tutorials.

0 ·

question