How to make sure error of different replication results are small enough?

Question

question

shanice.c asked Oct 4, '21 Ryan Clark commented Oct 5, '21

How to make sure error of different replication results are small enough?

I've gone through the manual, the Experimenter is a tool that enables you to run the same simulation model multiple times, changing one or more parameters each time to see its effect on the performance measures.

But I'd like to know what if I don't need to change any variables(only 1 scenario), I simply want to run a model several times(ex: 10 replications) and then want to know the error between these 10 replications to make sure the error is small enough to prove the simulation result is strong enough to reflect the real world system? I'm having the question because generally or academically, we couldn't use the result of 1 replication to conclude to simulation result right? So I would like to know what tools or modules I could use to do I've said above in FlexSim. Thank you.

Software Version:

FlexSim 21.2.0

flexsim experimenter

· 1

Ryan Clark commented · Oct 05, 2021 at 05:48 PM

Hi @Fiona C, was one of Joerg Vogel's or David Seo's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

0 ·

Answer 1 · 2021-10-04T00:51:06Z

David Seo answered Oct 4, '21

@Fiona C

What do you mean exactly about 'error' during run FlexSim (simulation in general)?

For example, the result of each simulation is different from each or the tiny error during running like warning message? The model suould not have any error message in the model logic.

Your 'result error' means 'distribution of the result value' after running?

If so, it means 'distribution of the result value'.

If your current model result is exactly same with each running, make 'repeat random streams' uncheck. The option is located in 'Statistics>Repeat Random Streams'.

The option makes the result value distributed.

2021-10-04-094943.jpg (27.6 KiB)

· 3

shanice.c commented · Oct 04, 2021 at 07:27 AM

@David Seo If all processing time and flow item arrival time are constant, even if unchecked the"repeat random streams", the 10 replications result would have no difference?

0 ·

David Seo shanice.c commented · Oct 04, 2021 at 09:13 AM

@Fiona C Yes. If you set all processing time and arrival time to be constant, the results of the repeated replications are same. No difference. If the input value is contant and then the result is constant not deviation.

0 ·

shanice.c David Seo commented · Oct 04, 2021 at 09:42 AM

@David Seo Thanks for the reply. I'll keep this in mind.

0 ·

Answer 2 · 2021-10-04T00:44:23Z

Joerg Vogel answered Oct 4, '21

Your thoughts are correct. And you can do it exactly as you describe it. You let run your model several times in the experimenter.

· 3

shanice.c commented · Oct 04, 2021 at 07:24 AM

Hi,@Joerg Vogel I think what I want is the KPI like picture below(with average, standard deviation, Min, Max).

In order to gain this, is it necessary to use statistics collector before, so that I can use calculated table to receive KPI I want, and then I could use performance measurement, finally to get so socalled "data summary"?

I understand maybe these tools provide flexibility for users to collect whatever they want, but I'm a kind of unaware of where to start from. After reading the manual I feel like confused about using these tools. Is there before&after relationship using these tools? Actually I would like to collect very simple KPIs, such as throughput, Queue waiting time, Processor Utilization, AGV Utilization. Would you give me some more example of how use these tools?

0 ·

1633332095412.png (26.9 KiB)

Joerg Vogel shanice.c commented · Oct 04, 2021 at 07:47 AM

What do you want to achieve depends on the distribution sequence of input and the process duration of your model. If you create all items at model run start the efficiency is poor. If you create only items if the level of availability of your processes is good the efficiency rises. If you want to compare different option setting you need to control parameters in different scenarios in the experimenter.

Maybe you want to evaluate your model after a warmup has been finished and all involved stations have something to do.
Perhaps you can work with the tutorials to gather data on a small and easy model of your own first. Then if you have problems with your model, you come here back and tell us about them and share your testing model with us.

0 ·

shanice.c Joerg Vogel commented · Oct 04, 2021 at 09:41 AM

@Joerg Vogel Thanks for explanation. I'll go through tutorials.

0 ·

question