Ankur A3 avatar image
Ankur A3 asked Andrew O commented

Experimenter Scenario Replication Error?

Hi Team,

While running multiple replications using experimenter, some replications are throwing errors which can be seen in console output tab. Still run is completed and results are generated for the scenario.

I have 2 questions:

1. Is it okay to believe on results in this case?

2. How to deep dive on errors for particular replication?

Thank you!

FlexSim 21.1.5
· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jacob W2 avatar image Jacob W2 ♦ commented ·

Hi @Ankur A3,

It's hard to know how to help without looking at your model. To receive a more accurate solution, please post your model or a sample model that demonstrates your question.

Proprietary models can be posted as a private question visible only to FlexSim U.S. support staff. You can also contact your local FlexSim distributor for phone or email help.

0 Likes 0 ·
Andrew O avatar image Andrew O commented ·

Hi @Ankur A3, was one of Jordan Johnson's or Jason Lightfoot's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

0 Likes 0 ·
Jordan Johnson avatar image
Jordan Johnson answered

1) If some replications have exceptions and others don't, I wouldn't trust the results from the replications that had exceptions. However, you can always run the replication interactively.

2) To run a specific replication of a scenario interactively, hover over that replication on the chart, and click on the test tube button:
That button applies the parameter values, warmup time, stop time, and replication number to the model, and it resets the model. You should be able to run the model and get the exact same behavior interactively as you got in the experiment run.

1667415429668.png (42.7 KiB)
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jason Lightfoot avatar image
Jason Lightfoot answered Jason Lightfoot edited

1. It Depends. If it's less than the required confidence level for the runs and/or you assume they are not biased then it might be okay. (I meant it might be okay to beleive the remaining results if you have enough confidence remaining - the error replications I would discard)

2. Set the "Repeat streams of Replication" number to one of the troublesome replications and run the model to see the problem unfold. Store the state of the model and load that if you think it will help you understand how the problem occurred by looking at the end state.

· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Ankur A3 avatar image Ankur A3 commented ·
Hi @Jason Lightfoot,

I am not clear on my 2nd question. Let me ask with some more clarity.

I am running 10 replications for scenario1 using experimenter and scenario 1 is throwing error in replication #5. How to deep dive in replication 5?

The reason of asking this question is that we don't know which seed value has created that particular replication so how to run that replication in 3D environment to check the errors?

Can you add some images or model for easy explanation?

Thank you!

0 Likes 0 ·
Jason Lightfoot avatar image Jason Lightfoot ♦ Ankur A3 commented ·

Under the scenario tab set the model to scenario 1 by selecting the column and pressing the button to the right.


Then under the Advanced tab set the replication to repeat to 5:

1667374106815.png Now run the model - you should see the same results as the experiment - if not check for repeatability.

0 Likes 0 ·
1667374039587.png (10.2 KiB)
1667374106815.png (4.9 KiB)