question

Noah Z avatar image
2 Likes"
Noah Z asked Jason Lightfoot edited

Experimenter replications repeat exactly after "Reset Experiment"

I ran the experimenter for 10 replications of a scenario and then looked at the reported performance data. One of the ten replications reported a value for a performance measure that was well outside of an expected value. I decided to rerun the experimenter to see if the behavior showed up again when running 10 more replications. Well it did show up again but it was in the exact same replication. Repeating this one more time gave me the same result again. When we "Reset Experiment" and then re-run, is the experimenter supposed to produce the exact same replication results as last time if we haven't changed anything? Is there a way to change this so it doesn't repeat and instead generates new data?

FlexSim 16.0.1
experimenterreplicationsrandom stream
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Matthew Gillespie avatar image
2 Likes"
Matthew Gillespie answered Jason Lightfoot edited

This is by design. Your model should run the same way every time if you don't change anything. This makes it possible to debug your model when it's not doing what you want it to. This extends to the experimenter too. Each replication should run exactly the same each time you run the experimenter. This way you can choose a specific replication and run it in your model (using the same scenario variables and random stream numbers) when you need to debug or you want to see how a replication produced a certain result.

If you need more data you can increase the number of replications.

There is a way to stop the model repeating random streams (Statistics > Repeat Random Streams), but in general we don't recommend this as it makes it nearly impossible to debug models.

· 5
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Noah Z avatar image Noah Z commented ·

Thanks for clarifying!

1 Like 1 ·
Tee Hiett avatar image Tee Hiett commented ·

Matthew, Is there a way to eliminate the Repeat Random Streams option for Experiments? I have unchecked the Repeat Random Streams in the model under the Debug tab but that does not seem to work with the Experiment.

Thanks, Tee

1 Like 1 ·
Matthew Gillespie avatar image Matthew Gillespie ♦♦ Tee Hiett commented ·

@Cliff King

0 Likes 0 ·
AD1 avatar image AD1 Matthew Gillespie ♦♦ commented ·

@Matthew Gillespie , @Cliff King , Was this question about preventing repeat streams for an experiment answered? Running into a similar issue.


Thank you

0 Likes 0 ·
Show more comments