Experimenter not making it through all replications

Question

question

Noah Z asked Oct 26, '20 Noah Z commented Nov 13, '20

Experimenter not making it through all replications

I have a model that I set to run 50 replications in the experimenter. It gets through 17-20 replications on the single scenario I am running and then the remaining replications never start.

Unfortunately I can't upload the model to this public forum or by private question due to it's sensitive nature but I'm wondering if there are any ideas from the community about what might be going on and what might be done to mitigate. I do realize not having the model to look at significantly reduces the chances of this getting figured out here.

My hunch is that the replications that are being run in the background while experimenter is working aren't terminating properly as when I reset the experimenter and try to re-run the experiment again, no replications begin at all. It takes me restarting my computer to be able to get any replications working again with the same outcome seen again (i.e. stopping at ~17 or so replications).

Any ideas?

Software Version:

FlexSim 20.0.0

experimenter flexsim 20.0.0

Answer 1 · 2020-10-28T14:21:27Z

Jordan Johnson answered Oct 28, '20 Noah Z commented Oct 28, '20

@jason.lightfoot's answer is an excellent guide to troubleshooting the experimenter. Another possibility is that you might be experiencing a bug with using the "restore original state" checkbox, which we fixed in version 20.0.7. If you upgrade to version 20.0.9 (the latest bug fix at time of writing), do you still experience the issue?

· 1

Noah Z commented · Oct 28, 2020 at 06:40 PM

Just ran the model with V 20.0.9 and it worked with no issues or hangups. Thanks for your help.

0 ·

Answer 2 · 2020-10-27T12:38:37Z

Jason Lightfoot answered Oct 27, '20 Noah Z commented Nov 13, '20

The replications are likely crashing and therefore not terminating correct as you guessed. If you start with 8 cores running then as each replication fails you have fewer cores picking up jobs - you only need 8 to fail and the experiment will stop.

Can you first check that the model is repeatable so that say replication 1 always gets the same result? Then check that you can run that replication interactively and get the same result? If you can then it means we should be able to diagnose each failing replication. If not then we need to diagnose why the model isn't repeatable.

In case you've not seen the option to select the replication number to run interactively, it's in the experimenter's advanced tab:

The next step is to then interactively run the replications that fail and look for problems that cause it to crash. I'd start with the replication that crashes earliest - so the one with the shortest green line on the progress indicator.

I'd try removing all running FlexSim instances using the task manager and see if that then allows you to start a new experiment without rebooting your machine. If you add the field 'commandline' to the task manager detailed process view you may be able to see which if any are child processes that have got stuck.

Please come back if you want more help with each step in this process and we'll see what we can do to guide you through it.

experimenteradvancedrepeatspecificreplicaiton.png (6.6 KiB)

· 7

Noah Z commented · Oct 28, 2020 at 12:29 PM

Thank you Jason for sharing that troubleshooting method. I'll follow this procedure today and update this thread once I have some new insights. One thing to note is that in experimenter, all the first 17 successful replications have a "full" green bar. They don't stop in the midway through their run.

0 ·

Jason Lightfoot ♦ Noah Z commented · Oct 28, 2020 at 01:47 PM

If an interactively run replication runs to that simulation end time without errors popping up or printing to the console (or an outright crash), then it sounds like something could be causing a problem in the PFM definitions and/or the process that gathers results from the child instances.

0 ·

Noah Z Jason Lightfoot ♦ commented · Oct 28, 2020 at 06:41 PM

I was previously using FS 20.0.0. Per Jordan's comments I upgraded to 20.0.9 and the experimenter ran with no issues. Thanks again for your insights.

0 ·

Ignacio commented · Nov 06, 2020 at 02:59 PM

The same thing is happening to my model experiment.

How do you go to a specific replication and run it interactively? @jason.lightfoot

0 ·

Jason Lightfoot ♦ Ignacio commented · Nov 06, 2020 at 03:15 PM

Just go to the Advanced tab of the experimenter and change the number against "Repeat Streams of Replication" - which means "set the interactive run to use the randome stream seed for this replication."

1 ·

Ignacio Jason Lightfoot ♦ commented · Nov 13, 2020 at 02:30 PM

I am running 30 replications and after 8 it stops. I changed the " "Repeat Streams of Replication" to 9, 10 and 11 and they all run without issued in interactive view. What could it be?

0 ·

Show more comments

question