Idea

Pedro Ortelani suggested Nov 23, '20 Joerg Vogel edited Nov 23, '20

Experimenter execute scenarios before replications.

Hello,

Our team is facing a situation when running long models on Experimenter, it takes a lot of time, so I asked if it was possible to run each scenario before running all replications, this way we could get partial results while the model is still running.

I got an answer that solved the problem (thank you @jason.lightfoot :)) but with a workaround that can be not very user friendly:
https://answers.flexsim.com/questions/93802/experimenter-run-scenarios-before-every-replicatio.html?childToView=93882#answer-93882

When running models via Experimenter, it always execute all replications from one scenario before running the next scenarios. We think it would be a good idea to have an option inside Experimenter, that switch the execution order from "executing all replications of one scenario first" to "execute one replication from each scenario before executing all replications".

Thanks!

experimenter

· 3

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Paula Carneiro Martins commented · Nov 23, 2020 at 03:54 PM

@Jörg Vogel, the solution works but it isn't user friendly. Our team use to have a lot of scenarios those they want to avaliated but they need more than 1 replication to present the variability of that. As the model takes long, it is interesing that they can following the results for each scenario during run time.

This can help them to change the strats based in partial results or check if they have a configuration error or making sure about the results before all run time.

The solution puting just 1 replication works, but in case to have 20 replications, you need to put 2 models for running, and for avoiding the duplicates, you need to abort the 1st replication of the second model... What is possible, but is not user friendly .

So, we believe if we had an option for choose "How do you want to run your model by experimenter" could be a good approach.

If you have a computer with 40 threads available and you have a configuration error in scenario 5, using the standard option you will check the problem just in the third cycle of running,what can take long .In the second option, you are able to see in the first cycle, because of that could be helpful.

2 ·

Joerg Vogel commented · Nov 23, 2020 at 02:56 PM

What is the problem to set the number of replications to "1".

0 ·

Jason Lightfoot ♦♦ Joerg Vogel commented · Nov 23, 2020 at 03:20 PM

I can see the benefit of gradually building a picture, across scenarios - whether that's getting an idea of the result or detecting problems with a particular scenario - the earlier you can see something the less time wasted doing relatively worthless replications.

0 ·

anthony.johnson commented Nov 23, '20

We are in the process of redesigning the experimenter at the moment. One of the target features will be that you can "continue" an experiment, i.e. add more replications to various scenarios. This should fulfill your goal. You can initially run just one replication of each scenario, then narrow down and run more replications, add scenarios, etc. once you have a baseline of results.

· 1

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jason Lightfoot ♦♦ commented · Nov 23, 2020 at 09:30 PM

A quick thought - If a user has more cores on their PC than scenarios, then it might make more sense to tie the child instances to scenarios until all replications are done reducing the scenario setups

1 ·

Idea

Experimenter execute scenarios before replications.

1 Comment

Things to know…

Your Opinion Counts

IDEA DETAILS

Related Ideas