Idea

Pedro Ortelani avatar image
12 Likes"
Pedro Ortelani suggested Joerg Vogel edited

Experimenter execute scenarios before replications.

Hello,

Our team is facing a situation when running long models on Experimenter, it takes a lot of time, so I asked if it was possible to run each scenario before running all replications, this way we could get partial results while the model is still running.

I got an answer that solved the problem (thank you @jason.lightfoot :)) but with a workaround that can be not very user friendly:
https://answers.flexsim.com/questions/93802/experimenter-run-scenarios-before-every-replicatio.html?childToView=93882#answer-93882

When running models via Experimenter, it always execute all replications from one scenario before running the next scenarios. We think it would be a good idea to have an option inside Experimenter, that switch the execution order from "executing all replications of one scenario first" to "execute one replication from each scenario before executing all replications".

Thanks!


experimenter
· 3
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Paula Carneiro Martins avatar image Paula Carneiro Martins commented ·

@Jörg Vogel, the solution works but it isn't user friendly. Our team use to have a lot of scenarios those they want to avaliated but they need more than 1 replication to present the variability of that. As the model takes long, it is interesing that they can following the results for each scenario during run time.

This can help them to change the strats based in partial results or check if they have a configuration error or making sure about the results before all run time.

The solution puting just 1 replication works, but in case to have 20 replications, you need to put 2 models for running, and for avoiding the duplicates, you need to abort the 1st replication of the second model... What is possible, but is not user friendly .

So, we believe if we had an option for choose "How do you want to run your model by experimenter" could be a good approach.

If you have a computer with 40 threads available and you have a configuration error in scenario 5, using the standard option you will check the problem just in the third cycle of running,what can take long .In the second option, you are able to see in the first cycle, because of that could be helpful.

2 Likes 2 ·
Joerg Vogel avatar image Joerg Vogel commented ·

What is the problem to set the number of replications to "1".

0 Likes 0 ·
Jason Lightfoot avatar image Jason Lightfoot ♦ Joerg Vogel commented ·

I can see the benefit of gradually building a picture, across scenarios - whether that's getting an idea of the result or detecting problems with a particular scenario - the earlier you can see something the less time wasted doing relatively worthless replications.

0 Likes 0 ·

1 Comment

·
anthony.johnson avatar image
3 Likes"
anthony.johnson commented

We are in the process of redesigning the experimenter at the moment. One of the target features will be that you can "continue" an experiment, i.e. add more replications to various scenarios. This should fulfill your goal. You can initially run just one replication of each scenario, then narrow down and run more replications, add scenarios, etc. once you have a baseline of results.

· 1
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jason Lightfoot avatar image Jason Lightfoot ♦ commented ·

A quick thought - If a user has more cores on their PC than scenarios, then it might make more sense to tie the child instances to scenarios until all replications are done reducing the scenario setups

1 Like 1 ·

Write a Comment

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Your Opinion Counts

Share your great idea, or help out by voting for other people's ideas.

Related Ideas

Experimenter results sometimes do not display correctly in FlexSim 2017 (17.0.0)

Create a performance measure from statistic of every object in model, group, or class

Performance Measure from Statistics Collector and Calculated Tables

Chinese characters show error in Performance Measure Results

Experimenter Statistics - Save Only Select Data

Allow all PFM Names and Labels to be changed