question

Leo L6 avatar image
0 Likes"
Leo L6 asked Natalie White commented

Experimenter Replications Stuck on 'Recording'

Hello,

What steps shall I take to debug this error? image: 1687284780302.png

When I start the experimenter the progress of each replication seems good until it completes and moves into the 'recording' phase which is when it freezes. Each replication runs fine separately in the model view (no errors), it is just in the experimenter which this happens. Unfortunately I'm not able to post the model file but what steps would I take to start debugging this?

FlexSim 22.0.16
experimenter
1687284780302.png (25.9 KiB)
· 1
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Natalie White avatar image Natalie White commented ·

Hi @Leo L6, was Jordan Johnson's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

0 Likes 0 ·

1 Answer

·
Jordan Johnson avatar image
1 Like"
Jordan Johnson answered Jordan Johnson commented

During the Recording phase, many things happen:

  • In the child process that ran the model:
    • All Statistics Collectors (including those inside charts) calculate values for any "When value is accessed" column.
    • All Calculated Tables (including those inside charts) are updated, if they were out of date. For Calculated Tables set to "By Lazy Interval" or "Always", this means the query for that table runs again.
    • If you are saving state, the state file is saved at this point.
    • All result data is copied into a special tree, including output/system console stuff.
    • The special tree is saved and send to the main process
  • In the main process that is running the Experimenter:
    • The tree from the child is loaded
    • The result data is copied to the database
    • The status of the replication is changed to Complete
    • The tree from the child is destroyed

Usually, issues with the recording phase happen because of a Calculated Table, or a chart with a Calculated Table inside. If there are issues in that stage, the child process can simply exit before it can transfer results.

To debug the issue, try these steps:

  1. Use the test tube button to apply the replications that get stuck to your model.
  2. Reset and run your model to the end time. Watch for exceptions in the system console.
  3. Once the model stops running, open every dashboard in your model. Watch for exceptions in the system console, or crashing, or any issues. Alternatively, see how long it takes to open the dashboard. There is a chance that some replications generate significantly more data than others, although this seems less likely.
  4. If that doesn't reveal issues, view the table for every calculated table in your model. Again, watch for delays or exceptions.
· 6
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Leo L6 avatar image Leo L6 commented ·
Thanks for the insight!

So I do have a lot of dashboards/table in this model, but I did try running the experimenter after deleting all the dashboards and tables in the model and the same error happens. Would this eliminate tables/dashboards as the cause of this error? If so, what could be the other causes of this error?

I also used the test tube button to apply replications but the individual replications run fine with no exceptions/errors showing up in the system console.

One more thing that's frustrating about this error is that after it gets stuck in the recording phase, I can't stop the model run. Clicking the stop/start buttons on the experimenter don't appear to do anything. Closing the FlexSim application doesn't stop the processes from running in the background either when I check task manager. I need to force quit through task manager and re open the model to run it again.

0 Likes 0 ·
Jason Lightfoot avatar image Jason Lightfoot ♦ Leo L6 commented ·

You could try increasing the size of your non-interactive desktop heap as described in the webserver pages of the manual.

0 Likes 0 ·
Leo L6 avatar image Leo L6 Jason Lightfoot ♦ commented ·
I upped the heap size 4x and still got the same error
0 Likes 0 ·
Jordan Johnson avatar image Jordan Johnson ♦♦ Leo L6 commented ·

Hmm. I don't know how much more we can debug without a model. Here's one more thing to try, that's maybe more direct that what you tried before:

  1. Open your original model. Save it as a new model, for this debugging test.
  2. Apply the scenario/replication that gets stuck, as you did before
  3. Close all the windows except the FlexSim application.
  4. Run the model to the end at max speed.
  5. Run this script in a Script Console:
Map task;
task.scenarioID = 1;
task.replication = 1;
function_s(Model.find("Tools/Experimenter"), "packResults", task);

This is the exact code that fires at the end of a replication. Does it throw exceptions? If not, then you'll need to provide your model for further debugging.

Another thought I had while writing this is that deleting charts doesn't always delete all the parts of the chart. Sometimes, invisible charts are running in the background. Here is a script to delete those. Then try running the experiment again, to see if the issue is resolved.

treenode statisticsFolder = Model.find("Tools/Statistics");
Array usedStatsObjects;
var dashboards = Model.find("Tools/Dashboards").subnodes;
for (int d = 1; d <= dashboards.length; d++) {
    var widgets = dashboards[d].subnodes;
    for (int w = 1; w <= widgets.length; w++) {
        Object widget = widgets[w];
        treenode saver = widget.attrs.find("saver");
        treenode storedNode = widget.attrs.find("stored");
        if (saver && saver.value == "DashboardWebStatGraph" && storedNode) {
            var storedLinks = storedNode.subnodes;
            for (int s = 1; s <= storedLinks.length; s++) {
                treenode linkedNode = storedLinks[s].find("+/~");
                if (linkedNode && linkedNode.up == statisticsFolder) {
                    usedStatsObjects.push(linkedNode);
                }
            }
        }
    }
}
var statsObjects = statisticsFolder.subnodes;
for (int i = statsObjects.length; i >= 1; i--) {
    treenode statsObject = statsObjects[i];
    if (usedStatsObjects.indexOf(statsObject) <= 0) {
        print(statsObject);
        // statsObject.destroy();
    }
}
0 Likes 0 ·
Leo L6 avatar image Leo L6 Jordan Johnson ♦♦ commented ·

With this script:

  1. Map task;
  2. task.scenarioID = 1;
  3. task.replication = 1;
  4. function_s(Model.find("Tools/Experimenter"), "packResults", task);

No exceptions are thrown.

Your second script to delete the charts doesn't seem to work though. It's throwing 'undefined variable objs'

0 Likes 0 ·
Show more comments

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.