Error in initializing the environment using the flexsim_env.py file

Question

question

Lorenzo_pio G asked Oct 16, '22 Felix Möhlmann commented Oct 24, '22

Error in initializing the environment using the flexsim_env.py file

Hi. I have developed a model in flexsim in which I need to optimize the idle rate of a processor by applying RL to a robot. It has to pick up the boxes from the right queue and decide on which combiner to leave it, based on the number of boxes present on the pallet placed on the top combiner (Combiner1). I used a process flow to manage the movement of the robots and defined all the parameters of the RLToolbox. However, initializing the environment via the flexsim_env.py file I get the following messages on the terminal:

The model in flexsim opens correctly; however, it appears that parameters regarding observations and actions are not being read correctly. At the end of loading, in fact, the terminal should return me the message "Waiting for input to close flexsim". Can you tell me what this may be due to? Is there some error in the initialization of the parameters?

progettoTesi.fsm

Software Version:

FlexSim 22.2.2

robot reinforcement learning flexsim_env.py

1665915821188.png (46.0 KiB)

progettotesi.fsm (439.1 KiB)

Answer 1 · 2022-10-21T10:58:56Z

Felix Möhlmann answered Oct 21, '22 Felix Möhlmann commented Oct 24, '22

It seems using the "OnEntry" event of a process flow activity as the decision trigger doesn't work properly. If you use the "OnLoad" event of the robot instead it works. (Sorry, that I didn't check this before saying you could use the PF activity. I didn't expect there to be a (likely) bug)

You should also adjust the code that sets the observation parameters. You are currently not dividing the idle time by the model time, so the observation wouldn't be the rate but the total time in the state. Also, since you are using an integer parameter, the calculated value should be rounded before applying it to the parameter.

if(Model.time == 0)
{
    Model.parameters["IdleRate"].value = 0;
}
else
{
    Model.parameters["IdleRate"].value = Math.round(Model.find("Sigillatura").as(Object).stats.state().getTotalTimeAt("idle")/Model.time*100, 0)/;
}

· 12

Lorenzo_pio G commented · Oct 21, 2022 at 11:17 AM

So do I need to change the Decision Events of the RL Tool? Should I delete the Event1 event and sample the Robot to which I need to add the OnLoad trigger?

0 ·

Lorenzo_pio G Lorenzo_pio G commented · Oct 21, 2022 at 11:23 AM

What is the purpose of the "/" character in line 7? Because it gives me an error the moment I enter the ";".

0 ·

Felix Möhlmann Lorenzo_pio G commented · Oct 21, 2022 at 11:40 AM

Yes, use the sampling tool to select the new event from the robot.

I don't know how the "/" at the end of the line got there. There shouldn't be one.

0 ·

Show more comments

Answer 2 · 2022-10-20T22:11:15Z

Kavika F answered Oct 20, '22 Kavika F converted comment to answer Oct 21, '22

Hey @Lorenzo_pio G, when I opened your model, it said that it couldn't set your parameters to 0. I checked them and they had a lower bound of 1. If you change their lower bounds to 0, the program should be able to set them to 0 and allow you to run the simulation. Let me know if that helps.

1666303863815.png (8.5 KiB)

· 1

Lorenzo_pio G commented · Oct 21, 2022 at 08:59 AM

Yes, I have already changed those parameters, but it still doesn't work.

0 ·

question