Error training with flexsim_training.py

Question

question

Lorenzo_pio G asked Oct 14, '22 Jacob W2 answered Oct 26, '22

Error training with flexsim_training.py

Hi. I have developed a model in flexsim in which I need to optimize the idle rate of a processor by applying RL to a robot. It has to pick up the boxes from the right queue and decide on which combiner to leave it, based on the number of boxes present on the pallet placed on the top combiner (Combiner1). However, the moment I go to train the agent through the code in flexsim_training.py, this writing remains perpetually and the agent is not trained.

progettoTesi.fsm

Software Version:

FlexSim 22.2.2

robot reinforcement learning training

1665770307006.png (56.2 KiB)

progettotesi.fsm (439.0 KiB)

· 3

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jacob W2 ♦ commented · Oct 20, 2022 at 06:57 PM

@Lorenzo_pio G,

Are you able to run the flexsim_env.py code? It seems like visual studio is unable to connect to FlexSim.

0 ·

Lorenzo_pio G Jacob W2 ♦ commented · Oct 21, 2022 at 09:03 AM

The connection to FlexSim is made correctly. In fact, FlexSim opens the moment I run the code. I also tested the flexsim_env.py file on the fsm ChangeoverTime file, which I implemented to follow the RL tutorial, and everything works correctly. There must be an error in initializing the RL parameters in my project, but I can't figure out what it is.

0 ·

Jeanette F ♦♦ Lorenzo_pio G commented · Oct 25, 2022 at 10:39 PM

Can you post your model? It looks like you may have tried to but there is not an attachment.

0 ·

Answer 1 · 2022-10-26T18:53:16Z

Jacob W2 answered Oct 26, '22

Hi @Lorenzo_pio G,

It looks like the reason your model wasn't able to train is because the reinforcement learning was never able to make a decision with the "On request action" activity. If you run your model you will see that the value in your Action parameter table will never change.

The reason behind this is that reinforcement learning decision events cannot be linked to object flows. You can however use the custom code activity to act as request decision itself. This option can be found by selecting "Control," Reinforcement learning," and then request decision. Select the reinforcement learning tool that you will be using, and then whenever a token enters that activity it will drive the decision making process in the reinforcement learning.

Once that is fixed, your model should be ready to start training.

RL_JW.fsm

reinforcement-learning.jpg (83.2 KiB)

rl-jw.fsm (320.1 KiB)

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

question