question

Lorenzo_pio G avatar image
0 Likes"
Lorenzo_pio G asked Jacob W2 answered

Error training with flexsim_training.py

1665770307006.png

Hi. I have developed a model in flexsim in which I need to optimize the idle rate of a processor by applying RL to a robot. It has to pick up the boxes from the right queue and decide on which combiner to leave it, based on the number of boxes present on the pallet placed on the top combiner (Combiner1). However, the moment I go to train the agent through the code in flexsim_training.py, this writing remains perpetually and the agent is not trained.

progettoTesi.fsm

FlexSim 22.2.2
robotreinforcement learningtraining
1665770307006.png (56.2 KiB)
progettotesi.fsm (439.0 KiB)
· 3
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Jacob W2 avatar image
0 Likes"
Jacob W2 answered

Hi @Lorenzo_pio G,

It looks like the reason your model wasn't able to train is because the reinforcement learning was never able to make a decision with the "On request action" activity. If you run your model you will see that the value in your Action parameter table will never change.

The reason behind this is that reinforcement learning decision events cannot be linked to object flows. You can however use the custom code activity to act as request decision itself. This option can be found by selecting "Control," Reinforcement learning," and then request decision. Select the reinforcement learning tool that you will be using, and then whenever a token enters that activity it will drive the decision making process in the reinforcement learning.

reinforcement-learning.jpg

Once that is fixed, your model should be ready to start training.

RL_JW.fsm


rl-jw.fsm (320.1 KiB)
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.