question

Lorenzo_pio G avatar image
0 Likes"
Lorenzo_pio G asked Jacob W2 answered

Error training with flexsim_training.py

1665770307006.png

Hi. I have developed a model in flexsim in which I need to optimize the idle rate of a processor by applying RL to a robot. It has to pick up the boxes from the right queue and decide on which combiner to leave it, based on the number of boxes present on the pallet placed on the top combiner (Combiner1). However, the moment I go to train the agent through the code in flexsim_training.py, this writing remains perpetually and the agent is not trained.

progettoTesi.fsm

FlexSim 22.2.2
robotreinforcement learningtraining
1665770307006.png (56.2 KiB)
progettotesi.fsm (439.0 KiB)
· 3
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jacob W2 avatar image Jacob W2 ♦ commented ·
@Lorenzo_pio G,

Are you able to run the flexsim_env.py code? It seems like visual studio is unable to connect to FlexSim.

0 Likes 0 ·
Lorenzo_pio G avatar image Lorenzo_pio G Jacob W2 ♦ commented ·

The connection to FlexSim is made correctly. In fact, FlexSim opens the moment I run the code. I also tested the flexsim_env.py file on the fsm ChangeoverTime file, which I implemented to follow the RL tutorial, and everything works correctly. There must be an error in initializing the RL parameters in my project, but I can't figure out what it is.

0 Likes 0 ·
Jeanette F avatar image Jeanette F ♦♦ Lorenzo_pio G commented ·
Can you post your model? It looks like you may have tried to but there is not an attachment.
0 Likes 0 ·

1 Answer

·
Jacob W2 avatar image
0 Likes"
Jacob W2 answered

Hi @Lorenzo_pio G,

It looks like the reason your model wasn't able to train is because the reinforcement learning was never able to make a decision with the "On request action" activity. If you run your model you will see that the value in your Action parameter table will never change.

The reason behind this is that reinforcement learning decision events cannot be linked to object flows. You can however use the custom code activity to act as request decision itself. This option can be found by selecting "Control," Reinforcement learning," and then request decision. Select the reinforcement learning tool that you will be using, and then whenever a token enters that activity it will drive the decision making process in the reinforcement learning.

reinforcement-learning.jpg

Once that is fixed, your model should be ready to start training.

RL_JW.fsm


rl-jw.fsm (320.1 KiB)
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.