question

Clara avatar image
0 Likes"
Clara asked Clara commented

Reinforcement learning

I am trying to simulate model with AI using reinforcement learning. I am following your tutorial of flexsim 2022 (https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/Training/Training.html) . But I am facing issue in python It cant able to fetch observation data. It did not update further value.

1706610806990.png

1706610817993.pngChangeoverTimesRL.fsm

Version 24.0.2


FlexSim 24.0.1
reinforcement learning
1706610806990.png (308.2 KiB)
1706610817993.png (13.7 KiB)
changeovertimesrl.fsm (29.8 KiB)
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Nil Ns avatar image
0 Likes"
Nil Ns answered Clara commented

Hello,

I've reviewed your model and it appears that it should be working perfectly. However, it seems there might be a bug causing the issue you're experiencing.


I'm sending you a modified model that resolves this problem. I believe this is a better approach and I highly recommend it. The problem lies in the Decision Events within the Reinforcement Learning tab. We’re going to delete them all (as they seem to be failing) and we’ll call the RL from another place.


The solution involves using ProcessFlow to create a token each time an event triggers, and then calling the RL from a customCode. The default code for this is located in control. After this, we remove the token.


I hope this helps!

changeovertimesrl.fsm



changeovertimesrl.fsm (34.8 KiB)
· 4
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Clara avatar image Clara commented ·
Thank you so much!!
0 Likes 0 ·
Clara avatar image Clara commented ·

Hello, even tough your answer solved the issue, i have kept going with the training part and i'm facing another problem now.

It seems that, because we changed the way observation and action space are defined, it's not able to inherit from the gymnasium class correctly. (Error image attached).
1706769166412.png
Thanks in advance!

0 Likes 0 ·
1706769166412.png (22.7 KiB)
Nil Ns avatar image Nil Ns Clara commented ·

Hello Clara,

We will try to look at it as soon as possible.

However, since it’s not the same problem as before, I strongly recommend that you create a new question, so it will have more visibility for all users

0 Likes 0 ·
Clara avatar image Clara Nil Ns commented ·
Ok, thanks for you time!
0 Likes 0 ·