Reinforcement learning

Question

question

Clara asked Jan 30, '24 Clara commented Feb 1, '24

Reinforcement learning

I am trying to simulate model with AI using reinforcement learning. I am following your tutorial of flexsim 2022 (https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/Training/Training.html) . But I am facing issue in python It cant able to fetch observation data. It did not update further value.

ChangeoverTimesRL.fsm

Version 24.0.2

Software Version:

FlexSim 24.0.1

reinforcement learning

1706610806990.png (308.2 KiB)

1706610817993.png (13.7 KiB)

changeovertimesrl.fsm (29.8 KiB)

Answer 1 · 2024-01-30T13:04:21Z

Nil Ns answered Jan 30, '24 Clara commented Feb 1, '24

Hello,

I've reviewed your model and it appears that it should be working perfectly. However, it seems there might be a bug causing the issue you're experiencing.

I'm sending you a modified model that resolves this problem. I believe this is a better approach and I highly recommend it. The problem lies in the Decision Events within the Reinforcement Learning tab. We’re going to delete them all (as they seem to be failing) and we’ll call the RL from another place.

The solution involves using ProcessFlow to create a token each time an event triggers, and then calling the RL from a customCode. The default code for this is located in control. After this, we remove the token.

I hope this helps!

changeovertimesrl.fsm

changeovertimesrl.fsm (34.8 KiB)

· 4

Clara commented · Jan 31 at 09:19 AM

Thank you so much!!

0 ·

Clara commented · Feb 01 at 06:35 AM

Hello, even tough your answer solved the issue, i have kept going with the training part and i'm facing another problem now.

It seems that, because we changed the way observation and action space are defined, it's not able to inherit from the gymnasium class correctly. (Error image attached).

Thanks in advance!

0 ·

1706769166412.png (22.7 KiB)

Nil Ns Clara commented · Feb 01 at 06:40 AM

Hello Clara,

We will try to look at it as soon as possible.

However, since it’s not the same problem as before, I strongly recommend that you create a new question, so it will have more visibility for all users

0 ·

Clara Nil Ns commented · Feb 01 at 06:54 AM

Ok, thanks for you time!

0 ·

question