question

Maryam H2 avatar image
0 Likes"
Maryam H2 asked Maryam H2 commented

RL model reward

I followed all the steps in the reinforcement learning tutorial, but I'm facing an issue and that is the reward is not updating like it's supposed to. Plus, when I open the model, the time inside it skyrockets to something huge. Can't figure out what's going wrong here. Can anyone help on this?

* I've got all the Python packages updated and in place, and the model is set up in meters, so that's why the Boxes might seem a bit on the small side.

Socket connected
Waiting for READY message
READY
Waiting for ActionSpace message
Waiting for ObservationSpace message
Sending Reset message
Waiting for Observation message
{"done":0,"reward":0,"state":2}
Sending Action message: 3
Waiting for Observation message


Also, is this the last version of env.py after update to gymnasium from gym?example-flexsim-env.py

ChangeoverTimesRL_1.fsm

FlexSim 24.0.0
reinforcement learning
changeovertimesrl-1.fsm (29.9 KiB)
example-flexsim-env.py (7.5 KiB)
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Nil Ns avatar image
0 Likes"
Nil Ns answered Maryam H2 commented

Hello Maryam

I just saw your question and it seems you're encountering the same issue as Clara (Reinforcement learning - FlexSim Community).


I hope the solution provided to Clara will be helpful to you as well.



· 3
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Maryam H2 avatar image Maryam H2 commented ·
@Nil Ns Thanks! yes, this is very helpful, I have one question:

how do you know all decision events in RL failing? was it just by looking at the stack up issue in the visual model or is there any other way to check these things?



0 Likes 0 ·
Nil Ns avatar image Nil Ns Maryam H2 commented ·

Hello,

I realized that the events were not triggering because in the model, the `LastItemType` parameter only updates when this event triggers. When it wasn't updating, I began to suspect there was an issue. To confirm this, I opened the code of the request action and placed a debug point (the model stops each time it's going to execute that line). Seeing how it only stopped at the reset, I realized there was a problem.

1706628371461.png

1706628379850.png


Another option could have been to print to the Output Console every time a call is made to the RL (in the On Observation code).

1706628397638.png

0 Likes 0 ·
1706628371461.png (16.5 KiB)
1706628379850.png (12.9 KiB)
1706628397638.png (18.8 KiB)
Maryam H2 avatar image Maryam H2 Nil Ns commented ·
@Nil Ns got it, thanks for your clarification!
0 Likes 0 ·