Reinforcement learning - problem by test runs

Question

question

katerina-fratczak asked Jul 3, '24 katerina-fratczak commented Jul 4, '24

Reinforcement learning - problem by test runs

Hello,

I have problem by running the training test runs of my model with push logic and two decision events:

In python stops it like this:

The python scripts (modified long time ago by us because the original training scripts did not run well) run by the training model well. I am not sure now, if the problem is in FlexSim or in python.

Thank you

15_our_3_model_push.fsm

flexsim_env.py

flexsim_training.py

reinforcement learning

1720002556798.png (49.7 KiB)

1720002593625.png (18.7 KiB)

15-our-3-model-push.fsm (35.0 KiB)

flexsim-env.py (8.4 KiB)

flexsim-training.py (2.4 KiB)

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Answer 1 · 2024-07-03T19:45:41Z

Jordan Johnson answered Jul 3, '24 katerina-fratczak commented Jul 4, '24

I did a little debugging. I think something may have changed with stable_baselines3 since we wrote the demo script. It looks like the return value of model.predict() is a tuple containing the array of actions, rather than just the array of actions.

To fix it, I changed:

action = model.predict(observation)

to this:

(action, _) = model.predict(observation)

The second version "unpacks" the tuple and saves the first value in the action variable.

I'll add an issue to our bug list to investigate this issue, to see if we need to adjust to changing software.

· 1

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

katerina-fratczak commented · Jul 04 at 06:25 AM

Hello, thank you very much! I have changed it in the training.py as you suggested and it works now :-)

0 ·

1720074256648.png (25.2 KiB)

question