Reinforcement learning - Error by more decision events

Question

question

katerina-fratczak asked Jul 2, '24 katerina-fratczak commented Jul 3, '24

Reinforcement learning - Error by more decision events

Hello,

I have created my own model with two decision events in queues and push logic. Both events are in one RL window. Observation, action space I have set Multidiscrete. But when trying to run the flexsim_training.py there is an error in python and FlexSim freezes. Could you please tell me if there is anything wrongly defined in FlexSim or if it is necessary to adjust any of the python scripts?

Thank you

15_our_3_model_push.fsm

reinforcement learning

1719916027919.png (187.9 KiB)

1719916841971.png (70.1 KiB)

15-our-3-model-push.fsm (32.2 KiB)

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Answer 1 · 2024-07-02T11:36:46Z

Felix Möhlmann answered Jul 2, '24 katerina-fratczak commented Jul 3, '24

The relevant part of the error message is right at the bottom:
"Expected: int64, actual dtype: float32"

Try specifying the data type in the flexsim_env file.

See also this question:

https://answers.flexsim.com/questions/140869/issue-with-observation-space-in-reinforcement-lear.html

1719920100052.png (43.8 KiB)

· 5

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

katerina-fratczak commented · Jul 03 at 07:40 AM

Thank you, Mr. Möhlmann,

I have specified int64 here in env file:

Now the training runs, but when running the test runs, this happens:

And in python it stops like this:

How can this be solved please?

0 ·

1719992160710.png (19.9 KiB)

1719992256667.png (32.3 KiB)

1719992332683.png (32.7 KiB)

1719992361213.png (9.8 KiB)

Felix Möhlmann katerina-fratczak commented · Jul 03 at 08:45 AM

Hm, based on the error message I would have expected you'd need to set the dtype as float32.

I don't know why that error would occur. I was able to train and run the model you uploaded in the question. Did you make any changes to it?

0 ·

katerina-fratczak Felix Möhlmann commented · Jul 03 at 09:39 AM

No, I have not done any changes to the FlexSim file. dtype float32 does not work, with int64 it works, but still the problem by test runs. And mainly on 204 there must be int64, (on row 193 maybe not necessary, it works also without dtype here) But we have changed some things in the python scripts long time ago (because the original did not work - using gymnasium etc.), but it worked well with training model. Maybe there could be the problem.

flexsim_env.py

0 ·

1719999415193.png (79.2 KiB)

flexsim-env.py (8.4 KiB)

Show more comments

question