Reinforcement learning training problem

Question

question

mark zhen asked Sep 11, '23 Jeanette F commented Sep 22, '23

Reinforcement learning training problem

When deciding the upper and lower limits, it seems that you cannot set the interval to be very large, otherwise you will run env again.

This result will appear when

In the picture above, I set the upper and lower limits very large.

The picture below shows the normal upper and lower limits.

So I want to understand the meaning of the state he gave back.

Also, I have been stuck here during training without any error message. Is it a problem with my computer? Or have I set the wrong settings in this model (I have no problem training other models) )

Software Version:

FlexSim 22.0.16

reinforcement learning python

1694416762216.png (98.2 KiB)

1694416777652.png (427.2 KiB)

1694416908817.png (183.7 KiB)

· 7

Kavika F ♦ commented · Sep 11, 2023 at 05:56 PM

Hey @mark zhen, could you please upload a model with these settings and your python files so we can review their configurations? Thank you.

0 ·

mark zhen Kavika F ♦ commented · Sep 11, 2023 at 09:30 PM

@Kavika F I tried changing to a new set of reward functions and status as attached

He shouldn't be 20. I feel like my judgment may not work.

Or if my writing is wrong, please help me correct it. The way I want to write it is that I want to calculate my completion. If he completes it within the deadline, I will give him +1. If it exceeds the time limit, I will give him -1.0911.fsm

0 ·

1694467361735.png (6.7 KiB)

0911.fsm (394.9 KiB)

mark zhen mark zhen commented · Sep 13, 2023 at 12:46 AM

@Kavika F @Felix Möhlmann @Jason Lightfoot

Basically, my model should be fine now, but I can't train smoothly and the images are all stuck in the same place. 0912.fsm

0 ·

1694566025271.png (185.0 KiB)

Show more comments

Jeanette F ♦♦ commented · Sep 22, 2023 at 02:35 PM

Hi @mark zhen , was Kavika F's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

0 ·

Answer 1 · 2023-09-18T21:53:39Z

Kavika F answered Sep 18, '23

Hey @mark zhen, after some debugging, it seems that it gets hung up on training because you have more "total_timesteps" than the model runs for. That means that the model finishes running before it can reach the total amount of time steps it wants to perform.

To solve this issue, you can do a few things.

(1) You can Repeat the Source Table schedule so the model will run indefinitely, that way you're sure to hit the specified number of timesteps.

You could also expand how big the schedule is so there's less repeat in the data.

(2) You can decrease the number of timesteps to run within the amount of time your model runs.

I recommend the first one.

1695073978778.png (8.1 KiB)

question