Reinforcement Learning with FlexSim Model. Fps decreases with training time

Question

question

Nitin S2 asked Sep 23, '22 Jeanette F commented Sep 29, '22

Reinforcement Learning with FlexSim Model. Fps decreases with training time

I am following the tutorial (link) on the FlexSim website to integrate RL with an existing FlexSim model. Using the codes provide there too for training and inference.

As I was training the RL agent, I could see that the fps (frames per second) showing a consistent decrease (see graph) with model training time. High fps means more iterations of FlexSim environment (faster training) and it usually should fluctuate around a constant value.
I was wondering if this is a known case and if someone could point me to a way to fix is.

fps-drop.PNG

Software Version:

FlexSim 22.0.0

reinforcement learning

fps-drop.png (8.5 KiB)

· 1

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jeanette F ♦♦ commented · Sep 29, 2022 at 04:33 PM

Hi @Nitin S2, was Jacob W2's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

0 ·

Answer 1 · 2022-09-23T19:12:38Z

Jacob W2 answered Sep 23, '22 Nitin S2 commented Sep 26, '22

Hi @Nitin S2,

In reinforcement learning the fps generally will decrease over time because the algorithm takes more and more time to think about what to do. There is a possibility that we have a bug FlexSim that slows FlexSim down, but you'd probably notice that running the model over and over. Example: run an experiment with 50 replications, only using 1 core. Does the model run slower and slower? If not, then the slowdown must be the RL tool just thinking a lot.

· 1

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Nitin S2 commented · Sep 26, 2022 at 03:56 AM

Thanks for the response.

I am not sure that fps decreasing with time is an RL exploration issue. Will run the experiment and find more on it.

0 ·

question