Reinforcement learning training problem

Question

question

mark zhen asked Sep 11 2023 at 7:22 AM Jeanette F commented Sep 22 2023 at 2:35 PM

Reinforcement learning training problem

When deciding the upper and lower limits, it seems that you cannot set the interval to be very large, otherwise you will run env again.

This result will appear when

In the picture above, I set the upper and lower limits very large.

The picture below shows the normal upper and lower limits.

So I want to understand the meaning of the state he gave back.

Also, I have been stuck here during training without any error message. Is it a problem with my computer? Or have I set the wrong settings in this model (I have no problem training other models) )

Software Version:

FlexSim 22.0.16

reinforcement learning python

1694416762216.png (98.2 KiB)

1694416777652.png (427.2 KiB)

1694416908817.png (183.7 KiB)

· 7

Kavika F ♦ commented · Sep 11 2023 at 5:56 PM

Hey @mark zhen, could you please upload a model with these settings and your python files so we can review their configurations? Thank you.

0 ·

mark zhen Kavika F ♦ commented · Sep 11 2023 at 9:30 PM

@Kavika F I tried changing to a new set of reward functions and status as attached

He shouldn't be 20. I feel like my judgment may not work.

Or if my writing is wrong, please help me correct it. The way I want to write it is that I want to calculate my completion. If he completes it within the deadline, I will give him +1. If it exceeds the time limit, I will give him -1.0911.fsm

0 ·

1694467361735.png (6.7 KiB)

0911.fsm (394.9 KiB)

mark zhen mark zhen commented · Sep 13 2023 at 12:46 AM

@Kavika F @Felix Möhlmann @Jason Lightfoot

Basically, my model should be fine now, but I can't train smoothly and the images are all stuck in the same place. 0912.fsm

0 ·

1694566025271.png (185.0 KiB)

Show more comments

Jeanette F ♦♦ commented · Sep 22 2023 at 2:35 PM

Hi @mark zhen , was Kavika F's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

0 ·

______

Cookie preferences

Your privacy is important to us and so is an optimal experience. To help us customize information and build applications, we collect data about your use of this site.

May we collect and use your data?

Learn more about the Third Party Services we use and our Privacy Statement.

Strictly necessary – required for our site to work and to provide services to you

These cookies allow us to record your preferences or login information, respond to your requests or fulfill items in your shopping cart.

YES

Improve your experience – allows us to show you what is relevant to you

These cookies enable us to provide enhanced functionality and personalization. They may be set by us or by third party providers whose services we use to deliver information and experiences tailored to you. If you do not allow these cookies, some or all of these services may not be available for you.

YES

NO

Customize your advertising – permits us to offer targeted advertising to you

These cookies collect data about you based on your activities and interests in order to show you relevant ads and to track effectiveness. By collecting this data, the ads you see will be more tailored to your interests. If you do not allow these cookies, you will experience less targeted advertising.

YES

NO

Are you sure you want a less customized experience?

We can access your data only if you select "yes" for the categories on the previous screen. This lets us tailor our marketing so that it's more relevant for you. You can change your settings at any time by visiting our privacy statement

Your experience. Your choice.

We care about your privacy. The data we collect helps us understand how you use our products, what information you might be interested in, and what we can improve to make your engagement with Autodesk more rewarding.

May we collect and use your data to tailor your experience?

Explore the benefits of a customized experience by managing your privacy settings for this site or visit our Privacy Statement to learn more about your options.

Answer 1 · 2023-09-18T21:53:39Z

Kavika F answered Sep 18 2023 at 9:53 PM

Hey @mark zhen, after some debugging, it seems that it gets hung up on training because you have more "total_timesteps" than the model runs for. That means that the model finishes running before it can reach the total amount of time steps it wants to perform.

To solve this issue, you can do a few things.

(1) You can Repeat the Source Table schedule so the model will run indefinitely, that way you're sure to hit the specified number of timesteps.

You could also expand how big the schedule is so there's less repeat in the data.

(2) You can decrease the number of timesteps to run within the amount of time your model runs.

I recommend the first one.

1695073978778.png (8.1 KiB)

question