Reinforcement Learning Training Actions Not Updating

Question

question

janders1 asked Mar 5, '24 janders1 commented Mar 7, '24

Reinforcement Learning Training Actions Not Updating

I am doing the initial RL training here. As far as I know, I do everything exactly as it says, but when I get to step 29, ItemType does not change, and the processor does not chose random values. Model is attached. I have tried the tutorial from the beginning twice in a row. Maybe a version issue?

jordan.fsx

I also compared it to a working project, from someone who did the tutorial on an earlier version of FlexSim and noticed the FSX was different. The working one (right) seems to have some type of coupling object that mine doesn't. Not sure if that is helpful.

Software Version:

FlexSim 24.0.2

reinforcement learning

1709676515505.png (85.0 KiB)

jordan.fsx (668.9 KiB)

· 1

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Joerg Vogel commented · Mar 06 at 09:08 AM

@janders1, can you insert the keyword “gym” into the search tool line of this site. And you order the results by time. You do this by activating the clock icon. Then I suggest you look for answers related to your question. Please keep in mind that tutorials have been written when involved features were different in comparison to today. It is just a hint to solve your problem faster yourself. But you can also wait to get an answer here.

0 ·

Answer 1 · 2024-03-06T21:10:59Z

Jordan Johnson answered Mar 6, '24 janders1 commented Mar 7, '24

I think the issue is that the event listening mechanism on the RL tool isn't working correctly at the moment.

I was able to work around this issue by calling requestdecision() directly instead of listening for it.

In the attached model, I deleted the event in the RL tool and I added a call to requestdecision() in the Pull Strategy code on the Processor. Let me know if that works for you.

jordan_1.fsx

jordan-1.fsx (668.9 KiB)

· 2

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jordan Johnson ♦♦ commented · Mar 06 at 09:28 PM

Update: it looks like there is an issue with the UI when sampling the Pull Strategy event. As an alternative to calling requestdecision(), you can change the event in the tree from "Pull Strategy" to "PullStrategy" (no space).

Fix the node at MODEL:/Tools/ReinforcementLearning/ReinforcementLearning1>variables/events/Event1/eventName

1 ·

1709760508624.png (48.7 KiB)

janders1 Jordan Johnson ♦♦ commented · Mar 07 at 08:38 PM

This worked. Thank you Jordan.

0 ·

question