Maryam H2 avatar image
Maryam H2 asked Logan Gold commented

RL: Observation parameters

Hello -

If my observation space consists of the contents of several queues (multi-discrete) in the RL tool, is it correct to set the observation parameters as follows?

Model.parameters["Queue1_Content"].value = Model.find("Queue1").subnodes.length;
Model.parameters["Queue2_Content"].value = Model.find("Queue2").subnodes.length;

The action space is updating the content coming onto the queue, so I'm wondering what should be in Decision Events (Event 1 for Queue 1, Event 2 for Queue 2, etc.). Is "On Content Change" for the object Queue correct?

@Felix Möhlmann can you take a look at this one?

FlexSim 24.1.0
reinforcement learningobservationdecision event
· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Felix Möhlmann avatar image Felix Möhlmann commented ·

I'm not really understanding what role the RL agent is supposed to fill, so I can only give general advice.

A decision event, as the name suggests, should fire when a decision from the RL agent is required on how to proceed. Meaning before something happens in the model that depends on one or multiple action parameters.

You can set the observations parameters like this in the "On Observation" trigger, or you continually update them while the model is running. The latter would be slightly faster if the content changes less often than a decision is made.

0 Likes 0 ·
Logan Gold avatar image Logan Gold ♦♦ commented ·

Hi @Maryam H2,

We haven't heard back from you. Were you able to solve your problem? If so, please add and accept an answer to let others know the solution. Or please respond to the previous comment so that we can continue to help you.

If we don't hear back in the next 3 business days, we'll assume you were able to solve your problem and we'll close this case in our tracker. You can always comment back at any time to reopen your question, or you can contact your local FlexSim distributor for phone or email help.

0 Likes 0 ·

0 Answers