How do I get environmental observations of temporary entities?

Question

question

Kuilin S asked Jul 21, '22 Kuilin S commented Jul 22, '22

How do I get environmental observations of temporary entities?

Hello everyone,

Now I'm dealing with a reinforcement learning model. How do I get environmental observations of temporary entities? For the "box" generated from the Source, how do I get its information into the observation parameter table? If I generated 100 "boxes", it is impractical to record its information with 80 lines of parameters. I have built a very simple model, it's incomplete, just to represent how to get the observations of BoxStatus into the parameter table. "0","1","2" are the Status.

temporaryEntities.fsm

Software Version:

FlexSim 22.0.4

reinforcement learning observation temporary entities

temporaryentities.fsm (267.0 KiB)

Answer 1 · 2022-07-21T08:06:27Z

Felix Möhlmann answered Jul 21, '22 Kuilin S commented Jul 22, '22

From my limited research into the topic when trying out the functionality myself, I don't believe most (if any) reinforcement learning algorithms deal with a variable amount of parameters or array values.

The only information these status labels allow to gather seems to be "how many items what have status" anyway. So you could have one parameter for each possible value (0, 1, 2) that shows the number of items with that status.

You could either adjust the parameter values together with the labels.

Or use the 'On Observation' trigger to update the parameters when an observation is made. For this I would push all items to a global variable array and then sum up the labels in the trigger. (The code below automatically removes empty entries (deleted items) from the array.)

temporaryentities_fm.fsm

1658390359801.png (17.4 KiB)

1658390649047.png (27.0 KiB)

temporaryentities-fm.fsm (267.0 KiB)

· 5

Kuilin S commented · Jul 21, 2022 at 09:18 AM

In this way ("how many items what have status"), it may make sense ..But if I want to get observation -- the actual location.x of each temporary item, how will it be?

0 ·

Felix Möhlmann Kuilin S commented · Jul 21, 2022 at 10:06 AM

As I hinted at above, if the RL-algorithm you are using supports arrays/varying variable counts you can create your own custom observation space to pass the needed values on.

If not, each observation variable needs its own parameter.

Depending on what you want to achieve you could potentially observe each item individually with a fixed set of parameters.

0 ·

Kuilin S Felix Möhlmann commented · Jul 22, 2022 at 01:13 AM

I see. I just want to use the RL using Flexsim, I have already get several observation parameters for the fixed resources. My observation type is "multidiscrete". For the "Status" of temporary items, I have already get the observation on "how many items what have status" . I think about to have the location.x of each temporary items. It can be "Scroll to update", have you ever met this kind of problem before? Please help me..

0 ·

Show more comments

question