question

russ.123 avatar image
0 Likes"
russ.123 asked russ.123 commented

Using Reinforcement Learning to train a queue "send to port"

sequencing reinforcement learning.fsm
Hello!

After queue6 in my model, there are 4 processors that are able to process all 4 different items, but each of them "specializes" in processing one item type faster than all the others (eg. processor 1 processes all items, but processes item type 1 the fastest as compared to all other processors). I am trying to use reinforcement learning to train queue6 which processor to send the item to such that the process time of a batch (one of each item, for example) is as short as possible. How do I do this? Below is a snapshot of my model for reference.

1733978195228.png

FlexSim 23.2.0
reinforcement learning
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Felix Möhlmann avatar image
0 Likes"
Felix Möhlmann answered russ.123 commented

All that seems to be missing is:

- Fixing the code in the entry trigger of the sink (the label name is part of the path in the Model.find() the first pickoption.

- Setting the observation variable for the Send to Port decision. Since there is no delay before the queue would send the item, we can assume that the sent item is the last item to enter the queue and thus set the parameter in the queue's On Entry trigger.

- Set the Send to Port field to use the action parameter.

· 3
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

russ.123 avatar image russ.123 commented ·

thank you for the reply!

i have fixed the code mentioned in your first point.


as for the second point onwards, I am unsure of what you mean.

With reference to the model attached, "ReinforcementLearning2" controls "Queue6" which is supposed to direct each flow item in it to the processor that takes the least time to process the item depending on its type. As mentioned, each processor is able to process a certain item type the faster than the other processors, and the agent is supposed to learn that with the goal of reducing processing time as much as possible.

Are you referring to the part that I have circled in the picture below? and if so, how am I to configure it? 1734012036900.pngsequencing reinforcement learning.fsm


0 Likes 0 ·
Felix Möhlmann avatar image Felix Möhlmann russ.123 commented ·

You can use that field, but it's slightly more convenient to set the observation in the On Entry trigger of the queue in my opinion, since you have a reference to the item available there.

capture1.png

And for the last point set the port the item is send to to the action parameter.

capture2.png

Though the parameter names don't really make sense for this context. Calling the observation parameter "ItemType" and the action parameter "Port" or something similar might be better.

0 Likes 0 ·
capture1.png (7.6 KiB)
capture2.png (4.5 KiB)
russ.123 avatar image russ.123 Felix Möhlmann commented ·
Thank you so much for the help Felix!
0 Likes 0 ·