question

Vothan Salomão avatar image
0 Likes"
Vothan Salomão asked Felix Möhlmann commented

AGVs and ReinforcementLearning

I'm trying to apply reinforcement learning in my FlexSim model using AGVs. The idea is for them to learn how to assign AGVs to tasks more effectively. I'd like to know if anyone has any tips on what I can do in this case. I've built the model, and I want to use a globalTable as an observation for reinforcement learning, which contains the following information about the AGVs: current control point (Cp), last Cp, destination Cp, and the distance between the current AGV and the destination Cp. However, I noticed that FlexSim's Reinforcement Learning module is not able to read my ParameterTables. I've made various modifications, and as a last resort, I converted everything to integers, but even then, Reinforcement Learning can't read these ParameterTables. I'd like to know if someone could help me, whether it's possible to achieve this learning, and what might be a better approach to this problem. Thank you for your attention.

My model: ReinforcementLearning_AGVsv11.fsm

My GlobalTable :1696884951224.png
ReinforcementLearning: 1696885017399.png

FlexSim 23.2.1
agvreinforcement learningglobal tablesparameters table
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

1 Answer

Felix Möhlmann avatar image
0 Likes"
Felix Möhlmann answered Felix Möhlmann commented

The parameter name must be a string (text) value, not a pointer. It also has to be unique across all parameter tables. So you could name them CP_CAR2, LastCP_CAR2, Destination_CAR2 and so on.

· 2
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Vothan Salomão avatar image Vothan Salomão commented ·

Hello @Felix Möhlmann , how are you? Even after making these changes, I still can't understand why my parameters table is not showing up in the reinforcement learning module, even when I input integers as requested in the documentation
ReinforcementLearning_AGVsv12.fsm

0 Likes 0 ·
Felix Möhlmann avatar image Felix Möhlmann Vothan Salomão commented ·

Thanks, I'm doing fine. I hope so are you.

Sorry that I missed the time I checked your model. Your parameters are set to "Passthrough". The Reinforcement Learning tool requires one of the types listed under observation space.

capture1.png

The boundaries should also be properly set, so the RL algorithm knows how many possible values to expect for that parameter.

This does also mean that you have to copy the table values to the parameters. Depending on what happens more often - the table values changing or the RL algorithm making a decision - you can either update the parameter whenever the table is updated. Or you copy the entire table every time an observation is made (in the On Observation trigger).

capture2.png

That would look something like below.

Table agvTable = Table("AGVs");
for(int row = 1; row <= agvTable.numRows; row++) {     string agvName = agvTable[row][1];     Model.parameters["CP_" + agvName].value = agvTable[row][2];     Model.parameters["LastCP_" + agvName].value = agvTable[row][3];     Model.parameters["Destination_" + agvName].value = agvTable[row][4];     Model.parameters["Distance_" + agvName].value = agvTable[row][5]; }
0 Likes 0 ·
capture1.png (16.0 KiB)
capture2.png (2.4 KiB)