Reinforcement Learning for a Simple Problem - Random Actions Test not working

Question

question

your_boy_gorja asked Jul 17, '22 your_boy_gorja commented Jul 19, '22

Reinforcement Learning for a Simple Problem - Random Actions Test not working

I am trying to use Reinforcement Learning functionalities in FlexSim for a test problem in my case. Attached is the FlexSim model. Essentially the task is to find an optimal vessel shipping schedule from Production Tank to the two customer tanks. My goal was to use reinforcement learning to do this and followed the tutorial here.

The actions for me are the destinations to which the vessels are shipped and the states are the tank levels at Cylindric1, Cylindric2 and Cylindric3. For now I put a fixed reward just to get started and kept 100 days for a single episode. I have some attached images for what I have implemented.

What I see is something I am not able to understand. For some reason I am not able to see the random actions being selected i.e. ships being shipped randomly to the two destinations. Also, while trying to test the model with flexsim_env.py, I get a message "Waiting for Observation message" with the code not running further.

I guess for some reason, the agent is not able to find observations and hence its not able to run further. Can anyone please help me by having a look at it? Thanks in advance.

Software Version:

FlexSim 22.0.0

processflow reinforcement learning scheduling reward action

observations.png (18.0 KiB)

actions.png (15.2 KiB)

rl-propertiesz.png (44.0 KiB)

flexsim-env.png (33.9 KiB)

demo-sc-rl.fsm (318.1 KiB)

Answer 1 · 2022-07-18T16:50:50Z

Jordan Johnson answered Jul 18, '22 your_boy_gorja commented Jul 19, '22

The issue is that in order to request a decision, you are listening to the OnEntry of an activity in Process Flow. There is a known bug that prevents this from working.

However, to work around it, you can add a Custom Code activity previous to the Assign Labels activity that you are listening to, and you can have that Custom Code call the requestdecision() command. There is a pickoption for that, under Control, and then under Reinforcement Learning.

· 1

your_boy_gorja commented · Jul 19, 2022 at 07:34 AM

Thanks a lot.

This solved the issue for me.

0 ·

question