question

Scarlett X avatar image
0 Likes"
Scarlett X asked Jordan Johnson commented

Questions about reinforcement learning example

Hi everyone, https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/Training/Training.html

I have same questions about reinforcement learning in flexsim,


  1. If I reset and run model 1000 times, it means I had trained model for 1000 times?
  2. If Q1 is true, how can I use the trained model which had been trained for 1000 times to another model?
  3. If I want to increase processor from 1 to 3, I only need to set 3 decision events in reinforcement learning?

Thank you!!

FlexSim 22.0.0
reinforcement learning
· 1
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jason Lightfoot avatar image Jason Lightfoot ♦♦ commented ·

Hi @Scarlett X, was Jordan Johnson's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

0 Likes 0 ·

1 Answer

Jordan Johnson avatar image
2 Likes"
Jordan Johnson answered Jordan Johnson commented

To train the model, you need to run configure the Reinforcement Learning tool. Then you need to run flexsim_training.py. When you run the model yourself, using the reset and run buttons, the model will fire the OnRequestAction trigger, whenever you request an action. But this does not train the AI. When you run flexsim_training.py, the Python script will launch the model and run it. In this case, when you request an action, the OnRequestAction trigger does not fire. Instead, FlexSim sends the observations to the RL algorithm, and gets an action in response. The RL algorithm uses FlexSim as the environment. Running flexsim_training.py is how you train the AI.

Once you have run flexsim_training.py, you'll see that it produces a .zip file. That .zip file contains the AI. You can use the trained brain by running flexsim_inference.py, which becomes a server. You can then configure your OnRequestAction to send the observations to that server, and get an action back.

https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/UsingATrainedModel/UsingATrainedModel.html

To increase the number of processors, you would need to make sure the LastItemType parameter matches the last item processed. For example, suppose Processor1 works on a Type 2 item, and Processor 2 works on a Type 5 item. Then, when Processor1 requests an action, the LastItemType value needs to be 2. And when Processor 2 requests an action, the LastItemType value needs to be 5. And yes, then you can listen to the three decision events.

· 7
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Scarlett X avatar image Scarlett X commented ·

Thank you for your prompt reply.Does it means I had trained the model when I run flexsim_training and python show "Saving model...

Waiting for input to do some test runs..." ?

0 Likes 0 ·
Jordan Johnson avatar image Jordan Johnson ♦♦ Scarlett X commented ·

Yes, that is correct. When the Python scripts we provide show "Saving model...", that means that python is saving the agent as a .zip file.

0 Likes 0 ·
Scarlett X avatar image Scarlett X Jordan Johnson ♦♦ commented ·

I'm sorry ,what's "Waiting for input to do some test runs" ? What it want me to do?

0 Likes 0 ·
Show more comments
Show more comments