question

Arthur Ml avatar image
0 Likes"
Arthur Ml asked Jeanette F commented

Setting a parameter when resetting the environment in Reinforcement Learning

Hi everyone,

I am doing Meta Reinforcement Learning for production scheduling. Therefore, I predefined several job lists (n=50) that should be scheduled optimally with RL. In MetaRL, the algorithm regards each list as "task" and the algorithm should be able to select a specific task (i.e. 1 list of the 50 lists) when resetting the environment. I want realize that, by changing the value of a global parameter, which then leads to the usage of the specific list. I.e.: episode is finished -> MetaRL selects. e.g. task=14 for the next episode -> parameter is set to 14 -> flexsim is resetted and job list 14 is used in the model now.

Currently, my code for resetting flexsim looks like this:

def _reset_flexsim(self):
    if self.verbose:
        print("Sending Reset message")
    resetString = "Reset?"
    if hasattr(self, "seedNum"):
        resetString = "Reset:" + str(self.seedNum) + "?"
    self._socket_send(resetString.encode())

Can anyone help me please how I can set the parameter value when resetting flexsim?

Thanks in advance!

FlexSim 22.2.1
reinforcement learning
· 1
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Jeanette F avatar image Jeanette F ♦♦ commented ·

Hi @Arthur Ml, was Kavika F's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.

0 Likes 0 ·

1 Answer

Kavika F avatar image
0 Likes"
Kavika F answered Arthur Ml commented

Hey @Arthur Ml, one way you can set the parameter's value on model reset is using the "OnModelReset" trigger found in the Toolbox:

1679003018177.png

This gives you a code editor that will execute when the model is about to Reset. Here you can grab a reference to the Parameter's value cell and set it with the task for the next run. In my example I simply set it with the output count of a Processor.

change-param-on-reset.gif

set-param-on-reset.fsm

Hope this helps!


· 5
5 |100000

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Arthur Ml avatar image Arthur Ml commented ·

Dear @Kavika F ,

thanks for the quick response.

Sorry, my question was not specific enough. How can I set a parameter value from Python, when resetting the model/ RL environment? I use a socket connection for connecting Python with Flexsim as presented in the documentation (https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/KeyConcepts/KeyConcepts.html). So far, with this socket connection, I can launch, reset, and close Flexsim, and I can send actions and receive observation from the model. But how can I set a specific parameter value when resetting the model?

Thanks and best regards,
Arthur


0 Likes 0 ·
Kavika F avatar image Kavika F ♦ Arthur Ml commented ·

Oh okay. Once you reset your model, you can use the following command to set a parameter:

controller.setParameter("ParameterName", value)

If you look at the FlexSimPy.cpp, you can see a list of commands you can use from the controller (and within the FlexSim):

1679097754270.png

0 Likes 0 ·
1679097754270.png (196.9 KiB)
Arthur Ml avatar image Arthur Ml Kavika F ♦ commented ·

Is there a way to set a Parameter in Flexsim with the help of the "flexsim_env.py" script you provided in the documentation (https://docs.flexsim.com/en/22.1/ModelLogic/ReinforcementLearning/KeyConcepts/KeyConcepts.html)? Because I am using the scripts from the documentation to connect to Flexsim and do the training.

0 Likes 0 ·
Show more comments