question

WillRS avatar image
0 Likes"
WillRS asked WillRS edited

RL Tutorial with gymnasium environment?

I'm trying to run the RL tutorial, with the migration from gym to gymnasium environment, but the files don't work. I tried running both FlexSim 23.0 and FlexSim 23.2.0, but the files don't work!


I am unable to adjust the parameters for the gymnasium environment

FlexSim 23.2.0
reinforcement learning
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

2 Answers

Kavika F avatar image
0 Likes"
Kavika F answered WillRS commented

Hey @WillRS, what kind of errors are you getting? I was able to get it working and posted an answer about it in this post. Things to look out for:

  • Including the correct packages
  • Changing function parameters (and tuples that receive results from them)
  • Making sure you're using the right Python versions and your pip is up to date
· 3
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

WillRS avatar image WillRS commented ·

Thank you very much for your response!

Now I was able to run the training model correctly!

1 Like 1 ·
WillRS avatar image WillRS commented ·

Thank you Kavika for your reply!

I saw your answer in the post. I used your environment file and tried to tweak the training file (I don't know if I did it correctly).

I used python 3.10.13, stable-baselines3 (2.1.0), gym 0.26.2, gymnasium 0.29.1 and Shimmy 1.2.1.

I also attached the two python configuration files that I used.


When executing the training file, it runs until the training file is saved, but then the error below appears.

1697236030376.png

example-flexsim-env.pyflexsim_training_update.py

0 Likes 0 ·
1697236030376.png (16.6 KiB)
example-flexsim-env.py (7.6 KiB)
flexsim-training-update.py (1.5 KiB)
Kavika F avatar image Kavika F ♦ WillRS commented ·

@WillRS, after downloading the files fresh from the website, changing the file paths to match mine, making the changes to gymnasium.Env, and running them, I was able to train without errors. Here are some things you may want to check:

  • Make sure your "reset" function has the "options" optional parameter (flexsim-env.py, line 29)
  • Run the flexsim_env.py program to ensure it works locally (I noticed on line 215 in
  • flexsim-env.py you didn't change the number of received results from env.reset().)
  • Try setting the reset optional parameter "seed" to default to 0 rather than None; that may cause an issue; especially if in the flexsim-training.py (line 34-ish) you remove the env.seed(i) line.
    • Also around this section, you changed a few lines that may have an effect in the future, but shouldn't be causing this issue you're seeing
  • Keep the check_env(env) (flexsim-training.py, line 18) - that's important to ensure your FlexSimEnv follows the gymnasium.Env specs properly

Let me know if any of these help

0 Likes 0 ·
WillRS avatar image
0 Likes"
WillRS answered WillRS edited

It worked, thank you very much!

5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.