Reinforcement Learning's versioning problem

Question

question

mark zhen asked Aug 8, '23 Jordan Johnson commented Jul 26, '24

Reinforcement Learning's versioning problem

When I was running the new py environment, I found that the training could not be executed smoothly. I don’t know which link is the problem. I bought a new computer and built a new vscode, but it couldn’t be executed smoothly.

PS C:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL> & C:/Users/R2504_111/AppData/Local/Programs/Python/Python311/python.exe "c:/Users/R2504_111/Documents/FlexSim 2022 Projects/RL/flexsim_training.py"

Initializing FlexSim environment...

Traceback (most recent call last):

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 402, in check_env

env.reset(seed=0)

TypeError: FlexSimEnv.reset() got an unexpected keyword argument 'seed'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 53, in <module>

main()

File "c:\Users\R2504_111\Documents\FlexSim 2022 Projects\RL\flexsim_training.py", line 19, in main

check_env(env) # Check that an environment follows Gym API.

^^^^^^^^^^^^^^

File "C:\Users\R2504_111\AppData\Local\Programs\Python\Python311\Lib\site-packages\stable_baselines3\common\env_checker.py", line 404, in check_env

raise TypeError("The reset() method must accept a `seed` parameter") from e

TypeError: The reset() method must accept a `seed` parameter

Software Version:

FlexSim 22.0.16

reinforcement learning python

· 4

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Joerg Vogel commented · Aug 08, 2023 at 08:59 AM

You are sure, you did all needed upgrades to use Python 3.11 on FlexSim 22.0 yourself?

0 ·

python-flexsim-release-notes.jpg (13.6 KiB)

mark zhen Joerg Vogel commented · Aug 08, 2023 at 10:17 AM

Can you show me all the necessary upgrades?

0 ·

Julie Weller commented · Aug 08, 2023 at 03:10 PM

Hey @mark zhen! This looks like more of a python question than a FlexSim question. You can try searching other websites for answers or posting there. Let us know if you have a question related to FlexSim!

0 ·

Julie Weller commented · Aug 14, 2023 at 01:43 PM

Hi @mark zhen , was one of Phil BoBo's or Jordan Johnson's answers helpful? If so, please click the "Accept" button at the bottom of the one that best answers your question. Or if you still have questions, add a comment and we'll continue the conversation.

If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always comment back to reopen your question.

0 ·

Answer 1 · 2023-08-08T16:36:04Z

Phil BoBo answered Aug 8, '23

The provided python scripts are example code to demonstrate how to communicate with FlexSim from a reinforcement learning algorithm. If they don't work perfectly with a particular version of python or another language, library, package, or implementation, then you should customize them according to the needs of your project. As explained in the first paragraph of Getting Started with OpenAI Gym (flexsim.com).

It looks like the version of stable_baselines you are using requires a "seed" parameter on the environment's reset() function. In flexsim_env.py, try adding a seed parameter to that function. Something like this:

    def reset(self, seed=None):
        self.seedNum = seed
        self._reset_flexsim()
        state, reward, done = self._get_observation()
        return state

Alter the example python script to work if it isn't working. That's why the script is provided.

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Answer 2 · 2023-08-08T17:28:22Z

Jordan Johnson answered Aug 8, '23 Jordan Johnson commented Jul 26, '24

Is the issue resolved by commenting out the check_env() line? I think I have seen that come up before.

· 5

5 |100000

Attachments: Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

mark zhen commented · Aug 09, 2023 at 12:36 AM

I have no way to solve the problem I am encountering now

0 ·

1691541410873.png (22.4 KiB)

Joerg Vogel mark zhen commented · Aug 09, 2023 at 10:11 AM

@mark zhen , you checked if a path description allows characters of more bytes length. Maybe there are only characters allowed of one byte length like an ASCII set. You are dealing with a programming language coming from an origin completely different to a windows system

https://docs.python.org/3/tutorial/interpreter.html.

0 ·

Phil BoBo ♦♦ mark zhen commented · Aug 14, 2023 at 02:00 PM

Your comment doesn't describe the problem you are encountering now, just a screenshot of a code snippet.

0 ·

mark zhen Phil BoBo ♦♦ commented · Jul 18 at 12:27 PM

@Phil BoBo @Jordan Johnson @Joerg Vogel @Kavika F @Felix Möhlmann

I'm having the same problem, but there is still no good way to solve it, I'll provide my code and related versions, I hope it can help me solve the problem, thank youflexsim_training.py flexsim_training.py flexsim_env.py

0 ·

1721305561181.png (20.0 KiB)

1721305666664.png (20.6 KiB)

flexsim-training.py (1.7 KiB)

flexsim-env.py (7.5 KiB)

Show more comments

question