question

Steven Chen avatar image
0 Likes"
Steven Chen asked Phil BoBo edited

Reinforcement Learning and Parameter Table Bound

Hello,

The observation variables in parameter table are not really bounded in the case I working on.

It's fine to set a large value as bound but I always wonder why parameter table must be strict? If unbounded variables need to save in global table, isn't it against the purpose to expose variables, because those bounded and unbounded variables were suppose to be on same table.

I think it's better to make optional lower bound and upper bound. What do you think?


By the way it will be nice to have these features in future, is this correct space to propose request?

Add "copy parameter" button to parameter table.

Listen to group instead object in Decision Events of Reinforcement Learning tool.

FlexSim 22.1.0
reinforcement learningparameters table
· 2
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Iago MF avatar image Iago MF commented ·

Hi Steven,


If you need a solution right now, you can get the maximun allowed range (limited by the number of bits for each type) writting -1/0 for lower bound and 1/0 for upper bound.


I hope it helps you.

1 Like 1 ·
Matthew Gillespie avatar image Matthew Gillespie ♦♦ commented ·

@Steven Chen For submitting feature requests you should create an idea on this site so that other users can vote and discuss the idea. You can also send the request to [email protected]. See How do I submit feature requests or bug reports?

0 Likes 0 ·

1 Answer

Phil BoBo avatar image
0 Likes"
Phil BoBo answered Phil BoBo edited

You should normalize the observation space parameters when training RL algorithms. See Reinforcement Learning Tips and Tricks — Stable Baselines documentation (stable-baselines.readthedocs.io)

Machine Learning algorithms will learn better and faster if your observation space variables are normalized between [0,1] or [-1,1]. Using unbounded variables or variables with large, unknown ranges will cause the training to not work very well.

Machine learning isn't magic; it's math. Parameter tables are strict with bounds set on the variables so that optimization algorithms can work well.

You can copy/paste with Ctrl+C/Ctrl+V in a Parameters table to copy parameters.

I'll add a case to the dev list with the suggestion for listening to Groups.

· 2
5 |100000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 12 attachments (including images) can be used with a maximum of 23.8 MiB each and 47.7 MiB total.

Steven Chen avatar image Steven Chen commented ·
How to use normalized value in parameter table?

I use continuous type but it is invisible in Reinforcement Learning tool.

0 Likes 0 ·
Phil BoBo avatar image Phil BoBo ♦♦ Steven Chen commented ·

If you want to use a continuous parameter, then you need to select a space type that supports continuous parameters. Use a Box space type. See The Reinforcement Learning Tool (flexsim.com)

To normalize a value, divide the difference between the value and the minimum by the difference of the maximum and the minimum:

double value = 17;
double min = 10;
double max = 20;
double normalized_0_1 = ((value - min) / (max - min)); // range [0, 1]
double normalized_neg1_1 = ((value - min) / (max - min)) * 2 - 1; // range [-1, 1]

See Feature scaling - Wikipedia

0 Likes 0 ·