What I want to accomplish in the model now is to find the final scheduling order within a fixed time, but regarding the observation space, I don’t understand why there will always be problems, and the example of SET UP TIME mentioned in the case, I want to understand if I want to use PROCESS TIME for the most similar thing is it ok?0312test.fsm
currently it looks like your observation space is throwing an error because of your code in line 15.
int stationNum = mostRecentFinishProcessor.name.slice(-1).toNum();
This is always returning 0 because none of your processors actually have a number at the end of their name. The slice function is pulling the letter "e" or "g".
Since I am not sure what case you are referring to, it is hard to help you regarding the rest of your question. I do however recommend looking up other forums on AI training specifically those dealing with openAI gym, which is what FlexSim uses, because users on those forums will generally have more experience with AI training than users on this forum.
Hey @mark zhen, you're getting an error in your "On Observation" because of your stationNum code.
int stationNum = mostRecentFinishProcessor.name.slice(-1).toNum(); Model.parameters["LastItemType"+stationNum].value = getvarnum(mostRecentFinishProcessor, "f_lastlabelval");
This would be valid if your stations had names that ended in numbers (i.e., Processor1, Processor2, etc.). However, you changed the names of your processors to be more accurate to their job (like "cut to length line"), which I think is good design, but if you'd like to grab a number from them you can instead set a label on each processor and pull that to get the "stationNum".
@Jacob W2 @Kavika F @Felix Möhlmann @Jason Lightfoot
i have a question now
My model does not need set up time, so do I need to rewrite my observation space?
Because I noticed that the sentence treenode curvar = assertvariable(current, "f_lastlabelval"); did not appear in the processing time'
Hi @mark zhen , was Felix Möhlmann's answer helpful? If so, please click the "Accept" button at the bottom of their answer. Or if you still have questions, add a comment and we'll continue the conversation.
If we haven't heard back from you within 3 business days we'll auto-accept an answer, but you can always unaccept and comment back to reopen your question.
If you are going to remove the type dependent setup time the "LastItemType" is probably not going to be a useful observation anymore.
I've implemented alternative observations in the attached model (and fixed up the model, properly integrating the fourth processor for example). The observations provide information about which processor is currently querying and how many items of each type/step combination are available in the queue. They don't show the actual quantity but instead a very rough estimate of none/few/many (relative to other types) to keep the amount of different states low so the RL algorithm can hopefully learn faster.
I also changed the reward. It now gives out a value between 0 and 1 that reflects the utilization across all processors since the last query.
Finally, I deleted most of the previous code/triggers/variables that are not used in the model to hopefully make it easier to understand how the model now works.
I want to know about the settings you made in the observation space! And the logic of the code
In addition, there are three problems that must be defined about reinforcement learning
s state (observation space)
a action
r reward (reward)
Can you explain, by the way, how you define these three questions?
The model keeps track of how many items of each type/step combination by incrementing/decrementing entries in a global variable array in the OnEntry/OnExit trigger of the queue.
The "OnObservation" code then uses a simple heuristic to translate these numbers into one of three possible values. If there are no items of the respective combination the parameter is set to 0. Otherwise the code compares the ratio of items to the ratio you would have if each combination had the same amount. With sixteen possible values (currently only eight are used due to how the steps are defined) you would on average expect 6.67% of items to be of a given type/step combination. If the actual ratio is lower than that, the parameter is set to 1, otherwise to 2.
This is only a rough example and could probably be improved (for example not count "inactive" combinations when determining the expected ratio).
To get an idea of what observations I want to implement, I just ask myself: What information would I, as a person, want in order to make an informed decision about what item to process next?
Similarly for the reward function: How would I measure how well I am doing at any given moment? This could for example also be the achievend throughput over the last X minutes.
17 People are following this question.
FlexSim can help you understand and improve any system or process. Transform your existing data into accurate predictions.
FlexSim is a fully 3D simulation software environment. FlexSim can be used to simulate any process in any industry.
FlexSim®, FlexSim Healthcare™, Problem Solved.®, the FlexSim logo, the FlexSim X-mark, and the FlexSim Healthcare logo with stylized Caduceus mark are trademarks of FlexSim Software Products, Inc. All rights reserved.
