question

Properly tuning the reward in a Reinforcement Learning model is very difficult, and each unique model needs to be individually tunes. The only way to find the correct rewards is to iterate through the following process over and over:

1. Plan out a set of observations, actions, and a reward system.
2. Train an AI using that reward system. Be sure to train long enough to be sure that it's working or not working. For small models like this, I'd say observe for 1 million steps to see if the policy learns something. In that time, the AI should learn to improve its reward. If the average reward has not improved in that time, it usually means the AI doesn't have the right set of observations.
3. Use the model from the above training to see what the AI did learn. How did it maximize reward? It is very common for an AI to maximize reward and perform in an unexpected way. Observe what behaviors the AI uses to maximize its reward.
4. You may discover that the reward system you created incentivized improper behavior, or that the AI was unable to learn anything at all. You may need major changes, or you may need only minor changes, depending on how your AI performs. Using what you have learned, return to step one.

Also keep in mind this rule about reward: reward should not accumulate between actions. For example, if I want to reward throughput.

It is very helpful to keep a journal or log of what you tried and how it performed. This process is time consuming and difficult, and a log can help you remember what you learned from previous attempts.

If you get "inf" in your reward value, that usually means that somewhere is a divide by zero. I note that the Sink updates its Reward label with this value:

1000/(Model.time-current.LastTime)

In your model, it is possible that two items enter the Sink at exactly the same clock time, so that the denominator is zero.